Destination and renaming of uploaded files
Hy insterested ones,
I added for you two new improvements to Zend_File_Transfer or if you work with Forms, to the file element.
The first one is that Zend_File_Transfer allows now the usage of file filters. Note that, of course, file filters work differently to normal string filters. They do not return the content, but the location of the file, aka the real location and filename. It should also be noted that file filters will be attached AFTER the file has been received. This is due to the fact that we are not allowed to change the file before it has been received, as we would then get a attacker exception from PHP itself. Input elements, for example, will do filtering before validation. File elements will always do validation and afterwards filtering. So you could now add your own filters if you are in need. Or you wait for me to do this.
The second improvement is the first file filter.
I am happy to announce that the “Rename” filter is ready in core.
What can it do ?
Well, it will allow you to have much more influence of the behaviour of uploaded files.
Let’s see an example:
$filter = new Zend_Filter_File_Rename('C:\pictures\newpics', true);
or when you use the adapter
$adapter->addFilter('Rename', array('C:\pictures\newpics', true));
All uploads will now be moved to the directory C:\pictures\newpics and will be overwritten if they exist.
The default is false, which means that the filter will not overwrite existing files. Beware: If the file exists, you will get a false as soon as you receive the file with an error, that the file already exists if you do not set the overwrite property.
Of course the rename adapter allows much more.
You can define the location for each single file separatly.
You can even define a new filename and extension for the uploaded file.
$adapter->addFilter('Rename', 'C:\pictures\newname.txt');
For more details about supported notations look into the manual.
I hope you find this addition usefull and use it in your own projects.
PS: If you have a good idea for a additional feature to any of my components dont be shy and share your idea.
Greetings
Thomas, I18N Team Leader, Zend Framework
26 comments on Destination and renaming of uploaded files
El_geronimo on Tuesday, 9 September 2008 at 10:25:57 says:
Thank you so much for adding this! Just what I needed for my new project :)
Kieran on Tuesday, 9 September 2008 at 14:11:42 says:
When dealing with file uploads, I’ve found it to be desirable to store the file in an unadulterated state, so to retain the possible semantic qualities that the file name holds. Obviously, storing several files with the same name in the same directory is impossible. I solve this my storing duplicate file names in incremented sub-directories.
eg. image.jpg would be stored as /path/to/uploads/image.jpg, however, a file of the same name, uploaded at a later time, would be stored in /path/to/uploads/0/image.jpg and so on.
Additionally, I also disperse my uploaded files in sub-directories according to the first and (optionally) second characters of the file name. Such as: /path/to/uploads/i/m/image.jpg.
These are just my methods, and I’ve had several discussions with colleagues as to optimal storage strategies. I suppose it leaves a lot open to question, but perhaps that is something for you to think about.
Thanks.
thomasw on Tuesday, 9 September 2008 at 17:53:54 says:
Generally, creating subdirectories is no good idea. Keep in mind that Zend_File_Translate is not only working on HTTP and of course not only on the local machine.
When we have 100 files of the same name, and have to create 100 different subdirectories it would really be a big performance problem.
When would be possible is to define alternate targets… when the same file exists in target 1 it will be stored in target 2 (directory).
Anyhow… Zend_File_Transfer should (and will for now) not add subdirectories at a location.
Kieran on Friday, 12 September 2008 at 20:27:50 says:
I don’t see why the protocol used to transfer the file to the server has any baring on creating sub directories. Surely if you have permission to rename files on the server, you also have permission to create a directory in the same location.
Granted, when there are many files uploaded with the same name, you incur a deficit on performance. Although, this can be reduced by indexing the uploaded files and determining the destination directory of any newly uploaded files from the index.
Having several targets sounds like a finite solution, where as if you have the ability to create directories, you would never run into the issue of overwriting files that you didn’t intend to overwrite.
I should mention that I don’t believe this is an optimised solution, it’s just a method for handling file uploads without the need for renaming. This, I feel, is something clients (in my experience anyway) appreciate, and is quickly deployable.
In a large scale solution, then I would want to rename the file to some unique index, and store in a directory, until that directory reaches it’s optimal capacity (dependent on the system); at which point, the storage target would change to a new directory. These files would then be cataloged in a database. I’d also (possibly) be tempted to rename the files as they are returned to the end user, according to their original filename, to retain any useful quality the filename contains (although I’ve never tried to do this).
Thanks.
Alexander on Wednesday, 17 September 2008 at 09:23:26 says:
The problem with using filters is that very often a file name is formed in the action script after processing other POST variables (e.g. a new row to the db is added and the file name should contain the id of that row). In this situation the whole potential of the file transfer filters becomes completely useless…
thomasw on Thursday, 18 September 2008 at 10:02:58 says:
I dont get the point.
Because you want to change the filename afterwards manually, we should not add filters at all ?
I am working on filters which change imagesize, add watermarks, compress files and so on… and all this should be throwed away ?
Sorry, but I think this is like break a fly on the wheel. Completly nonsense. And many people find this feature more than usefull, they are in need of it.
Alexander on Thursday, 18 September 2008 at 11:10:12 says:
Oh, no, that’s not what I meant!
I am really very grateful for your work, find this filters very useful, and almost all of them announced in the issue tracker could be applied in some of my projects, so I’m looking forward to see them in action.
But the problem is that their applicability is limited with the cases where their behaviour is independent from the other fields of the form, if I correctly understand the way they work. Probably there is a way to improve their functionality adding some kind of parameters which could be assigned during the processing of the form?
And I want to apologize once more if I offended you… That’s absolutely not what I wanted to do.
thomasw on Thursday, 18 September 2008 at 12:14:41 says:
You have not offended me. There are always people which have a different opinion… this would not have stopped me to develop this new filters. :-)
Actually, what you need is already supported.
You can add filters and validators any time… this is not limited to the form creation. When you add your form, for example and then validate it. When the validation is done, your form is already submitted and in processing state. Now you could add new filters after validation.
Keep in mind, or read in the manual, that file filters are applied “after the file was received” due to the fact that files can not be manipulated, but read, while they stay in the temporary cache of php.
Of course you can also use the file filters manually everywhere in your code. See the unittests for how ot do this. But alone, they are not as comfortable as in combination with the file transfer component.
Anyhow, this is already supported.
Sam on Saturday, 20 September 2008 at 21:58:57 says:
I’m looking for a way to convert the name of the uploaded file to lowercase.
Straight in PHP that would be done by changing the $_FILES[0][’name’] value before performing move_uploaded_file to avoid another call to rename().
Do you think something like this will be possible?
Ludo on Tuesday, 23 September 2008 at 17:35:27 says:
Hi,
At first, thanks for this very necessary update of this Plugin & filter.
But: If I’m not wrong, the $adapter requires a destination for the file via setDestination(), then if we want to move (rename, in fact) the uploaded file, it requires to use the ‘Rename’ filter, right? If this is true, then the file is always first moved to the first destination (setDestination()) then moved to the second via the filter… What if two uploads occur at the same time, with the same file names? Since setDestination() only accepts a directory, the files will override each other in the first step… If I’m not wrong…
thomasw on Tuesday, 23 September 2008 at 20:15:43 says:
Wrong assumption in both cases.
You are not required to call setDestination(). If you set no destination then the system uses it’s temp path as defined by PHP.
Zend_File_Transfer knows it and moves the file by the filter anyway.
The reason is more a technical one. Before you have not accepted the upload you can not edit the file. And moving the file is also a sort of editing.
Related to same time I see no problem. PHP does not save the file with it’s real name but with a internal hash. So as soon as you don’t set the same directory for both uploads and both uses the same filename in the same millisecond there is no problem.
When this really occurs, then the second upload will fail because the file can not be moved to it’s final destination. Plain setDestination does not overwrite existing files. This feature is only available by using filters.
Ludo on Tuesday, 23 September 2008 at 22:49:25 says:
Thanks for the clarification! If my 1st assumption was not right, the second had no more sense.
Bao on Friday, 26 September 2008 at 00:01:13 says:
Hi
I try to use this feature with Zend_Form_File_Element
Something like this:
$this->addElement(’file’, ‘uploadedfile’, array(
‘label’ => ‘upload file’,
‘filters’ => array(’Rename’, array($uploadDir,true));
I got an error that say: Zend_Loader_PluginLoader_Exception: Plugin by name Rename was not found in the registry. I’m using ZendFramework 1.6.1
Can you tell me what i did wrong? what am i missing?
Thank you
thomasw on Friday, 26 September 2008 at 01:05:08 says:
First please ask in the mailing list and not in my private blog… this is only related to features and not to bug solving.
Second you are not using Zend_Form_File_Element. I am sure by looking at your code that you are using Zend_Form instead. So better ask a Zend_Form specialist, shouldn’t you ? ;-)
mat on Monday, 29 September 2008 at 13:59:02 says:
Hi,
is it possible to get Zend_File_Transfer doing the following steps?
- uploading a _variable_ number of files (works)
- applying validators to every file (works)
- rename each file to a unique id (works for me somehow just for one file (the first one))
- get a list of all files that have been renamed (doesn’t seem to work since currently not implemented exception)
thanks in advance!
thomasw on Monday, 29 September 2008 at 15:49:08 says:
- yes
- yes
- no (issue already available)
- yes (getFileInfo method in trunk)
Tom W on Friday, 17 October 2008 at 13:10:35 says:
Good work on file uploading. However, I have a really simple question:
How do you get the file name once it has been uploaded?
thomasw on Friday, 17 October 2008 at 16:20:13 says:
Have you tried getFileName() ?
I thought that the name clearly defines what it returns…
brandon on Friday, 14 November 2008 at 22:05:19 says:
Is there a way to apply the rename filter to change the filename but preserve the file extension without knowing what it will be? For example, i allow the upload of all types of images so the extension could be .jpg,.gif,.png, etc but I want to normalize the filenames to be image1.xxx, image2.xxx, imageN.xxx
Thanks for your hard work.
thomasw on Friday, 14 November 2008 at 22:12:31 says:
For now you will have to do this manually.
I am already working on this feature but it will not be available before the next release.
Deon on Tuesday, 18 November 2008 at 19:09:58 says:
Thanks for this - its what I needed. Want to move files to another directory.
However when implementing it in 1.7.0 I had to add indexes my array when creating the filter, e.g.
$adapter->addFilter(’Rename’, array(’source’=>’*', ‘target’=>’/new/path/’, ‘overwrite’=>true));
Else the code as per your example would return the plain filename in the filter() function of Rename ($file = $this->_getFileName($value))
which would mess up the rest of the functions logic.
The source would then be what you want for your target value (the new directory location).
Did i initialise it incorrectly or soemthing? Got it working with the code above though.
thomasw on Tuesday, 18 November 2008 at 22:44:39 says:
Reading this blog you would have noticed that it was written 9th September. 1.7 was released 17.November, 2 months after this blog entry. Which means that this blog entry describes the behaviour of 1.6 and not 1.7
In the manual section there is a migration notes chapter for Zend_File_Transfer where you would have found that behaviour.
Simply said… please read the manual :-)
Marco Peverelli on Wednesday, 17 December 2008 at 15:31:31 says:
You wrote: “Actually, what you need is already supported.
You can add filters and validators any time… this is not limited to the form creation. When you add your form, for example and then validate it. When the validation is done, your form is already submitted and in processing state. Now you could add new filters after validation.”, but it doesn’t work for me…
After the creation of the form, I run a statement like this “if($form->isValid($_POST))” and, inside the if-block, first insert other data into db, retrieving auto-incrementing key, then try to add a filter so $form->getElement(’profilepic’)->addFilter(’Rename’,'picUser’.$id)
Could you kindly explain me what I’m doing wrong?
Thanks,
M.
thomasw on Wednesday, 17 December 2008 at 16:15:29 says:
Not really because I’ve answered to Alexander before. :-)
There have been 2 or 3 mails in past on the mailinglist from people doing exactly the strategy I’ve described.
Also it depends on the used release.
Be aware that you don’t receive the files and add the filter afterwards. You can check this by using isReceived().
Because some methods receive in background. getValues() is an example.
Also to mention: This article describes Zend_File_Transfer… I don’t know if the described solution works also on Zend_Form.
Beside that I would say that you should ask in the mailinglists or on IRC #zftalk because I can’t and also won’t solve bugs within my blog as you may understand. :-)
Marco Peverelli on Wednesday, 17 December 2008 at 18:25:40 says:
Sure, I understand. But I didn’t mean that behavior to be a bug in your code, I mean the problem is my lack of knowledge! ;)
BTW, which mailing list are you talking about? So that I can search and learn…
Thanks again,
M.
thomasw on Wednesday, 17 December 2008 at 18:53:36 says:
Look here http://framework.zend.com/community/overview for the mailing lists. Or use http://nabble.com to search them online.