Zend_File_Transfer examples or using validators to increase security
Hy interested ones,
the new Zend_File_Transfer component is growing day for day.
As incredible new feature this component allows to use file validators.
These are necessary to increase security and allow to define rules for file uploads (and also downloads in future). So let’s see some examples to get a feeling:
$upload = new Zend_File_Transfer_Adapter_Http();
$upload->addValidators('Size', '50kB)
->setDestination('C:/uploads')
->receive();
What we’ve done so far is to limit all sent files to 50kB filesize. Any file which is uploaded and exceeds the size of 50kB will throw an exception we can catch.
The more rules we define the more secure our upload will be.
So which other validators are supported until now:
- Size: We already know this validator. He checks the filesize of single file. You can set a minimum and a maximum filesize.
- Count: You should set this validator to represent exactly the amount of files you expect. He has also a mimimum and a maximum filecount. If this validator throws an error you are probably having an attack. But you can also limit the number of files to receive with this validator.
- Extension: This validator checks for the extension of files. You can set multiple extensions to be checked. But remember that an evil user can manually change the extension so you should not rely only on the extension.
- FilesSize: This validator also checks for the size of files. But different to the Size validator it checks for the size of ALL files. You could for example define that a single file must not exceed 50kB. But all files in sum must not exceed 200kB.
- ImageSize: The ImageSize validator checks the size of given files when they are images. You can define a mimimum and a maximum image size for width and height.
So let’s see a full example of validators and a more secure upload:
$upload = new Zend_File_Transfer_Adapter_Http();
$upload->addValidators('Size', '250kB')
->addValidators('Count', 5)
->addValidators('FilesSize', '1MB')
->addValidators('Extension', 'gif, jpg, png')
->addValidators('ImageSize', array(10, 10, 1024, 768))
->setDestination('C:/uploads');
if (!$upload->isValid()) {
print_r($upload->getMessages());
die();
}
try {
$upload->receive();
} catch (Zend_File_Transfer_Exception $e) {
$e->getMessage();
}
So what we’ve created now is a fileupload for images.
Each imagefile can have 250kB maximum filesize. We allow in sum 5 images but all images im sum are not allowed to exceed 1MB. Additionally we allow gif, jpg and png files and define a imagesize of 10×10 up to 1024×768. All files are uploaded to ‘C:\uploads’.
As you see it’s not complicated to define a more secure upload then just using php’s move_uploaded_file.
Feel free to play around with this example.
If future there will be additional validators like MimeType and FileName.
Also filter will be added which allow you to change uploaded files on the fly before they are stored.
Filters could contain the automatic change of imagesize or changing textfiles to have a proper lineending and much more.
Greetings
Thomas, I18N Team Leader, Zend Framework
48 comments on Zend_File_Transfer examples or using validators to increase security
Ota on Monday, 21 July 2008 at 12:40:39 says:
Hey thomas,
seems nice and easy to use but how do you map the http file upload fields to zend file transfer?
thomasw on Monday, 21 July 2008 at 17:21:54 says:
This is done internally by Zend_File_Transfer at initialization. So Zend_File_Transfer has to be the target of your file upload form to work. This is because the temporary files will be deleted immediately after the next call of a php script (the target of your form).
You can use both syntax… single files or the multifile syntax. Both are recognised by Zend_File_Transfer.
Ota on Tuesday, 22 July 2008 at 02:50:03 says:
So its not possible to define what files should be handled/excluded by Zend_File_Transfer?
To make it more clear, as soon as i initiate ZFT HTTP Upload it takes “controll” of my HTTP uploads?
BTW: Do you have magic quotes gpc turned on? My quotes got escaped as i forgot to enter a name.
thomasw on Tuesday, 22 July 2008 at 10:38:53 says:
As you may already expect the API can much more than I’ve described. :-)
You can also handle single files… each method can get a files string or array. Define your files within there and only this files will be processed. Just keep in mind that after the executed script is finished ALL not received files are gone. But you could execute the receive method multiple times. Simplest way is just to give an array with the files.
Yes, when using the HTTP Adapter, it will automatically take you uploaded files without any further methodcall.
Related to matic_quotes_gpc… I’ve only tested with default settings of PHP. But keep in mind that it’s not possible to inject code within here. When a filename can not be found due to false characters like ” ‘ / and so on it will just return an validation error. So there is no security leak for file uploads.
Ota on Tuesday, 22 July 2008 at 17:43:44 says:
Its obvious that uploaded files are deleted by the php garbagecollector after the script finished executing.
I am just asking if it would theoretical be possible to handle one file via ZFT and another via the build in php file functions?
About the magic_quotes problem: I meant your blog ;) The quotes in comments get escaped when there is an user input error.
thomasw on Thursday, 24 July 2008 at 14:38:12 says:
It would theoretically be possible… but in my eyes completly useless. :-)
David on Thursday, 24 July 2008 at 15:18:49 says:
Did the Zend_File_Transfer component make it into the 1.6.0 RC1 ? I can’t seem to find it, but it’s mentioned in the release notes
thomasw on Thursday, 24 July 2008 at 16:48:03 says:
You can find it in the incubator. It will be officially released with RC2 because the necessary View Component was not finished as the release was made.
queefsniffer on Friday, 25 July 2008 at 01:05:58 says:
what about .jpeg extensions and the mime/type “image/pjpeg” that IE sends
queefsniffer on Friday, 25 July 2008 at 01:09:31 says:
I also remember a situation we ran into where we checked mime type instead of extension and there was a certain executable mime type that we didn’t every allow but then a new ms office format of some sort had it’s file uploaded with this same forbidden extension and all our clients got pissed because they could no longer upload excel or powerpoint or something kind of files.
What’s the best way around this problem?
thomasw on Friday, 25 July 2008 at 01:52:33 says:
What should be with such an extension ?
You can set any extension you need. The Extension validator validates only the file extensions you set. The MimeType validator will follow in a few days. It will allow to validate against mimetypes.
This situation is quite simple… why should a new software be supported by an old application ? This is the reason why customers should go for a support contract.
There is no way to automatically support things which are not known at creation of a component. This is a generic problem, not a file upload specific one.
Hugo on Friday, 25 July 2008 at 13:24:23 says:
Unfortunatelly, this component doesn’t work well. I explain. I’ve tried the code below :
<?php
$oFileTransfer = new Zend_File_Transfer();
$oFileTransfer->addValidators(’Size’,'50kB’)
->addValidators(’Extension’, ‘gif, jpg, png’)
->setDestination(dirname(__FILE__).’/uploads’);
?>
But I’ve the error that I can’t apply the addValidators() method on my object because this one isn’t a Zend_File_Transfer_Adapter_Abstract type. So I was forced to add a getAdapter() method to the Zend_File_Transfer class, which explicitly returns the $_adapter attribute.
So, it works with the code :
<?php
$oFileTransfer = new Zend_File_Transfer();
$oFileTransfer->getAdapter()
->addValidators(’Size’,'50kB’)
->addValidators(’Extension’, ‘gif, jpg, png’)
->setDestination(dirname(__FILE__).’/uploads’);
?>
Then there a big problem with the Zend_Validate_File_Extension validator, which doesn’t work because in you Zend_File_Transfer::isValid() method, you try to validate the extension as the same way as the other validators. All validators are applied on the $_FILES[’file’][’tmp_name’] value. But unfortunatelly, this value looks like something like that : /tmp/1235ShdeZe8
There is no extension for a temporary file, that’s why the extension validator provides an error (the warning that the index ‘extension’ doesn’t exist in the $infos array of the Zend_Validate_File_Extension::isValid() method).
So, you should apply the Zend_Validate_File_Extension::isValid() method on the $_FILES[’file’][’name’] value, which contains the “trully” name of the file stored in the client hard drive. This value contains the extension of the file and so can be checked.
I’m very impatient to discover the Zend_File_Transfer class in Zend Framework 1.2 because I really need it for my future projects.
I have two question for you Thomas. The first one is :
Will the Zend_File_Transfer component be coupled with the Zend_Form component ?
In fact, It would be great if we can have a Zend_Form_Element_File element that we can instanciate and add to the Zend_Form object. Then, we could instanciate a Zend_File_Transfer object and defined its differents parameters (validators, destination…). When this is done, we would associate this Zend_File_Transfer object to its Zend_Form_Element_File object.
So in the finish, it could be nice to have a code, which looks like this :
<?php
// File field
$oFileField = new Zend_Form_Element_File(’my_file’);
$oFileField->setLabel(’Your file’);
// Submit field
$oSubmitField = new Zend_Form_Element_Submit(’my_submit’);
// File transfer
$oFileTransfer = new Zend_File_Transfer();
$oFileTransfer->addValidators(’Size’,'50kB’)
->addValidators(’Extension’, ‘gif, jpg, png’)
->setDestination(dirname(__FILE__).’/uploads’);
// Associate the File Transfer object to its Element_File object
$oFileField->setFileTransferObject($oFileTransfer);
// Form
$oForm = new Zend_Form(’my/action’);
$oForm->setAttrib(’enctype’,'multipart/form-data’);
$oForm->addElement($oFileField);
$oForm->addElement($oSubmitField);
if(!empty($_POST) {
if($oForm->isValid($_POST, $_FILES)) {
// Checks, validates and receives in the fly the transmitted file
echo ‘Success’;
}
}
?>
My second question is :
Does the Zend_File_Transfer object accept filters ? Because it could be nice to add filters to the object in order to rename properly its name for example. If a user tried to upload a file named like this (french filename) : “dernière journée à l’ile Maurice.pdf”.
An associated filter could rename it on the fly in one of the formats below :
- derniere_journee_a_l_ile_maurice.pdf
- derniere-journee-a-l-ile-mauriece.pdf
Or simply with the sha1() or md5() encoded filename string. What do you think about my proposition?
Thanks for your work.
Hugo.
Hugo on Friday, 25 July 2008 at 14:06:26 says:
The Extension validator doesn’t work anymore. I can upload .exe files on my server even if I declared only jpg, png and gif as valid extensions…
thomasw on Friday, 25 July 2008 at 14:26:42 says:
Hugo: Use the adapter and not the base class… until now the base class is not in a finished state. See the example code above.
Related to your problems… please write issues you have within Jira. Others might also have such problems and will not find a solution when they do not read my blog… well they probably should ;-) Also this component is not finished for now… there is much in change.
As you may have noticed in the ZF lists, a form element will follow soon. But this is something which Matthew has raised his hand for, so I will not do it. He’s the Form-U-Nator :-)
Related to filters… when you look at the code you will recognise that there is a getFilters, setFilters and a addFilters method. They are not coded for now, as other things are more important. So they throw an exception. But they will come in future.
But filters are not meant to change filenames they are meant to change the content… for example automatically resize a image, change content encoding, revert line endings and so on… there are much ideas on this point.
Greetings
Thomas, I18N Team Leader, Zend Framework
Hugo on Friday, 25 July 2008 at 15:45:13 says:
Hello thomas,
Oops I didn’t see that I had to use the Zend_File_Transfer_Adapter_Http… I will do it now :)
Thank your answer. I hope ZF 1.6 RC 2 and stable will be soon unveiled. Actually I’m not using the ZF at work (only for my personnal projects) but some components separetly whose Zend_Form, Zend_View, Zend_Config, Zend_Loader, Zend_Registry or Zend_File.
Thanks.
Hugo.
thomasw on Friday, 25 July 2008 at 15:50:59 says:
No problem at all…
I’ve also added some minor changes, so you should update to the latest incubator version. The extension validator for example takes now also ‘name’ in account. So just get the latest version and keep on testing ;-)
Greetings
Thomas, I18N Team Leader, Zend Framework
Hugo on Friday, 25 July 2008 at 17:33:22 says:
I’m now registered at Jira issue tracker. I have started to report the bugs I’ve discovered by testing your component ;)
SayMyName on Monday, 4 August 2008 at 09:35:36 says:
Is it possible to give the extension validator an Array instead of an comma separeted string?
->addValidators(’Extension’, ‘gif, jpg, png’);
->addValidators(’Extension’, array(’gif’, ‘jpg’, ‘png’);
thomasw on Monday, 4 August 2008 at 10:18:03 says:
Of course is this syntax also supported. Just read the documentation within the incubator to see what’s possible.
Julien on Tuesday, 9 September 2008 at 15:18:01 says:
Hi,
Is it possible to upload several files from one single page/form? Could you please give a working example ?
Thanks in advance for your help;
Julien
thomasw on Tuesday, 9 September 2008 at 17:47:08 says:
Simply define several file elements in your form.
Zend_File_Translate (or the file element) will always handle all given files. There is no code to change for Zend_File_Transfer to support this.
For informations how to define multiple form elements, please look into the ZF manual.
jsnod on Sunday, 14 September 2008 at 02:56:53 says:
Hi, is there any way for the adapter to return the filenames of the successfully uploaded files? I suppose I could use $_FILES directly, but it would be much nicer to have the adapter return an array of files uploaded.
thomasw on Sunday, 14 September 2008 at 12:31:03 says:
Use the getFileName method which is described in the manual.
dawid on Monday, 22 September 2008 at 03:24:17 says:
Hi, great article:)
I have 2 questions:
1. Default desitnation of Zend_File_Transfer_Adapter_Http is tmp dir where files are uploaded but what about when users upload two difrent files with the same name? I look into code but i didnt find any solutions there.
2. What is better for sth like resizing images( ie. 2 resolutions), add watermark, or renaming(unique file name with id or in subdirectories) its better to write own ZF_Transfer_Adapter or writes filters for this actions? (i think more on philosphy doing this, more about approach)
thomasw on Monday, 22 September 2008 at 10:39:20 says:
1.) Simply use the Rename filter. Each file is targeted by the name of it’s form element.
2.) These are 2 different things. I am planning a ImageResize and a Watermark filter. But resizing image + adding a watermark changes content, and renaming only changes name and does not touch content. So it’s always a matter of what you want. Generally the component supports both ways so that writing a own adapter is not necessary.
Ekoka on Sunday, 23 November 2008 at 19:23:31 says:
Very nice initiative, although I’m having some issues with this component. My comments are meant as constructive criticism. I wish it was simple enough to send comments directly to the doc and dev teams, but Zend requires a series of preliminary steps for contributions for which I simply don’t have time atm. Maybe later, under more forgiving circumstances.
1-) My first (and potentially grave) problem is the fact that Zend_File_Transfer writes to the directory specified by setDestination, *before* renaming the file. This is a problem as it exposes that directory to overwrites operations.
e.g. Lets say that i have a self describing photo called family_picnic.jpg, that I upload through my cms. I employs a scheme to rename the file and link them to a specific db record, so that at the end the photo will be renamed something like ‘img_456.jpg’. If I then upload another picture, displaying some elephants in Africa and actually named img_456.jpg, that photo will first be saved - effectively overwriting my picnic photo - then renamed to reflect to which record it belongs to (e.g. img_998.jpg).
2-) I’ve noticed that when accessing the Zend_File_Transfer object through Zend_Form_Element_File, some of the accessor methods are lost. e.g. “method getHash does not exist”, “method getFileInfo does not exist”.
One of two things should be done:
- either allow access to all of the Zend_File_Transfer public methods and properties, through Zend_Form_Element_File,
- or In the Zend_Form_Element_File section of the manual, explicitly document the getTransferAdapter method of that element.
3-) I’m a bit uncomfortable with the Rename filter. I don’t understand why it is required to (again) provide the path to the file (which has already been set with setDestination). All which should be required is the actual name of the new file. If the concern is multiple files uploads, then I believe it would probably be more appropriate to have an additional parameter in setDestination to distinguish each file’s destination.
4-) Finally, the Rename filter should offer the option of appending an extension to the new file name, using the original name’s extension as a default.
e.g.
$newFileName = ‘img_’ . $record->id;
->addFilter(’Rename’, $newFileName, array(’extension’=>’jpg’)) // all images uploaded will have a .jpg extension
->addFilter(’Rename’, $newFileName) // all images uploaded will use the same extension as the original name
Just my 0.02$
thomasw on Sunday, 23 November 2008 at 20:35:04 says:
It is as easy… you could eighter write a mail to the mailinglist, or write a issue to the issuetracker. Both require no CLA.
To your points:
1.) This is more a handling problem of you. You should not set the temporary directory and the final directory the same. setDirection will be used as temporary directory when the rename filter is applied. And if the file should be overwritten, or if you want to have an failure returned can be decided with the option ‘overwrite’.
Anyhow, as with every OS and every language you can not store 2 same named files in the same directory.
2.) It is not possible to have all methods of Zend_File_Transfer integrated in the file element. The transfer class was build to support uploads, download, multiple protocols… the file element is not able to support these.
New methods are, of course, firstly available in the transfer class. But if they are proved by a wider comunity and no problems are found they will be available within the next subrelease in the file element.
You could simply add a issue to force this feature enhancement. :-)
3.) No it is not necessary to set the source file. It is optional as stated in the manual. But as all file filters they can also be used standalone, and then you will need also the source option when you have multiple file definitions.
4.) This is an already existing idea… a issue already exists, but as you can mention there are much more. There are also other feature enhancements which have to be integrated for the rename filter, and I will work on one after the other. :-)
There has really much changed in past weeks. Therefor I always recomment to use the latest release for all who have problems or are missing features. ;-)
Tomasr on Sunday, 11 January 2009 at 14:51:51 says:
Hello,
I’m trying test Zend_file_transfer_Adapter_Http(). I have made upload file with validators. And I’m going to do multiple files upload. I know that in manual is described some method setMultipleFiles(), but this method can create only identical file elements. I would like create two file elements. One of them is element for a picture and second for archive. It means, each element has own validators (own adapter), but it isn’t possible. Adapter automatical contain both files and validators does’t work how they should. Is possible to solve it?
I use ZF 1.72.
Thanks for advice.
Tomas
thomasw on Sunday, 11 January 2009 at 20:33:22 says:
It’s not necessary to test Zend_File_Transfer_Adapter_Http. ZF provides it’s own testbed and all components must be 80% unittested before they are allowed to be in core.
setMultiFile() creates, as you already noted, ONE file element which allows multiple files to be uploaded within the same element. There is of course no way to provide different validators within the same element.
But you can simply create two different single file elements as you also create two button elements when you need two buttons.
How ever, the Adapter itself, which should normally not be used standalone, contains ALWAYS all uploaded files. Looking at HTTP itself you will of course only have one FILES array provided from the webserver and not one per file.
Tomasr on Wednesday, 14 January 2009 at 00:03:40 says:
I didn’t think test like test in right sense, but like play with it. Sorry for my bad explain skills in english:).
I’m not sure if I understand to last paragraph, because when I had just one adapter and used isValid method on one file (false), it doesn’t work right on second one (should be true, was false).
So I was making two adapters (with own validators), one for each file element. Now it’s working well.
Krish on Saturday, 28 February 2009 at 08:19:27 says:
How can i rename and resize image, ulpoad via
form (http)
thomasw on Saturday, 28 February 2009 at 11:50:56 says:
Read the blog examples and comments. It was already mentioned. Also in the ZF mailinglists this question has been answered multiple times.
Piter Vergara on Thursday, 12 March 2009 at 00:28:13 says:
Hi;
I’m trying to create a form, using Zend_Form_Element_File, which can receive several image files. I can do this calling the setMultiFile(10) method, but this will create all the 10 inputs in the form. So, I’m looking for a way to use the filters and validators avaliable in Zend_Form_Element_File but having the inputs created on the client side, using javascript (”jQuery Multiple File Upload Plugin”) in order to create a “gmail-like” file upload.
Do you have any tips on how to do this?
Thanks in advance;
thomasw on Thursday, 12 March 2009 at 09:18:29 says:
As you may have seen, Zend_File_Transfer is a server component to simplify the usage and increase the security.
For clientside with jQuery you should ask jQuery gurus or within the mailinglist. I am sure some of them can give you an answer.
nitin on Thursday, 19 March 2009 at 14:57:22 says:
I used u r example but it get me the error like
i am new to php and zend framework
Notice: Undefined index: tmp_name in C:wampwwwjplnewslibraryZendFileTransferAdapterAbstract.php on line 589
Notice: Undefined index: tmp_name in C:wampwwwjplnewslibraryZendFileTransferAdapterAbstract.php on line 589
Notice: Undefined index: name in C:wampwwwjplnewslibraryZendValidateFileSize.php on line 398
Notice: Undefined index: tmp_name in C:wampwwwjplnewslibraryZendFileTransferAdapterAbstract.php on line 589
Notice: Undefined index: name in C:wampwwwjplnewslibraryZendValidateFileExtension.php on line 228
can u plz guide me
thomasw on Thursday, 19 March 2009 at 15:54:40 says:
Please look into the ZF-FAQ or the manual.
Often made mistakes are:
* Deleted default file decorator when using Zend_Form_Element_File
* Forgot to set the encoding of the form
* Using GET instead of POST
Also to note: This article is related to the manual usage of the transfer adapter. People without experience should use Zend_Form_Element_File instead, for simpler behaviour and reduced coding.
Greetings
Thomas
nitin on Friday, 20 March 2009 at 11:19:55 says:
Dear Thomas
I Trying to much but not get answer yet for file uploading by using zend_form_element_file
if possible please send me code
Greetings
nitin
thomasw on Friday, 20 March 2009 at 11:40:02 says:
Within this blog you can find several examples. Also the manual and other blogs show examples.
A simple search shows 334 results: http://www.google.com/search?hl=en&q=Zend_Form_Element_File+upload+example&meta=
Give it a try.
When you have problems with existing code, please ask within the mailinglist.
nitin on Tuesday, 24 March 2009 at 09:43:44 says:
dear Thomas
is it possible to upload file using zend framework?
Nitin
thomasw on Tuesday, 24 March 2009 at 10:08:59 says:
Support for File uploads is on the roadmap for Zend_File_Transfer, but actually I am not finished with it.
Zend_Http_Client supports it within trunk since 4 days, but it needs to have the cURL extension installed.
Within Zend_File_Transfer there will be support for multiple ways/extensions and of course the ability to work as wrapper and as gateway which both are actually not supported by ZHC.
Huy on Wednesday, 25 March 2009 at 15:47:30 says:
hi Thomas,
I’m using ZF 1.7.7 , Zend_File_Transfer_Adapter_Http to upload some files as recommended by the manual and this article.
Basically I have a form with a input type file and when the user browse for a file it automatically upload the file via AJAX. I have set up the request correctly (to point to the controller with the upload action)
In my controller, I had something similar to what you posted in the article.
………………………
try {
$upload->receive();
var_dump($upload->isUploaded());
}
……………………………..
That calls gave no error and the dump gives a true result yet no file was uploaded!!! I’m positive the path I set was correct.
The var_dump of $this->_request->getPost(); returns an array from the input file with just the file name
e.g <input type=”file” name=”photo” id=”photo” />
the var_dump gave:
array(1) {
[”photo”]=>
string(15) “wall_ff_318.jpg”
}
Do you know what went wrong?
Thanks
Huy on Wednesday, 25 March 2009 at 15:58:51 says:
I forgot to say $_FILES was also empty when I did the var_dump before and after the try block
thomasw on Wednesday, 25 March 2009 at 16:38:37 says:
Huy, please use the official Zend mailinglist for help or bug reports.
A blog is the wrong channel for help because this normally involves several steps, code listings and much more. It would bloat this entry unnecessary.
Regarding manual:
You may have noted that receive() returns a boolean value within the actual release, and no longer an exception.
Also to mention: When you set the element to be optional, no error will be triggered.
Huy on Thursday, 26 March 2009 at 02:18:00 says:
Thanks Thomas,
I’ll subscribe soon.
Huy on Thursday, 26 March 2009 at 04:17:13 says:
Thomas,
just to confirm, is the mailing list at the following address?
http://framework.zend.com/archives/core
I saw some lists there and subscribed for that one. Are these lists related to Nabble? Sorry I haven’t used mailing lists before :P
Huy
Mikec on Friday, 3 April 2009 at 14:30:53 says:
Hi Thomas,
Aplologies if this is the wrong place for help,
I’m using Zend_File_Transfer_Adapter_Http (1.7.8) with 2 file elements in the form, both set required to false, if user enters a file for both the upload is successful for both, if user enters a file for the first element only, the upload is successful, but if a file is only entered for the second element, the file is not uploaded.
However if I use the getFileInfo() method and call isUploaded() for both elements it tells me that the second file was uploaded even though it actually doesn’t. actually arrive. The isValid() method also returns no value for either element.
I am calling receive() for each element, by iterating through the array returned by getFileInfo(), but the upload of the second element still fails if the first element was left blank.
thomasw on Friday, 3 April 2009 at 16:33:48 says:
Mikec, please read the comments.
Mikec on Friday, 3 April 2009 at 17:43:11 says:
Thanks Thomas,
I got it to work with the “magic” addition of:
$adapter->setOptions(array(’ignoreNoFile’ => ‘true’)); - works fine now.