File Upload problems and my fix

[eluser]crojac[/eluser]
Nothing like an inconsistent bug to get the ole sluething chops wet. I scoured the forums yesterday
regarding issues related to the file upload library and its purported 'go - no go' attitude it cops from time to time but none of threads beared any fruit.

My file upload method would sometimes would take a pooh on PDF's for no apparent reason. Very very frustrating.

The do->upload method was returning FALSE via the is_allowed_filetype call. Simple enough. (Dive, Dive, Dive)

After triple checking the mimes.config and my upload config ( btw I changed the order of the allowed_types. - which mysteriously worked on occassion but not reproducible, I have no reason why ), I was still scratching my head.

So off to the is_allowed_filetype method. Simple straight forward code, throw a few debugs down and banzai the in_array is NOT throwing TRUE as it should if it the file_type was in the mime array. Interesting.

Now the good part. I set up a '==' comparison, file_type vs mime array element. First code run - good (match), Second code run - good (match), Third code run - bad (WTF), Fourth - bad, Fifth bad, then good, then bad. yahda, yahda, yahda. I had found the exception.

So I printed the file_type and the mime element and sure enough it WAS different - there were double qoutes surrounding the file_type value ( example - "application/pdf" ). I'm assuming the in_array function executes some type of string comparison so this was probably the problem.

Ok, what was causing the inconsistent file_type == mime element match, mismatch? Time to fire up the Tamper Data plugin for Firefox. Navigate to page, Start Tamper, do upload, examine POST REQUEST specifically the Request Header and look at the POSTDATA.

Lo and behold there before me in its full glory was the ' Content-Type: "application/pdf" ' string. The double qoutes had been found and they were making there way into the $_FILE hash where they would eventually bomb the Upload.php library.

But wait! Why did the upload work a few times. I ran a few more tests and the double qoutes disappeared from around the Content-type string of the POSTDATA allowing the Upload class to work. ( a big WTF ). Again what was causing the appearance disappearance. Lets see if RFC - 2046 Multipurpose Internet Mail Extensions can explain things such as syntax and the like. I didn't find anything conclusive ( the need for quoting ) and this was in the POSTDATA anyways. Dang. Now where. Hey I know, back to the browser.

Edit->Prefences->Applications and scroll down the Content-Type/Action list and here is what I found

PDF document ("application/pdf")
PDF document (application/pdf)
PDF document (text/html)

I will stop there. I didn't have the time or the energy to research how the 'Content-type' string gets set in the browser request POSTDATA and or if the Content-Type/Action list has anything to do with the double qoutes. Thats for the next person to hunt down.

I went back in and stripped the double qoutes from the file_type and it works 100% of the time for me.

I don't know why double qoutes show up in the file_type string on occassion, but I do know that if they are present the is_allow_filetype returns false and do_upload croaks. Sorry for the story. I hope this helps others.

Peace - Out.

[eluser]crojac[/eluser]
Ah.. so the saga continues. After my initial fix ( see above post ) and dozens of tests. I encountered another failure. This time Firefox sent a Content-Type: 'text/html' in the POSTDATA instead of 'application/pdf' a quick check of the Edit->Preferences->Applications Content Type/Action list showed a PDF document with an action of (text/html) associated with it. Check your mimeType.rdf file and see whats going on there.

Below is a link on how Mozilla determines MIME Types:

https://developer.mozilla.org/en/How_Moz...MIME_Types

After googling firefox mimetype bug I came accross this page:

http://techblog.procurios.nl/k/news/view...refox.html

( please clink on the links in the article that will take you back to the bugzilla posts. Don't forget to read the threads )

This has now cleared the waters once and for all. There is a know issue in the way Firefox associates MIME Types to files. This issue directly contributes to the 'hit and miss' of the Upload.php library.
Read, read, read! I'm not going to describe it here.

I suppose Codeigniter could call out to the OS and do a 'file' check or use the PECL Fileinfo library on the incoming file. Thats of course if there is a OS compatible 'file' option and or they ( hosting company, your company, etc ) have the Fileinfo library installed.

I would certainly like to hear the opinions of others on this issue. I believe I'm on the right track with this and have identified the problem. I don't know if the issue plagues IE or not. I'm going to be testing that later.

Have fun

[eluser]crojac[/eluser]
Ok.. since we have 'file' hanging around on our web front ends im going to be using it to do a passive check and set of the $this->file_type. I hate using 'exec' but CI uses it a once so oh well. I've place my new fix below where $this->file_size is set in the do_upload method of the Upload.php class

$rv = 1;
$cmd = "/usr/bin/file -bi " . escapeshellarg($this->file_temp);
exec( $cmd, $my_file_type, $rv );
if ( $rv == 0 && ! empty($my_file_type[0]) ) $this->file_type = $my_file_type[0];

You could use the native 'Fileinfo' (my first choice but not available to me) which is available in PHP >= 5.3.0 OR roll your for a lesser verion of PHP.

My code seems to be working just fine. BTW.. If you experiencing the intermitent upload problems from your dev box using a moz based browser. Just delete your mimeType.rdf file. This will force Mozilla to go to the OS for mime help if it can.

What an ordeal this has been, but not one that will soon be forgotten. I'm going to perfect a general solution in the future for all in the coming days.

[eluser]darkhouse[/eluser]
Thanks for all of your investigation into this. I didn't go as deep as you, but I did find and fix a couple of the others.

I didn't realize Firefox would also send incorrect mime types some of the time. I've extended the upload library to hopefully fix all of our invalid filetype issues. I know this "problem mimes" solution isn't the greatest, but it shouldn't cause any issues.

Code:

class MY_Upload extends CI_Upload {
    
    var $problem_mimes = array('text/plain', 'text/html');
    
    /**
     * Verify that the filetype is allowed
     *
     * @access    public
     * @return    bool
     */    
    function is_allowed_filetype()
    {
        if (count($this->allowed_types) == 0 OR ! is_array($this->allowed_types))
        {
            $this->set_error('upload_no_file_types');
            return FALSE;
        }
        
        if(in_array('*', $this->allowed_types)) return TRUE; //allow all
        
        $this->file_type = str_replace(array('"', "'"), '', $this->file_type); //fix firefox mime type bug

        $image_types = array('gif', 'jpg', 'jpeg', 'png', 'jpe');
        
        foreach ($this->allowed_types as $val)
        {
            $mime = $this->mimes_types(strtolower($val));
            
            // Images get some additional checks
            if (in_array($val, $image_types))
            {
                if (getimagesize($this->file_temp) !== FALSE) //check to see if it's not false and return true
                {
                    return TRUE;
                }
            }

            if (is_array($mime))
            {
                if (in_array($this->file_type, $mime, TRUE))
                {
                    return TRUE;
                }
            }
            else
            {
                if ($mime == $this->file_type)
                {
                    return TRUE;
                }
            }    
            
            if(in_array($this->file_type, $this->problem_mimes, TRUE))
            {
                return TRUE;
            }
        }
        
        return FALSE;
    }
    
}

[eluser]BaRzO[/eluser]
Thanks for this thread... I was getting mad

[eluser]crojac[/eluser]
Well a few days ago I was alerted that my file upload fix using FILE was borking. It appears that FILE's output is somewhat different depending on your flavor of linux. The mime type string was listed twice in the output from the exec command ( the first element of the $my_file_type array contained 'application/msword application/msword' why I have no idea. I'm only assuming it has to do with the magic files on the box or that implementation of FILE for that distro.)

Code:

$rv = 1;

$my_file_type = array();

$cmd = "/usr/bin/file -bi " . escapeshellarg($this->file_temp);

exec( $cmd, $my_file_type, $rv );

When I run my test script ran against ubuntu 9.10 ( i know.. do shoot me ) it returns a single mime type as expected but when ran against a CentOS/Xen instance I get the double mime type return. This all of course fails in the reverse lookup using the CI mime.php file as a temporary fix I've inclued the double mime string for now. Remember drop in some logging to catch what mime type arrived vs what FILE deciphed to catch this slight bug and last but not least. I created several variations of MSWord docs in OpenOffice, did the upload and it works. I had a fellow coder send a test file MSWord (2003) from his windows box and guess what. Its the one that threw the double mime type. LOL.. Take care friends. One day we will find the promise land regarding this issue.

[eluser]BaRzO[/eluser]
Hi all,

I am havin problem with upload lib maybe this is a bug i don't know...

I run Ubuntu 9.10 and Firefox
I created an empty file with gedit and I renamed it to test.gif my upload conf is jpg||png|gif
when I try to upload this file I get errors because of getimagesize trying to get its size...