Avoiding URL Forbidden Characters By Using urlencode()?

[eluser]Matt Egan[/eluser]
Hey there! While working on my most recent project, I have encountered some odd results while combining Codeigniter's uri features with the file and download hellers...

I am creating (For my local technology fair) an application that allows groups to upload files and then distribute a URL for the project with which the other team members can upload and download files also.

Uploadih is done using the file upload class, the filename is saved into a database along with a name that the user can specify. I also save the file's upload timestamp, and the project it is associated with.

My model can generate links to the file download function of my projec controller that calls a function to force the download. This is where my problem lies. This link is formatted as follows;

..index.php/project/download/projectname/filename.txt

This model that generates the function uses PHP's urlencode() functions to replace any disslowed characters. This is my first problem, even if I encode the filename, Codeigniter interprets it as containing dissallowed characters. I'm not sure how to prevent this, is this expected behavior? If so, how can I encode the URL to not contain any dissallowed characters?

Anywho... Then, if I allow the characters that are restricted, read_file function fails to read the file. If the filename contains no strange characters, it works. I thought to replace the function with PHP's native file_get_contents, but it seems to replace the strange characters with HTML entities, and is then unable to locate the file. Again, it works of the filename does not contain anything strange. What should I do here?

I've been unable to fix these problems. I was thinking about using the security class to encode thr URL, but I kind of wanted the URL to download the file be nice and readable. In was thinking of using an auto incremented, ID field in the file database so the URL would be more like:

.../index.php/project/download/projectname/261

That might be easier, and I may end up going down this path, but I'd like to know why I'm running into these problems for future reference. Thanks so much!

By the way, I am terribly sorry for any spelling or gramatical errors, I'm in study hall typing this out on my iPhone.... 

[eluser]danmontgomery[/eluser]
urlencode() encodes URLs, not filenames. The rules for allowed characters are different... Google for a list of forbidden filename characters for whatever OS your webserver is running on, and just strip those characters.

[eluser]Matt Egan[/eluser]
But the filename is being passed through the URL, this is my reasoning behind my use of urlencode().

Is this wrong?

[eluser]danmontgomery[/eluser]
Yes, because you can use any character in a url if it's encoded, and there are characters that you cannot use in a filename, depending on the operating system. For example, urlencode() will replace non-alphanumeric characters with %XX, and % is a forbidden character in windows filenames.

[eluser]Matt Egan[/eluser]
I understand that and am already cecking for permitted filename characters. My problem is that all characters that are permitted in a filename are not permitted in a URL. When using urlencode(), Codeigniter for some reason says my URL contains forbidden characters...

Is this the way it is supposed to be?

[eluser]danmontgomery[/eluser]
CI's allowed URL characters are defined in config.php

Code:

/*
|--------------------------------------------------------------------------
| Allowed URL Characters
|--------------------------------------------------------------------------
|
| This lets you specify with a regular expression which characters are permitted
| within your URLs.  When someone tries to submit a URL with disallowed
| characters they will get a warning message.
|
| As a security measure you are STRONGLY encouraged to restrict URLs to
| as few characters as possible.  By default only these are allowed: a-z 0-9~%.:_-
|
| Leave blank to allow all characters -- but only if you are insane.
|
| DO NOT CHANGE THIS UNLESS YOU FULLY UNDERSTAND THE REPERCUSSIONS!!
|
*/
$config['permitted_uri_chars'] = 'a-z 0-9~%.:_\-';

You can modify that regular expression, or replace the invalid characters in the url and do some substitution to translate the url to the filename.

[eluser]Matt Egan[/eluser]
I realize this. But by using urlencode() my URL contains no restricted characters, yet Codeogniter is still telling me that my URL does contain them, technically, it does, but the characters are encoded. I do not want to allow any more characters, because there should be an easy solution to my problem, and I don't wan to give in and just allow some extra characters. Also, I am not a securty "expert" by any means.

Thanks again,
Matt

[eluser]danmontgomery[/eluser]
Nothing else I can do for you unless you post the url

[eluser]Matt Egan[/eluser]
Here is my "download" function inside my "project" class. Pretty simple.

Code:

function download()    {
    
    if($this->projectPoint->loggedIn())    {
        
        if($this->projectPoint->downloadFile())    {
            
            redirect('project/index/'.$this->session->userdata('projectDirectory'));
            
        } else {
            
            $this->session->set_flashdata('message', 'sorry, there was an error downloading your file');
            
            redirect('project/index/'.$this->session->userdata('projectDirectory'));
            
        }
        
    } else if($this->uri->segment(3) != "")    {
        
        redirect('project/login/'.$this->uri->segment(3));
        
    } else    {
        
        redirect('create/index');
        
    }
    
}

Inside my model, I have two functions that aid in downloading of the file the first is the "downloadFile" function. Again, this is where I am encountering my problems. This works and the file is downloaded if the file doesn't contain any funky characters..

Code:

function downloadFile()    {

    $this->load->helper('url');
    $this->load->helper('download');
    $this->load->helper('file');
    
    $filename  = urldecode($this->uri->segment(4));        
    
    if(!$data = file_get_contents(base_url().'projects/'.$this->session->userdata('projectDirectory').'/'.$filename))    {
        
        return FALSE;
        
    } else{
        
        force_download($filename, $data);

        return TRUE;
        
    }
    
}

Here is the function inside my model that generates links to download the file, "downloadLink" The global variable "files" is a query row of the files database. It creates a link that looks like this - "../index.php/project/download/projectname/filename. extension"

Code:

function downloadLink()    {
    
    $this->load->helper('url');

    echo site_url('project/download/'.$this->session->userdata('projectDirectory').'/'.urlencode($GLOBALS['files']->fileFilename));
    
}

Here is a sample url.

Code:

http://localhost/projectpoint/index.php/project/download/wolff/21_fix_you__coldplay_(1).pdf

I haven't added or taken away from any of the allowed characters Codeigniter came with. I get this error.

Code:

The URI you submitted has disallowed characters.

This is strange, the URI I am submitting does not have any disallowed characters, it does, but they are encoded, the filename is "21_fix_you__coldplay_(1).pdf" so unencoded, it does contain disallowed characters. If I add parenthesis to the allowed characters, I then get this error.

Code:

Message: file_get_contents(http://localhost/projectpoint/projects/wolff/21_fix_you__coldplay_(1).pdf) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found

This is also weird, where did those odd characters come from in the filename, I've echoed out the decoded filename and passed it into the function, and it does the same thing, even though the decoded filename is returned correctly. I'm really stumped here. Any ideas? Again, it all goes down without a hitch if my file contains only characters that Codeigniter allows....