get_dir_file_info() hangs when run on a large directory

#1
[eluser]Unknown[/eluser]
Hi, I have made a little function that deletes files based on date. Prior to doing the deletions, it lets the user choose how many days/months back to delete files, telling them how many files and how much memory it would clean up.

It worked great in my test environment, but when I attempted to test it on a larger directory (approximately 100K files), it hangs.

I've stripped out everything else from my code to ensure that it is the get_dir_info() function that is causing the issue.

Code:
$this->load->helper('file');
$folder = "iPad/images/";
set_time_limit (0);
echo "working<br />";
$dirListArray = get_dir_file_info($folder);
echo "still working";

When I run this, the page loads for approximately 60 seconds, then displays only the first message "working" and not the following message "still working".

It doesn't seem to be a system/php memory problem as it is coming back after 60 seconds and the server respects my set_time_limit() as I've had to use that for other processes.

Is there some other memory/time limit I might be hitting that I need to adjust?




#2
[eluser]CroNiX[/eluser]
Not sure how to solve your exact problem there, but wouldn't it be much simpler to just store the filename, filesize and the upload date in a datebase and then query the database and use the filenames as the ones to delete? There's an awful lot of overhead in what you're doing, and it will only take longer and longer and require more memory as the number of files in the dir grow since you have to scan each file individually to check it's filedate/last modified time. A db query combined with deleting the files would take tenths of a second or so no matter how large the dir is.

#3
[eluser]Unknown[/eluser]
I received a suggestion over on stackoverflow that worked.

Instead of pulling all of the information in one command, I ended up breaking it up into two commands get_filenames and get_file_info (which is essentially what you were suggesting):

Code:
$allFiles = get_filenames($folder);
foreach ($allFiles as $fileName)
{
   $fpath = $folder.$fileName;
   $finfo = get_file_info($fpath, array('size','date'));
}

Thanks for replying, I appreciate the help!


Digg   Delicious   Reddit   Facebook   Twitter   StumbleUpon  


  Theme © 2014 iAndrew  
Powered By MyBB, © 2002-2020 MyBB Group.