A cronjob to extract the contents of a zip file into a DB.

[eluser]gh0st[/eluser]
I've got a cronjob controller up and running fine (it just sends an email at the moment).

My situation is as follows;

1. Every night a new ZIP file, which contains a CSV file, which has 500,000+ rows will be uploaded to a specific directory outside of web root.
2. A cronjob runs a controller method (inaccessible via a web browser) that will:
a. Extract the contents of the ZIP file and then perform a few validation rules and then insert them into the Database.

But CI does not come with a UNZIP extractor, and I'm wondering is there a faster, easier way of handling such amounts of data.

Thanks

[eluser]xzela[/eluser]
why not do it all in the cronjob? why does CI have to do the work? Wouldn't it be easier to just create a shell script that does exactly what you are describing?

[eluser]gh0st[/eluser]
I never thought of that! Doh! I'll have to look into that

[eluser]gh0st[/eluser]
I've been trying to run both a unix gunzip and unzip via cron on my system, but regardless of what I do it keeps saying it cannot find the file, despite it existing in the said directory.

I've even tried a 1KB zip file as well as a 1KB tar file to see what would happen, but I keep getting this error message and I don't know how to fix it.

# the error message I get
unzip: cannot find or open /home/somepath/file.zip
, /home/somepath/file.zip
.zip or /home/somepath/file.zip
.ZIP.

Code:

#!/bin/sh
# my bash script
unzip -t ~/somepath/file.zip
echo "done"

I've checked the path outputted by ~/somepath/file.zip and it is the same as /home/somepath/file.zip

What else could I be missing? Any help on this would be great!

[eluser]TheFuzzy0ne[/eluser]
Does your path contain any spaces? It may also serve you better to try using an absolute path rather than a path relative to your home directory. Another thing to check are the permissions on the file you're trying to use, as well as ensuring that you can manually unzip the file in question.

[eluser]gh0st[/eluser]
Hi there.

1. I tried absolute paths and it made no difference. The same error message kept appearing
2. Permissions are 777, including the folder (this is for test purposes, I'll change it later to something more secure)
3. I've tried manually unzipping the file and it seems fine.

I think the problem could be one of two things.

1. I'm not allowed to use the unzip command outside of web root.
2. The zip file is causing errors.

[eluser]TheFuzzy0ne[/eluser]
Perhaps you need to cd into the directory where you want to unzip the zip, first?

[eluser]Hockeychap[/eluser]
At the risk of teaching you to suck eggs, the home directory "~" depends upon the using starting the shell script.

So if it's running from crontab it will run as the user who's crontab it is. If you are triggering / executing the script from the web server the shell will run as the user of the web server process.

Cheers
Justin

[eluser]TheFuzzy0ne[/eluser]
That's assuming the suPHP module is installed running. Without suPHP, I'm sure that PHP runs as user "nobody".

[eluser]yannyannyann[/eluser]
ghost, could you explain how you did the cron for this ?
I'd like to call a php via CRON as well, but I'm a bit confused with what exactyl I should write (command line as well as php side.. )

do you use things like curl or wget ?


thanks in advance