Insert in chunks

[eluser]codex[/eluser]
[quote author="TheFuzzy0ne" date="1239049255"]
I don't fully understand how you're processing your data, but I'm sure that with a little work, and a lot of shouting, you can get this to work in your set-up.[/quote]

Again Fuzz, I gotta thank you. This setup works really well for my needs. I slightly altered the output though: instead of outputting insert statements I output the data as csv rows (delimited by ';') so I can load the file using LOAD DATA INFILE, which works really, really fast.

I'm still experimenting, but this seems like the way to go.

[eluser]TheFuzzy0ne[/eluser]
I'm glad it's working for you. I had been thinking about using the CSV data as-is, but I wasn't sure about what other data might have been in there. Good thinking though, as it saves you even more processing time.