CSV Importing Advice

The problem looks like you're doing several db interactions at each point during this loop. You're checking to see if the company exists, potentially inserting a new department and updating the user to show they belong to that department. That many db interactions is going to cause a slowdown.

The best thing you can do is to break that down into a few individual steps to minimize the number of times you hit the db. It might mean looping over the CSV and collecting different arrays of info, like users, departments, whatever. Then you could pull all departments in one query. You could collect the department/user correlations and perform a single insert_batch. Things like that.

You're right. I always felt that doing a db interaction for each line wasn't the right choice. I will have to have a good think about how to improve it. Thanks.