CodeIgniter Forums
Managing large recordset and maintaining database (MySQL) - Printable Version

+- CodeIgniter Forums (https://forum.codeigniter.com)
+-- Forum: Using CodeIgniter (https://forum.codeigniter.com/forumdisplay.php?fid=5)
+--- Forum: Best Practices (https://forum.codeigniter.com/forumdisplay.php?fid=12)
+--- Thread: Managing large recordset and maintaining database (MySQL) (/showthread.php?tid=72789)



Managing large recordset and maintaining database (MySQL) - kaitenz - 02-12-2019

Hi,

Need some help about databases. I'm currently using MySQL as my database and CI as my framework. I created a web page where I can upload a CSV file and insert them to my table. I have CSV file with 90K+ lines in it.

After the upload and insert process, and checking the dashboard of my app, it became slow. My app took around 5 seconds load time.

How can I manage large database without stalling or slowing the application? Is there any special technique on database or source code?

Thank you.

By the way, I use DataTables + Ajax to display records.


RE: Managing large recordset and maintaining database (MySQL) - php_rocs - 02-12-2019

@kaitenz,

What is your setup? PHP version? CI version? MySQL version? Have you debugged your code to see if there are any bottle necks? Have you tested your queries to see if they need to be refined. Does your tables have indexes?


RE: Managing large recordset and maintaining database (MySQL) - gondwe - 02-22-2019

When records hit this high, resolve to using custom pagination libraries other than datatables