Welcome Guest, Not a member yet? Register   Sign In
Having lots if insert into database for thread analytics
#2

Hi,

I think you are worrying unnecessarily. An insert is not slowed down by the size of the table. Counting results is pretty quick too. DB's were built to do these tasks efficiently and quickly, in fact it never ceases to amaze me how they perform so well.

You should be wary of table size when you start reaching the limits of your DB and environment limits, usually, or so I have read but never experienced, when you have start reaching millions of records. But this is usually due to the indexes no longer fitting into your RAM, and although I have no experience of doing this, very big data sets can be managed in a variety of ways that is much more about how you are hosting and configuring the database rather than any actual DB limitations.

However, you probably also should be doing some clean up on those tables. For instance, summarizing historical data into a summary table and cleaning out your thread table. I suppose it depends on your usage of the data.

My biggest tables were done for a site where I was checking which items a user had viewed. It never seemed right logging potentially 'every resource' times 'every user' just for a viewed change of style. But I never found a cleverer way to do it. The other one was a history table of user actions, that became enormous, easily racking up 200 rows per user per visit. That one was more easily solved with with a clean up and archive routine though.

There are some really interesting reads about all this although for me I quickly get lost when they start talking about stacks and server environments.
Reply


Messages In This Thread
RE: Having lots if insert into database for thread analytics - by PaulD - 10-09-2016, 03:13 AM



Theme © iAndrew 2016 - Forum software by © MyBB