Optimizing MySQL Database structure for big big data

[eluser]Crafter[/eluser]
This si where the advise of a good DBA might come in very handy .

Did I hear you righjt when you said you will create a database for each entry. Sound like overkill to me.

Did you benchmark the possible overheads of connecting to all databases in a round robin manner compared to drawing a query from a single monolithic database?

You coulkd use contructs like view to reduce the complexity of the database structure.

In addition, I would look closely at the queries you are running and run an EXPLAIN command on them. Sometimes just placing an index in the right place, or even changing the order fo fields or where clauses might create huge performance gains.