Mutli Server Questions

[eluser]domainsvault[/eluser]
I am building a system that I know will eventually reach and exceed load limits on a singular server environment. A cloud rig is not an option (currently). So I am wondering before I get really heavy into the development if its possible to use multiple servers for various aspects.

Examples of "Various Aspects": Multiple DB Servers, Multiple Picture/Storage Servers, Multiple Servers for the actual app itself.. and so on as needs begin to force the hands to fit the enlarging expected population we hope to have one day soon.

I guess what I am mostly curious about is, can I through CI build a system that can handle what is hoped to be an ever lasting need for expansion hardware wise. I mean I may be taking a look at the need for multiple servers in the wrong context and devising a game plan for expansion wrong (which if that be the case, please by all means let me know)

All I know is I need to build a solution early in prep for the hopeful need of one day soon. The particular type of service I am building once launched and running will build steam quick this much we know based on prototype concepts we have launched with a similar concepts in the past but this particular one is much larger scale. So with that we know one machine will not handle the load for to long, and need to have something stable in place for when we go outward from one machine to many.

This is mostly an informational quest than it is one to seek answers of how to based on whats wrong. But anything anyone has to offer on this concept, from there experience to whether or not its even possible where if it is, how.. would be greatly appreciated.

[eluser]n0xie[/eluser]
This is my standard reply to every entrepreneur I meet who thinks he's going to have Facebook like scaling issues.

- Build 'the next big thing' first, worry about scale later.
- Scale up then out.
- Caching is the key
- Hardware is cheap
- Look at what others are doing

If you are absolutely convinced that you will need to scale soon and fast there are some simple technologies that scale quite well: Take at look a Memcached, Redis and any of the big distributed data stores (most notably Cassandra).

So far I seldom come across a system where a few loadbalanced webservers plus some simple MySQL Cluster couldn't handle the load, so before you even begin thinking about 'scaling' issues, my advice would be to just not worry about scaling and just worry about getting your app online.

[eluser]domainsvault[/eluser]
Oh no, definitely not thinking anything as big as facebook, myspace or the like, and very likely no where near millions of concurrent users per second. However we do expect 6 digits easy within a year and a half or so. More so with the companies we are currently in talks with for partnerships. Hence why I am seeking to find out just how extensible CI really is, as we don't really want to build up a service that we will ultimately have to rebuild in favor of something else later. We would rather worry about enhancing instead of reinventing for at least a good while. That said, I know we will be needing multiple servers. My team has built a few social service like sites. Where we have ran into issues with bottlenecking our servers and causing them to crash due to improperly handling the loads (which I know balancing is all the server end and not the script end, however there is the rubber band effect, you pull a file to many times within any given second you can cause it to snap temporarily and result with a 404 like error).

[eluser]domainsvault[/eluser]
noted to the last post of mine, with everything we are doing this is serivce we are building will rely heavy on user input and the main backbone will be mySQL, there will also be high volumes of file storage from pictures and other file types. Again why the need for multiple servers comes into play. I'm no datacenter/networking engineer, so I am not 100% as how the balancing really works on the backend per say (yet). I do try to do my fair share of reading on the subjects as far as what I should worry about, and other practices other sites have used. Though just not sure how to incorporate them into the development exactly yet. But like I said in my first post. I am just curious right now, with anything I should think about putting in now to help save headache in the future when and if the time comes to expand out further. Not looking to build the end all be all solution for it, but you know something that could be considered good practice to begin with. If it we're just me I'd build an entirely proprietary MVC and forego systems like CI, Zend, other. But we are preping a team of 15 to come aboard in the next 3 weeks. So in the name of good practices as far as coding standards within our company goes we feel its best to use a system like CI

[eluser]n0xie[/eluser]
CodeIgniter has a very small footprint which makes it very suitable for scaling. In the end it's mostly a request router, some MVC and some database abstractions. It should be fine for what you need. For serving large amounts of files you will probably not use the framework at all, instead opting to serve it directly via the webserver.

The only caveat I see is if you want to connect to multiple databases at once. While this is no problem in CI, it's not something that works easily out of the box. It's pretty trivial to make it work though.

[eluser]domainsvault[/eluser]
Sweet, thats all I ultimately need to know for now. I appreciate it.