web crawler

[eluser]jcavard[/eluser]
[quote author="Kamarg" date="1265688329"]If you are wanting to continue with your php version, look into pooling. This link is for thread pooling but the same idea applies substituting sockets/connections for threads.[/quote]

Well, at this point I'm considering all avenues, so if PHP is not best, I might just rewrite the whole thing.

The only requirements are:
- must run on a shared hosting
- possibility to limit concurrent connections

maybe the a C++ extension might be a good idea. but my main concern is the darn concurrent connection...


Anyone knows how I can monitor/limit the concurrent connection?