running a separate process to run reports

I have just about finished my app after years of effort. But in load testing I have found a significant problem. I can't afford the vcpu's needed to have a reasonable service. I am using RDS and with the type of dbase instance I can afford, I can run a single user session, at a time. This just won't work on a business level.

Of course I can run many users who are doing data entry, but the creation of the report is a severe bottleneck and the report is the final output of the system ie the reason users will use it. My ultimate goal was to allow 25 users to run reports at the same time. There is no way I can afford a large enough dbase server to do that (the project is freemimum) it would require a dbase server with at least 32 vcpu's. 

So I am considering alternatives. My current thought is to separate the report generation section and set it up via something like AWS SQS which will queue requests to run a report. To do this I will need to take my report code and stick it in a separate process, and have the process just wait for a report request. Once the report is done I can stick the report in a non SQL  datastore such as S3 so the user can get it the next day. 

At first I was thinking to use Lambda which is AWS's non server based procedure, but I use Koolreport  heavily which is a php tool plus php is not supported on Lambda. 

Is there any type of 3rd party app that will allow me to queue a request and run a separate process in php? I am still running CI3. Any guidance is appreciated.

proof that an old dog can learn new tricks

I have decided to just allow one user at a time to run the time consuming function. I know there is no semaphore in CI3. I guess I could use $_GLOBALS or $_SERVER variable. Which of these can be seen on the server by all the instances?

proof that an old dog can learn new tricks

I guess I could use $_GLOBALS or $_SERVER variable. Which of these can be seen on the server by all the instances?

No.

See https://tideways.com/profiler/blog/php-s...-downsides

ci-phpunit-test | CodeIgniter Testing Guide | CodeIgniter 3 to 4 Upgrade Helper

Very helpful!

Now things make more sense. I think that if I used a large rds instance with say 64 virtual cpus I could run 64 users. Of course that is a worst case. I will try it out for a few minutes.

I *think* that is the issue. So by serializing access to the rds server I will severely slow down my system. Btw, my sql on the server sucks but from my understanding it is the number of vCPUs.

I know it is not the best solution but setting an env variable with getenv and putenv might possibly work?

---

Very helpful!

Now things make more sense.  I think that if I used a large rds instance with say 64 virtual cpus I could run 64 users. Of course that is a worst case. I will try it out for a few minutes.

I *think* that is the issue. So by serializing access to the rds server I will severely slow down my system.  Btw, my sql on the server sucks but from my understanding it is the number of vCPUs.

I know it is not the best solution but setting an env variable with getenv and putenv might possibly work?

BTW, I estimate that users will be only running reports about 10% of the time. So running 10 report at the same time would be just fine.

proof that an old dog can learn new tricks

I am creating a bunch of temporary tables. If they already exist then I delete them. I am running on AWS RDS. So if user A creates his temp tables and is still using them and user B tries to create his own temp tables, but sees (on the RDS server) that they already exist, so deletes them (but they are user A's temp tables), this could be an issue. I need to be certain that user A's temp tables and user B's temp tables do no collide?

proof that an old dog can learn new tricks