When Queue Fails

Hi,
Please note my application is hosted on a shared hosting platform.

I am using the Queue package to process 3rd Party Webhooks, and for the most part, the Queue package is working like a dream.

However, if a queued job fails, the Queue class writes the queue data to the queue_jobs_failed table, then seems to just exit.

This is causing problems on the shared hosting platform. As per the documentation I have a CRON job executing the following every 1 minute:

Code:

php spark queue:work myqueuename

Because a failed job will just exit after writing to queue_jobs_failed table and not return the cron, the CRON does not know the job has finished executing and leaves the cron task open on the server. If the issue persists, the servers task queue quickly becomes overloaded as it starts a new cron every minute, then will eventually "crash".

I have implemented an exception handlers, where I can, to return the cron, but was wondering if there are recommendations on how to exit the Queue class (return the cron) if/when the queue fails?

When running the queue I'm considering using the suffix

Code:

--stop-when-empty

But, would the Queue class recognise the queue an empty and return the cron even after the case of a failed queue?

Note: 
I'm not sure about other environments, but if your application is hosted on a shared server and you are implementing Codeigniter Queue Package I would advise that for any development or testing you use the servers cPanel - Terminal (or similar)  to run the queue, rather than cron. That way the servers task queue does not become overloaded if the queue fails.

Because a failed job will just exit after writing to queue_jobs_failed table and not return the cron, the CRON does not know the job has finished executing and leaves the cron task open on the server. If the issue persists, the servers task queue quickly becomes overloaded as it starts a new cron every minute, then will eventually "crash".

Exactly this problem is described in the dedicated section: https://queue.codeigniter.com/running-queues/#with-cron

If you set CRON to run the "queue:work" every minute, then you should definitely use "--stop-when-empty" plus some other flag, like "-max-jobs" or "-max-time" and make sure that the values used will not allow the worker to exceed the time that will trigger the next CRON task.

Here, you can check the available options: https://queue.codeigniter.com/commands/#queuework

But, would the Queue class recognise the queue an empty and return the cron even after the case of a failed queue?

If after a failed job, there are no other jobs to process, then yes - the queue will exit.

If you want to stop the queue when the job fails, then you can simply call this command from your job before throwing an exception.

PHP Code:

command('queue:stop myqueuename') 


Many thanks michalsn, this is very helpful.