Welcome Guest, Not a member yet? Register   Sign In
Any thought of having an email queue or "spooling" in CI4?
#1

(This post was last modified: 03-03-2016, 04:39 PM by skunkbad.)

Over the years, I've come to realize that an email queue is pretty much essential for any website that sends email, and in my experience they all do.

By email queue, I mean:

  1. Emails are not immediately sent, but instead stored in database records.
  2. Any attachments that go along with the email are stored on the filesystem, and referenced in the queue record.
  3. A cron checks for unsent email and sends X amount of emails every N minutes.

I'm sure there are many ways to do this, but there are many advantages of having this type of queue, including having a history that admins can look at, the ability to re-send emails if somebody says they never got it, being able to view emails on development systems that don't have the ability to send email, etc.

Any chance CI4 might have such a thing built in?
Reply
#2

I don't think it's been discussed, actually.

I have thought for a while that some form of Queue integration should be there. It's present in too many of the discussions I've seen over the last few years on building performant sites that scale. And can be especially noticeable doing image processing or email sending. Helps keep the site speedy, also.

So, I guess the answer for now is, it's likely, but I can't say for certain at this point. Smile
Reply
#3

(This post was last modified: 03-05-2016, 12:30 AM by sv3tli0.)

I think that such thing should be additional package to CI.
Its something that many devs may use but its not required for all..
I already started a thread for building CI community apps and addon packages perhaps this can be there too
Best VPS Hosting : Digital Ocean
Reply
#4

I think I agree with sv3tli0 that this would have to be an add-on. Although I do agree that email queues are very important. But as already mentioned so are queues for image processing, or file uploading.

Any queue for CI would have to be a generalized queue not built specifically for email. It would have to have a queue name (for different queues), and some way for timing the queues, and some generalized way of triggering different events, since not every host has access to cron jobs like some shared hosting platforms. In fact I have no idea how all that could actually work.

So although I like the idea of a generalized queue, or a specific email queue, I think that like the shopping cart class or user management, it is not something that can be easily generalized to suit everyone or anyone, and most people needing such functionality would use either a more specific third party app or build their own.

Just my thoughts,

Best wishes,

Paul.
Reply
#5

It is what you make it. I'm not asking for some cheesy, half thought out, or rigid (you must do it my way) type of solution. Both Symfony and Laravel have solutions, so why not CodeIgniter?

Laravel: https://laravel.com/docs/4.2/mail#queueing-mail
Symfony: http://symfony.com/doc/current/cookbook/...spool.html

I built my own solution that is also a driver so I can switch between CI's email class and Swiftmailer. Nice options might be that one could use file, memory, or database type storage for the queue, although mine is only set up for database storage.

You are saying that you might think the solution needs to be generalized, but why not just make it highly flexible and configurable? Also, if you're not using the queue, it's not like it has to be a hindrance or some sort of obstacle. You would still be able to send email without using it.

Again, once I started using an email queue, I can't see building a website without one, even if the website is small. At least for me, since I feel that it is really a feature that belongs in CodeIgniter, that's why I've brought it up here. I could just keep using mine, but it is a highly desirable feature for a number of reasons.
Reply
#6

My thoughts were that queues are important for many types of sites today, moreso now than in the past, since different forms of architecture abound, and performance is becoming key for many, many sites. That's why I'd love to see a queue system in place. Laravel was where I first really thought about a generalized one in a framework.

So, for me, it's important enough of a concept for modern applications that it should be included, and a standardized API across a handful of queues, like Laravel does, really makes sense to me.

Again - this hasn't been discussed among the council yet so we'll have to have that discussion and see where the rest of the team lies on this.
Reply
#7

+1 for a solution built within the framework.
Reply
#8

(03-05-2016, 07:27 AM)skunkbad Wrote: It is what you make it...

You are saying that you might think the solution needs to be generalized, but why not just make it highly flexible and configurable...?

... I can't see building a website without one, even if the website is small...

Excuse my curtness but am on my phone.

I have been thinking about what you said and looking at Symfony'Spool', and I have changed my mind. I think you are right and it would be, assuming it is configurable, a very good addition to the core. My vote is for a Symfony'Spool' like queue to be created for CI.

It's a great idea :-)

Paul.
Reply
#9

This and events should be considered for CI4, on my honest opinion.
Best regards,
José Postiga
Senior Backend Developer
Reply
#10

Feels like it probably should be additional optional package, because of different app and SMTP supplier requirements.

However, if you are planning to add email queuing, and brief look at both Laravel and Symphony documentation they don't seem to have it, can you please add "priority".

We are using AWS SES to send all our web app emails and user newsletter through same bespoke CI email system.

AWS has timed send limit, and their guidelines suggested on failure to try and send email again in 5 minutes. We ended up adding date field to DB, a bit like reverse expiry, emails are only considered ready for sending if time in that field has passed ( NOW() + 5 minutes ). Technically this also can help with Google Inbox "Undo send" feature.

There are 7-10K newsletters going out in rate of about 50 a minute. So we also implemented "priority" for messages, where lets say password reset request or very important app notification is priority #1, and newsletter is priority #7. Higher (or mathematically speaking, lower) priority emails are always sent before lower priority ones.

In future version we also want to implement sent email archive, probably 1 DB table for email body and description, a table for actively queued email IDs, "ready-for-send" dates, and a table for events (sent, failed /w reason/debug, ...).

Interesting that Laravel uses one shared queuing system.
Reply




Theme © iAndrew 2016 - Forum software by © MyBB