Hi there,
is there a solution to send emails through the Mandrill REST API?
Sending emails by SMTP is takes ~50msec which at first feels like short time but if you have thousands of users its not.
If there is not a solution what would be the best way to start creating a solution?
Do we have to create a new provider? A new PHPMailer class?
Thank you in advance
Sorry for not being clear.
I was wondering if there is another option than the SMTP protocol.
If i understand correctly, when i set a forum and users are subscribed, Moodle sends email notifications to forum members one by one. We can of course set SMTP Keep alive to a high number, and this is what we did, but even with a fast service such as Mandrill SMTP it takes around 50ms per email. Given the fact that we have thousands of users it will take a lot of time to send emails on every forum post.
That is why i have send the question to ask if there is another option.
An option that will use modern REST API to send emails to Mandrill.
Or a solution where email notifications for Forums will be grouped together and send as bulk (e.g. use X-MC-PreserveRecipients)
If there is no option for this, how would i proceed to create an option? Do i need to create a new Provider?
Thank you in advance
Yes, you need to create your own custom provider. Than you can disable standard SMTP provider and just use custom plugin.
Thank you Darko for the response.
Moodle has no plans to change the way Forum notifications via email are delivered?
Wont be better choice if instead of sending mails to users one by one to bulk send to users that are in that Forum?
In case we consider to create a new Provider we will end up with the same situation 1 email per user. Am i correct?
Is there a better option that i miss?
Moodle has no plans to change the way Forum notifications via email are delivered?
Most likely not. You can always open an issue in tracker but the timeline for something like that could be from here to eternity.
Wont be better choice if instead of sending mails to users one by one to bulk send to users that are in that Forum?
Probably, but that is not how they implemented it.
In case we consider to create a new Provider we will end up with the same situation 1 email per user. Am i correct?
Yes. It does not mean your provider MUST send everything as it is being received from Moodle. Being the implementer you can also implement the desired behavior. For example whenever a send message method in your provider is being called you could just store it somewhere and process it using custom scheduled task whenever you need it.
Hi Nikolas,
No, Moodle has no plans to change the way in which Forum notifications are delivered.
It is not possible to send e-mails in bulk.
You will still end up with the same solution of one transaction per user.
Hello Nikolas,
There are other alternatives to SMTP but you would need to write a Moodle Message provider as others have said.
Although Moodle does send an e-mail notification to all forum members one-by-one, it would categorically not be possible to convert this to use a service like Mandrill. There are many parts to the sending process and each e-mail is unique to the user.
At the very minimum, different users receive different e-mail notifications:
In addition, each user will receive different versions of the notification:
Furthermore if the site has e-mail replies enabled then a unique reply address is inserted into each e-mail. That token is per-user/per-message and must be unique.
Therefore I do not think it will be practical to send Mandrill a template and tell it to send that template to a list of participants. You could only really do so where users make use of the same language, and mail format, and probably where mail replies are disabled.
Whilst you could convert many of these to variables, the complexity to do so dynamically for every notification would be extremely high. You'd probably only want to do so for the access tokens and some of the links, and possibly the reply address,. You would therefore still have to batch notifications by language (there would just be too many variables for this to make sense), mail type (HTML vs Plain, though post prefer HTML these days), and forum digests would need to continue to be treated completely separately.
The complexity of this set of restrictions would likely make this far less worthwhile.
Instead, I would recommend that you run a local Mail Transfer Agent (MTA) -- this can utilise SMTP, or it can be via the sendmail socket. A local MTA queues the mails and sends them for you, rather than having Moodle connect to a remote SMTP server. This is preferential because the latency for a local MTA is far lower than connecting to a remote SMTP server, and an MTA is far more efficient at sending mails to the next hop (this is literally its entire job). Use of a local MTA will also help to smooth out load concerns because it will automatically (well, when configured correctly) reduce the speed at which it sends mail under high load, and increase it under lower load.
I will also point you at MDL-46881 where we have completely rewritten the way in which we queue forum notifications to be more efficient. Unfortunately this change also has the effect of making your proposal even harder, but it will allow generally faster delivery of forum notifications.
Please note that the "X-MC-PreserveRecipients" header is a Mandrill-specific header.
Hope this helps,
Andrew
Thank you both for your detailed answers.
Sorry for asking again but i want to be sure that i understand it correctly.
1. No matter what setting i have or what technology is use, emails will always be send to each user individually when using the plugin "FORUM".
2. I can achieve better performance If we use an MTA (e.g. Postfix)
3. Not quite clear for me what the MDL-46881 issue solves. Sorry i am not a core Moodle developer.
Hi Nikolas,
1) correct. Each user’s notification is personalised to that user.
2) Yes. This is something I would strongly recommend and it should make a huge difference. I’ve always used exim but that’s because I worked with the mail experts in my team favoured it.
3) the changes in that issue modify the way in which Moodle sends emails. Rather than one process determining which forum posts are eligible and then determining which users to send to, and then sending them one by one, we now queue an additional task for each user. The main intnent of this change is to reduce the reliance upon a single and complex process, but it does allow for parallelisation of the queue.
As a very first step though I would strongnly suggest setting up and spending the time to configure your MTA.
Andrew
Thank you Andrew.
I have a Postfix setup running in my server for other services. It is tested and works with Mandrill .
For Moodle configuration then, will be just to send the mails to the local server (localhost) on the port postfix listens to?
Hi Nikolas,
You have two options:
Best wishes,
Andrew
Thank you very much for your answers. Once i get back to office i will try the solutions.