Sending emails using the Amazon Simple Email Service

Sending emails using the Amazon Simple Email Service

by Marius Butuc -
Number of replies: 14

I've recently migrated the Moodle 2.0.2 enabled site I'm working on to a new server (new Amazon EC2 instance), and I notice that the emails sent from that new IP end up 83% of the time being marked as Spam, since the server is now white-listed (yet).

Now, to counteract this, I'm looking into integrating Amazon Simple Email Service (SES) as the default emailing option.
My take is to use Amazon-SES-Mailer-PHP - a drop-in replacement of PHPMailer which sends email via Amazon SES.

And here is where my questions come in:

  1. Has anybody tried so far to implement an alternative to the way Moodle sends out the emails?
    Are there any hints/guidelines for doing this?

  2. Would this be a good time for me to contribute back to Moodle and - when successfully solving this - to turn it into a plug in / module?
    If so, where should I start to learn how to contribute? 
Average of ratings: -
In reply to Marius Butuc

Re: Sending emails using the Amazon Simple Email Service

by Dan Poltawski -
Hi Marius,

Thanks for sharing your plans - to be honest I think you are going about this the wrong way.

I would recommend putting a local mta (exim/postfix) on your EC2 instance and configure that to send mail to SES. There seem to be a few gudies out there about configuring postfix/exim to do this if you google around. Then configure Moodle to talk plain old smtp to your local mailserver. This has the advantage of your local server doing the right thing if it can't contact the amazon service.

I recommend this sort of setup for any resonable sized moodle site in any case because it just allows moodle to fire off the emails and forget whilst the mta does its job of managing the email delivery.

I don't think there would be much interest in seeing this core for that reason (moodle just sends emails with plain old SMTP)

cheers,
Dan
In reply to Dan Poltawski

Re: Sending emails using the Amazon Simple Email Service

by Ben Bowler -

Personally I'd be very interested in this type of plugin. I'm moving a lot off apps to "cloud infrastructure" so it's be good to have multiple email providers configured in Moodle and it fall back from there.

I'm currently trying to implement this by adapting these instructions to no avail. Have you had any luck?

http://docs.moodle.org/22/en/Email_setup_gmail

Ben

In reply to Ben Bowler

Re: Sending emails using the Amazon Simple Email Service

by Ben Bowler -

I've managed to connect to Moodle to Amazon SES through the SMTP settings.

I followed these instructions: http://docs.moodle.org/22/en/Email_setup_gmail

But used email-smtp.us-east-1.amazonaws.com:465 as the domain and the user and password I created in the Amazon Web Services console.

In reply to Ben Bowler

Re: Sending emails using the Amazon Simple Email Service

by Ben Bowler -

I've managed to connect to Moodle to Amazon SES through the SMTP settings.

I followed these instructions: http://docs.moodle.org/22/en/Email_setup_gmail

But used email-smtp.us-east-1.amazonaws.com:465 as the domain and the user and password I created in the Amazon Web Services console.

In reply to Ben Bowler

Re: Sending emails using the Amazon Simple Email Service

by julian maya -

Ben:

Could you describe the steps that you follow in order to make work Amazon SES. 

I have tried everything, included install postfix and I cant make it send mails.

In reply to julian maya

Re: Sending emails using the Amazon Simple Email Service

by Mika Seppala -

The problem seems to be that moodle sets the sender email address to be the actual address of the sender. Hence Amazon will realize, in most cases, that the sender is not authenticated, and the message will not be delivered. A remedy to this is to hide the email addresses of all users. In this case moodle will use a default no-reply address as the sender. Assuming that this is set correctly, i.e., so that the no-reply address is of a verified domain, messages will get delivered. But one cannot reply to such messages.

In reply to Marius Butuc

Re: Sending emails using the Amazon Simple Email Service

by Richard Phillips -

I tried several approaches recomended by others - all worked to a point but had one drawback or another. 

Finally hit upon using emailrelay which is a simple SMTP relay.  Very simple to set up and works perfectly. 

yum -y install openssl-devel

Download source for emailrelay,  and compile with openssl so the encrypted connection to SES will work. 

./configure --with-openssl

make & make install

I put my SES SMTP credentials (not amazon aws keys) into a file: eg /etc/amazon-ses as a single line thus: 

login client <username> <password>

Then run email relay as service using the following... 

/usr/local/sbin/emailrelay --as-server --port 10025 --spool-dir /tmp/spool --poll 30 --forward-to email-smtp.us-east-1.amazonaws.com:587 --client-tls --client-auth /etc/amazon-ses --spool-dir /tmp/spool

This will listen on port 10025 for incoming SMTP reqests, spool them to /tmp/spool and every 30 seconds attempt to send them through to amazon SES for delivery. Of course changing these settings is trivial. 

In centos I put this in /etc/rc.d/rc.local so it runs at startup. Check your machine is firewalled so external connections to port 10025 are not permitted. 

Then, within Moodle, I set up the email plugin to use SMTP address 127.0.0.1:10025 and no authentication details. 

One gotcha is that Moodle uses the 'support contact' email address as the sender (not the no-reply address), so this MUST be an address verified through your SES account with amazon. 

One of this benefits of this approach is that your Moodle PHP can 'send' email almost instantly - there is no pause waiting for SSL handshaking with a distant machine - and the emails are then spooled to be sent in an orderly way using a different process. 

This approach will also work with Gmail and avoids any core hacks. It only takes ten minutes to set up and seems rock solid (so far). 

My machine is on Amazon EC2 and Apache is Chrooted which created challenges for communicating with local MTA (no localhost).  

Hope it helps...

In reply to Richard Phillips

Re: Sending emails using the Amazon Simple Email Service

by R.T. Brown -

I have tried the the E-MailRelay method, but continue to receive the follow errors in .bad files. This is odd because I have both a verified domain and email address configure in AWS SES:

~~~~

X-MailRelay-Format: #2821.3 X-MailRelay-Content: 7bit
X-MailRelay-From: verified@domain.edu
X-MailRelay-ToCount: 1
X-MailRelay-To-Remote: student@someaddress.com
X-MailRelay-Authentication: servername
X-MailRelay-Client: 127.0.0.1
X-MailRelay-End: 1
X-MailRelay-Reason: smtp client failure: Message rejected: Email address is not verified.
X-MailRelay-ReasonCode: 554

In reply to Marius Butuc

Re: Sending emails using the Amazon Simple Email Service

by Kyle Goslin -

I have been working with Amazon EC2 based Moodles this semester and have been happy with everything until it hit the mail!

I used Jango SMTP for a while, they had a nice mail relay service, and very little effort was envolved. 

But being hosted on the Amazon, I wanted to change over to the SES. And after a little head banging, I managed to get it working!

Here is what I've done:

1. Signed up for the SES and created the auth key pair needed.

2. I decided to go with one email address that will be used for email, so I verified the email address I wanted to use with the SES.

3. Just for kicks I verified the domain also, which wasn't needed. Just by chance I may need it later.

4. Add the details to the Plugins -> Message Outputs -> Email.

The address I used was email-smtp.us-east-1.amazonaws.com:587

This has the port number for TLS.

5. I enabled TLS and added the username and password auth details.

6. I updated the reply to address, to the one I verified.

7. Then I went to the support email address and updated it (admin/settings.php?section=supportcontact) to the verified email also. This is very important because this is the address things are often sent from!)

 

And then It was ready to roll!

Another little pointer was, ensure you have your cron set up, because things like the forum post emails rely on this!

 

- Kyle.

In reply to Kyle Goslin

Re: Sending emails using the Amazon Simple Email Service

by Kyle Goslin -

This method though, has proved to show issues when emails are sent from user addresses, as they can not be verified, e.g. a gmail domain. So on further thought, the previous post of the mail relay may be worth a shot!

In reply to Kyle Goslin

Re: Sending emails using the Amazon Simple Email Service

by Kyle Goslin -

Just another little follow up on this.

As each address needs to be verified by Amazon, which is impossible for student emails (e.g gmail.com addresses), Amazon will not allow it to be sent out.

As a hack on this,

inside the moodle/lib/moodlelib.php folder

In this function:

 

function email_to_user($user, $from, $subject, $messagetext, $messagehtml='', $attachment='', $attachname='', $usetrueaddress=true, $replyto='', $replytoname='', $wordwrapwidth=79) {

global $CFG;

$from = 'outgoingmailaddress@domain.com';
$replyto = 'outgoingmailaddress@domain.com';

 

I added those two lines just below the CFG, and that was the address I verified with the Amazon SES.

Bit of a hack, because all mail goes though one address. But it works for cases when students send an email, which because it was from their address would otherwise be rejected by the Amazon SES for not being a verified send address.

In reply to Marius Butuc

Re: Sending emails using the Amazon Simple Email Service

by Ian McNaught -

I've been running Moodle on AWS for nearly 2 years, and email was the biggest headache until I discovered Critsend email relay service. It's not free, but incredibly cheap (and you get 50,000 mails for free to start you off). It will take you about 5 minutes to get Moodle on AWS setup to use critsend, and you get useful reports and statistics too (e.g. how many users open the emails sent).

We've used them for 2 years now, sent about 500,000 emails and not had any problems.

I wanted to use SES to keep it all in AWS, but it won't send on behalf of others, only on authorised email addresses - which isn't very suitable for Moodle.

Average of ratings: Useful (2)
In reply to Ian McNaught

Re: Sending emails using the Amazon Simple Email Service

by Patrick Weller -

Ian,

Could you provide additional information on Critsend? I registered for the service, but for the last several days I get this message, "

Your Account Is Pending Approval

We will approve it in the next twenty four hours max. In case of emergencies, please contact us"

Can you personally contact or get response from their Support team?

Does the Support team respond to your emails?

This is not a slam on the company. I am literally wondering if this is an anomoly or if the service is not viable.

Also, are you aware of any other comparable services?

 

In reply to Patrick Weller

Re: Sending emails using the Amazon Simple Email Service

by Bora Uslusoy -

Same here. I keep getting the same message. Is there any altervative services for Critsend. Desperately seeking...