How do find where the email process breaks down?

How do find where the email process breaks down?

by Jurgen Oschadleus -
Number of replies: 13

I am new to Moodle and have managed to get my test site largely set up - but I simply cannot get email functionality to work and am at wits end. I believe I've done everything outlined in the setup documentation for 2.8.1; I've checked all the email-related settings (Site admin/"incoming mail" and "server") and have searched the discussion forums for days. I can create messages which are visible to users when they access the messages site, but I simply cannot get Moodle to send out email messages.

From my understanding of the Cron log files, the process runs and connects to the email server, but it never seems to find any messages to be picked up and sent out.

In all probability it's a small setting that I've missed somewhere, but I simply don't know where to look anymore, and hope someone out there can give me some suggestions. I've attached the latest cron log file in case that helps.

Incidentally, the system runs on an Apache server.

Any suggestions would be appreciated.

Regards, Jurgen

Average of ratings: -
In reply to Jurgen Oschadleus

Re: How do find where the email process breaks down?

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

The email settings are under Site Admin/Plugins/Messages/Email - make sure that is set up correctly..

Average of ratings: -
In reply to Emma Richardson

Re: How do find where the email process breaks down?

by Jurgen Oschadleus -

Hi, Emma - thanks for providing the full link to the site. To my knowledge the settings are correct. Cron reports that it can connect to the email server. But I still not see any evidence that emails are actually being distributed. I've tried sending messages, resetting passwords, etc - the screen tells me the messages have been set, but nothing gets delivered. I've checked my spam folders to make sure the messages are not being trapped. But I suspect that they're not even getting that far since the cron log file shows 0 messages.

Any other place you can think of where I should look?

Regards, Jurgen

Average of ratings: -
In reply to Jurgen Oschadleus

Re: How do find where the email process breaks down?

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Have you checked your notification settings?  Do you have email set up as a notification method for yourself?

Average of ratings: -
In reply to Emma Richardson

Re: How do find where the email process breaks down?

by Jurgen Oschadleus -

Hi, Emma - yes, email notification is selected for myself and all other users.

Jurgen

Average of ratings: -
In reply to Jurgen Oschadleus

Re: How do find where the email process breaks down?

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Some of the email notifications go out automatically and don't wait for cron.  I would suspect that something is not set up correctly with your email settings. I would double check all your settings and authentication methods.  You might trying using the php mail through the server and see if that works.

Average of ratings: Useful (1)
In reply to Emma Richardson

Re: How do find where the email process breaks down?

by Jurgen Oschadleus -

Hi, Emma - I appreciate your effort to help.

This is my frustration - I believe I have checked all the settings and authentication methods that I'm aware of, and it's still not working. So there is clearly something I'm missing somewhere, but I just can't figure out where or what.

I'll see if I can figure out the whether php mail is an option. Thanks again.

Average of ratings: -
In reply to Emma Richardson

Re: How do find where the email process breaks down?

by Jurgen Oschadleus -

Emma - here's a virtual hug for you! Simply remove any references to mail server and let Moodle do it for you. Why didn't I think of that?

It seems the problem is now solved. Thank you for your persistence.

Jurgen

Average of ratings: Useful (1)
In reply to Jurgen Oschadleus

Re: How do find where the email process breaks down?

by Gary Benner -

HI Jurgen,

 

Came in late in this discussion, so for completeness am adding my 2c worth.

It is important to know that Moodle has the ability itself to send emails direct to end user mail servers using inbuilt smtp send routines.  It can also send the emails out via a nominated SMTP server.

Now a number of factors can effect the delivery of emails. Here are some from my experience:

 

1. When using the inbuilt SMTP (no email server specified) the path out from the server (on port 25 ) may be blocked, or the IP address may be blacklisted. Try Senderbase if you think it is a problem.

2. In specifying an external mail server you may need to check which port it received email on - usually it is port 25, sometimes port 587 or it may be another. You will also need to check if you require smtp authentication (username and password) to access the server. Again you need to ensure you have firewall rules allowing access.

3. In both cases you also need to check if the domain from which you are sending ( the part of your email address after the '@' - in my case "benner.co.nz") does not have a DNS SPF record limiting which IP Addresses may send emails on behalf of your domain. This is a security mechanism, and will cause every well configured email server to reject your outgoing emails when they receive them if the IP Address of your server is not listed in the SPF Record. This stops spammers sending emails with your email address as the return address.

 

To test your Moodle server's capability to send emails you can first test using your own email address, then some others, and if there are still problems, you can install a dummy SMTP server such as FakeSMTP. This will receive your emails but not send them anywhere - it will just list them for you.

 

Moodle also will send some emails immediately from the screen you are on, and others are relegated to the regular CRON Job that gets run every few minutes. The latter is harder to debug, but I  usually disable the background CRON and run it manually from a browser, and so you can see the log of what happens. You can see when emails are sent.

 

If you are really stumped, then there is Wireshark and other tools for analysis of the network traffic out of your server, but you need to be keen, or a professional at that point.

 

Finally, email is increasingly subject to tightening of security, due to the unpleasant side effects of spamming. If you are responsible for a Moodle server, then do some study on SMTP, or get alongside someone who does it for a day job.

 

kind regards

Gary

 

 

 

Average of ratings: Useful (2)
In reply to Gary Benner

Re: How do find where the email process breaks down?

by Jurgen Oschadleus -

Thanks, Gary.

I have email sort of working through the default PHP mailing at the moment, but I'm not wildly confident that I have it under control.

Your notes are definitely helpful - although the main thing I took out of all of the above is that I should get alongside an SMTP expert and let him or her figure it out for me  smile That will be my next step.

Regards, Jurgen

Average of ratings: -
In reply to Gary Benner

Re: How do find where the email process breaks down?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

"It is important to know that Moodle has the ability itself to send emails direct to end user mail servers using inbuilt smtp send routines.  It can also send the emails out via a nominated SMTP server."

Untrue... if you don't do anything, Moodle will use the mail delivery settings in php.ini. This usually means using the Unix mail 'standard' which is Sendmail (or something that emulates it). That is, the mail software on your server is sending the email. This definitely won't work on a Windows box as Windows has no such equivalent. 

The critical thing is to understand what your options are for getting mail from your server to the outside world. This is nothing whatever to do with Moodle and depends on knowledge of your system's networking setup. Ask your network administrator.

My advice is always, "never mind Moodle can you send email from an email client installed on your server?". If there's problems, you usually can't. 

Average of ratings: Useful (2)
In reply to Howard Miller

Re: How do find where the email process breaks down?

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Yes. People confuse delivering mail to the local SMTP as _Moodle sending_ mail.

The documentation doen't help either:
===
SMTP stands for Simple Mail Transfer Protocol. The SMTP host is an email relay that will take the email from Moodle and send it to users. You will need to set this only if your server does not allow mail relay. Otherwise, PHP will send out the mail using its built-in mail server. All the email sent by forums and other modules will be sent through this host.
===
https://docs.moodle.org/en/Messaging_settings#Email_settings
Average of ratings: -
In reply to Howard Miller

Re: How do find where the email process breaks down?

by Gary Benner -

Thanks Howard.

Sorry if I have introduced some confusion, as I had tried to keep the conversation as non-technical as possible.

Here's a more in depth and technically complete explanation.

 

From the MoodleDocs:  https://docs.moodle.org/28/en/Messaging_settings

"SMTP stands for Simple Mail Transfer Protocol. The SMTP host is an email relay that will take the email from Moodle and send it to users. You will need to set this only if your server does not allow mail relay. Otherwise, PHP will send out the mail using its built-in mail server. All the email sent by forums and other modules will be sent through this host. "

Moodle uses an internal library called PHPMailer (another open-source project) . Moodle customises this using a subclass so it can retain the vanilla PHPMailer code and easily update the base class.

The method of sending emails is determined by what is entered in the Site Administration Email parameter smtphosts. This is set in <yourDomain>/admin/settings.php?section=messagesettingemail, along with the parameters such as security and smtp authentication username and password parameters.

What happens to email is coded in the moodlelib.php file, and from reading the code it has three main options:

 

1. if smtphosts is empty - uses the inbuilt PHP mail() command

This is configured in php.ini as Howard notes. For Unix and Linux servers the mail is delivered by a local binary such as Sendmail, Qmail etc.

The PHP reference site php.net states: http://php.net/manual/en/function.mail.php

The Windows implementation of mail() differs in many ways from the Unix implementation. First, it doesn't use a local binary for composing messages but only operates on direct sockets which means a MTA is needed listening on a network socket (which can either on the localhost or a remote machine).

So for Windows servers you need to configure php.ini to point to a SMTP server (can be localhost if there is on on your server).

2. if smtphosts is set to 'sendmail' or 'qmail' then PHPMailer will use a unix shell command to communicate directly with the relevant binaries.

3. if smtphosts is set to a valid smtp server address (an IP Address or a FQDN- fully qualified domain name - such as mail.mydomain.com), then PHPMailer talks directly to that SMTP server using it's inbuilt SMTP class.

 

So there it is, hopefully I haven't lost too many minds along the way.

 

Finally, what is the best method to use? Well that depends upon your server resourcing, and size of your site. In larger sites it is best to simplify the operations on the main server and delegate as much as you can, in which case using the external smtp server, where all email sending, retries etc, are managed by the smtp server, and not the local server. Personally I would suggest this is the best option, as there are other support functions that may be better managed through a dedicated server for this purpose.

For a small, simple site, then you may use any of the options above. That is of course until you hit a problem, and you need to delve deeper smile Like James Cameron when he went to the deepest part of the oceans and saw many "interesting things", so will you.

Have fun and enjoy the process.

Gary

 

 

 

 

Average of ratings: Useful (1)
In reply to Gary Benner

Re: How do find where the email process breaks down?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Almost always your host, ISP or whatever will have a 'mail relay' that you can use. That is, you just need to put in the hostname of that in Moodle's SMTP setting. This is usually the most likely thing to work. Just ask your network administrator for the details of your mail relay. Only if that fails do you need to start understanding how mail is configured.

I remember the first time I built an MTA from scratch myself - lots and lots of stress wink

Average of ratings: -