Good Morning Dear Moodlers,
Any suggestion to my problem? Moodle spams all accounts every time the cron job runs. Below see a screenshot of my contab -e of my www-data user.
I am running moodle 2.7 on a Ubuntu 12.04 server.
Hi David
Sorry to hear that your odyssey is not over: "Cron Command not working" https://moodle.org/mod/forum/discuss.php?d=278566. (Please put a note there saying that the follow up is in this new thread.)
Couple of things:
- to display a crontab use 'crontab -l' (el as list)
- the screen-shots of shell users are text! You simply cut and paste the screen in to the forum. For example:
- What is going wrong right now? What does "Moodle spams all accounts every time the cron job runs" means?
Sorry to hear that your odyssey is not over: "Cron Command not working" https://moodle.org/mod/forum/discuss.php?d=278566. (Please put a note there saying that the follow up is in this new thread.)
Couple of things:
- to display a crontab use 'crontab -l' (el as list)
- the screen-shots of shell users are text! You simply cut and paste the screen in to the forum. For example:
# comments trimmed
[...]
0 2 * * 0-6 /usr/bin/php /var/www/moodle/admin/cli/automated_backups.php - What is going wrong right now? What does "Moodle spams all accounts every time the cron job runs" means?
Thanks Visvanath,
What I mean by "Moodle spams all accounts every time the cron job runs" is that, every time the moodle cron job runs (ever minute) each and every single one of my users registered in my moodle site get an empty e-mail message from my moodle site. The message has no subject and no content. It comes from my moodle@noreply address.
It is going to be total chaos when my production site goes live and 10K users get 60 blank emails per hour when cron runs. I don't know what switch or option to use in the command to make this silent. This is the line I have setup in my crontab -e file:
* * * * * /usr/bin/php /var/www/moodle/admin/cli/cron.php >/dev/null 2>&1
I thought the 2>&1 would prevent it from emailing everyone... but I have not been so lucky.
Is it maybe that I have a setting turned on somewhere else in my server that is forcing this e-mail to go out?
I appreciate your help.
DM
A wild guess from a Moodle newby.
If Moodle's cron is sending emails I can't see why redirecting the standard and error output of the cron entry for moodle's cron.php would make any difference to them being sent.
The comments in /<path to moodle code>/admin/cli/cron.php include:
"This script looks through all the module directories for cron.php files and runs them."
I found all the cron.php files in my moodle with:
find /<path to moodle code> | grep cron.php
and found 4 of them.
It would be possible to figure out if one of these is sending emails by temporarily renaming them (so the main cron doesn't run them) and testing them out one by one, assuming of course that it is not a production site.
Thanks for the suggestion. I tried renaming one by one the cron.php files, but it still sent an email every time I ran cron...
Many modules have the cron function inside their lib.php files, so there are more places that cron executes than just the explicit cron.php files.
It sounds like you may have a mail stuck somewhere, some corruption most likely, which causes the sending to fail and Moodle keeps resending it. Do you have or had a forum (news?) on the front page which subscribes all users automatically? A post in such a forum would go to all your users, for example.
Your problem is not likely with the crontab file itself. The redirection there applies to cron's output not what it does.
Have you tried i run cron manually and see whether you get any useful messages? You may need to crank up the debugging level and check also the server logs.
It sounds like you may have a mail stuck somewhere, some corruption most likely, which causes the sending to fail and Moodle keeps resending it. Do you have or had a forum (news?) on the front page which subscribes all users automatically? A post in such a forum would go to all your users, for example.
Your problem is not likely with the crontab file itself. The redirection there applies to cron's output not what it does.
Have you tried i run cron manually and see whether you get any useful messages? You may need to crank up the debugging level and check also the server logs.
Hey Robert,
This is a my test environment which I created so that I could practice before building my production server. Maybe there is some sort of corruption that I hope is not there when I build my production environment from scratch. I don't think I have any forums where I subscribed anyone but I will take a look. Perhaps something like that was setup in one of the courses I migrated, and since my courses were created in Moodle 1.9 something gets corrupted in the conversion process from 1.9 to 2.7.
I may have to rebuild my test server to see if the problem is related to a corruption. Since this is the first time I build a moodle server from scratch and I have tried many things to make things work, maybe I inadvertently turned on something I wasn't supposed to.
Regards,
DM
Hi
> every time the moodle cron job runs (ever minute) each and every single one of my users registered in my moodle site get an empty e-mail message from my moodle site. The message has no subject and no content. It comes from my moodle@noreply address.
Now I got it!
Since you say every minute, it could only be the line:
To be sure, comment out that line and see whether the stream of mail stops.
In you last post you wrote:
Are you sure, "* * * * *"? Not "*/1 * * * *"?
> every time the moodle cron job runs (ever minute) each and every single one of my users registered in my moodle site get an empty e-mail message from my moodle site. The message has no subject and no content. It comes from my moodle@noreply address.
Now I got it!
Since you say every minute, it could only be the line:
*/1 * * * * /usr/bin/php /var/www/moodle/admin/cli/cron.php > /dev/null 2>&1 To be sure, comment out that line and see whether the stream of mail stops.
In you last post you wrote:
* * * * * /usr/bin/php /var/www/moodle/admin/cli/cron.php >/dev/null 2>&1 Are you sure, "* * * * *"? Not "*/1 * * * *"?
Thanks for the reply Visvanath,
I've tried both ways. Someone in a previous post told me the minimum by default is once a minute, so I guess
*/1 * * * * or * * * * * is the same thing.
DM
Consider whether you really want to run cron every minute. It may be too often. Try 10 or 15 minutes.
Robert
The original problem the OP had was that every time the cron.php runs _every_ user of his Moodle site gets an _empty_ e-mail - which is weired. He is not converging on it because he has no strategy or rather not following any one strategy people have provided, rather randomly trying things out, and worst of all trying to supress the reports cron tries to send by all means instead of reading them.
The original problem the OP had was that every time the cron.php runs _every_ user of his Moodle site gets an _empty_ e-mail - which is weired. He is not converging on it because he has no strategy or rather not following any one strategy people have provided, rather randomly trying things out, and worst of all trying to supress the reports cron tries to send by all means instead of reading them.
It seems to me that if your Moodle cron.php is sending unwanted emails, it is nothing to do with the operating system's scheduler, so you won't solve it by fiddling around with the crontab entry.
You can stop sending them by removing the crontab entry for moodle but Moodle needs to run it's cron.php to maintain the system so that is not a solution.
Are you able to disable the Moodle crontab entry and run moodle's cron.php manually to verify that each execution results in an empty email being sent?
Robert is right in saying that many of Moodle's cron functions are in library files so it would be a bit of a needle in a hay stack trying to solve it by rummaging through all of moodle's cron related files.
You could look through the logs (Site Admin>Reports>Logs) for 'message sent' events and maybe get a clue.
I wish I could help more.
Greetings Bruce,
You are correct, removing the crontab entry is not an option as cron.php needs to run. I have tried running cron.php manually and every time it sends me a blank message. I will definitely start looking in the logs. I really appreciate your input.
Any other suggestion is appreciated.
Regards,
DM
This is the output of my crontab when I manually run it via https://my.moodle.address/admin > click on the "cron has not been ran for at least 24hrs."
(my apologies for such a large entry, maybe your seasoned moodle eyes will see something I'm missing) Perhaps that part I colored in red is the problem...
Server Time: Mon, 02 Feb 2015 10:59:28 -0700
Execute scheduled task: Cleanup old sessions
... started 10:59:28. Current memory use 4.2MB.
... used 26 dbqueries
... used 0.07210898399353 seconds
Scheduled task complete: Cleanup old sessions
Execute scheduled task: Delete unconfirmed users
... started 10:59:28. Current memory use 4.7MB.
... used 1 dbqueries
... used 0.00071096420288086 seconds
Scheduled task complete: Delete unconfirmed users
Execute scheduled task: Delete incomplete users
... started 10:59:28. Current memory use 4.7MB.
... used 0 dbqueries
... used 4.4107437133789E-5 seconds
Scheduled task complete: Delete incomplete users
Execute scheduled task: Clean backup tables and logs
... started 10:59:28. Current memory use 4.7MB.
... used 2 dbqueries
... used 0.0044000148773193 seconds
Scheduled task complete: Clean backup tables and logs
Execute scheduled task: Background processing for tags
... started 10:59:28. Current memory use 4.7MB.
... used 8 dbqueries
... used 0.01850700378418 seconds
Scheduled task complete: Background processing for tags
Execute scheduled task: Cleanup contexts
... started 10:59:28. Current memory use 4.7MB.
Cleaned up context instances
... used 11 dbqueries
... used 0.017915964126587 seconds
Scheduled task complete: Cleanup contexts
Execute scheduled task: Remove expired cache entries
... started 10:59:28. Current memory use 4.8MB.
... used 1 dbqueries
... used 0.0031180381774902 seconds
Scheduled task complete: Remove expired cache entries
Execute scheduled task: Background processing for messaging
... started 10:59:28. Current memory use 4.8MB.
... used 1 dbqueries
... used 0.0032949447631836 seconds
Scheduled task complete: Background processing for messaging
Execute scheduled task: Send new user passwords
... started 10:59:29. Current memory use 4.8MB.
... used 1 dbqueries
... used 0.00075793266296387 seconds
Scheduled task complete: Send new user passwords
Execute scheduled task: Send failed login notifications
... started 10:59:29. Current memory use 4.8MB.
... used 0 dbqueries
... used 4.3153762817383E-5 seconds
Scheduled task complete: Send failed login notifications
Execute scheduled task: Create missing contexts
... started 10:59:29. Current memory use 4.8MB.
Created missing context instances
... used 11 dbqueries
... used 0.018260955810547 seconds
Scheduled task complete: Create missing contexts
Execute scheduled task: Legacy cron processing for plugins
... started 10:59:29. Current memory use 4.9MB.
Running auth crons if required...
Running enrol crons if required...
Running cron for enrol_manual...
Verifying manual enrolment expiration...
...manual enrolment updates finished.
Processing manual enrolment expiration notifications...
...notification processing finished.
Running cron for enrol_self...
Verifying self-enrolments...
...user self-enrolment updates finished.
No expired enrol_self enrolments detected
Processing self enrolment expiration notifications...
...notification processing finished.
Running cron for enrol_cohort...
Starting activity modules
Processing module function assign_cron ...
... used 5 dbqueries
... used 0.14033699035645 seconds
done.
Processing module function chat_cron ...
... used 8 dbqueries
... used 0.0090107917785645 seconds
done.
Processing module function quiz_cron ...
Looking for quiz overdue quiz attempts...
Considered 0 attempts in 0 quizzes.
Starting quiz reports
Processing cron function for quiz_statistics...
... started 10:59:29. Current memory use 8.1MB.
Cleaning up old quiz statistics cache records...done. (1 dbqueries, 0 seconds)
Finished quiz reports
... used 5 dbqueries
... used 0.13069987297058 seconds
done.
Processing module function scorm_cron ...
Updating scorm packages which require daily update
... used 5 dbqueries
... used 0.0084381103515625 seconds
done.
Processing module function workshop_cron ...
processing workshop subplugins ...
Starting workshop allocation methods
Processing cron function for workshopallocation_scheduled...
... started 10:59:29. Current memory use 8.3MB.
... no workshops awaiting scheduled allocation. done. (1 dbqueries, 0 seconds)
Finished workshop allocation methods
... used 6 dbqueries
... used 0.03252100944519 seconds
done.
Processing module function checklist_cron ...
No automatic update checklists found
... used 3 dbqueries
... used 0.0065701007843018 seconds
done.
Processing module function questionnaire_cron ...
... used 2 dbqueries
... used 0.028783082962036 seconds
done.
Processing module function journal_cron ...
... used 2 dbqueries
... used 0.0034370422363281 seconds
done.
Processing module function turnitintooltwo_cron ...
... used 1 dbqueries
... used 0.00069093704223633 seconds
done.
Finished activity modules
Starting blocks
Processing cron function for recent_activity....done.
Processing cron function for rss_client....
0 feeds refreshed (took 0.00053700000000001 seconds)
done.
Finished blocks
Starting admin reports
Finished admin reports
Starting course reports
Finished course reports
Starting gradebook plugins
Finished gradebook plugins
Starting repository plugins
Processing cron function for repository_dropbox...
... started 10:59:30. Current memory use 14.2MB.
done. (1 dbqueries, 0 seconds)
Processing cron function for repository_filesystem...
... started 10:59:30. Current memory use 14.2MB.
done. (10 dbqueries, 0.02 seconds)
Finished repository plugins
Starting admin tools
Processing cron function for tool_inactive_user_cleanup...
... started 10:59:30. Current memory use 15.9MB.
Hey, admin tool
inactive user cleanup is running
Error: lib/moodlelib.php email_to_user(): User 1 (Guest user ) email (root@localhost) is invalid! Not sending.
id
2
inactivity
29
email sent
done. (179 dbqueries, 10.1 seconds)
Finished admin tools
... used 264 dbqueries
... used 11.263548135757 seconds
Scheduled task complete: Legacy cron processing for plugins
Execute scheduled task: Background processing for gradebook
... started 10:59:40. Current memory use 16.3MB.
... used 2 dbqueries
... used 0.0016520023345947 seconds
Scheduled task complete: Background processing for gradebook
Execute scheduled task: Background processing for events
... started 10:59:40. Current memory use 16.3MB.
... used 58 dbqueries
... used 0.055353879928589 seconds
Scheduled task complete: Background processing for events
Execute scheduled task: Calculate completion data
... started 10:59:40. Current memory use 16.3MB.
... used 7 dbqueries
... used 0.043071031570435 seconds
Scheduled task complete: Calculate completion data
Execute scheduled task: Background processing for portfolio plugins
... started 10:59:40. Current memory use 16.7MB.
... used 0 dbqueries
... used 4.3153762817383E-5 seconds
Scheduled task complete: Background processing for portfolio plugins
Execute scheduled task: Background processing for plagiarism plugins
... started 10:59:40. Current memory use 16.8MB.
... used 0 dbqueries
... used 0.0049159526824951 seconds
Scheduled task complete: Background processing for plagiarism plugins
Execute scheduled task: Send calendar notifications
... started 10:59:40. Current memory use 16.8MB.
Updating calendar subscriptions:
... started 10:59:40. Current memory use 17.7MB.
Finished updating calendar subscriptions.
... used 1 dbqueries
... used 0.03209400177002 seconds
Scheduled task complete: Send calendar notifications
Execute scheduled task: Sync external blogs
... started 10:59:40. Current memory use 17.7MB.
... used 2 dbqueries
... used 0.0066781044006348 seconds
Scheduled task complete: Sync external blogs
Execute scheduled task: Background processing for question engine
... started 10:59:40. Current memory use 18.1MB.
Cleaning up old question previews...done.
Cleaning up old question statistics cache records...done.
... used 6 dbqueries
... used 0.022587060928345 seconds
Scheduled task complete: Background processing for question engine
Execute scheduled task: Site registration
... started 10:59:40. Current memory use 18.3MB.
... used 3 dbqueries
... used 0.011619091033936 seconds
Scheduled task complete: Site registration
Execute scheduled task: Check for updates
... started 10:59:40. Current memory use 18.3MB.
Outdated or missing info about available updates, forced fetching ... sending notifications ... done
... used 7 dbqueries
... used 1.2934060096741 seconds
Scheduled task complete: Check for updates
Execute scheduled task: Background processing for caches
... started 10:59:41. Current memory use 19.9MB.
Cleaning up stale session data from cache stores.
... used 0 dbqueries
... used 0.00080704689025879 seconds
Scheduled task complete: Background processing for caches
Execute scheduled task: Automated backups
... started 10:59:41. Current memory use 20MB.
Checking automated backup status...INACTIVE
... used 0 dbqueries
... used 0.26963901519775 seconds
Scheduled task complete: Automated backups
Execute scheduled task: Award badges
... started 10:59:42. Current memory use 22.8MB.
Started reviewing available badges.
Badges were issued 0 time(s).
Sending scheduled badge notifications.
... used 9 dbqueries
... used 0.017699003219604 seconds
Scheduled task complete: Award badges
Execute scheduled task: Delete stale temp files
... started 10:59:42. Current memory use 23MB.
... used 0 dbqueries
... used 0.068091154098511 seconds
Scheduled task complete: Delete stale temp files
Execute scheduled task: Cleanup files in trash
... started 10:59:42. Current memory use 23MB.
Deleting old draft files... ... started 10:59:42. Current memory use 23MB.
done.
Deleting orphaned preview files... ... started 10:59:42. Current memory use 23MB.
done.
Cleaning up files from deleted contexts... ... started 10:59:42. Current memory use 23MB.
done.
Deleting trash files... ... started 10:59:42. Current memory use 23MB.
done.
... used 295 dbqueries
... used 0.64563584327698 seconds
Scheduled task complete: Cleanup files in trash
Execute scheduled task: Background processing for statistics
... started 10:59:42. Current memory use 23MB.
...preventing stats to run, more than 4 hours since scheduled time.
... used 2 dbqueries
... used 0.033348798751831 seconds
Scheduled task complete: Background processing for statistics
Execute scheduled task: Cleanup password reset attempts
... started 10:59:42. Current memory use 23.7MB.
Cleaned up old password reset records
... used 1 dbqueries
... used 0.00049185752868652 seconds
Scheduled task complete: Cleanup password reset attempts
Execute scheduled task: Forum mailings and maintenance jobs
... started 10:59:43. Current memory use 23.8MB.
Starting digest processing...
Cleaned old digest records
Removing old forum read tracking info...
... used 6 dbqueries
... used 0.005856990814209 seconds
Scheduled task complete: Forum mailings and maintenance jobs
Execute scheduled task: Legacy log table cleanup
... started 10:59:43. Current memory use 23.8MB.
... used 0 dbqueries
... used 4.1961669921875E-5 seconds
Scheduled task complete: Legacy log table cleanup
Execute scheduled task: Log table cleanup
... started 10:59:43. Current memory use 23.8MB.
... used 0 dbqueries
... used 0.00019502639770508 seconds
Scheduled task complete: Log table cleanup
Execute scheduled task: Update question creation activity grades
... started 10:59:43. Current memory use 23.9MB.
... used 1 dbqueries
... used 0.0050909519195557 seconds
Scheduled task complete: Update question creation activity grades
Execute scheduled task: Synchronize students questions access capabilities
... started 10:59:43. Current memory use 24.2MB.
... used 1 dbqueries
... used 0.00028514862060547 seconds
Scheduled task complete: Synchronize students questions access capabilities
Cron script completed correctly
Cron completed at 10:59:43. Memory used 24.2MB.
Execution took 14.313595 seconds
Kind Thanks,
DM
Greetings Visvanath, and thank you for taking the time to reply.
What do you mean when you say "He is not converging on it?", Also, what do you mean when you say "He has no strategy?" what part of I am a noob with moodle is not clear? I am definitely following what all of you moodlers suggest. Randomly trying things out? Of course I am, I am trying things out as I get the suggestions from all of you helpful moodlers.
Also, I am not trying to suppress the reports, I am just trying to make moodle stop spamming all my moodle users. It would be nice if I could get moodle to send that report to a repository or even one e-mail address, not everyone in my environment.
Thank you for your time helping me in this matter.
Regards,
DM
Hi David
> What do you mean when you say "He is not converging on it?",
I mean this running thread and its first chapter "Cron Command not working" https://moodle.org/mod/forum/discuss.php?d=278566 and the one before "cron job help" https://moodle.org/mod/forum/discuss.php?d=276696 have now a grand score of 43 forum posts! Well, I see Ken at the helm and the ship heading somewhere. So don't give up!
> Also, what do you mean when you say "He has no strategy?" what part of I am a noob with moodle is not clear?
The cause of the problem is not expected in Moodle but in your system software: handling cron in Linux and, in recent posts, sending mails.
> I am definitely following what all of you moodlers suggest.
In this post https://moodle.org/mod/forum/discuss.php?d=276696#p1189400 I suggested a couple of things. How many of them have you covered in your answer https://moodle.org/mod/forum/discuss.php?d=276877&parent=1190421 ?
> Randomly trying things out? Of course I am, I am trying things out as I get the suggestions from all of you helpful moodlers.
Oh, I see. I am a part of a random idea generator!
> What do you mean when you say "He is not converging on it?",
I mean this running thread and its first chapter "Cron Command not working" https://moodle.org/mod/forum/discuss.php?d=278566 and the one before "cron job help" https://moodle.org/mod/forum/discuss.php?d=276696 have now a grand score of 43 forum posts! Well, I see Ken at the helm and the ship heading somewhere. So don't give up!
> Also, what do you mean when you say "He has no strategy?" what part of I am a noob with moodle is not clear?
The cause of the problem is not expected in Moodle but in your system software: handling cron in Linux and, in recent posts, sending mails.
> I am definitely following what all of you moodlers suggest.
In this post https://moodle.org/mod/forum/discuss.php?d=276696#p1189400 I suggested a couple of things. How many of them have you covered in your answer https://moodle.org/mod/forum/discuss.php?d=276877&parent=1190421 ?
> Randomly trying things out? Of course I am, I am trying things out as I get the suggestions from all of you helpful moodlers.
Oh, I see. I am a part of a random idea generator!
Thank you Robert,
I will take your advice and change it to 10 minutes or so. My apologies for being new to this program and having to bug the community with my forum posts.
Regards,
DM
Ok ...
test environment running ubuntu server in what environment? (a VM?)
on what network? (corp network or home network?)
Test server has an IP but does it have a fully qualified domain name ... even if being used right now to test?
You've taken 1.9 backups and restored them to the 2.7 test environment.
1.9 backups, even if they contained users, the users shouldn't come into the 2.7.
So those courses shouldn't contain any users except those you have assigned.
So in one of those courses that was restored from 1.9 backup, are there any users enrolled?
The error you see about root@localhost ... is accurate. There are no servers on internet that have localhost as a domain that can be reached by internet. Localhost is the local machine ... associated with the loopback address every computer on the internet has: 127.0.0.1.
By chance have you assigned **all users** in your test environment as admin level users?
If so, then all users will get notifications when the cron job runs - exception would be anyuser@localhost.
Is it possible to see the header of one of these 'spam messages'?
Then there is the other end ... the mail server that accepts mail from your test server.
That server actually might be rejecting all mail from your test server.
What does your test server mail logs show?
'spirit of sharing', Ken
Greetings Ken,
Fist of all, my apologies because I am new at moodle and some of the things I say will make you want to kill me and think I have an ID10T problem. But I am new and I don't know of any other way of learning except by trying,reading documentation and asking the experts.
That being said:
Test environment running ubuntu server 12.04 on a VM in a corporate network.
The site does not have a fully qualified domain, it is only accessible from inside our corporate network.
I have taken some 19 backups and used this 3rd party tool called "conversion thingy" to convert them to moodle 2.x. I have also created the backup, changed the extenssion from .zip to .mbz and they restore successfully (no I am not all goofy and happy that this change of file extension made it work work and I know there is corruptions and problems by just doing things like this, but that is the purpose of a test environment right?). When I backup my courses, uncheck the "user data" option, I uncheck the "course files" option, and I also select "non" under "users". So no users should be backed up or enrolled.
I only assigned 4 people as admins in my environment, other than that the other 175 are registered as regular users, which some have teacher level access to their course.
The header of one of the messages is:
Received: from svr.ad.my.domain (2002:cec0:bd0c::cec0:bd0c) by
mail.svr1.ad.my.domain (2002:cec0:bd09::cec0:bd09) with Microsoft SMTP
Server (TLS) id 5.0.13.2 via Mailbox Transport; Mon, 2 Feb 2015 10:56:24
-0700
Received: from svr.ad.my.domain (2002:cec0:bd22::cec0:bd22) by
svr.ad.my.domain (2002:cec0:bd0c::cec0:bd0c) with Microsoft SMTP
Server (TLS) id 5.0.13.2; Mon, 2 Feb 2015 10:58:11 -0700
Received: from corporatefirewall.my.dom.ain (X.X.X.X) by
svr.ad.my.dom.ain (X.X.X.X) with Microsoft SMTP Server (TLS)
id 5.0.13.2 via Frontend Transport; Mon, 2 Feb 2015 10:56:47 -0700
Received: from unknown (HELO my-relay-1.my.dom.ain) ([X.X.X.X])
by corporatefirewall.my.dom.ain with ESMTP; 02 Feb 2015 10:56:46 -0700
Received: from tes-moodleserver (tes-moodleserver.my.dom.ain [X.X.X.X)
by my-relay-1.my.dom.ain (Postfix) with ESMTP id DC49B140849
for <user@my.dom.ain>; Mon, 2 Feb 2015 10:56:45 -0700 (MST)
Date: Mon, 2 Feb 2015 10:59:30 -0700
To: Test Moodle Admin <admin@my.dom.ain>
From: <noreply@test-moodleserver>
Reply-To: Do not reply to this email <noreply@test-moodleserver>
Subject:
Message-ID: <a60eb92af9ead717bb7ac29c470b0a60@test-moodleserver>
X-Priority: 3
X-Mailer: PHPMailer Moodle 2014051203.03 (https://github.com/PHPMailer/PHPMailer/)
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Return-Path: admin@my.dom.ain
X-MS-Exchange-Organization-Network-Message-Id: 3e76c0c2-70fe-41b0-ef7e-08d20d28f142
X-MS-Exchange-Organization-AVStamp-Enterprise: 1.0
X-MS-Exchange-Organization-AuthSource: svr.ad.my.domain
X-MS-Exchange-Organization-AuthAs: Anonymous
My test server is registered with our production mail relay servers, so e-mail should be coming through with out a problem.
I don't have a test mail server, the mail function work with the built in PHP mailer and our corporate mail relay server.
Hope this helps narrow down the problem.
Thank you for taking the time.
Kind Regards,
DM
No need to apologize ... as moodle marches ever forward new issues arise and *ALL* of us could use some help from time to time.
Interesting approach to restoring 1.9's ... gonna have to try that one myself ... thanks for sharing that back.
Now to the mail
Read the EMail headers (the Received: lines) from bottom upwards. That's the path the message is taking to get to destination.
On your Moodle server the PHP Mailer is using Postfix and sending to my-relay-1.my.dom.ain and as you can see it does take a hop through more than one server. **At any one of those hops, mail could be filtered/changed.** so one might have to check what's happening to the mail on each of those.
See the: 'Received: from unknown' line? Something couldn't be found via DNS?
I see it going to the corporate firewall ... any reason it has to do that? Since all of it is internal, can't mail go directly to the mail server desired?
Those 4 users who are admin levels ... are their EMail addresses FQDN's ... ie, @atruemailserveraddy?
And those "other 175 are registered as regular users" ... is there addy FQDN's also?Check one of those 175 for their roles then check the role definitions ... what that role can do/not do.
They shouldn't be getting such notifications (that's what you are calling 'spam' right?).
There are no messages in the mailque of the Moodle server? No bounced messages for postmaster or root user on the moodle server? Did you check mail server (postfix) logs ON the Moodle server?
Have you checked the config of output messages?
http://site/message/defaultoutputs.php
'spirit of thinkering', Ken
Ok, here's a test senario ... one has to figure out IF it's moodle process or Postfix that's stripping out stuff.
On the Ubuntu server create a shell account called 'hpotter'. Then, in Moodle, manually create an 'hpotter' account and for that users EMail address, point it to the moodle server itself - from what I've read from headers that should be:
hpotter@tes-moodleserver.my.dom.ain
Postfix on that server shouldn't relay but deliver that message locally to hpotter account.
Create a minimal "test" category and course. The 'test' course has only one student enrolled ... that of hpotter.
Subscribe hpotter to the announcements forum in the 'test' course.
You, as an admin level user, should be able to post to that forum.
Run the cron job from command line ... look for indication that forum postings are being sent to hpotter. If you send immediately, check server logs ... the mail server logs.
su to hpotter's account and check the mail.
If the message is delivered in tact ... ie, the posting shows as it should, then Moodle code *IS* attempting to deliver the content of the message as it should ... ie, nothing wrong with Moodle code and the sending of the message.
The only other thing I see that *could* cause Windows systems issues is that of the 'admin' user addy being used.
admin@my.dom.ain - what of changing to moodleadmin@my.dom.ain (that account must exist in the Windows mail systems for it to be used).
'spirit of thinkering', Ken
This part ...
"changed the extenssion from .zip to .mbz and they restore successfully (no I am not all goofy and happy that this change of file extension made it work work and I know there is corruptions and problems by just doing things like this, but that is the purpose of a test environment right?"
A .mbz is a .zip by another name:
If you have a .mbz file on the Ubuntu server, one could get brief file info by:
file -b nameofmbzfiles.mbz
The restore process unzips .mbz's or .zip into moodledata/temp/backup/
Changing the extension doesn't, however, actually change the key .xml file Moodle uses to restore courses and their contents.
In a 1.9 backup, that file is called moodle.xml and it must reside a the root of the extraction.
In a 2.x backup. that file is called moodle_backup.xml ... ditto on location when extracted.
The other key .xml file is called users.xml. IF there are users in a backup.
So *if* did get users into the newer moodle their preferences/settings, etc. might be frapped up and some how, some way, a flag is set in the DB so that all of them get notifications. In affect, you've in-adavertently created your own 'spam' issue.
Since it's a test site ... remove all users who are students.
Using bulk upload, create some students via csv file. Or just manually create a couple of more hpotters ... hpotter1, hpotter2, etc.. as students.
Assign those students to some courses, run your cron job and let's see what happens!
Those notification messages shouldn't be sent to those hpotter1 users ... but if message attempts are sent to them they would bounce because they do NOT exist in your true Mail server.
Mail server logs would/should show that.
Gonna have to cut this one loose cause of the 'non-standard' way of doing things there is probably no way another person could replicate your problem. But continue to 'play' in your sandbox! But know that no one here on these forums may be able to help. At least you're learning what works and what doesn't! ;)
'spirit of sharing', Ken
Thank you Ken, I will give your suggestions a Try. I will definitely not use non-standard procedures in my production environment.
Regards,
DM
Thank you all for your time helping me with this.
DM
I would very much like to better understand how Moodle sends emails.
I've used php's mail function by configuring the sendmail_path in php.ini.
I've then left the settings in Site administration>Plugins>Message outputs>Email blank and my moodle seems to send emails OK.
How are these settings in Moodle used? I can't see any overloaded form of php's mail function where they can be used.
Does moodle use MTA's or mail clients in some way outside of php's mail functions?
I found this
https://docs.moodle.org/dev/Email_processing
but it still doesn't really clarify it for me. (It's also a little old 2.5)
I'd be grateful if anyone could shed light on this for me.
First, wonder if the moderator of this forum might consider moving your question to a thread of it's own, rather than piggy-back on Cron issue.
It's been my experience, that when one uses Linux (pick your distro) and elects to run a web server + options, what gets installed is the AMP stack but also, either sendmail (the granddaddy of them all) or postfix or mail server option of your choice (Exim, other).
Sendmail, for example, is really two parts ... the sending part is the MTA and apache will use as defined in your php.ini. It can send mail out but not configured to receive mail other than from itself (localhost). And that's all you really want Moodle to be able to do ... send the mail (not receive it like a full blown mail server).
As far as your Moodle not having issues with the sending of mail without configuring any SMTP server, sendmail/postfix itself could be configured to send all mail via an SMTP server. Hosting providers would more than likely confiigure the packages rented to customers like that.
A standalone server ... one installed via net or CD/DVD from scratch (the kind I usually do/have done) requires one to install other than the default MTA ... CentOS 5 and below pre-installed sendmail. CentOS 6/7 has changed to Postfix. It's up to the OP, however, to configure it as per 'taste'. Me ... I have some boxen receiving mail but only from restricted IP addresses/FQDN's. Would not recommend that for most Moodle server implementations as it's one more thing to configure correctly/protect and maintain. Keeping up with the "Moodle March" is enough for most folks ... including me! ;)
Realize this is not a really technical explanation, but hope it's enough to answer the question (given the information shared).
'spirit of sharing', Ken
Hi Ken,
Thanks for your reply.
I appreciate that it's getting a little off topic but thought it was somewhat relevant to the direction the discussion was heading.
I've previously posted a question about this and got no reply. I'm reluctant to do a duplicated post.
I've installed a Ubuntu LAMP and put Moodle on it.
To get email working I used MSMTP (which is just an email client), got that working with php by putting the msmtp info in apache's php.ini (sendmail_path=<MTA/Client or whatever the server is using>) then Moodle could send emails with no further configuration, presumably by calling php's mail function. The email client I used had it's own config file where I put in SMTP server address, username, password etc.
What I don't get is how Moodle can use the info in Site Administration>Plugins>Message Outputs>Email
(ie SMTP Host, Username etc)
to send emails.
My understanding is that php's mail function (http://php.net/manual/en/function.mail.php), which fairly obviously wraps whatever MTA or client is specified in sendmail_path in php.ini, does not provide arguments where these things can be specified when mail is called.
Am I wrong? Otherwise how can moodle use the mail configuration info to control how emails are sent? Does Moodle bypass php's mail and send emails some other way, maybe using environment variables?
I appreciate your help.
Thanks
Bruce
Oh, well ... looks like it's not moving anyway .... sooooooooo ....
That MSMTP (according to sourceforge) is a specialized MUA/MTA for SMTP server. Interesting. As long as it works for ya, great!
How does Moodle know ... well, it checks for STMP settings it has in it's config. If found, Moodle uses code to build the messages (whatever they are ... forum posting, etc.) and then sends to SMTP server via PHPMailer.
Next time you install a LAMP box (Ubuntu if ya like) check to see what comes with their 'web server' package. Even if sendmail or postfix not included, they are easy installs. The hard part is config.
Since I started with sendmail I have a tendency to understand what's needed on a CentOS boxen for mail. And that's what the following dribble is about .... sendmail ...
submit.cf is for localhost and sendmail.cf is for true sending and possibly receiving mail.
In the sendmail.cf one could use a 'smart' relay host rule and defining that all mail not destined for a local user account (like root or other you've established) automagically goes to the SMTP server you point it to as the 'smart' relay host. Of course, one has to config that 'smart' relay host and have it allow relay from the Moodle servers sendmail.
Postfix is similar.
Advantage in using what you did, it's very limited but I would assume very secure ... but did require change to php.ini. Advantage in using sendmail/postfix/exim ... well, you can do more with it IF so inclined - no change/addition to php.ini.
Most folks just want it to work.
Check the headers of messages you get from your server via Moodle. That should show exactly what's going on with Moodle. Or, from terminal sessions/ssh, install pine or alpine - text based MTA ... those are configurable ... can set smtp host in them and test sending messages as root user or other user to/trough the SMTP server.
Check the header of messages you get from Moodle.org forums.
Due to spam/virus, etc. ... almost all mail servers will check sending servers for things like SPF and DKIM records in DNS .... checks for the validity of mail servers for a domain or server within a domain.
'spirit of sharing', Ken
Hi Ken,
Thanks for your explanation.
Aha phpmailer! I see that gives a whole bunch of options so Moodle can control how emails are sent.
I used msmtp because I found an explanation of how to set it up with gmail and it seems to work.
I agree totally that it's preferable not to manually edit php.ini (or any config files) if it can be avoided but wouldn't the apache php.ini have to point to the right email program regardless of how moodle sends emails? I can't see any way of specifying client/MTA in Moodle's settings.
I don't think my ubuntu LAMP put anything on for sending emails during the install. I didn't check the mail server option when I installed it because I don't want it to be a mail server. I just want a light weight client that moodle can use.
I'm sure it's easy to install sendmail, postfix or whatever with apt-get so I'm just trying to figure out the best practice. My knowledge of sendmail is that it's that really annoying thing that makes linux take ages to boot! But hey if it's the best option ...
As for "Most folks just want it to work", count me in! but if I'm the guy that has to make the Moodle server work it's probably best I try to know a bit about what's going on under the hood for when it stops working.
I really appreciate you sharing your knowledge Ken.
Hakuna Matata
Bruce
First, didn't mean to sound like 'sendmail' *IS* a best practice. It appears that is somewhat decided by the distributor of the OS. I mentioned that CentOS =< 5.x used sendmail, but CentOS => 6.x now chooses Postfix ... sendmail because it was readily available and, of course, because of license and price (which is $0 - like that of Postfix/Other open sourced daemons).
IF, right after one installs Ubuntu Server with no Email server option selected upon initial installation, one might see in phpinfo a couple of things:
sendmail_path (normally set to /usr/sbin/sendmail -t -i) and others related ... like: sendmail_from (default is no value). There is one for SMTP and SMTP port. Of course it would be a good idea to check official PHP docs for what those options do - specifically.
And a trick learned with 'java' ... when there is an update to java ... new version ... to use the new version one simply changes the 'symlink' /usr/bin/java to point to the new version. I do this with 'alpine' and 'pine'.
Symlinks, BTW, can be used in Moodle data as well.
PHPMailer is a library which can be plugged into many online apps like Moodle. Programmers don't have to re-invent the wheel then, just make appropriate calls to the library.
As far as 'sendmail' causing 'slow' boot times .... I've not ever really baselined that ... do you mean start up? It can be called as it is called in apache which would then appear to be slow or it could be run as a daemon (a server service). Obviously, if sendmail running as a service/server daemon, faster ... it's already in memory of server. Re-config of sendmail is the 'scary' part - but it has tons of rules. Editing the config file isn't any harder than editing php.ini manually. And launching/restarting the sendmail server doesn't take all that long. If you are cramped for memory, check out which uses more memory ... sendmail vs postfix.
One thing about using Sendmail/Postfix/Exim/other ... it's maintained and updated by the package manager of your operating system. If there is a security issue, the OS makers will eventually push out an update. Your MSMTP doesn't have a repo ... but then again, it may not need it ... although if I were using, I'd check back with the sourceforge page for that to see if there are updates available.
Think I've seen this advice on choosing an OS for Moodle more than once ... the 'best OS for Moodle is the one with which you are most familiar' (although I'd exclude Windows Server, myself). Same is true of the 'best mail server'. Pick your 'poison' and learn it!
'spirit of sharing' ... (too much here!),
Ken
Thank you all, looks like the problem was my moodle code, some how I broke it. By upgrading 2.7 to 2.8 the cron problem got fixed.
Regards,
DM