The mod_customcert\task\email_certificate_task is experiencing significant delays in sending emails, sometimes taking several hours and I noticed it takes up to 9 days after a student meets all requirements and passes restrictions for certificate issuance to receivethe email. As indicated in the screenshot, the task continues to run for almost four hours and dtill running. How can I optimize and speed up this process?
回复Mohamed Atia
Re: Optimizing Email Delivery Time for mod_customcert Certificate Task in Moodle
回复Mark Nelson
Re: Optimizing Email Delivery Time for mod_customcert Certificate Task in Moodle
Thank you for providing the link. Based on our usage data, I believe optimizing the email certificate task could benefit from implementing caching and batching mechanisms, especially for sites with a substantial number of users. Here are some key statistics from our site:
- Number of Certificates: 5,188+
- Number of Issues: 241,132+
- Number of Courses: 2,991+
- Number of Users: 334,217+
- Number of Enrollments: 519,938+
- Context Table Length: 270,480+
- Moodle Version: 4.2.2+ (Build: 20230929)
- Custom Certificate Version: 4.2.4 (Build: 2023042407)
It seems that the current email certificate task processes all certificates, issues, and enrolled users, and verifies user accesses during each run, causing a considerable delay before initiating email sending. In our case, with a four-hour task runtime limit, the task is terminated before any emails are dispatched.
I propose considering the following workarounds if you think they are useful:
1- Batch Processing:
-Modify the task to handle certificates in batches during each run.
-Implement a mechanism to resume processing from the last stopped point in subsequent runs.
2- Admin Configurable Options:
-Allow administrators to exclude certain older courses from certificate checks, especially those that have been closed for an extended period.
-Introduce parameters allowing administrators to specify the number of certificates processed per run, with a default processing rate (e.g., 10 to 50 certificates) that can be handled in a minute before the next run.
3- Two-Task Approach:
-Divide the task into two components:
a. One task for processing a limited number of certificates at regular intervals.
b. Another task for periodic, comprehensive checks of all certificates, perhaps on a weekly basis.
These suggestions aim to enhance the efficiency of the email certificate task, providing administrators with more control over the processing load based on their environment and requirements.
- Number of Certificates: 5,188+
- Number of Issues: 241,132+
- Number of Courses: 2,991+
- Number of Users: 334,217+
- Number of Enrollments: 519,938+
- Context Table Length: 270,480+
- Moodle Version: 4.2.2+ (Build: 20230929)
- Custom Certificate Version: 4.2.4 (Build: 2023042407)
It seems that the current email certificate task processes all certificates, issues, and enrolled users, and verifies user accesses during each run, causing a considerable delay before initiating email sending. In our case, with a four-hour task runtime limit, the task is terminated before any emails are dispatched.
I propose considering the following workarounds if you think they are useful:
1- Batch Processing:
-Modify the task to handle certificates in batches during each run.
-Implement a mechanism to resume processing from the last stopped point in subsequent runs.
2- Admin Configurable Options:
-Allow administrators to exclude certain older courses from certificate checks, especially those that have been closed for an extended period.
-Introduce parameters allowing administrators to specify the number of certificates processed per run, with a default processing rate (e.g., 10 to 50 certificates) that can be handled in a minute before the next run.
3- Two-Task Approach:
-Divide the task into two components:
a. One task for processing a limited number of certificates at regular intervals.
b. Another task for periodic, comprehensive checks of all certificates, perhaps on a weekly basis.
These suggestions aim to enhance the efficiency of the email certificate task, providing administrators with more control over the processing load based on their environment and requirements.
回复Mohamed Atia
Re: Optimizing Email Delivery Time for mod_customcert Certificate Task in Moodle
回复Mark Nelson
Re: Optimizing Email Delivery Time for mod_customcert Certificate Task in Moodle
Thank you for your response. As I am a Moodle administrator, not a Moodle developer, I appreciate the consideration of the suggestions. While I'm open to learning more about Moodle Development, I can't commit to direct development work at this point. After studying Moodle Development, I'm uncertain about my capability to perform such tasks, but if I find myself capable, I will do so. Your understanding is appreciated.
回复Mark Nelson
Re: Optimizing Email Delivery Time for mod_customcert Certificate Task in Moodle
Hi Mike,
I wanted to update you on the progress I've made in response to our recent discussion. After setting up a Moodle development environment and familiarizing myself with Moodle and the email task within your plugin, I've implemented the following changes:
Scheduled Task Configuration:
- Added a new settings header "Scheduled Task Configuration."
- Introduced three new settings:
- a. Certificates Per Run: Enter the number of certificates to process per scheduled task run (0 for all certificates).
- b. Include Certificates in Not Visible Courses: Checkbox to include certificates in courses not visible to the user. (in our production site there are1000+ certificates in hidden courses)
- c. Certificate Execution Period: Specify the period for certificate execution based on their end date (0 for all certificates).
Database Table for Tracking:
- Added a new table in the database to track the current position of the certificate to be processed.
for some reason when I updated the plugin default data was not created so I created it using the below query after upgrade, I will troubleshoot this laterINSERT INTO customcert_task_progress (taskname, last_processed, total_certificate_to_process, usermodified, timecreated, timemodified)VALUES ('email_certificate_task', 0, 0, 2, UNIX_TIMESTAMP(), UNIX_TIMESTAMP());
Future Enhancements:
- Considering an option in settings to send the certificate URL instead of the PDF (I will work on it).
- Exploring the possibility of dividing the task into two separate tasks: one for issuing certificates and one for email.
- Investigating techniques for running the certificate issuing task in parallel for improved efficiency.
You can access my changes at this fork from your plugin: https://github.com/mohamedmohamedatia/moodle-mod_customcert.git
I'm excited about these developments and would appreciate your feedback.