Ad hoc tasks convert_submission fail

Ad hoc tasks convert_submission fail

由Steve Brisco發表於
Number of replies: 4
I am system admin who has been handed the role of managing the system side of managing Moodle.

The OS is Ubuntu 20.04 LTS and is running PHP 8.0.  I have taken our Moodle server, which was running Moodle 4.1.3 and updated it to 4.1.4+ and then when all seemed well, upgraded it to 4.2.1.

After a seeming successful upgrade to Moodle 4.2.1, I started perusing the different Admin console screens and noted in the Ad hoc tasks that the convert_submission component had over 2000 failed tasks with the earliest dating back to June 24, 2023.  
Looking at the task logs, it seemed they all had the same error: 

Adhoc task failed: assignfeedback_editpdf\task\convert_submission,Error generating image with ghostscript, debugging info: <pre>Command:
'/usr/bin/gs' -q -sDEVICE=png16m -dSAFER -dBATCH -dNOPAUSE -r'100' -dFirstPage='1' -dLastPage='1' -dDOINTERPOLATE -dGraphicsAlphaBits=4 -dTextAlphaBits=4 -sOutputFile='/tmp/requestdir/Wkkt/64b6c5c5ea33c/64b6c5d262bbe/image_page0.png' '/tmp/requestdir/Wkkt/64b6c5c5ea33c/64b6c5d262bbe/error.pdf'
I installed the package ghostscript and configured the path to ghostscript in the Moodle System Paths configuration page (/usr/bin/gs) and also the path to the PHP CLI.

I then manually re-ran some of the failed tasks and after confirming that they completed successfully, proceeded rerun all the failed tasks.  As of now, all the failed tasks now ran successfully and cleared out except for two.  Those are failing with the same error and I am unable to find anything in the forums to give me an idea what needs to be fixed.   Here is the log from the failed task:

Execute adhoc task: assignfeedback_editpdf\task\convert_submission
Adhoc task id: 29422
Adhoc task custom data: {"submissionid":"18913","submissionattempt":"0"}
... started 16:57:57. Current memory use 22.3 MB.
Debugging increased temporarily due to faildelay of 86400
Converting submission for user id 1274
... used 6 dbqueries
... used 0.0047640800476074 seconds
Adhoc task failed: assignfeedback_editpdf\task\convert_submission,You don't have permission to access this page.
Backtrace:
* line 336 of /mod/assign/feedback/editpdf/classes/document_services.php: call to assignfeedback_editpdf\document_services::get_combined_document_for_attempt()
* line 84 of /mod/assign/feedback/editpdf/classes/task/convert_submission.php: call to assignfeedback_editpdf\document_services::get_combined_pdf_for_attempt()
* line 508 of /lib/classes/cron.php: call to assignfeedback_editpdf\task\convert_submission->execute()
* line 348 of /lib/classes/cron.php: call to core\cron::run_inner_adhoc_task()
* line 368 of /lib/classes/cron.php: call to core\cron::run_adhoc_task()
* line 154 of /admin/cli/adhoc_task.php: call to core\cron::run_failed_adhoc_tasks()

Any insights would be helpful.

Thank you,

Steve




評比平均分數： -
In reply to Steve Brisco

Re: Ad hoc tasks convert_submission fail

由Visvanath Ratnaweera發表於
Particularly helpful Moodlers的相片 Translators的相片
Hi

The adhoc tasks are difficult to debug since they happen in the background, detached from the calling PHP thread. But as the task assignfeedback_editpdf\task\convert_submission successfully ran, check whether moodledata/filedir/ and moodledata/temp/ have grown in size. I'm referring to MDL-71909.
評比平均分數： -
In reply to Visvanath Ratnaweera

Re: Ad hoc tasks convert_submission fail

由Steve Brisco發表於
Hi Visvanath,

Sorry for the delayed response. The moodledata/temp/ is very small at 196K and does not appear to be growing. /moodledata/filedir is 24G and that does not appear to be growing. Unfortunately since I have inherited this system, I do not have any historical information except overall filesystem utilization. These two ad hoc tasks are still failing but everything else in the task logs appears to be running successfully.

As an aside: In looking at the number of courses being backed up and accounting for 2 backups for each course, there are 354 created courses so I am not sure that 24G for the course data is too big.

Based on the information in the log, can you confirm that user id in the error log is that same as the column named id which exists in the database table mdl_user? Using this query, I matched user id 1274 to id 1274 in the mdl_user table using this query:

mysql> select id,username from mdl_user where id = 1274;
+------+----------+
| id | username |
+------+----------+
| 1274 | muire |
+------+----------+
1 row in set (0.00 sec)

I am not sure what my next step would be though. Is it possible for the student or instructor to remove the student's submission and then resubmit it?

Gratefully,

Steve
評比平均分數： -
In reply to Steve Brisco

Re: Ad hoc tasks convert_submission fail

由Visvanath Ratnaweera發表於
Particularly helpful Moodlers的相片 Translators的相片
Hi

> The moodledata/temp/ is very small at 196K and does not appear to be growing.

That is a good sign.

> /moodledata/filedir is 24G and that does not appear to be growing.

Whether all the 24 GB are genuine in the sense, what the users uploaded, no outsider can gauge. One need a deeper analysys of the Moodle file repository.

> Unfortunately since I have inherited this system, I do not have any historical information except overall filesystem utilization.

Although monitoring is generally good, we are talking about the file system utilization. Anything odd? Do they correlate with user actions?

> These two ad hoc tasks are still failing but everything else in the task logs appears to be running successfully.

I would investigate that further.

> As an aside: In looking at the number of courses being backed up and accounting for 2 backups for each course, there are 354 created courses so I am not sure that 24G for the course data is too big.

Check the automated backup settings whether you've instructed to keep two copies. Whether 24 GB too big, see above. Still in absolute number it is not big. I am dealing with Moodle instanced where filedir/ is hundreds of GB and Terra bytes!

> Based on the information in the log, can you confirm that user id in the error log is that same as the column named id which exists in the database table mdl_user? Using this query, I matched user id 1274 to id 1274 in the mdl_user table using this query:

mysql> select id,username from mdl_user where id = 1274;
+------+----------+
| id | username |
+------+----------+
| 1274 | muire |
+------+----------+
1 row in set (0.00 sec)

Almost every table in the Moodle database has an 'id' field. But in the context you mention, I think the id is the one from the mdl_user table.

> I am not sure what my next step would be though. Is it possible for the student or instructor to remove the student's submission and then resubmit it?

Either follow up the failing ad-hoc task or from the front-end delete the offending files. The latter is easier, if it is feasible.

P.S. If you have automated course backups activated and no external target set, that is a bad idea. The backups land in the sea of files under filedir/. Teachers take course backups and don't clean the originals which gets backed up again, etc. Pretty tedious to clean even if you change the target to an external directory. Those are the downsides of the Moodle's repository concept.
評比平均分數： -
In reply to Visvanath Ratnaweera

Re: Ad hoc tasks convert_submission fail

由Steve Brisco發表於
Visvanath,

Thank you for taking the time to assist.

>> Check the automated backup settings whether you've instructed to keep two copies. Whether 24 GB too big, see above. Still in absolute number it is not big. I am dealing with 
Moodle instanced where filedir/ is hundreds of GB and Terra bytes!

I can confirm that the course backups are being placed in a separate location and that the default of 2 backups are being retained.

>>Either follow up the failing ad-hoc task or from the front-end delete the offending files. The latter is easier, if it is feasible.

I will pursue understanding the ad-hoc task queue and if that does not reveal any information, try to work with the student / course instructor to clear the offending submission.

Thank you again,

Steve
評比平均分數： -