General developer forum

storedfilecannotcreatefile

 
Picture of Tom Smith
storedfilecannotcreatefile
 

Hi all

I have an unusual error when marking assignments. I have posted in the main user forums but received no joy; I think this is more of a developer-level question.

When marking 'assignments' using the graphical interface (the one that puts PDFs on the left and marking tools on the right), I am now getting a 'storedfilecannotcreateacess' error. This seems to be the case in particular for assignments that do not have an upload element (i.e. where marking 'offline' work.) We still want to use the graphical interface as we are using marking rubrics etc. This always worked fine, it is only this week that this error has been seen.

We're using moodle 3.4 on an Ubuntu 16.04 server, Apache2/MySQL etc. I've enabled debugging and here's the output (via graphical window to user):

Error code: storedfilecannotcreatefile
* line 466 of /lib/filestorage/file_system_filedir.php: file_exception thrown
* line 1803 of /lib/filestorage/file_storage.php: call to file_system_filedir->add_file_from_string()
* line 1435 of /lib/filestorage/file_storage.php: call to file_storage->add_string_to_pool()
* line 331 of /mod/assign/feedback/editpdf/classes/combined_document.php: call to file_storage->create_file_from_string()
* line 214 of /mod/assign/feedback/editpdf/classes/combined_document.php: call to assignfeedback_editpdf\combined_document->store_empty_document()
* line 310 of /mod/assign/feedback/editpdf/classes/document_services.php: call to assignfeedback_editpdf\combined_document->combine_files()
* line 349 of /mod/assign/feedback/editpdf/classes/document_services.php: call to assignfeedback_editpdf\document_services::get_combined_pdf_for_attempt()
* line 193 of /mod/assign/feedback/editpdf/locallib.php: call to assignfeedback_editpdf\document_services::page_number_for_attempt()
* line 7940 of /mod/assign/locallib.php: call to assign_feedback_editpdf->is_feedback_modified()
* line 8108 of /mod/assign/locallib.php: call to assign->apply_grade_to_user()
* line 1559 of /mod/assign/externallib.php: call to assign->save_grade()
* line 228 of /lib/externallib.php: call to mod_assign_external::submit_grading_form()
* line 59 of /lib/ajax/service.php: call to external_api::call_external_function()
Simialr lines are appearing in /var/log/apache2/error.log.


This might suggest user permissions or lack of space, but this seems unlikely:

/srv/moodledata is the data dir with read and write permissions for group www-data [/srv/moodledata/filedir is a link to /srv/mnt/moodledata which is remote; has always worked fine in current setup]

/var/www/moodle is the web root with read permissions for www-data

There is plenty of space left on this server:

 user #$ df -h
Filesystem                              Size  Used Avail Use% Mounted on
udev                                    2.0G     0  2.0G   0% /dev
tmpfs                                   396M   41M  355M  11% /run
/dev/mapper/<serveridentifier>--PP--vg-root   28G   15G   12G  56% /
tmpfs                                   2.0G     0  2.0G   0% /dev/shm
tmpfs                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                   2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/sda1                               472M   58M  390M  13% /boot
<internal 10.... IP>:/MoodleData                 14T  3.6T   11T  26% /srv/mnt/moodledata
tmpfs                                   396M     0  396M   0% /run/user/1001


Today, just to be sure, I ran a chgrp -R www-data and chmod g+w on /srv/moodledata. As a last-ditch attempt, I even tried re-starting the webserver but still get the same error. Would really appreciate any help that could be offered.

 
Average of ratings: -
Renaat
Re: storedfilecannotcreatefile
Core developersParticularly helpful MoodlersPlugin developers

What happens when you add $CFG->preventfilelocking = true; to your config.php file?

 
Average of ratings: -
Picture of Tom Smith
Re: storedfilecannotcreatefile
 

What happens is that the error has now gone away and all seems to work perfectly. Thanks!

Would you mind answering a couple of follow-up questions?

  1. Is this 'file locking' the same sort of thing as the traditional filesystem file-locking on windows (and usually not) unix systems?
  2. Why is it that this error only appeared a week or so ago? What is likely to have changed in my setup to result in this error?
  3. Given that I was marking an assignment, surely there should be no 'file storing' going on, it should just be a database write; so why is Moodle complaining about file creation/storage of some sort?
  4. Does adding this setting to $CFG likely have any negative consequence?


Thank you very much

 
Average of ratings: -
Renaat
Re: storedfilecannotcreatefile
Core developersParticularly helpful MoodlersPlugin developers

The error message is gone but all is not working perfectly.

There is a reason why this error started popping up : another program started locking files on your network file system. An antivirus program - a hardware error - another Moodle accessing the same network file system - a stalled cron task - an external backup program -  ... Find out what is locking your files and refresh your Network File System documentation readings.

 
Average of ratings: -
Picture of Tom Smith
Re: storedfilecannotcreatefile
 

I see, thank you for the info. I will meet with our network engineers to get them to investigate what's going on in the background. It might be a backup program gone wrong or something as no-one has intentionally changed anything on the network file system.

Thanks again.

 
Average of ratings: -
Picture of Matteo Scaramuccia
Re: storedfilecannotcreatefile
Core developersParticularly helpful MoodlersPlugin developers

Hi Tom,
$CFG->preventfilelocking should be set to false i.e. you should get rid of it from your config.php file; otherwise, please change the file system used to host the Moodle storage pool since proper file locking support is more than an optional requirement, for several reasons when managing big workloads.

That being said, double check that every Moodle task activated by cron runs under the same user running the web server hosting your Moodle istance i.e. www-data. Usually a change in the permissions set is due to the cron process touch-ing that file - which is common in Moodle tasks - under a different user.

HTH,
Matteo

 
Average of ratings: -
Picture of Tom Smith
Re: storedfilecannotcreatefile
 

My thanks to Matteo and Renaat who responded to this post. We have not managed to work out precisely which processes had caused the file locking, but we have at least removed any file locks and set moodle back to using file locking. Our network admins are monitoring for the next few days to try to work out what, if anything, might be trying to do file locking; it was the advice on this forum that pointed us in the right direction.

 
Average of ratings: -