Files missing.

Files missing.

by Walter DeMoss -
Number of replies: 9


On December 4 I upgraded from 3.9+ to 3.10+ on moodle.msgcnx.com.  right after that files started randomly dissappearing.  For a given assignment 10 students may have submitted.  5 of them have missing files. The submission page looks normal.  BUT when you click on the file you get an error. The error message is below. I have searched for hours in the Moodle database in the Mdl-Files.  The submission is properly received and hashed.  HOWEVER when you follow the content hash to the path in moodledata/filedir there is nothing there.  The other student's whose files work look the same in the database and their files are in the filedir.  

We are running everything selfhosted.  Ubuntu Server 20.04 with Kubuntu Desktop. MariaDB 10.5.8  all updated.

It just seems that either Moodle never puts the file where it belongs or the database blinks. 

Thanks in advance for any and all help.

ERROR MESSAGE >>>>>>>>>>>>>>>>>>

This site can’t be reachedThe webpage at https://moodle.msgcnx.com/pluginfile.php/2493/mod_folder/content/0/Module%204%20Journal%20Topics.pdf might be temporarily down or it may have moved permanently to a new web address.

ERR_INVALID_RESPONSE


Attachment Selection_166.png
Average of ratings: -
In reply to Walter DeMoss

Re: Files missing.

by Randy Thornton -
Picture of Documentation writers
What I find curious here is that you say these were Assignment submissions, but yet the URL in the error message is pointing to a file in a mod_folder Folder activity. The pluginfile process checks that the activity type and its component in the file table match, otherwise it will toss a file not found error. A file attached to an assignment submission will have a assignsubmission_file in its url and files table entry as its component type.

In the context table, is id 2493 pointing to the right place?
Average of ratings: Useful (2)
In reply to Randy Thornton

Re: Files missing.

by Walter DeMoss -
Excellent observation. I have noticed the same. All student submissions were regular assignments. Some of the folders with resources that teachers posted were also corrupted. Always the same general error flagging the pluginfile.php.

I have removed the broken assignments.  Next time we have an error I will look for the context id as you asked.

One other detail is this. About two months prior to the error I installed an access restriction plugin(https://moodle.org/plugins/pluginversions.php?plugin=availability_password). I upgraded manually as I always have. I forgot the plugin and that flagged an error. I copied it back into the right directory. Everyting seemed fine. After the error I removed the plugin. The error remains.

Yesterday I spend hours cleaning up my apache directories. 4 classes submitted test assignments that all worked. I am tracking them to see if they vanish.
Attachment Selection_133.png
Average of ratings: -
In reply to Walter DeMoss

Re: Files missing.

by Randy Thornton -
Picture of Documentation writers
Walter,

I think you are right when you say "corrupted." Clearly, something is very wrong here. From what you say, it looks like the context ids and the file table are not matching anymore, as they should.

So, if you add a new file right now, what should happen is that the context id and the file entry with that context id should match. This may work okay for new entries, since you are watching. Does it increment the numbers properly? is something to look for. It could be the numbers are okay but the component type is wrong, as well, since as you say, both assignment submission and file folder has some corruptions. I am wondering if this may have been an issue only with some past data already in the table, that happened during the upgrade, but now is working fine for new file additions.

I'd be surprised if that plugin was the culprit. Availability restrictions are pretty innocuous; they don't have anything to do with files that I know of.

Did you do anything to the database during the upgrade, like update the SQL version or change collations or anything? The db may need some optimizing or repair. But it may be just that some of the context id numbers are somehow now wrong.
Average of ratings: Useful (1)
In reply to Randy Thornton

Re: Files missing.

by Walter DeMoss -
Randy,

Thanks for your thorough answer. I have been watching carefully. Since Jan 8 we have not seen any bad files. I did nothing to the database during or after the upgrade. We had moved from mysql to mariadb in September via the repo. We upgraded in November from 18.04.x to 20.04.1.

After locking down the server it does seem that we are ok. May have been malicious? We are still waiting. This Friday I will bump the version again and see what happens. Thanks for clearing up the plugin for me. I will look into optimizing the database and checking on collation versus what config.php says.

I am writing a script now that looks for new files in filedir and copies them into a backup folder and keeps them for a week. That way corrupt files can be retrieved base on mtime. Moodle tracked the upload times well and I could line them up perfectly with files.

Thanks again,
Walt
Average of ratings: -
In reply to Walter DeMoss

Re: Files missing.

by Walter DeMoss -
This is incredibly embarrassing. A tribute to Moodle's reliability. The whole thing was pilot error. Myself being the pilot. When I upgraded from 18.04 I cloned the old server, upgraded it and when it worked great I used it. BUT somehow my old /home was still mounting because of some random script. That caused some files(only God knows how it chose some) to be written to the old /moodledata filedir and some to the new one. Verified by finding the missing files by their timestamp. The old drive has now been more PERMANENTLY removed. No more mystery. Many thanks to those who responded and helped. I hope to return the same favor to the Moodle community.
Average of ratings: -
In reply to Walter DeMoss

Re: Files missing.

by Randy Thornton -
Picture of Documentation writers
Walter,

Well, the good news is wasn't an issue in the database done by the upgrade smile Glad the mystery is resolved.
Average of ratings: Useful (1)