Okay, I just got through troubleshooting issues on our uploading files for doing course restores. We tried using mod_security on our site and it was killing file uploads halfway through, which cause partial uploads in our <MOODLE_DOC>/filedir structure. I need someone to explain how the file picker uploads/stores files. I think I understand some of it. A hash of the file is created, for example 3d15123456789. This seems to upload the file in the following location on the OS file system:
I think I get that, BUT as I mentioned I had this Moodle behind mod_security for some time and has 900MB of 1.6GB uploaded and stored as the file listed above. After removing mod_security I attempted toupload the same file as before, andinstead of mod_security stopping it, Moodle eventually barked about incorrect pool file content. I had plenty of disk space, max uploads was plenty high, and web server has write permissions, but this pool error kept coming up. My only recourse was to delete the 900MB file listed above and reupload, which was successful. I'm owrried about similar issues cropping up once we go live. Can someone walk me through how files are stored and why delteing was the only thing that would fix my issue? Is there another way I should have done that??
Moodle files are stored based on the SHA1 hash of their contents.
What that means is that the strange key you see is uniquely related to the contents of the file. It is impossible that a 900Mb file and a 1.6Gb file will have the same hash value, so they will end up with different names and different storage locations in Moodle. The error means that the contents of the file stored in a location does not match the name that it was given.
Theoretically, you could use some software to calculate the SHA1 hash of the file, rename it to that, save it in the correct location and then update the Moodle database 'mdl_files' table to point to the right place. In practice, what you did was probably the right way to solve it.
The story behind the new file system is twofold.
On the one hand, and as you already know, Moodle calculates the SHA1 hash of the file content and uses this hash to rename the file and to create the two-level directory path (with the first 4 characters of the hash) where it will be saved.
For example, if the hash of a file is "96d142ae...", then the path "96/d1" will be created under moodledata/filedir, like so: moodledata/filedir/96/d1.
This hash is saved in the contenthash field of the mdl_files table.
On the other hand, Moodle creates another SHA1 hash based on the pathname (or context) in which the file is being used (inside Moodle). Specifically, this hash is created based in the following info: contextid, component, filearea, itemid, filepath and filename.extension.
For example, if a file named bgd1l1.swf is located under contextid=24, component=mod_scorm, filearea=content, itemid=0, and filepath="/data/swf/", then Moodle will calculate the SHA1 hash for the string "/24/mod_scorm/content/0/data/swf/bgd1l1.swf".
This hash is saved in the pathnamehash field of the mdl_files table.
My guess is that the incorrect pool file content error you had was because eventhough the contenthash value of both, the 900 MB and the 1.6 GB files was different, they actually had the same pathnamehash value, something that is not allowed because that is a uniquely indexed field. And so, that is why you first had to delete the 900 MB file.
Hope this helps.