Scalability - mdl_files entries optimization

Scalability - mdl_files entries optimization

by Tomasz Muras -
Number of replies: 2
Picture of Core developers Picture of Plugin developers Picture of Plugins guardians Picture of Translators

Hello !

Each time a file is created in Moodle mdl_files table, there is an extra entry created there - the current "." directory. This means for example that for each Moodle file resource, there are 2 entries created in mdl_files table.

This introduces some unnecessary overhead in big Moodle sites. I've seen Moodle sites with millions of file resources. So for the site with 10 million file resources, mdl_files will have 20 million entries - 50% of them redundant ones (not bringing any new information into the database).

I've implemented a patch to change see - please see https://tracker.moodle.org/browse/MDL-65857 . Any comments (and help!) welcome!

Tomek

Average of ratings: -
In reply to Tomasz Muras

Re: Scalability - mdl_files entries optimization

by Paul Holden -
Picture of Core developers Picture of Moodle HQ Picture of Moodle Workplace team Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

That doesn't seem right - if there are multiple files in a resource, there is still only one '.' record created in the files table.

It would be more accurate to say for each context filepath there is a single '.' record, e.g. a folder resource with paths multiple levels deep, each containing multiple files will only have the number of '.' entries as there are distinct filepaths.

In reply to Paul Holden

Re: Scalability - mdl_files entries optimization

by Tomasz Muras -
Picture of Core developers Picture of Plugin developers Picture of Plugins guardians Picture of Translators

You're correct - the overhead of 1 mdl_files entry is per context (per resource created).

The scenario I described is the worst case - it's when a site has course with file resources that contain only 1 file. Unfortunately, on live installations this is the most common use case.