Moodle performance vs. large courses

Moodle performance vs. large courses

by Callahan Harry -
Number of replies: 7

Hi,

My moodle platform contains very large courses (900 sections and 5000 modules on average). Course caching takes a while (several minutes and several thousand of database queries). On any course modification, the cache is rebuild.

I increase the values of MAX_MODINFO_CACHE_SIZE and CONTEXT_CACHE_MAX_SIZE to limit the amount of database queries:


// config.php

define('MAX_MODINFO_CACHE_SIZE', INF);

define('CONTEXT_CACHE_MAX_SIZE', INF);


But there is still costly repeated database queries on all pages, when I used section options :


$section_info->{option_name};


# ...

SELECT id,name,value FROM mdl_course_format_options WHERE courseid = '206' AND format = '...' AND sectionid = '2202'

SELECT id,name,value FROM mdl_course_format_options WHERE courseid = '206' AND format = '...' AND sectionid = '2207'

SELECT id,name,value FROM mdl_course_format_options WHERE courseid = '206' AND format = '...' AND sectionid = '2212'

SELECT id,name,value FROM mdl_course_format_options WHERE courseid = '206' AND format = '...' AND sectionid = '2216'

# ...


or when I want to get user-depended data :


get_fast_modinfo($course, $user);


How to improve the cache performance and how to limit the amount of repeated database queries?  


My configuration :

- Moodle: 2.9 (Build: 20150511)

- Database: MariaDB Galera Cluster

- Cache stores: Memcached server for application and session caches



Average of ratings: -
In reply to Callahan Harry

Re: Moodle performance vs. large courses

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

"900 sections and 5000 modules on average"

That's insane and/or you're on your own. My suspicion is that nobody has tested Moodle with courses that big. 

Why would you do that?

Average of ratings: Useful (3)
In reply to Howard Miller

Re: Moodle performance vs. large courses

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

That really would be the scroll of death

Average of ratings: Useful (3)
In reply to Marcus Green

Re: Moodle performance vs. large courses

by Derek Chirnside -

Don't listen to these guys Harry, they are just a bit grumpy before their morning coffee.  They are imagining being a student with a 9,000 section course.

You are basically doomed.  Moodle pays attention to all the sections in a course in an update.  The more sections, the longer.  Most people will have less than 30 sections.  I try to have less than 10.

I guess there must be a reason for 9,000 section.  Someone did it for you and you are now stuck with it?  Someone has designed a competency test with 9,000 items in it?  Can you tell us more?

-Derek

Average of ratings: -
In reply to Derek Chirnside

Re: Moodle performance vs. large courses

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

The thing is, Moodle really should not be rebuilding the cache very often.

Are you sure the the cache store you are using is working reliably?

It seems that, at the OU, the biggest course we have is 727 sections and 5385 activities. (That is significantly huger than I would have guessed. I blame mod_subpage.)

When I just went to look at it, the first page load was quite slow:

12.604354 secsRAM: 63.4MBRAM peak: 64.9MBIncluded 409 filesget_string calls: 1102 DB reads/writes: 10847/2
But then the second load from the cache was not too bad.
1.960997 secsRAM: 61.4MBRAM peak: 62.3MBIncluded 383 filesget_string calls: 379 DB reads/writes: 968/4

(Mind you, 968 DB reads is not good. Someone should look at that.)

However, as Derek says, something like this is highly atypical. Also, or server are pretty powerful to cope so well with abuse like that.

Average of ratings: -
In reply to Tim Hunt

Re: Moodle performance vs. large courses

by Derek Chirnside -

Now I know why I sometimes read unusual posts like this.  I learn things along the way.

Re mod_subpage

Google has taken me to this: https://moodle.org/mod/forum/discuss.php?d=254737

Is this likely to ever get onto the plugins database and get updated to 3.0?

Is there a picture of what actually it does?  How it works?  Anywhere?

The real question I'm asking is "Is it worth us looking at this in my perpetual quest to find a solution to the scroll of death that can be used by regular ordinary tutors and is not clumsy, rocket science, time consuming or ugly?"

-Derek



Average of ratings: -
In reply to Derek Chirnside

Re: Moodle performance vs. large courses

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Don't use subpage. We are in the process of getting rid of it. When sam marshall made it back around Moodle 2.1 time, it require various core Moodle hacks, which we subsequently tried to get into Moodle core. However, Moodle core eventually want in a different direction with its improved support for what course formats can do.

So, our replacement for mod_subpage is now a custom course format (with an associated module). I believe the new version will not require any core changes. However, I don't know if we will every share all that code. Bits of the course format are rather idiosyncratic (the work great for OU courses, but no-one else would want some of the functionality).

Hopefully, at some point, sam can write a blog about how he built the new version, and someone can rebuild something more generic.

Average of ratings: -