Moodle 2.6 running very slow

Moodle 2.6 running very slow

by Michael K -
Number of replies: 8

Hello,

We have recently upgraded from Moodle 2.1 to Moodle 2.6 and we are finding that some things are very slow.

The worst pages are: 

course/view.php which is taking about 10 times as long to load as it used to (we do have a large number of resources/activities in the course but it is very fast on 2.1).

mod/page/view.php is also going very slow, as well as quiz/attemptquiz.php.

 

We are using a PostgreSQL database and we have reindexed the database. We have tested some large queries in the database and found that the execution time is very similar to when we run these queries on the old database. 

We have also installed OPcache.

Is there any way with Moodle that we can determine which database queries are taking a long time?

We have checked everything in Performance Overview, etc.

Any help is greatly appreciated! Thank you!

 

Average of ratings: -
In reply to Michael K

Re: Moodle 2.6 running very slow

by Michael K -

I edited lib/dml/pgsql_native_database.php and echoed out every query in get_records_sql(), get_recordset_sql(), insert_record_raw() and update_record_raw(). I then ran every query in psql shell and none of them took a long time. I noticed that mdl_cache_text is getting updated a lot. Is this correct for the highest performance?

Average of ratings: Useful (1)
In reply to Michael K

Re: Moodle 2.6 running very slow

by Michael K -

Well after using microtime on each function, I eventually found that the cause of the problem was that Moodle is loading 50,000 "manual enrolment" links in the settings block. I had to comment out enrol_add_course_navigation around like 3665 in lib/navigationlib.php.

Although these are manual enrolment links should not be for students, the system was running very slow for students, until I removed the enrolment from course navigation. 

Average of ratings: Useful (1)
In reply to Michael K

Re: Moodle 2.6 running very slow

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi Michael,

Search for the langstringcache option in the admin menu, and disable it.

The langstringcache has been removed in 2.7 and most large production systems have it disabled for performance reasons.

Best wishes,

Andrew

Average of ratings: -
In reply to Andrew Lyons

Re: Moodle 2.6 running very slow

by Michael Aherne -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Andrew

That's very interesting - I hadn't come across this before. Is this because the language strings will generally be cached by opcache, or is there more to it than that? Can you point us at some documentation about it?

Cheers

Michael

Average of ratings: -
In reply to Michael Aherne

Re: Moodle 2.6 running very slow

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi Michael,

The issue which removed the previous caching was MDL-43524 and technically I was thinking about global text caching ($CFG->textcache) rather than langstringcache - my apologies. You can see the commit which removed that caching in http://github.com/moodle/moodle/commit/62c8032509a0496d17d8fb4233077e2c3559215b (note the MDL in the commit message is incorrect).

There's also a very brief discussion on it at https://moodle.org/mod/forum/discuss.php?d=251831#p1091891 (note that the OU are one of the largest Moodle installations), and Petr has outlined some of the major issues with the old style caching at https://tracker.moodle.org/browse/MDL-43524?focusedCommentId=263251&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-263251. You'll also find that most (and probably all) of the major installations have it disabled.

The language strings are currently cached by MUC and should be reasonably fast, but you should look into something like memcached as Howard suggested. Take a look at your MUC config.

Best wishes,

Andrew

Average of ratings: Useful (2)
In reply to Michael K

Re: Moodle 2.6 running very slow

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Also, note that you can modify your postgresql configuration to write queries to it's own log - see the log_min_duration_statement setting. There's also a config setting to save logs to a Moodle database ($CFG->logsql = true;) but see the information in config-dist.php for warnings on use of this one (e.g. filling up your disk).

Andrew

Average of ratings: -
In reply to Michael K

Re: Moodle 2.6 running very slow

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Where is your 'moodledata' files area? Is it on something shared (e.g. NFS or SAN) or are you on a VM with poor disk IO performance (most VMs).  Additional caching was added in 2.4 and this is VERY sensitive to disk performance. The answer being to set up a dedicated cache (usually memcached).

Average of ratings: -
In reply to Michael K

Re: Moodle 2.6 running very slow

by Michael K -

Hi all, thank you for all of the suggestions.

I am quite happy with the speed of the system now.

I made the following changes:

1. In lib/navigationlib.php I removed enrol_add_course_navigation($coursenode, $course); (around line 3665 in load_course_settings). This improved the average page creation time from 7 seconds to 1 second.

2. I disabled the Activity names auto-linking filter. Now after clicking "Attempt Quiz" the quiz is made in about 5-6 seconds instead of 25+ seconds.

Average of ratings: Useful (1)