best way to find which moodle components are the most resource consuming?

best way to find which moodle components are the most resource consuming?

by Patrick Scott -
Number of replies: 17

Hi

I'm aware of all kinds of software that can monitor server performance and find bottle necks in resources (like database, memory... etc.)
but is there something that can help me figure out which moodle components is resources heavy?

it's one thing to know which resources are not enough at what times of usage, but it's a different thing to detect which moodle component is more consuming and what I need is a moodle context point of view for this.

I've seen threads about this from 2012 but want an up to date information about this

i have moodle 3.5.1 installed

Average of ratings: -
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Joffrey Pannequin -

Hi Patrick,

I can think to a few different ways to investigate:

Hoping that helps!

Average of ratings: Useful (1)
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators

May be I don't understand your question. Chat and Quiz are treated as the most resource heavy components for many many years.

Or, is this a different approach to your pending problem readfile slow in filelib.php?

Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Re: best way to find which moodle components are the most resource consuming?

by Patrick Scott -
we install all kinds of Moodle plugins that don't come with moodle and we would like to know if any of those are resource heavy as well.
in case they are - we might stop using them, but we first need to know if they are resource heavy.
Average of ratings: -
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
It might help to give us some examples of plugins you are concerned about. I'd still like to understand what 'resources' you are talking about, too? Do you have an actual problem you are trying to fix? It's all very vague.
Average of ratings: -
In reply to Howard Miller

Re: best way to find which moodle components are the most resource consuming?

by Patrick Scott -
this is a plugin that every time we delete a lot of items, the whole site becomes slow: https://moodle.org/plugins/block_massaction
another thing is the quiz.
we uses quizzes a lot lately and while a lot of users are trying to do quizzes at the same time, again the entire site because slow - every page we try to load on moodle takes like 30 seconds or so.
I can't tell what kind of resources are running out, I'm not a system admin and currently don't have an answer to that.

the quizzes are really a problem because the grades are based on them... any idea what quizzes take so much resources ?
Average of ratings: -
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Ken Task -
Picture of Particularly helpful Moodlers

Have looked over your previous postings and at one time you were asked to provide specs of server.  You don't have access at that level and your true server admin evidently asked which/what specs.

About the only thing shared is version of Moodle.  Not enough.

No one in these forums can do 'Vulcan Mind Melds', so you are going to have to provide more info about server setup.  No sense in responding to questions as those responses would be examples and total guesses. smile

So ... typically ...

Is server standalone ... web/DB/moodledata on same server? or are services split ... dedicated DB server, dedicated server for moodledata?  IF split specs on DB server.

operating system((s) and versions of,  memory of server(s), ...

no logins/passwords, for obvious reasons.

Now if you want to look at something that will show only counts of instances of mods, one could go to: /admin/modules.php

You've already shared 'lots of quizzes' ... understandable given uptick of online learning systems ... there could be others that are used.

Those numbers however are kinda tip of the iceburg, however, when it comes to quizzes.   What does a typical quiz look like ... questions and types, how many questions on a single page, are they scheduled such that students must complete within a short time frame, include audio/video?

What's 'largest' class ... number of students?

And so on an so forth ...

'SoS', Ken


Average of ratings: Useful (1)
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
I don't want to sound like we're ganging up on you, but Ken speaks the truth. We simply can't help you with the amount of information you have given.

It won't surprise anybody that large quizzes can make a site slow. I think you're trying to turn this on its head, though. You need to be working with your server administrators. They need to be monitoring the server and telling you what's going wrong during large quizzes.

We also need to look at the configuration of your installation to recommend ways that it might be improved. But we can't do that without more information.

Regarding the massaction block - it's quite old now with no updates. Plus there isn't a version for Moodle 3.5. Worse, there are other people complaining that it's very slow in 3.5. On the face of it - bad plugin. 
Average of ratings: -
In reply to Howard Miller

Re: best way to find which moodle components are the most resource consuming?

by Patrick Scott -
this is the answer I got from the system administrator:
The webserver is on a dedicated group of servers behind a load balancer (reverse proxy) each has 4 cores an 8GB RAM.
The database is on a separate server and currently has 8GB RAM and 4 cores and a 200GB disk.
We are considering increasing the RAM to at least 32GB thou it already handles the current load easily
The OS is centos7 on the web servers and redhat 7 on the DB server.
(I also replied the other thread about the files)

hope that's enough, if there is more information that might be relevant, just say so, I'll ask him.

the massaction block we should stop using by the way, that's for sure.

something I still can't pin point: seems everybody is saying that quizzes are one of the most resource heavy components in moodle.
why are quizzes so resource heavy? what differ them from other moodle components ?
Average of ratings: -
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
The thing about quizzes is that they are the place in where students do think that need to be stored in the database.

Sudents reading a book or page resource, or downloading a file is all read-only, and it is easy for the database to execute reads in parallel (but also, probably, the student will do one thing, then spend a lot of time reading it.)

Even with forums, students are mostly reading, and only occasionally posting, and it takes a certain amount of time to write one forum post.

With a quiz, it might have many questions, and the response to each question is a thing to save in the database. Writes to the database are trickier to do in parallel. I mean it can be done, but there are locks involved. This is one part.

The other issue with a quiz, is what happens if you have a stricly timed quiz. Suddenly, not only do you have lots of difficult-to-parralelise writes, but then someone says "everyone start now" that that just ensures that everyone tries to do things at exactly the same time. Compare this to the forum where probably people write forum posts more spread out in time. The single most helpful thing you can do if you need to run a big quiz is to spread the start times out a bit. If students start their attempts spread over 5-10 minutes, rather than everyone clicking start in the same 5-second period, your server will handle many more students. (Once sutdents are into the body of the quiz, they will move between pages in a more randomly distributed way.)

It is worth noting that that the two most expensive operations are when the student starts the quiz, and when they submit it. Both those operations have to do an update for every question in the quiz, so they are particualrly write-heavy.

Having said that, although quiz does place *more* load on the server that other ways that students can interact with Moodle, it is still reasonably efficient. A lot of what I have seen people say about quiz performance recently strikes me as FUD. No-one is giving real data (for example "I ran a quiz with this many users, on this much hardward, and the (web servers/db servers) ran out of (ram/cpu).")

Anyway, the bottom line is: the only way you will discover what size quiz your server can handle is to test your server. It is perfectly possible to get JMeter to load-test Moodle quiz, but I am not sure if that is built into the load-test script generator that you find in Moodle's admin menu.
Average of ratings: Useful (4)
In reply to Tim Hunt

Re: best way to find which moodle components are the most resource consuming?

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
To follow up from Tim's comments, I have had surprisingly high numbers of people do a quiz simultaneously on a raspberry pi and have heard from other people who have the same experience. The pi is not a high power computer.
Average of ratings: Useful (2)
In reply to Marcus Green

Re: best way to find which moodle components are the most resource consuming?

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators

Hi Marcus

I didn't the connection to Raspberry Pi. Where did it come up? The OP runs "a dedicated group of servers behind a load balancer" https://moodle.org/mod/forum/discuss.php?d=398827#p1613371.

Do you mean to say, the Moodle quiz should be so lean to support N simultaneous quiz takers on a Raspberry Pi? Or the other way around, how many simultaneous quiz takers would a RPi support. This and many other interesting topics are continuously discussed in the community forum of MoodleBox https://discuss.moodlebox.net/.

Average of ratings: -
In reply to Visvanath Ratnaweera

Re: best way to find which moodle components are the most resource consuming?

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
'I didn't the connection to Raspberry Pi'
It is an example of running quiz on a very resource constrained machine. See https://moodle.org/mod/forum/discuss.php?d=395329&parent=1594200
Average of ratings: -
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
You need to load-test all the components you plan to use... although you need to define what you mean by 'resource consuming' because you're going to need to measure those resources as you test.

I'm not sure it's worth the trouble. The reverse is usually true. If you want to do huge quizzes then typically you need to build a site to handle that. Which isn't any easier, of course. 
Average of ratings: -
In reply to Patrick Scott

Re: best way to find which moodle components are the most resource consuming?

by Tomasz Muras -
Picture of Core developers Picture of Plugin developers Picture of Plugins guardians Picture of Translators

Hi

One way to discover "which moodle components are resources heavy" is to use Moodle's built-in performance info statistics. If you can find a page that always loads too slowly then you are on a good way to fix the issue, see the blog entry on how to use it -  https://enovation.ie/moodle-performance-investigation-using-performance-info/

cheers,
Tomek

Average of ratings: Useful (2)
In reply to Tomasz Muras

Re: best way to find which moodle components are the most resource consuming?

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators

This is a treasure trove! I almost said, must go to the forum documentation. But that something which nobody reads (not without reason). I don't know, perhaps a sticky post? Or, a whole sticky thread as follow-ups to this?

Average of ratings: -