Hardware and performance

Hi

You have identified the difficulty of the problem:
> it's not very easy to emulate the users activity and test how the server can handle it in real situation.

But you expect the solution in a wrong place, I'm afraid:
> I think the main problem is how to configurate it properly.

Look at it this way. You said the server failed at 600 concurrent quiz users. Let's take 500 as a safe number. 500 concurrent quiz users are a huge load. You server is definitely not malconfigured. Then I would be hugely surprised if you double that number through fine tuning. Best case 10-20%. If it is more, the server was malconfigured.

> We have a VMWare and we have enough resource to give our VMs.

How much were they (during the 600 user session)? That was a single VM, right?

> Idea:
> Two Virtual Machines, one for web/moodle and another one for DB.

OK, that is a big change (not a mere configuration change). You can expect a big improvement.

> OS: Centos 7
> ..
> Web: Apache 2.4.41
> ..
> For high-traffic web it is recommended to use Apache (MPM event) + PHP-FPM

There is a raging debate whether Nginx with FastCGI perform better. That would be a big change which you need to experiment separately. Always the same question: How do you measure the improvement.

> I have run some tests with Apache JMeter, but I haven't done well yet.

I too think only JMeter tests on Moodle itself give comparable results. Recent Moodle releases come bundled with a whole package of tools for JMeter. They are not well documented though. I made a start here: Is moodle-jmeter-script-generator still maintained? then stalled. You are welcome to extend the document with your findings.

> Can someone give some examples (and hardware) how they have configured their settings for:
> Apache MPM event:
> ..

Since you already have decent server, please submit its values as an example. There are many visitors here looking for 500 concurrent quizz situations.

> Output, when users were taking a quiz:
> [root@moodle ~]# ps -ylC httpd | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Process Size (MB): "x/((y-1)*1024)}'
> Apache Memory Usage (MB): 4897.9
> Average Process Size (MB): 349.85

I think the process size is on the high side. Anyway, there is no single number to measure how a server is performing. Have a look as Munin output for example to see how many parameters are involved.

All that said, apart from RAM and fast database response, efficient cache systems improve Moodle performance best.