"400 user download at the same time, which is certaintly never the case
The problem occurs only with 20% of students, the server rejects them."
How are you able to determine (20%) X students attempt to play an mp3 the server rejects?
'Server rejects' meaning the operating system or web service rejects, or Moodle reports an error or times out?
In the mysqltuner output did it show any info about these:
Highest usage of available connections
Aborted connections
If you adjusted settings just what did you adjust?
"user / memory capping" describe what you mean here.
Are we thinking that Moodle restricts a user?
Or are we asking about where you host ... which suggest a memory cap.
With the quiz ... are you displaying all 50? questions with one click? Not sure if Moodle starts to push the .mp3 to clients or not but that could be a factor.
'spirit of sharing', Ken