I can assure you that your problem is *not* hardware--your CPU and memory are great. It is somewhere in software settings and tuning. Look below and see the results of a very similar server installation which achieved 60 simultaneous users doing very heavy audio quizzes. Sorry I can't help you with the tuning, we had a server specialist set up the LAMP+PHP accelerator.
Our school did an English placement test (Moodle Quiz Module, ver 1.4.4 with 30 audio questions) this past April 2005 for all 1100 entering freshmen to our university. If you check an earlier report, you will notice in April 2004, I used a single slow Mac as a server--and yes, another thread shows us OS X and MySQL don't love each other so well--no problem with Linux and MySQL. This is what we did to make our test a complete success.
- switched to cheap Linux servers (see specs below)
- divided the students into 5 sessions of up to 250 students each
- used four servers (one server per 55 students)
- Dell small business server--no OS, no monitor/keyboard
- Dual Xeon 2.8ghz CPU
- 2GB Memory
- 80GB hard drive
- Internal LAN--servers in another building on campus
- Fedora Core 1, Apache 2.0.50, PHP 4.3.1, MySQL 4.1.10a
- Ioncube PHP accelerator for PHP 4.3.0