Moodle requires some hefty memory sometimes. Quizzes/test is one of those mods that require memory.
Check the servers PHP settings viewable in System Admin -> Server -> PHPInfo
Look for things like memory_limit, max_execution_time, max_input_time ... anything involving memory usage and how long scripts (php) can run and how much memory php scripts can consume.
Those changes will take place in your php.ini and changes there will require a restart of Apache for them to take affect.
Basically ... increase them. Will say this ... a 3 GB RAM box is probably a little light on memory for Moodle to run many users.
There is a setting in Moodle you can also change ...
Site Admin Menu -> Server -> Performance ...
'spirit of sharing', Ken