Note that, while I was visiting Moodle HQ a couple of weeks ago, I was able to sit down the Sam Hemelryk and implement caching of question definitions using the new caching system. This should make a bit improvement to quiz performance, but we have not actually done any testing to prove this yet. See MDL-34399 for more details. Note that this has been integrated, so this will be included in Moodle 2.4 when it is realeased.
If anyone can do any performance testing on this, that would be really helpful.