Assuming that there is nothing wrong in the Moodle configuration and only the performance is bad and that nothing is broken, I would start with optimizing the database. I am assuming you have the entire Moodle installation on a single server, and are using the default LAMP stack. I moved my Moodle version 1.9 installation to a new server (Ubuntu Server 12.04 and upgraded to Moodle 2.0) during the summer of 2012 and when classes started again the new server just couldn't handle the load. The server was in a VMware virtual environment, so adding more RAM and processor was easy. Still, the server crashed under load a couple of times, and adding more RAM did not seem to help.
Installing mysqltuner and running it yielded a lot of hints to tweak how much memory could be allocated to the MySQL database. The stock configuration does not make use of very much available memory so running mysqltuner and then tuning the my.cnf file accordingly repeatedly over time yielded much better results and complete stability.
Further speed increases were had by installing and tuning the APC (Alternative PHP Cache). Now my benchmark is that pages should load in less than a second, and almost always are well under that. FYI, my Moodle installation has about 2000 users total and runs on a single virtualized server.