We are running with similar numbers over here. A few ideas:
- One server cluster with a very powerful DB+NFS server (we use a dual Opteron), many cheap webservers with a load balancer in front.
- The configuration is tricky -- make sure you have top gun sysadmins that understand linux, the load balancer, apache and database configs really well. And PHP's session management, which needs careful config on a cluster.
- Did I say run it on Linux? (or OS X)
- One Moodle setup per school, all sharing the same server cluster.
We have done a lot of performance work in Moodle that is in the queue for 1.4.3 and 1.5. If your timeframe is after Xmas, then you'll benefit from those a lot.
On the Database front you'll hear a lot of stories. Well tuned MySQL vs well-tuned PostgreSQL is a hard one. Our experience in the last few years is that Postgres is more _consistent_ (a bit slower with light load, but significantly better with high load) and less prone to failures.
Postgres _is_ finnicky with bad SQL or badly optimized queries. That's why our patches for 1.4.3 are full of SQL tweaks ;)