I am pretty sure that somewhere in Ross's blog is some more details about our hardware. I can't remember all the details, but some key points:

1. use Postgres for the database. If you tune it right, it scales better than anything else in high concurrency situations.
2. build your DB server with enough RAM so that essentially the whole database can be cached in memory.
3. then you can have multiple web-servers to share the PHP processing load. I think we have 5 web servers, and a 6th server just running the scheduled cron tasks. (Moving cron onto a separate server prevents long-running tasks from interfering with serving web pages.)
4. At the OU, we manage fine just using Apache + mod_php, however, I have seen reports of better performance with nginx + php-fpm.
5. Obviously, use a PHP op-code cache.
6. The standard Moodle database sessions are not good enough for large sites. You probably need to use memcache sessions. Moodle HQ really needs to do something with MDL-31501, and get it in to Moodle core.
7. So far we have talked about DB and web servers. The third part is file store. All the web servers need access to a networked file store, and that needs to be fast enough. (Sorry, I don't know so much about that bit.)

(I am really not an expert on any of this. I have just been hanging around for long enough that I have heard what a lot of experts say, and the above is a hopefully helpful summary.)

As you say, a key thing is testing before you go live. Then, once you are live, monitoring of everything so that you can find bottlenecks before they bring your system to its knees.