Thanks for the write-up!
Quite a bit of information there! If I understand right, you have a powerful server behind a slow connection.
a) Considering the slow connection you are postively surprised.
I can only wish, the rest of the world consume bandwidth rationally like your people!
b) The server itself was never really tested.
Doesn't surprise me, there was a serious bottleneck in front, the upload speed of 512 kbps! You have test the server in a 100 Mbps LAN!