Just to add my two cents, when we were tuning cache settings and db settings and all that, I used a jmeter test plan and graphs to measure outcomes of different settings. This can lead down a rabbit hole, since sometimes when you are tuning for one metric, another metric might go up, but what we tuned for was page load times and maximum # of concurrent users served, as defined by a certain # of fake accounts logging in an order and performing a series of 'representative student actions' such as browsing course, opening quiz, etc.
For ongoing we use a free service from gtmetrix.com that more simply measures public-facing page load times from an external location you can choose (so that we take into account internet latency issues that normal users are exposed to). It gives us a longitudinal measure when we are not actively testing/tweaking, so for example it lets us see that that spiffy new high-res image we added to the front page to feature the latest new course added 1 second on average to page load times.