Hardware and performance

I do scenario A for a living. Depending on how much money you want to spend, there are some great tools available. Where I work, we use Segue SilkPerformer as the driver, and a combination of Segue monitoring, Windows PerfMon, and some home-grown code for Linux, much of which I've personally written.

There are starting to be decent open source tools available, though. I've found that even with an excellent script capture tool like SilkPerformer, there's quite a bit of custom programming required on the driver side. So don't be afraid to use an open source driver tool if you understand the programming language it's based on.

As I mentioned (almost a dozen times in the last day or so ;) I'm building a Moodle-based community site for this, and one of the courses I want to put up is on open source load testing tools. Come on by http://linuxcapacityplanning.com

I don't have much faith in scenario B. There are good benchmarks available for processors, I/O subsystems and databases, but my experience has been that most performance problems are design issues in the application, not something in the underlying platform.

In the end, it becomes an economic question: which is cheaper, fix the application or throw hardware at the problem? If you decide to throw hardware at the problem, it's relatively simple to determine what kind of hardware -- processors, memory or I/O subsystems -- need to be beefed up.