You *should* be able to expect a lot better performance from your machine, I run a moodle 2.9 site on a Raspberry Pi and get sub 1s page load times. Moodle isn't a heave weight application, especially when running for one person of localhost!
You said you are using PHP 5.5 so you should already have the built in PHP opcache available.
In your moodle you can check the 'opcache.enable' option on the <yourmoodle>/admin/environment.php page.
If that is enabled check it is configured in a sensible way, use this guide https://docs.moodle.org/28/en/OPcache
Now check it's not your database that is the bottleneck, to do this enable performance reporting in you site, go-to, <yourmoodle>/admin/settings.php?section=debugging and tick the 'Performance Info' option. This will add some debug information to the footer of your site where you can see the 'DB queries time', if this is more than a few miliseconds your issue is the database not Moodle/PHP.