Hi
Another option to consider is using x-sendfile in place of pluginfile.php. See https://tracker.moodle.org/browse/MDL-26028
This wont per se speed downloads, but will reduce load on the server making your server overall more efficient, especially under higher workloads. Lots of posts here indicate moodle 2.x requiring more horsepower than 1.x code; this will help alleviate extra load on the web server allowing for either a smaller web footprint or extra horsepower for more users, etc.