Since the move to SCSS (and to some extent, although not quite as pronounced, with the LESS on-demand processing) fresh page loads have been very slow.
The best 'trick' is the one Emma recommended, simply leave designer mode off and then purge all caches only when you make a development change that may impact the cache (anything building css/js for example)