It's kind of common sense....
The main file area (that stores stuff for courses et al) doesn't matter that much when it comes to performance - within reason.
Things like sessions and caches clearly do. In the first place you should consider the benefit of storing sessions in the database. I am honestly not sure which is likely to be quicker. You should measure it!
Whatever remains in the files area will obviously benefit from being on the fastest storage that you have available. If high speed storage is at a premium then the caching/session parts can be symlinked to some other place than the main file store. This works fine.
As usual, it depends on your setup and how you use Moodle. You should strive to get some good baseline measures and then change things and see what happens.
I'm not sure what 'not separating the moodledata directory from Moodle's core installation' means? Moodle doesn't care - it's just a file area, there's nothing magic about it. Your file subsystem is either fast or it isn't.
The latest Moodle 2.4 has a substantially rewritten cache API. I don't know enough about it (yet) to say what difference it will make but it looks to give you more control and options.