I couldn't get Moodle on a stick to work correctly at first either, even with the Flash drive set as F:.
I believe the problem is due to a missing folder in the apache setup.
If you create an empty folder \apache2\logs I believe it will start.
At least that worked for me.
Also, you *must* put the Moodle files/folders in the root directory of the Flash drive, not in a subdirectory. In other words \apache2, \www, and start.exe must be located in the root of F:\, not in a subfolder.
Aside from those minor "gotchas", MoodleOnAStick seems to be working well.