I am running my Moodle site on a GoDaddy VPS with similar specifications, about half the student load that you are planning for. I find that I get pretty good performance with Moodle 2.6+. Most pages display in about 1 to 1.5 seconds (sometimes faster, sometimes slower) for me. Of course, being the administrator, I see slower times than students are probably seeing.
I am careful about distributing due dates among my 4 to 6 courses. For example, I do not make all assignments in all courses due on Friday night. If I did this, the load on my Moodle would be high on Friday evenings. Even for timed exams (quizzes), I give my students a window span of one day to do these.
This semester, I tweaked the mySQL settings, and am trying Opcache. I will have a chance to see how it goes.