If you recently upgraded Moodle or this is a new Moodle installation, note that the minimum system requirements is now 512 MB of memory. I find that this will let you run with about 5, maybe up to 10 concurrent students through it seems to slow down as you get towards the top end depending on what students are doing. If you can't get 2-5 concurrent users up and running on a basic installation, you may have more significant issues. I once had a site running Moodle 3.1 with 1-2 users on a server that only had 384 MB of memory and no swap file. It ran fine but would crash every once in a while due to insufficient memory.
While it really depends on the usage of your particular site, as a general rule, I usually recommend starting with 1 CPU core and 2 GB of RAM for every 20-25 concurrent students. Keep in mind that recent operating system security fixes to address Meltdown and Spectre vulnerabilities have reduced server CPU capacity by up to 25%.
Either way, make sure you have a swap file enabled. Although this is often not usually recommended if running on an SSD drive due to potential excessive wear - could lead to early hardware failure, if you truly have sufficient memory in the first place, the use of the swap file should be minimal and will help prevent your server from dying on the rare occasion when there is a sudden requirement for more memory above what is available.
For extra stability and flexibility in a production environment, it is highly recommended that you put your database server on a separate machine. This will also give you the ability to support additional students and make it easier to scale your Moodle site down the road to increase capacity as your needs grow.
If you have enough CPU and memory, it could also be the result of a combination of Moodle or web server configuration settings.
If your database is already on a separate physical server, it could be your router, switch or cabling.
If you are trying to make this work on Shared hosting and find that the problem is intermittent, it could be due to other things that are going on at the same time. For example, I had a client who was told they were on dedicated hosting. After further investigation, it turned out that their hosting reseller considered it dedicated because he had only his clients on the server. However it was in fact a shared hosting scenario.
Although a little dated (was written in 2014), you might find this 4 part series on optimizing a Moodle installation interesting. See http://www.iteachwithmoodle.com/category/speed-up-moodle/ . Also take a look at Moodle Performance Recommendations.
Hope you find something in all of this helpful.