You seem to know the workings of the web
server, the
Apache MPM prefork, very well. And only you know the load patterns of your Moodle. So I assume, there is nothing to see in this corner. Otherwise, that is the place to start.
Still, talking about "seeing", your conclusions are based purely on theory, estimations. Or, do you have actual measurements? Just snapshots are not sufficient, you need continuous measurements, a proper monitoring 24x7 and then relate the behaviour to user activity. For example, in this graph the
Apache has run out of available processes Tuesday at 22 hours:
That applies to all other parameters you were talking about.
BTW, the vertical load above was artificial, the web sever was hit with a benchmarking client.
Absolutely important: The
server should not swap. If it must, a couple of hundred MB are OK, if frozen. (The OS not releasing the memory after a short crisis.) But never grow, definitely not towards the GB range! Rebooting the server is a joke the Windows folks crack, Unixers avoid them at all cost!
Memory analysis is more complicated. I'm surprised that you said "doesn't ever seem to use more than 50% of this". What is used, what is unused? The OS fills the memory with all sorts of caches and buffers when free - "just in case" and release them when needed elsewhere. Do you count these as used or as unused? A good measurement tool will give you a full breakdown - in to a dozen of items, swap marked in red. ;)
I would follow up on that line. Starting with the size of your
database (or databases, if the DBMS has more)? 64 GB RAM is respectable, as long as the DBs don't add up to 50 GB.

And what about caching, Redis and the like? They save memory by reusing, but bring their own overhead. If you don't have one now, don't add until you sort out the current problem - it'll make debugging more involved.