The main request made when Enrol users is clicked is /lib/ajax/service.php?sesskey=…&info=core_get_fragment. Hopefully you can find this in the list of requests:
If you click on it in your browser's developer tools you should be able
to see the response and whether it contains something not JSON, possibly HTML as you say. It may even contains information that helps
solve this. If the response to that request looks okay, you'll have to check any other requests (there may only be this one).
If the certificate is in front of the server and internal connections are bypassing this then I had wondered if something could be inspecting and changing the contents mid-stream. That said Moodle should only have one URL and that would be HTTPS (ideally) or HTTP but not both, and HTTPS would prevent modification of the stream (unless there was some MITM involved).