In Moodle core, there is an enrol_paypal module. And, in the plugins database, there are plugins like https://moodle.org/plugins/enrol_stripepayment. I suggested you look at those, to get a feel for how they work, then do something similar with the API for your payment gateway.
General developer forum
payment enrollment plugin development
This discussion has been locked so you can no longer reply to it.