You need to put the quiz in a course that allows self enrollment with no key.
Then, what you could do is put enrollment keys in labels whose visibility is restricted by the score received in the quiz - perhaps with a link to the course they should enrol in.