Let's differentiate between authentication (user creation) and enrollment (assigned a role in a course).
When you mention students "outside the system" I assume you mean that they did not already have user accounts in Moodle. Enabling the Email-base Self-registration authentication method will, as you might imagine, allow users to create their own accounts and receive an automated confirmation email. They can now access Moodle but are not enrolled into any courses.
Enabling Self-enrollment (at the site level and for applicable courses) allows users to enroll themselves into courses with the click of a button. As part of configuring the enrollment for each course you will designate what role the system will assign to users who self-enroll into it. New enrollees will be sent a welcome email.
These email messages are only sent out if the user personally performs these actions. So if you manually create a user account and manually enroll them into a course, no emails are sent. So if you're trying to use self-registration and enrollment, you shouldn't be involved in either processes at all. It all should be automated..
And just to encourage you, MANY Moodle admins are instructional entrepreneurs, teachers, or like you and I, designers masquerading as sys admins. There's plenty of code heads and Linux junkies and server buffs on here, to be sure, and we less technical types lean on them for their strengths. But in not too long a time you may surprise yourself by posting a sage answer to a mind-numbing question in this very forum.