Is it possible to give students the ability to edit their own username once they're account has been created?
By default only admins and managers can edit usernames. It's controlled by the capability moodle/user:update. (See the documentation Capabilities/moodle/user:update for further details.) Students can however edit the name that is displayed in Moodle.
so how do I change that? I found where it lists it in capabilities, but can find where to override that.
Or, what could also solve the issue is if I could give that capability to one person but ONLY over a group of students. This person needs "non-editing teacher" rights to a catagory of courses but nothing else besides the ability to change the username.
Unfortunately I don't think it's possible to restrict the capability as you describe. It provides access to the editadvanced page where any profile setting can be changed, not just the username. It also needs to be applied in the system context i.e. a user must be assigned a system role.