You need these two capabilities at the sitecontext:
moodle/user:updatemoodle/user:delete
If you look at the Browse list of Users screen it always has the icons to update and delete users that is why it is so.
In the code admin/user.php where this is done it clearly askes for these two capabilities or the nopermssion error is returned.