So I've done some digging further into this. It appears that it linked to the capability mod/assign:grade If they can grade it then they can be send emails. There is no capability to let them grade it but not receive emails.
I'm going to add this to the tracker for a possible feature.