How many groups do you have?
If only a couple, maybe try setting up a forum just for that group? Then not only will the forum be restricted to those group members, the emails would only go to them then...
Another option (I've not tested this) is to group your assignment in the Forum Settings under Common Module settings (if enabled by admin at the site level) (this would act more like a sub group of the class though. Then when you create a new forum post, you can post to 'all' or post to the subgroup.