According to MDL-55981 if your Moodle 3.2 is a new installation then by default non-editing teachers should not be able to access all groups, but if it is an upgraded site then they can access all groups and you will need to, as Tim says, remove the capability moodle/site:accessallgroups
General developer forum
What rights do non-editing teachers have?
This discussion has been locked so you can no longer reply to it.