We have group assignments in the Dataform module, which allows for far more complex assignment structures, e.g. multiple submission types in the same entry and timed workflow. With this module we can deliver collaborative assignments such as Group Peer Review, alongside standard assignments.
Which module one uses for group assignments doesn't change the reasoning I have proposed. If you apply separate groups mode and allow content from one group to be available to a different group without the consent of all members of the originating group, you break the integrity of the assignment.