So I remember now why I had view hidden enabled; we have some hidden courses, which are in hidden categories. However I wanted enrolled students to be able to see the course if we manually enrolled them.
Now, I have configured a category override for the hidden course. Changing the student role here to allow view hidden courses seems to work as desired.