Thanks for taking the time to reply and adding your own experience.
I have managed to install a test instance of Moodle 3.2 to check this and you are correct. It does behave as you say so it seems my recollection was wrong. What I seem to have been recalling was likely to have been with the setting External unenrol action = Disable course enrolment where once the user was removed from the external database, and the enrolment sync script run, the option became available to delete them manually from the course. This I confirmed in the test Moodle 3.2 instance.
It seems a bit strange that there is no override possible if you somehow lose the enrolments in the external database allowing permanent enrolments as thereafter there is no option to unenrol.