You have more than one choice... slightly confusingly 'guest' is both a user (log in as 'guest') and a role (guest access in courses).
You can disable to enrolment plugin sitewide - Site administration > Plugins > Manage enrolments, then hide the guest plugin.
And/or - Site administration > Plugins > Authentication > Manage authentication, hide 'Guest login button'