@Nick
Have you tried going directly to the page that will allow un-install?
http://yoursite/admin/plugins.php
OR try removing what you attempted to install manually by hiding (a "dot" in front of the folder name via rename) or by moving that folder out into 'other space'. When doing plugins check then it might say 'missing from disk' but hopefully get by that issue to continue.
'spirit of sharing', Ken