Yes, that is possible but you need to be logged in as the person who added the tag to the course. Here's what to do:
1. click the edit my tags link by the add button in the block
2. select the tag you want to remove in the "delete tag from this course" drop-down
3. press submit.