To answer your question, blocks are stored in the block_instances table.
- blockname is the type of block. You are looking for 'html'
- parentcontextid is the main indication of where the block should appear. It is a foreign key link to the mdl_context table. That is a key table to understand. For course pages, contextlevel will be 50, and the instanceid will be the cousre id.
- configdata is the settings for the block, stored in a strangely obfuscated way.
Note that you should not just delete the block_instances row from the database. That will leave garbage data in other tables. Instead you should really write a script that calls the delete method in lib/blocklib.php to clean delete the blocks and all associated data.