If a qustion is used in a quiz, then when you try to delete it, Moodle does not actually delete the data from the database. It just makes the question as hidden. As you say, this is so that the quizzes using the question don't break.
Quiz
Deleting Quiz questions
This discussion has been locked so you can no longer reply to it.