If a qustion is used in a quiz, then when you try to delete it, Moodle does not actually delete the data from the database. It just makes the question as hidden. As you say, this is so that the quizzes using the question don't break.
Quiz
Deleting Quiz questions
Энэ мэтгэлцээн түгжигдсэн тул та цаашид хариулт өгөх боломжгүй