I've had to do this once. After some students started a homework assignment quiz, I found some particularly moronic page numbering put there by a teacher not thinking carefully at the time (me!). There were certain description boxes with instructions that were separated from the questions that they applied to. Since it was an untimed practice assignment and I discovered it really early on AND my kids were in class with me, I deleted the three attempts that had already started and fixed it.
Interestingly, using the quiz settings page to repaginate was an option I considered but it wasn't helpful since there were different sections of different numbers of questions that I needed to maintain. (This was my original mistake actually... sigh.)
Fixing pagination on the fly would be nice in that rare circumstance. But I don't think its worth sacrificing consistency.