Turn full debugging on and it should point you in the right direction. As the "proud" owner of a huge system level quiz bank, I have had so many issues with restoring that I am about to rebuild and get rid of my system level bank in favor on course level quiz banks.
I had the same issues with restores - I found a few corrupted questions were the culprit. Sometimes turning on debugging would actually allow the restore to complete (my theory is that the process of writing out the errors kept the process alive long enough to complete) without further interaction but on a few, I was able to trace the questions that were causing the errors through the debug output, fix them and then was able to get them to restore.