I think it will depend on the way each of those other modules connects to the question_attempts table. If a module uses the same method as the quiz and so question_usages.uniqueid leads you back to the module, then you can use the same method as with the quiz.
However, if the module use other methods, such as skipping the question_usages table entirely, then, yes, as you say, you may need to go through the contextid.
Modules may do this in ways very different from the way the quiz does it. So, how many ways are there for a module to use the question_usages and question_attempts tables?
I can't speak to the API here, since I don't know how it works, but if I were doing such a report in just SQL, I would first write a separate query for each module to get the results. Then I would understand each method. Then I would put them together into one report.