Quiz

The main problem with the old detailed statistics page was that it did not support all question types. The way I would like to see the new plug-ins work is that they call functions in the question types to perform the question type specific work. That way when new question types are added later they can provide support for existing report plug-ins simply by providing the required functions. Also the report plug-in author does not have to deal with all existing question types all at once, other people can contribute the necessary question type specific functions later.

And another thing about writing report plugins: you will often want to display the report in the form of a table. For this you should use the new Moodle table class because it automatically provides paging and sorting. Take a look at mod/quiz/attempts.php to see an example use of that table class. 

Hello Gustav,
let me say that I am totally in favor of your proposal.
One of the things i don't like about old detailed statistics report is that it mix 2 very different things
-detailed answers of the students (first table)
and real statistics (other tables)
I would really appreciate the separation.
Also before using Moodle I was using a test module made by some people of our University for the Claroline LMS. I really don't regret the move but for each test you could have a really neat graph. It would be better if I can post an image, but I will try to explain :
it was a bar graph with a bar for each question of the test and a percent vertical scale. Each bar was divided in 3 rectangular areas one red for false answers, one yellow for partial answers and one green for correct answers.
What i liked is that usually a simple look was enought to see the questions with too many green or too many red and after that look for the cause of these "accidents". It was very usefull.
Of course the first goal now is to upgrade the detailed statistics report, but maybe in the future if we create several report plugins for statistics, we can add that kind of feature and I am sure other Moodlers will have wonderfull ideas.

I'm all in favour of specific plugins too.

On a related point I was talking at a conference just now and someone asked "will Moodle 1.5 finally do item analysis?". I wasn't sure what to answer, I'm not too familiar with that kind of stats. I notice the language wonks   in Singapore were right into it so perhaps they could educate me on this?

I thought it was what the "detailed stats" was doing, didn't it?  If so then perhaps "item analysis" should be a new report tab name.   If not, there seems to be a demand we could work on later on.

As it turns out I like what you currently have, but that is probably because I have not seen another choice.  The one thing that I would like to have available is the true ability of extracting into Excel.  Whenever I attempt I do not get what I expect.  Maybe it's my problem.  Anyone else have any difficulty doing extracts into Excel?

Someone has mentioned maybe adding a new item analysis portion.  Seems to me that already exists in the Detail Statistics report, but maybe they are talking about something more refined/inclusive.  The only thing that I would like to see is some type of indicator, in the detail analysis report, showing the correct answer.  Here at a middle school you can't assume that the most answers are on the correct item.

bc

<>Hi,
>My first attempt on new quiz statistics have been to construct a table to summarize the responses to each question. This table is a combination of the tables 2&3 of the previous detailed statistics report. You may see the table below. I have called this report/table Item Analysis since this table may allow to judge on the performance of each item/question for testing purposes.

The table has all questions used in the quiz as rows. The first columns are for identifying the question: idnumber, name, question text. Then there is a column with the actual responses given by the students (including numerical and shortanswer) with the counts for each of the actual responses. In this way, it is possible to inspect what responses are more/less given by the students for a particular item. In addition, to other columns with the Facility Index (= % correct answers for that item) and the Discrimination Index (= (top_scores bottom_scores)/N, for top/bottom thirds of students in the overall marking distribution).

I have used fractional grade as the way to compute the statistics, rather than number of correct / false responses to that item. Much like in the Bluepaper 2 from CAA centre (thanks for the links, Jussi!!). This way the method is more extensible and can support analysis of multi-anwer MCQ questions of matching questions that are prone to partial credit.

The system can manage quizzes with random questions. The new class-based question types allow to go back easily to determine the actual question that was included in the test, and include it in the analysis. I think a very much needed feature. So there are provisions for not all questions having the same number of attempts (or not at all attempts having exactly the same questions).

But there are many bugs yet. Data is not collected correctly for some question types (matching, multianswer), improvements are needed for speedup statistical processing, some options are desires, not working yet, and the table display needs a more professional design.

I do have in my mind a mechanism to restrict the analysis to a subset of available attempts. Perhaps just the highest attempt per student, or excluding attempts with a too low overall marking (perhaps given up or playing attempts).

I am not ready to release the code (too many bugs). But I would like to know if I am in the right direction. Do you see this table as useful? Any other information should be included? (or splitted to other table?). Suggestions for presentation improvements, analysis options etc. All comments are welcome.

- Enrique -

Here's another report I've had more than one request for and it seems very useful. The client is willing to pay for the development for this so if you a programmer or know a programmer who wants to implement a report for this, please let me know: martin@moodle.com

---

The Quiz module in moodle currently supports categories for questions. What
we'd like to do is to be able to do some reporting based upon the results of a
quiz against those categories.

Let say for example a quiz that a user takes 5 questions in each of these
catagories:
Addition
Subtraction
Multiplication
Division

Phase 1:
A student may get 65% on the quiz overall, but we'd like to see a report that
further breaks it down by catagory with what the student got correct.
Addition: 100%
Subtraction: 80%
Multiplication: 30%
Division: 50%

Phase 2:
It would also be useful to compare how this student performed compared to the
entire class's average in the categories.