>My first attempt on new quiz statistics
have been to construct a table to summarize the responses to each
question. This table is a combination of the tables 2&3 of the
previous detailed statistics report. You may see the table
below. I have called this report/table Item Analysis since this
table may allow to judge on the performance of each item/question for
The table has all questions used in the
quiz as rows. The first columns are for identifying the question:
idnumber, name, question text. Then there is a column with the actual
responses given by the students (including numerical and shortanswer)
with the counts for each of the actual responses. In this way, it is
possible to inspect what responses are more/less given by the
students for a particular item. In addition, to other columns with
the Facility Index (= % correct answers for that item) and the
Discrimination Index (= (top_scores bottom_scores)/N, for
top/bottom thirds of students in the overall marking distribution).
I have used fractional grade as the way
to compute the statistics, rather than number of correct / false
responses to that item. Much like in the Bluepaper 2
(thanks for the links, Jussi!!). This way the method is more
extensible and can support analysis of multi-anwer MCQ
matching questions that are prone to partial credit.
The system can manage quizzes with
random questions. The new class-based question types allow to
go back easily to determine the actual question that was
included in the test, and include it in the analysis. I think a very
much needed feature. So there are provisions for not all questions
having the same number of attempts (or not at all attempts having
exactly the same questions).
But there are many bugs yet. Data is
not collected correctly for some question types (matching,
multianswer), improvements are needed for speedup statistical
processing, some options are desires, not working yet, and the table
display needs a more professional design.
I do have in my mind a mechanism to
restrict the analysis to a subset of available attempts. Perhaps just
the highest attempt per student, or excluding attempts with a too low
overall marking (perhaps given up or playing attempts).
I am not ready to release the code (too
many bugs). But I would like to know if I am in the right direction.
Do you see this table as useful? Any other information should be
included? (or splitted to other table?). Suggestions for presentation
improvements, analysis options etc. All comments are welcome.
- Enrique -