Hi everyone,
I've torn my quiz report module down to the ground and rebuilt it in a much more efficient manner. There is only one module now that is like a Swiss army knife. It does everything, but unlike the knife, all of the blades are always out -- you get the entire report whether you want it or not!
This version has the code that generates the data clearly separated from the code that generates the HTML so that it will be easier to upgrade in the future.
The plug-in covers these types of quizzes:
Short Answer, True-False, Multiple Choice, Matching and Numerical
It simply ignores any other types, although the code has been made so that other types can be easily added.
The report yields three tables:
Table 1 -- One line per student with the exact item input or selected for each question. Incorrect answers show in red.
Table 2 -- One column per question showing how many responses there were for each option selected, or each phrase input. The last row shows the percentage of students who got each item correct.
Table 3 -- A summary of the questions, with the choices for the multiple choice items. (Detail which would have made the other tables ungainly if placed in the cells.
How to install it:
Inside "moodle/mod/quiz/report/" create a folder with the name "fullstat".
Place the attached "report.php" INSIDE the fullstat folder. (Be careful not to overwrite the file by the same name that resides at "moodle/mod/quiz/)
If you don't wish to see fullstat as the link to access the plug-in, then place the following line of code in "moodle/lang/en/quiz.php":
$string['reportfullstat'] = "Detailed Statistics";
To use the plug in:
1) Click on the quiz.
2) Click on "Review reports for nn attempts".
3) Click on fullstat or "Detailed Statistics" at the top of the page.
4) If you wish an Excel spreadsheet or a text file, click on the buttons at the bottom of the page.
Please report your success or failure to me.
For those who wished to have more detailed statistical processing to evaluate the responses, I suggest that you download the file as an Excel spreadsheet, from which you can export the data to any stat program.
Thanks, Tom, wow it's huge! I've checked this in to CVS.
The code still needs a little work - I'll contact you privately about these - and I'd love it if you could maintain this directly in CVS!
The code still needs a little work - I'll contact you privately about these - and I'd love it if you could maintain this directly in CVS!
Hi Thomas,
as always, professionnal usefull tool working first time on Moodle 1.2 development (2003091800).
Thanks,
Bernard
as always, professionnal usefull tool working first time on Moodle 1.2 development (2003091800).
Thanks,
Bernard
FANTASTIC!! I like that red
Item respons analysis..
Does that mean that you use item-respons-theory to do that?
Can you explain that a little?
OOPS, I took one of the tests of my teachers (m.c. + short answers) and I got:
empty delimiter in report.php on line 772
(Could it be that only teachers did this test after it got new questons? so empty for the first students that did the test befor the change?)
>Does that mean that you use item-response-theory to do that?
No, all it does is present the percent correct. I think that at this point, to do a more thorough analysis is beyond the scope of what Moodle should do -- overkill for most users. A deeper analysis is useful if one is truly piloting items in order to improve the accuracy of the test, but this requires a larger pool of responses than the normal Moodle user would have. The results can always be downloaded to a spreadsheet and then fed into a program that can produce a detailed analysis.
I have one other report of that "empty delimiter" problem (in a WIN configuration) and am working on it.
I should have an improved version up in CVS in the next few hours.
Thanks, Ger!
No, all it does is present the percent correct. I think that at this point, to do a more thorough analysis is beyond the scope of what Moodle should do -- overkill for most users. A deeper analysis is useful if one is truly piloting items in order to improve the accuracy of the test, but this requires a larger pool of responses than the normal Moodle user would have. The results can always be downloaded to a spreadsheet and then fed into a program that can produce a detailed analysis.
I have one other report of that "empty delimiter" problem (in a WIN configuration) and am working on it.
I should have an improved version up in CVS in the next few hours.
Thanks, Ger!
I have now committed a revised version to CVS. I would appreciate it if people using a WIN server, in particular, would try it out to see if the "empty delimiter" bug has been resolved.
This version contains two major modifications:
1) The text of M/C choices is now displayed in the student-by-student summary. This may produce ungainly tables if the choices are extremely long. I hesitate to complicate matters by adding a preference for text vs. choice number, but I could make it so that if any choice were longer than, say 25 characters, then only the item numbers would be printed. Please let me know if this would be useful.
2) The final table used to only provide a listing of the questions, and for M/C items, the set of possible responses. (Since only the choice numbers were printed in the earlier tables this was necessary.) I have now added summary statistics for M/C and T/F items to the table to make it a bit more informative.
Note, that for this version to work properly, you will need to update your lang/en/quiz.php file since all labels now retrieve their text from there. Otherwise, you will end up with the ugly thisstring display.
I look forward to your feedback!
This version contains two major modifications:
1) The text of M/C choices is now displayed in the student-by-student summary. This may produce ungainly tables if the choices are extremely long. I hesitate to complicate matters by adding a preference for text vs. choice number, but I could make it so that if any choice were longer than, say 25 characters, then only the item numbers would be printed. Please let me know if this would be useful.
2) The final table used to only provide a listing of the questions, and for M/C items, the set of possible responses. (Since only the choice numbers were printed in the earlier tables this was necessary.) I have now added summary statistics for M/C and T/F items to the table to make it a bit more informative.
Note, that for this version to work properly, you will need to update your lang/en/quiz.php file since all labels now retrieve their text from there. Otherwise, you will end up with the ugly thisstring display.
I look forward to your feedback!
I've now added one more feature, one that Ger was hoping for -- a "Discrimination Index" which works under the assumption that effective test items should be answered correctly more often by the best students than the poorer ones. I've created a help file to explain the concept which I reproduce below.
These results print out in a row just below the percentage correct for each item.
--Tom
Discrimination Index
This provides a rough indicator of the performance of each item. It is calculated by counting the number of students who scored in top third on this quiz for each question and dividing this figure by the number of students who scored in the bottom third. For example, if 30 students took the quiz, there would be 10 students each in the top third and bottom third. If, on item #1, 9 of the best students got it right, but only 3 of the poorest third got it right, the discrimination index would be 9/3 = 3.0. The numbers in parentheses are--(top third correct responses/bottom third correct responses).
If the index goes below 1.0 it means that more of the weaker students got the item right than the stronger students. Such items should be discarded as worthless. In fact, they reduce the accuracy of the overall score for the quiz.
If no weak students got the item correct, the denominator would be 0 which would yield a result of infinity. The program substitutes a '10' for any such items.
Thanks, thanks, thanks, You earned your place in the gallery of Moodle heroes, but please don't retire, read until the end of this email.
I like your Moodle approach: make things for normal users as elegant ( I avoid the word simple!) as possible, realizing what their situation is....
....and still help them in a friendly way to raise the quality of their content.
I do not like to say it, but...
I did try every in between improvement of your report and saw sinking the error to the bottom of report.php, hoping that it would finally fall out the script. But now it stays hanging on line 992:
Warning: Empty delimiter in /var/www/elo/prod/moodle/mod/quiz/report/fullstat/report.php on line 922
This is on a Debian Linux machine, this script:
<?PHP // $Id: report.php,
// created from the above 2003/11/20 by Tom Robb tom@robb.net
// Version 2.2 Modified 2003/11/25 pm
// $showtext added to toggle display of full text of M/C questions
// Discrimination Index figures added
Only the pure 4-options-M.C. test (how to fly a plane) did not give this message
(If I scroll down along the messages in the other reports, sometimes only one error sometimes a lot, the calculated results are showing ok in all the reports)
Thank you,Ger! This was sufficient for me to track down the bug. It seems that null responses in Short Answers were the culprit. The report now tests for this for relevant answer types, so is "bug-free" - until someone finds another bug, that is!
I've committed the revised version to CVS.
--Tom
I've committed the revised version to CVS.
--Tom
Just to report that the fullstats report ( Version 2.3 Modified 2003/12/12 pm) gives me lots of warnings:
Warning: Empty Delimiter in c:\program files\easyphp\www\moodle\mod\quiz\report\fullstat\report.php on line 932
isn'that the old version on the top of this thread, not the one in cvs?
This appears to be due to empty data in the Short Answers. I've made yet another fix and sent it off to Gustav for testing. We shall see what happens.
For me the spreadsheet produced by the fullstats page is missing data in the third worksheet. That worksheet only lists the questions and possible answers but not the numbers of students that have chosen a particular answer and not the corresponding percentages.
That wasn't a bug, but rather an "unimplemented feature"! I hadn't built the percentage reporting into the Excel spreadsheet, only into the screen display.
It's there now, though. I've attached it here (to be put into quiz/report/fullstat) and have committed it to CVS as well.
The error due to no responses for short answers has also been fixed. Now "N/R" is printed out when no data is found.
It's there now, though. I've attached it here (to be put into quiz/report/fullstat) and have committed it to CVS as well.
The error due to no responses for short answers has also been fixed. Now "N/R" is printed out when no data is found.
I find it useful to have the data in the spreadsheet as numbers instead of a string. I therefore made the following small modifications:
I added the following after line 343
$formatbpct =& $workbook->add_format();
$formatbpct->set_bold(1);
$formatbpct->set_num_format('0.0%');
I replaced lines 542-544 by
$pct = $nowstat/$total_user_count;
$myxls->write_number($row,0,$nowstat,$formatb);
$myxls->write_number($row,1,$pct,$formatbpct);
Thanks, Gustav. I've redone things so that all percentages are displayed in a similar manner (for T/F items, for example). I am leary, however, of simply calculating the percent by dividing one number by the other, so I still use a small function to do it which tests for a 0 in the numerator.
I've committed the revision to CVS.
I've committed the revision to CVS.
Not sure if this is the best place to post this. I am running two Moodle system. 1st) Under windows for development and testing, using Easyphp 1.6. 2nd) A production system running Mandrake Linux 8.2. When using the export to Excel feature of the detail report I get the following error on my Windows machine
Works fine under Linux. This is a 1.2 install that I downloaded about a week ago.
Any ideas what could be causing this problem?
| <br /> | |||||||||||||||||
| <b>Fatal error</b>: Cannot instantiate non-existent class: format in <b>c:\program files\easyphp\www\moodle2\lib\excel\Workbook.php</b> on line <b>67</b><br /> | |||||||||||||||||
Works fine under Linux. This is a 1.2 install that I downloaded about a week ago.
Any ideas what could be causing this problem?
Jonathan,
I'm sorry that I didn't notice your original posting that is now two weeks old. Are you still having the problem?
Had you used the Excel spreadsheet feature under Linux, too? Does this file exist in your system? moodle/lib/excel/format.php
I'm sorry that I didn't notice your original posting that is now two weeks old. Are you still having the problem?
Had you used the Excel spreadsheet feature under Linux, too? Does this file exist in your system? moodle/lib/excel/format.php
I'm having trouble all of sudden getting the detailed report to work. It was working before on a quiz with 70 some questions, but now that I've got 48 responses on the quiz (it's actually a survey) I can't bring up the detailed report. Basically, it acts like it is going to give me the report and then just freezes with a blank page.
Any ideas?
It could well be that some 'bad data' is causing it to trip up.
What kind of items are they -- multiple choice, short answer, etc.?
What kind of items are they -- multiple choice, short answer, etc.?
I had a problem that sounds similar to what you describe.
In my case it was with the Journal module. Once I had over 30 or 40 student contributions, I couldn't open the report page. It started to open, but then ended up just showing a blank page -- or sometimes just the header.
It seemed to be a time-out error related to my browser client. I tried other browsers and discovered I could view the reports. I'm using Mozilla, Safari and Internet Explorer. It's been a while, so I don't really remember, but I think it ended up working best in Safari.
In my case it was with the Journal module. Once I had over 30 or 40 student contributions, I couldn't open the report page. It started to open, but then ended up just showing a blank page -- or sometimes just the header.
It seemed to be a time-out error related to my browser client. I tried other browsers and discovered I could view the reports. I'm using Mozilla, Safari and Internet Explorer. It's been a while, so I don't really remember, but I think it ended up working best in Safari.
I had that with previous versions, until our admin raised the processing time and maximum memory claim, so was not a browser problem. (See elsewhere in this forum)
Did you maybe change a question in the survey? (and forgot to give it a correct score - happened us because we did not need a good score in a survey...)
I have now received Dave Richards' data and tried it on my own machine. It works here, so I can only attribute his problem to some sort of limitation of his own machine -- most likely processor memory or speed. My "machine" however is simply a Macintosh Powerbook, so it isn't a lightening fast, super-duper server, either.
The 'detailed stats' program *does* create rather large arrays while it is processing, so a large number of items and/or test-takers could well cause it to choke.
The 'detailed stats' program *does* create rather large arrays while it is processing, so a large number of items and/or test-takers could well cause it to choke.
One of my users complains about the format of Excel, he says that "broken numbers" with a dot in it are transported to Excel as text.. I should tell that to my man in Japan, he said..
(By the way, are you working on this modul on this moment? I still hope for this claculated column like in the gradebook from Gustav)
This is the first that I have heard about 'broken numbers'. Could you please tell me where these appear (which kind of rows or columns) and for what kind of data (which kind of quiz item, etc.).
Concerning the 'calculated column', since I don't use the gradebook, I'm not sure what you mean. Please describe what it does in more detail and perhaps I can add it in.
Concerning the 'calculated column', since I don't use the gradebook, I'm not sure what you mean. Please describe what it does in more detail and perhaps I can add it in.
Well, the gradebook does a weighted final score for several scores in the gradebook.
If you use quizz for a survey, It would be nice if you could cluster answers in groups (5 questions about subject A, 5 about topic B etc.)
I will ask the teacher to show an example of the spreadsheet error
Could anybody tell me where I can find the latest version of this report.php file that Thomas Robb wrote and that was the topic of this thread?. Since this was announced quite a long time ago, I thought that this would have been incorporated somehow in the version of Moodle that I have (1.4.1). Today I created a quiz, however, and when I checked the grades I realized that only the total final grade was reported. Not the kind of detailed report that I was expecting. Whatever happened to that project? Can I still get that plugin somewhere and have it work with Moodle 1.4.1?
Josep M.
Josep M.
Hi Joseph,
Detailed statistics seem to tbe there in my 1.4.1.
http://url.com/moodle/mod/quiz/report.php?id=1&mode=fullstat
Please bear in mind however that it does not support random questions. This means that I can't use it since most of my questions are random.
Tim
Thanks for your answer Tim. I was a little behind keeping up with the Moodle forums. The URL you give me, though, takes me to some comercial site with some advertising.
Josep M.
Josep M.
Hello Josep
The URL I gave you was only an example. I am sorry it was confusing.
Please replace "url.com" with your site and "id=1" with the id of the quiz in question. Or you can get to full stats from the simple quiz results page.
http://url.com/moodle/mod/quiz/report.php?id=1&mode=fullstat
Only trouble is that it does not deal with random questions. It is a shame Gustav's mathematicians didn't need random.
Timothy
I tried to get the spreadsheet into Open Office and Apple Works, but the only thing that works is Excel. I also get a numeric type mismatch but it isn't important to me. I used to get that problem on SQL Server exports sometimes and I'm sure it is easy to fix, but can't remember how I used to do it. Exporting to another database was one way to do it. I probably wouldn't bother exporting it if the columns didn't default to the width of the text.
Hi everyone.
I need your help. I tried to apply fullstat for the quiz with questions, where two or three answers are good (the option "Multiple answers allowed" is turned on). The report not looks good, distribution of the frequency is wrong.
I have also troubles, when I try to use plug-in fullstat for quiz in which more than 1000 participants.
Best regards from Poland
Sorry, Marek, but the module does not (yet) know how to deal with multiple correct answers, so you will not be able to get correct results at this point in time.
As far as not working with 1000 students, I'm afraid that you have run up against the limits of your server's capabilities. You might try it again with other unneeded applications turned off to see if the memory that this frees up allows it to work.
When I created the Fullstat module, I didn't write it in such a way as to conserve memory, since I wasn't sure it this would be a problem or not. There may be some things I can do to it in the future to make it less greedy, but I am afraid that there isn't much I can do right now. I hope you can install your data on a machine with larger capacity and try it again.
As far as not working with 1000 students, I'm afraid that you have run up against the limits of your server's capabilities. You might try it again with other unneeded applications turned off to see if the memory that this frees up allows it to work.
When I created the Fullstat module, I didn't write it in such a way as to conserve memory, since I wasn't sure it this would be a problem or not. There may be some things I can do to it in the future to make it less greedy, but I am afraid that there isn't much I can do right now. I hope you can install your data on a machine with larger capacity and try it again.