Hi, there. I was working with Quiz module. I found some changes in the Item analysis table window.
I didn't find the following columns:
Answer's text, partial credit, r. counts, R%, %Correct Facility, SD
Instead of these, I found the following columns:
Attempts, Random guess score, intended weight, Effective weight
Does somebody know more about it?
TIA
Have you read the release notes?
Go to Moodle_2.0_release_notes#Quiz_module_and_question_bank and click on the "Quiz report enhancements" link.
Go to Moodle_2.0_release_notes#Quiz_module_and_question_bank and click on the "Quiz report enhancements" link.
Hi, Tim. I appreciate your reply. You are doing a great job in this moodle module. After I read Quiz notes I have some comments and questions:
Item statistics window
I didn't find facility index section for each item
Are you going to offer an export option in this section?
Also, I read Quiz item analysis calculation section.
Can you give me some bibliography to understand better the Discriminantive efficiency, Intended weight, and Effective weight concepts?
Finally, What is your opinion about Point Bi-serial correlation.
Cheers
Monico
Item statistics window
I didn't find facility index section for each item
Are you going to offer an export option in this section?
Also, I read Quiz item analysis calculation section.
Can you give me some bibliography to understand better the Discriminantive efficiency, Intended weight, and Effective weight concepts?
Finally, What is your opinion about Point Bi-serial correlation.
Cheers
Monico
Looking at the code, it seems that the facility index and standard deviation columns only appear if there is more than one attempt included in the stats.
That seems weird and confusing. I think those columns should always appear, and be blank if there is insufficient data to compute the value. Please could you file a bug at http://tracker.moodle.org/.
There is a download option for the Quiz structure analysis part of the report, isn't there?
From your post, I would guess you know more about statistics than me. (Once upon a time I did a maths degree, but I did not do much statistics. To decide what statistics to implement in the new Moodle stats report, we asked a statistician here, but that person has now retired.)
Discriminative efficiency I don't know about.
Intended weight is easy, it is just the maximum possible score for this question, as a percentage of the maximum score for the quiz. So, if you have a quiz with 10 questions, each worth 1 point, then the you intend that each question has an equal weight of 10%.
Effective weight is an attempt to estimate what weight each question had, when you look at the amount each question contributed to the variance in the final scores. So, if one question discriminates really badly, and the others are all pretty good, then in the above example, the bad question will have an effective weight close to zero, and the other questions will have an effective weight of around 11%. I am not sure of the derivation of this particular estimation formula. It may have another name in other statistical contexts.
I have got a book about test statistics at home. I'll try to remember to have a look in it tonight.
Point Bi-serial correlation: (once I had looked that up in wikipedia to remind myself). The issue here is how you model an assessment item. Is the score for a question just right/wrong, 1/0, or is it a continuous variable. Well, for some (most?) Moodle question types, fractional scores are possible. So it is easier to treat all question types the same, and process question scores as a continuous variable. That does not give optimal results for true/false and multiple choice (with a single correct answer) questions, but it still gives statistics that are good enough for what most people want to do with them.
That seems weird and confusing. I think those columns should always appear, and be blank if there is insufficient data to compute the value. Please could you file a bug at http://tracker.moodle.org/.
There is a download option for the Quiz structure analysis part of the report, isn't there?
From your post, I would guess you know more about statistics than me. (Once upon a time I did a maths degree, but I did not do much statistics. To decide what statistics to implement in the new Moodle stats report, we asked a statistician here, but that person has now retired.)
Discriminative efficiency I don't know about.
Intended weight is easy, it is just the maximum possible score for this question, as a percentage of the maximum score for the quiz. So, if you have a quiz with 10 questions, each worth 1 point, then the you intend that each question has an equal weight of 10%.
Effective weight is an attempt to estimate what weight each question had, when you look at the amount each question contributed to the variance in the final scores. So, if one question discriminates really badly, and the others are all pretty good, then in the above example, the bad question will have an effective weight close to zero, and the other questions will have an effective weight of around 11%. I am not sure of the derivation of this particular estimation formula. It may have another name in other statistical contexts.
I have got a book about test statistics at home. I'll try to remember to have a look in it tonight.
Point Bi-serial correlation: (once I had looked that up in wikipedia to remind myself). The issue here is how you model an assessment item. Is the score for a question just right/wrong, 1/0, or is it a continuous variable. Well, for some (most?) Moodle question types, fractional scores are possible. So it is easier to treat all question types the same, and process question scores as a continuous variable. That does not give optimal results for true/false and multiple choice (with a single correct answer) questions, but it still gives statistics that are good enough for what most people want to do with them.
Hi, Tim.
Facility index and SD columns appear already.
Download option appears on Quiz structure analysis window but, It doesn't on item analysis window. I attached a screenshot of this.
Statistics information.
I'm not sure if I know more than you about Statistics. But, my point of view is ask if I'm no sure about this statistics concepts. Of course, after to check other kind of Statistics sources. Thanks for your time to clarify all Statistics concepts mentioned above.
Cheers
Facility index and SD columns appear already.
Download option appears on Quiz structure analysis window but, It doesn't on item analysis window. I attached a screenshot of this.
Statistics information.
I'm not sure if I know more than you about Statistics. But, my point of view is ask if I'm no sure about this statistics concepts. Of course, after to check other kind of Statistics sources. Thanks for your time to clarify all Statistics concepts mentioned above.
Cheers
The book I have is "Essentials of Educational Measurement" by Ebel and Frisbie 5th edition. It does not go into statistics very deeply (It clearly pre-dates the wide availability of computers. Yes. (c) 1991.
To download the response analysis, you need to use the 'Download entire report' button (or whatever it is called) on the front page of the statistics report. That downloads the overall statistics, followed by the detailed analysis of each question.
To download the response analysis, you need to use the 'Download entire report' button (or whatever it is called) on the front page of the statistics report. That downloads the overall statistics, followed by the detailed analysis of each question.
Hi, Tim. Great book. As you know this book is one of most referenced in almost test validity research paper that I read so far.
I download the quiz report in ODS format. But, I couldn't find the detailed analysis of each question section. I attached a screenshot of this file.
Cheers
I download the quiz report in ODS format. But, I couldn't find the detailed analysis of each question section. I attached a screenshot of this file.
Cheers
I'm not sure if it is really a great book, or just the one everyone cites 
Judging by the abandoned bookmark I found it in, I did not get more than a few chapters into it before I left off and started reading something more interesting. Actually, it seems pretty comprehensive, but rather dry.
Did you click the "Download full report as" button near the top of the report, instead of the "Download table data as" button lower down?
I just did some attempts on http://qa.moodle.net/ and created the attached download.
Did you click the "Download full report as" button near the top of the report, instead of the "Download table data as" button lower down?
I just did some attempts on http://qa.moodle.net/ and created the attached download.
Hi, Tim.
Ebel's book
I haven't read this book, yet. But, I understand your point of view of it.
Download full report
Yes, you are right. I checked the Statistics window, again and I found the button near of the top of Quiz Structure analysis. I'll check the txt file, thanks.
Cheers
Monico
Ebel's book
I haven't read this book, yet. But, I understand your point of view of it.
Download full report
Yes, you are right. I checked the Statistics window, again and I found the button near of the top of Quiz Structure analysis. I'll check the txt file, thanks.
Cheers
Monico