We are searching for good practice strategies in administration of thousands of quiz questions.
Situation: There are thousand (more than 10.000) quiz questions. They should be stored in one course as a typ of quiz question library. We need several types of sorting: severity, topic, grammar, ...
Who has similar requirements and what are your strategies and experiences?
Tim, are there any plans for additional tools?
The OU will have similar requirements soon I hope, but we certainly don't have 10000 questions yet. Wow!
Currently, all Moodle has is the ability to publish question categories from one course, so they can be used in all other courses. This is a step in the right direction, but not nearly flexible enough for people who want to establish significant question banks.
More flexible sharing is certainly required. So you can share only with a certain set of courses, for example. I don't think we even have a clear picture how much flexibility is required here. It would be good if someone could have ago a drafting a requirements document. Then to implement it, we need to come up with a solution that works with the Moodle 1.7 roles system. Currently the roles system is not really aware of questions and question categories. The relevant capabilities are assigned a the coures level.
Then there is metadata on questions. That is definitely needed. There are some plans out there for adding metadata to everything in Moodle, but I don't remember the specifics. Anyway, we can either wait for one of those efforts to succeed, or we can write our own quiz-specific code. The first option is probably better, but who knows how long we will wait?
The one thing I think is a small change that I would like to do quite soon is full-text search on questions. There would be a simple text field on the question bank screen, and if you type a work in there, in only shows you quiz questions that contain that word somewhere in the question text.
So the summary is, we know there is a problem. We know roughly the sort of things that needs to be done to fix it. We don't have any detailed plans that are generally supported and ready to be implemented. And none of this is very high up my priority list, so if you want this now, don't wait for me to do it.
It would be helpful for people to discuss how they manage with the tools currently available.
Currently, all Moodle has is the ability to publish question categories from one course, so they can be used in all other courses. This is a step in the right direction, but not nearly flexible enough for people who want to establish significant question banks.
More flexible sharing is certainly required. So you can share only with a certain set of courses, for example. I don't think we even have a clear picture how much flexibility is required here. It would be good if someone could have ago a drafting a requirements document. Then to implement it, we need to come up with a solution that works with the Moodle 1.7 roles system. Currently the roles system is not really aware of questions and question categories. The relevant capabilities are assigned a the coures level.
Then there is metadata on questions. That is definitely needed. There are some plans out there for adding metadata to everything in Moodle, but I don't remember the specifics. Anyway, we can either wait for one of those efforts to succeed, or we can write our own quiz-specific code. The first option is probably better, but who knows how long we will wait?
The one thing I think is a small change that I would like to do quite soon is full-text search on questions. There would be a simple text field on the question bank screen, and if you type a work in there, in only shows you quiz questions that contain that word somewhere in the question text.
So the summary is, we know there is a problem. We know roughly the sort of things that needs to be done to fix it. We don't have any detailed plans that are generally supported and ready to be implemented. And none of this is very high up my priority list, so if you want this now, don't wait for me to do it.
It would be helpful for people to discuss how they manage with the tools currently available.
I have nearly 2000 math questions in a Moodle 1.6.3 question database. The questions are divided into 23 categories (and subcategories) such as "geometry/ lines and angles" and "algebra/special trinomials". The questions are used by tutors to create highly-targeted practice quizzes for specific students based on the student's specific strengths and weaknesses. For example, if a student is weak in "geometry/lines and angles", the tutor will create a quiz consisting of easy problems from this category. If another student is strong in algebra, the tutor will create a quiz consisting of more challenging algebra problems, etc.
In this pattern, which I call Tutoring Center, searching by content is useless. The tutor needs to be able to make a quick visual scan of the problems in a chosen category and select the ones having the desired level of difficulty. For this, Joseph Rezeau's category preview enhancement (MDL- 4793) is absolutely indispensable (see screenshot).
The ability to tag questions with user-defined attributes (i.e., metadata) would enable me to classify problems according to their level of difficulty. Currently, I determine difficulty by visual inspection.
As the screenshot shows, I added a cross-reference list to the category preview so that each question lists the quizzes in which it has been used within this course. This helps me avoid repeating the same question on different quizzes.
There is another requirement of any enterprise quiz database that is often overlooked. Question creation, update, deletion, and (re-)categorization should only be possible for users having the role of Question Database Administrator (QDBA). For other users, the questions should be read-only and export should be disabled. We don't want copies of questions to proliferate! We want to share questions in the true sense of sharing (copying is not sharing). We're trying to get away from the stovepipe model of teaching where every tutor has his own private question database! Errors in questions are fixed once and are instantaneously fixed everywhere the question is used. Category changes (a very common occurrence) are made in one place and everyone sees them. With these requirements met, we can begin to develop policies and procedures that will allow us to grow an enterprise question database to 100,000 questions. For example, we use a special forum for reporting question bugs to the QDBA team.
Here's how I implemented the Tutoring Center pattern (pre 1.7, without roles):
In this pattern, which I call Tutoring Center, searching by content is useless. The tutor needs to be able to make a quick visual scan of the problems in a chosen category and select the ones having the desired level of difficulty. For this, Joseph Rezeau's category preview enhancement (MDL- 4793) is absolutely indispensable (see screenshot).
The ability to tag questions with user-defined attributes (i.e., metadata) would enable me to classify problems according to their level of difficulty. Currently, I determine difficulty by visual inspection.
As the screenshot shows, I added a cross-reference list to the category preview so that each question lists the quizzes in which it has been used within this course. This helps me avoid repeating the same question on different quizzes.
There is another requirement of any enterprise quiz database that is often overlooked. Question creation, update, deletion, and (re-)categorization should only be possible for users having the role of Question Database Administrator (QDBA). For other users, the questions should be read-only and export should be disabled. We don't want copies of questions to proliferate! We want to share questions in the true sense of sharing (copying is not sharing). We're trying to get away from the stovepipe model of teaching where every tutor has his own private question database! Errors in questions are fixed once and are instantaneously fixed everywhere the question is used. Category changes (a very common occurrence) are made in one place and everyone sees them. With these requirements met, we can begin to develop policies and procedures that will allow us to grow an enterprise question database to 100,000 questions. For example, we use a special forum for reporting question bugs to the QDBA team.
Here's how I implemented the Tutoring Center pattern (pre 1.7, without roles):
- The question database lives in course 1
- The QDBA's are teachers in course 1
- all categories are published
- images used in questions are site files
In reply to Tim Hunt
這一討論區的貼文已經被移除
這一討論區的貼文已經被移除且無法再被存取
Ray,
It's not a complicated patch. Joseph attached patch files to MDL- 4793. Before category preview, I printed all 2000 questions (one per page) and filed then alphabetically within category in four thick 3-ring binders. In making up a quiz, I would flip through the binder looking for appropriate questions. However whenever I made a change to the Moodle question database, I had to make a parallel change in my filing system. For example, when I recategorized a question in Moodle, I had to physically move a page from one binder/section to another binder/section. Ugh.
It's not a complicated patch. Joseph attached patch files to MDL- 4793. Before category preview, I printed all 2000 questions (one per page) and filed then alphabetically within category in four thick 3-ring binders. In making up a quiz, I would flip through the binder looking for appropriate questions. However whenever I made a change to the Moodle question database, I had to make a parallel change in my filing system. For example, when I recategorized a question in Moodle, I had to physically move a page from one binder/section to another binder/section. Ugh.
I will be checking this out to see if it is helpful/useful to our users. I had some trouble finding the patch because I was looking at the tracker and the files were not located there. I took the liberty of copying them from http://moodle.org/bugs/bug.php?op=show&bugid=4793&pos= to http://tracker.moodle.org/browse/MDL-4793.
