Quiz

Administration of some thousands of questions

 
This discussion has been locked because a year has elapsed since the last post. Please start a new discussion topic.
Picture of Ralf Hilgenstock
Administration of some thousands of questions
Group Core developersGroup Particularly helpful MoodlersGroup Translators
We are searching for good practice strategies in administration of thousands of quiz questions.

Situation: There are thousand (more than 10.000) quiz questions. They should be stored in one course as a typ of quiz question library. We need several types of sorting: severity, topic, grammar, ...

Who has similar requirements and what are your strategies and experiences?


Tim, are there any plans for additional tools?
 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Administration of some thousands of questions
Group Core developersGroup Documentation writersGroup Particularly helpful MoodlersGroup Plugin developers
The OU will have similar requirements soon I hope, but we certainly don't have 10000 questions yet. Wow!

Currently, all Moodle has is the ability to publish question categories from one course, so they can be used in all other courses. This is a step in the right direction, but not nearly flexible enough for people who want to establish significant question banks.

More flexible sharing is certainly required. So you can share only with a certain set of courses, for example. I don't think we even have a clear picture how much flexibility is required here. It would be good if someone could have ago a drafting a requirements document. Then to implement it, we need to come up with a solution that works with the Moodle 1.7 roles system. Currently the roles system is not really aware of questions and question categories. The relevant capabilities are assigned a the coures level.

Then there is metadata on questions. That is definitely needed. There are some plans out there for adding metadata to everything in Moodle, but I don't remember the specifics. Anyway, we can either wait for one of those efforts to succeed, or we can write our own quiz-specific code. The first option is probably better, but who knows how long we will wait?

The one thing I think is a small change that I would like to do quite soon is full-text search on questions. There would be a simple text field on the question bank screen, and if you type a work in there, in only shows you quiz questions that contain that word somewhere in the question text.

So the summary is, we know there is a problem. We know roughly the sort of things that needs to be done to fix it. We don't have any detailed plans that are generally supported and ready to be implemented. And none of this is very high up my priority list, so if you want this now, don't wait for me to do it.

It would be helpful for people to discuss how they manage with the tools currently available.
 
Average of ratings: -
Picture of John Isner
Re: Administration of some thousands of questions
 
I have nearly 2000 math questions in a Moodle 1.6.3 question database. The questions are divided into 23 categories (and subcategories) such as "geometry/ lines and angles" and "algebra/special trinomials". The questions are used by tutors to create highly-targeted practice quizzes for specific students based on the student's specific strengths and weaknesses. For example, if a student is weak in "geometry/lines and angles", the tutor will create a quiz consisting of easy problems from this category. If another student is strong in algebra, the tutor will create a quiz consisting of more challenging algebra problems, etc.

In this pattern, which I call Tutoring Center, searching by content is useless. The tutor needs to be able to make a quick visual scan of the problems in a chosen category and select the ones having the desired level of difficulty. For this, Joseph Rezeau's category preview enhancement (MDL- 4793) is absolutely indispensable (see screenshot).

The ability to tag questions with user-defined attributes (i.e., metadata) would enable me to classify problems according to their level of difficulty. Currently, I determine difficulty by visual inspection.

As the screenshot shows, I added a cross-reference list to the category preview so that each question lists the quizzes in which it has been used within this course. This helps me avoid repeating the same question on different quizzes.

There is another requirement of any enterprise quiz database that is often overlooked. Question creation, update, deletion, and (re-)categorization should only be possible for users having the role of Question Database Administrator (QDBA). For other users, the questions should be read-only and export should be disabled. We don't want copies of questions to proliferate! We want to share questions in the true sense of sharing (copying is not sharing). We're trying to get away from the stovepipe model of teaching where every tutor has his own private question database! Errors in questions are fixed once and are instantaneously fixed everywhere the question is used. Category changes (a very common occurrence) are made in one place and everyone sees them. With these requirements met, we can begin to develop policies and procedures that will allow us to grow an enterprise question database to 100,000 questions. For example, we use a special forum for reporting question bugs to the QDBA team.

Here's how I implemented the Tutoring Center pattern (pre 1.7, without roles):
  • The question database lives in course 1
  • The QDBA's are teachers in course 1
  • all categories are published
  • images used in questions are site files
I want to reimplement Tutoring Center in 1.7 using roles, but I would definitely like to want to hear suggestions on how to do it.



 
Average of ratings: -
Ray Lawrence
Re: Administration of some thousands of questions
Group Particularly helpful Moodlers
Tim,

Is there any space in your schedule to incorporate the category view enhancement above?
 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Administration of some thousands of questions
Group Core developersGroup Documentation writersGroup Particularly helpful MoodlersGroup Plugin developers
Sorry, no. Not for the foreseeable future.
 
Average of ratings: -
Pierre Pichet
Re: Administration of some thousands of questions
Group Core developersGroup TestersGroup Translators

Back to work from two weeks in the caribbean seasmile , I will resubmit (2-3 weeks from now) an improve version of the category display once I will have assimilated the new HTML interface (http://docs.moodle.org/en/Development:lib/formslib.php).

Pierre

P.S. Tim, have you any technical constraints about this kind of display ?

 
Average of ratings: -
Picture of John Isner
Re: Administration of some thousands of questions
 
Ray,
It's not a complicated patch. Joseph attached patch files to MDL- 4793. Before category preview, I printed all 2000 questions (one per page) and filed then alphabetically within category in four thick 3-ring binders. In making up a quiz, I would flip through the binder looking for appropriate questions. However whenever I made a change to the Moodle question database, I had to make a parallel change in my filing system. For example, when I recategorized a question in Moodle, I had to physically move a page from one binder/section to another binder/section. Ugh.
 
Average of ratings: -
Anthony Borrow
Re: Administration of some thousands of questions
Group Core developersGroup Moodle HQGroup Plugin developersGroup Plugins guardiansGroup Testers
I will be checking this out to see if it is helpful/useful to our users. I had some trouble finding the patch because I was looking at the tracker and the files were not located there. I took the liberty of copying them from http://moodle.org/bugs/bug.php?op=show&bugid=4793&pos= to http://tracker.moodle.org/browse/MDL-4793.
 
Average of ratings: -
Picture of John Isner
Re: Administration of some thousands of questions
 
It reminds me of the time a friend of mine, a serious stamp collector, misplaced an extremely valuable stamp. After frantically searching for several hours, he found it stuck to the bottom of his shoe.
 
Average of ratings: -
Picture of Joseph Rézeau
Re: Administration of some thousands of questions
Group Core developersGroup Plugin developersGroup TestersGroup Translators
John > Joseph Rezeau's category preview enhancement (MDL- 4793) is absolutely indispensable

Sorry, John, not my own contribution. Render unto Caesar...wink

Joseph

 
Average of ratings: -
Picture of John Isner
Re: Administration of some thousands of questions
 
Sorry, Joseph. I meant Pierre Pichet !
 
Average of ratings: -