OU Plans for adaptive mode

OU Plans for adaptive mode

by Tim Hunt -
Number of replies: 9
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
At the OU, we have been thinking about adaptive mode. We think adaptive mode is really important for making good quizzes that teach people, as well as testing them. However, we are not quite satisfied with how it works at the moment, and so have been thinking about how to make it better. The result is summarised here: http://docs.moodle.org/en/Development:Plans_for_adaptive_mode

I would be really grateful if you could all take a look at these proposals and give us your comments. Obviously, with a fairly major change like this, it is important to get it right. I know that lots of you have lots of experience in using the quiz and will be able to say how well this might work in practice. And the chances are, you can think of ways to improve what we suggest.

We definitely want to achieve a consensus on this. We feel that these changes, or something like them, are essential to make Moodle quiz work effectively for us, but it is not feasible for us to make these changes in an OU-only way. (But surely, I'm quiz maintainer, I can do anything I want! evil evil power! power!!! Of course not.)


Those of you who want to find an ulterior motive in this, or a conspiracy theory, need look no further than our other online assessment system: OpenMark (which, incidentally, is now Free Software). What we are proposing does make the Moodle quiz more like OpenMark. We think this is a good thing. We have been using OpenMark, (or systems like it) for over 10 years, and found that they work really well in practice. We think we have taken the lessons learned from developing OpenMark, and and used them to suggest a way to improve the Moodle quiz in a way that is consistent with the traditions of the Moodle quiz.


Finally, I will just note that I have timed starting this discussion really badly, because I am off on holiday from tomorrow until next Wednesday, so I won't necessarily be able to join in any discussion that takes place. I just hope I have explained everything clearly enough.

(I am heading up to Dundee, Scotland. If there are any Moodlers near there who are interested in meeting up, email me before about 9:00am tomorrow, UK time.)
Average of ratings: -
In reply to Tim Hunt

Re: OU Plans for adaptive mode

by Pierre Pichet -
Hi Tim
As usual you are not removing but adding new features.smile
Just a first comment:

After the student has got a question right, they can no longer change their answer

This seems consistent with stopping them after three wrong tries.

However , the student may want to know the feedback from other answers to this question. There could be more than one 100% answer. As long as the grade is the highest value, he could experiment.

Pierre

P.S. I am using the quiz more as lessons or homeworks than exam.





Average of ratings: -
In reply to Tim Hunt

Re: OU Plans for adaptive mode

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Tim,

As you know I am all for using the quiz module as a learning, not a testing activity and for me a user-friendly adaptive mode is all-important. I had a look at http://docs.moodle.org/en/Development:Plans_for_adaptive_mode and will have some comments to make about it, later on. But first of all I'd like to say I'm disappointed to find all your illustrations taken from the maths world, which is a foreign land to me. Can't we have some questions examples from the "normal", non-mathematical world, i.e. history, geography, modern languages, theology, and the like?wink

All the best,

Joseph

PS.- Hope the Dundee holiday went well and you enjoyed some bracing breeze up there.

Average of ratings: -
In reply to Joseph Rézeau

Re: OU Plans for adaptive mode

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Apologies for the lack of diversity in the examples. It was not intentional. However, I started life as a mathematician so have a natural bias, and anyway, I just copied an pasted some simple examples that my colleagues had devised so it is not even my bias.

The examples were just meant to be simple illustrations of what I was really trying to say, and I hope that substance of what we are proposing is equally relevant to all subjects.
Average of ratings: -
In reply to Tim Hunt

Re: OU Plans for adaptive mode

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
Just teasing... Although I have to admit to a natural aversion for anything mathematical.sad
Joseph
Average of ratings: -
In reply to Tim Hunt

Re: OU Plans for adaptive mode

by Jeff Forssell -
I find myself uncertain whether we should discuss here or enter suggestions into the DOC wiki.

I find it hard to believe that JR, the wizard of regular expressions, really has a problem with the math problem there, I think he's just trying to see to it that language colleagues don't get scared off. A noble pursuit smile

I love the suggested goals of cleaning out the frustrating uncertainty after a wrong answer and the possibility of Hints which I have missed from a kind of questions I have used BM ("before Moodle").

As far as not having hints that are based on which wrong answer, I don't think that is a problem. The Moodle questions usually have specific feedback options for various answers and that kind of directed corrective feedback can be put there.

I'm not specially interested in grades and points, but for those that are, I had a suggestion that students could buy hints with % deductions that could be 10% default but adjustable. I would also like it to be possible to be able to limit hint offers till after submitting an answer.

"After the student has got a question right, they can no longer change their answer": Here I agree with Pierre that students should be allowed to "play with" the question some more if they want. Maybe one could have "Your answer is correct and your points for this question will be recorded as XXp. If you want, you may test how the question reacts to other answers."

Alleviating confusion after first unsuccessful answer: I would suggest that after submission the returned page could be:

#1 the question ... .... . . .....?
Last Submitted answer: wrong Answer (which would not be changeable)
[ the feedback .............................]
Incorrect marks for this submission 0/1 (!?) Hint? (-15%)
New answer [default filled with old answer but editable][Submit]

Here it would be clear that the feedback would be about the old wrong answer. A new answer would be submittable direct without new page download/ server load. It could also handle well saving without submitting.

What's that little (!?) ? Well, I'm glad you asked! smile Since it's meaning is not obvious (sugestions for a better text or icon are wellcome) it should have a OverLib tootip like: "Wellcome to give us feedback to make this question, feedback or page better. Click this link to open an easy form in a new window!" I think all questions (even all web pages) should invite feedback (or "challenge" as Dennis D calls it) where the user can easily give feedback to the question developer/teacher about the question. It should open a form, which automatically has: the URL, the identity and state of the question, the students name, mail, and Java Script harvested info about OS, browser, plug ins, monitor and window size, where the user can write his question, suggestion, criticism, praise etc.




Average of ratings: -
In reply to Jeff Forssell

Re: OU Plans for adaptive mode

by Mahmoud Kassaei -

We may allow students to change their answer while they have tries left (the maximum number of tries is not reached, default = 3). My question is why a student wants to risk his/her marks after answering a question correctly. I would understand that might be the case when a question has been answered as partially correct, the student has a number of tries left and therefore wants to get the correct answer (e.g.: Some student may think, “I have one try left, I would get a penalty for the next try, I may get it right if the given hint make sense to me …).

I like the idea of students buying hints for a certain penalty factor. You may even go further and offer a few hints at the same level, for instance:

…


Your answer is incorrect (choose a hint)


Hint1 (-%5)


Hint1 (-%10)


Hint1 (-%15)

I have originally suggested that each hint has its own penalty attached to it ( e.g.: the teacher might say: “my first hint has a 0.1 penalty factor for being very basic and only jogging the students’ memory, whereas the second one has 0.2 penalty factor, because it guides the students towards better understanding of the topic”). However, after a discussion with colleagues, for consistency reasons, we decided to use the same penalty factor for each hint.

Average of ratings: -
In reply to Mahmoud Kassaei

Re: OU Plans for adaptive mode

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
Mahmoud > My question is why a student wants to risk his/her marks after answering a question correctly.

This is because - like most quiz users - you are thinking in terms of testing, not learning! When using quizzes with my students I always insist that, after finding the correct answer they systematically try the "wrong" answers in order to confirm their opinion of why they are wrong. This strategy is only valid, of course, if each and every answer (correct or incorrect) has a meaningful feedback message attached to it. Obviously, for most teachers only interested in testing, not learning, feedback messages are either non-existent or of the "wrong, try again" type, which does not help. What I try to teach my students is that what matters is not finding the correct answer (and the marks attached to it) but to understand why that answer was correct, or more correct than others, and - equally important - why the incorrect answers were incorrect. Well, maybe that is another one of those lost causes for which I preach in the wilderness.dead

Joseph


Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: OU Plans for adaptive mode

by Jeff Forssell -
Hope I run into you in that wilderness big grin! I don't think we're the only ones, but wish there were more. But I suppose most teachers' bosses are more interested in grades. (Sure, if there's time for it, learning is OK too clown)
Average of ratings: -
In reply to Joseph Rézeau

Re: OU Plans for adaptive mode

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

they systematically try the "wrong" answers



I take your point that this is potentially a useful learning tool, but ...

First, it seems to imply multiple choice questions, since the number of possibilites for other questiontypes is unlimited.

Second, I think we would achieve the same learning opportunities using the general feedback, which we feel should be an overall summary of why the right answer is right and other answers wrong.

Something like this: http://www.open.ac.uk/openmarkexamples/p4_3.shtml (Sorry, it is a maths example again, that was not intentional wink )

This is certainly easier for the student than having to select all of the different options themself to see the different bits of feedback. Also the teacher can produce a neat couple of paragraphs summary, but this is more work.
Average of ratings: -