Feedback in Formulas question type

Feedback in Formulas question type

by Dominique Bauer -
Number of replies: 15
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

I think that this thread belongs to the Quiz forum and not the Mathematics tool forum. In fact, I am afraid some people perceive the Formulas question as being useful only for short basic mathematics questions. Having a default input box of 40px which allows entering only two or three digit numbers is a reflection of this.

The Formulas question has great potential in other sciences and in engineering where the problems to solve are more complex and considerably longer to solve. For these subjects, each part of a Formulas question actually represent a step in the solution, and each step by itself may be quite complex. Hence, each part should be as complete as possible, and it would be a great improvement to the Formulas question if each part could have the usual three feedbacks (correct, partially correct and incorrect) as well as a general feedback. This would allow the required flexibility of feedback for each step of the solution.

Average of ratings: -
In reply to Dominique Bauer

Re: Feedback in Formulas question type

by Dave Morgan -

I agree that this would be valuable functionality for those building questions with many parts. It would also be nice to be able to 'Check' each answer as it is input, rather than having to fill in all the boxes before you can get feedback. The Formulas question type is a powerful tool.

Average of ratings: -
In reply to Dave Morgan

Re: Feedback in Formulas question type

by Jean-Michel Védrine -

If your questions use the adaptive behaviour, you can easily check only the answer for one part :  just press the check button, you will not receive any penalty for parts where the input is left blank (and you will also not receive any penalty for a wrong answer that has already be given for that part).

This is why in my opinion it is not necessary to have a check button for each part (and I must add that some years ago I tried to make a version with a check button for each part, but I didn't succeed, maybe because I didn't saw the right way to do it ?)

Average of ratings: -
In reply to Dominique Bauer

Re: Feedback in Formulas question type

by Jean-Michel Védrine -

Hello Dominique,

I don't have the right to move a thread between 2 forums. Maybe a moderator with more powerful rights can do that ?

As I said in that thread it would be easy to change the way the part's feedback is displayed but my main concern about this change is that I am fearing it would break some existing questions

EDIT: Oops I missed that your proposal is in fact to have a combined feedback for each part. That would be a lot more work! Think: we must change the structure of one table, add code to upgrade, backup/restore and import/export also to the edit page and renderer, and finally all the needed code in the formulas question class... not counting all the tests to verify that this is working.

Having a per question combined feedback is easy because nearly all the code is already in the quiz engine but to have a per part combined feedback we would have to redo all that code !

Average of ratings: -
In reply to Jean-Michel Védrine

Re: Feedback in Formulas question type

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Hi Jean-Michel,

Although a full fledged combined feedback may not be an option at this time, each part already has a feedback that is displayed when the part's answer is incorrect. Would it be a lot of work to duplicate and modify the code so that a second feedback is displayed when the part's answer is correct?

Therefore, instead of having a single feedback for an incorrect answer, there would be two feedbacks: one for a correct answer and one for an incorrect answer.

---

I agree with you that it is not necessary to have a check button for each part. The adaptive behavior and the check button work perfectly as you describe it and, if you will allow me, I will put your exact words in the documentation.

The only comment I have is that, for long questions, there is a lot of scrolling up and down because the check button is at the bottom. A fixed button (style="display:fixed;") on the side of the window would be convenient.

Average of ratings: -
In reply to Dominique Bauer

Re: Feedback in Formulas question type

by Jean-Michel Védrine -

Even if I don't think I will find the time to code this in a very near future, I think that the idea to add a "Part combined feedback" is a very good one and should be done.

So I have created this tracker issue.

I will create a separate branch in my Github repo and try to work on it when I get some spare time until it is finished.

Note that as this feature needs a change in the database to add new fields it will only be available for the latest version of the Formulas question type (the one for Moodle 3.0 ... 3.4) and not for the Moodle 2.6 ... 2.9 version.


Average of ratings: Useful (1)
In reply to Jean-Michel Védrine

Re: Feedback in Formulas question type

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Hello Jean-Michel,

Let me ask you if, in addition to the "Part combined feedback (correct, partially correct, incorrect)", you could also include a "Part general feedback". Here's why.

My recipe for a complete feedback is now only possible for a single-part Formulas question. It would be nice if it were possible for every part of a multi-part Formulas question. Here is what could be the improved recipe:

  • Give the first half of the solution in the Part feedbacks for correct, partially correct and incorrect responses. For a partially correct response, a message such as "Check units" can be added (by the teacher).
  • Give the second half of the solution in the Part general feedback.

In this way, the same complete feedback is given to all students, regardless of whether their final response is correct or incorrect. However, only part of the solution is given as a hint for an incorrect response on the first tries.

Imho, it is important that all students get the same complete feedback as they proceed through the solution of the problem, i.e. through the several parts of the question, and also that they can refer to a complete feedback, depending on the review options, after the quiz is closed, for review and studying.

This is illustrated by the simple example below (problem type questions can be much longer):

QUESTION: Calculate the area, A, of a rectangle with sides a = 2 m and b = 3 m. Say only two tries are allowed.

1st try, if the response is correct:

  • Part feedback for any correct response
To calculate the area, multiply the length of side a by the length of side b.
  • Part general feedback
So we have A = a × b = 2 m × 3 m = 6 m2.

1st try, if the response is incorrect:

  • Feedback for any incorrect response
To calculate the area, multiply the length of side a by the length of side b.

2nd try, whether the response is correct or incorrect:

  • Part feedback for any correct response or part feedback any incorrect response
To calculate the area, multiply the length of side a by the length of side b.
  • Part general feedback
So we have A = a × b = 2 m × 3 m = 6 m2.
Average of ratings: -
In reply to Jean-Michel Védrine

Re: Feedback in Formulas question type

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Hello Jean-Michel,

I should have read your tracker before writing my previous post. I understand that you intend to add the three feedbacks (correct, partially correct and incorrect) as well as a general feedback to each part. It would be ideal!

Average of ratings: -
In reply to Dominique Bauer

Re: Feedback in Formulas question type

by Jean-Michel Védrine -

Hello Dominique,

Yes this is how it will work once it is done, there will be 4 editors for each part : the general feedback and the 3 combined feedback.

Can I ask your opinion about a detail : the "Feedback" field that already exists now will become the future "Part general feedback" but given that currently this feedback is only displayed for incorrect student's response, should I move the content to the new combined part incorrect feedback field during the database upgrade or let it as it is now ?

Thinking on how I will done this change I also realized that I don't know how it should work when the question is not using the adaptive behaviour.

The problem is that contrary to adaptive behaviour when the question is using another behaviour we don't have the details of the student attempt on a per question's part basis, we only have it on a per question basis, So I see only 2 solutions

  • Display all the part's combined feedback according to the question status (this means that for instance if the student's response to the question is partially correct the student will see all partially correct parts feedbacks even if the response to some part is completely correct or incorrect.
  • Don't display any part combined feedback if the behaviour is not adaptive
Unfortunately I think it is not possible to do better unless some developer publish "multipart aware" behaviours similar to what Tim Hunt has done for the adaptive behaviour.
Average of ratings: Useful (1)
In reply to Jean-Michel Védrine

Re: Feedback in Formulas question type

by Jean-Michel Védrine -

Well in fact I am stupid. As I have calculated the fraction for each part I am able to find the relevant combined feedback for each part even if the behaviour is not adaptive.

One day I absolutely need to refactor my spaghetti renderer code because I get lost in my own code, and if someone else want to modify it in the future he will certainly curse me evil

Average of ratings: -
In reply to Jean-Michel Védrine

Re: Feedback in Formulas question type

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Jean-Michel,

The "Feedback" field that already exists now will become the future "Part general feedback" but given that currently this feedback is only displayed for incorrect student's response, should I move the content to the new combined part incorrect feedback field during the database upgrade or let it as it is now?

As I understand it, from the teacher's point of view, the content should be moved to the new "combined part incorrect feedback". From the developer's point of view, you should do what is easiest and this should take precedence over the choice of teachers. I do not think it's a big deal for teachers to manually move the content of the new "part general feedback" to the "combined part incorrect feedback" field.

I don't know how it should work when the question is not using the adaptive behaviour.

It would be great even if it only works for adaptive behaviour.

As I have calculated the fraction for each part I am able to find the relevant combined feedback for each part even if the behaviour is not adaptive.

It would be great if it works just as well for other behaviours.

...he will certainly curse me.

You literally saved the Formulas question when Moodle upgraded to version 2.1, made great improvements to the question, and maintained it for many years. Nobody will curse you. On the contrary, people will say that you are the best.

Average of ratings: -
In reply to Dominique Bauer

Re: Feedback in Formulas question type

by Bernat Martinez -

Thanks,  Jean-Michel and Dominique, for your strong commitment to improve formulas question.

I have been following with great interest this thread. The role of the Feedback in formulas has been  an issue that I have avoided for many years (5?)  as I have considered me not being able to handle it.  Just 2 years ago I decided to discuss it in a training teachers course and in order to auto-clarify ideas I created this working document

Nowadays, I'm still not feeling confident I'm using  Feedback in formulas in an effective way, didactically speaking, that's to say in order to help students to improve understanding. 

Maybe we could recognise that the Formulas Feedback system has too many variables. For this reason I consider that we should reflect before adding more variables, and to think about the principle that sometimes "less is more" . 

In one week, we are going to start a training course in building formulas question and this is an issue I would like to discuss there with  more than 20 participants. So I would like to invite you both to follow the discussion there.

Average of ratings: Useful (1)
In reply to Bernat Martinez

Re: Feedback in Formulas question type

by Jean-Michel Védrine -

Hello Bernat,

I understand your concern about Formulas Feedback system having too many variables.

But at the same time Dominique has presented an interesting scenario to use part combined feedback with students.

In fact this new feature is nearly finished because it has been raining here in Montpellier so I was forced to stay at home and needed something interesting to do. The result is here  and you can see that a lot of changes were needed (more than 400 lines of code !)

But I don't think users of the formulas question should worry to much about this new feature, because if you don't click on the "Show more..." link there is absolutely no change in the Formulas question  edit page.

Of course if you click on that link there is now 4 editors instead of simply one:

Formulas part combined feedback

So I really think this is a good feature, and if you want you can completely ignore it.

But before releasing a new version with this feature, I need to do a lot of tests to be sure I haven't broken anything writing all these lines.

Average of ratings: Useful (2)
In reply to Jean-Michel Védrine

Re: Feedback in Formulas question type

by Bernat Martinez -

Thanks Jean-Michel for your clear explanation. 

In fact, the part feeback is the most I use and for me works fine as it is now. I understand Dominique position and agree that this will lead to the possibility of giving a more detailed feedback for student. But from the teacher point of view represents more potential work, .....and building formulas questions is so exhausting that I feel the newcomer teacher could feel overwhelmed. Anyway I will try to adapt to the new scenario and I'm sure that in the end I will use it.

On the other hand, I'm not very fond of using combined feedback. I try to use it few times as it has forced us to do some additional work.

It happens that , not always, just in some occasions ( I have not discovered why, maybe you could help me ), it adds some text to the feedback ( e.g. "your answer is not correct"). As we are using a multilang Moodle, it always happen that this text appears in a different language that the one used by the student. And teachers complain about the mixing of languages gives a bad impression. So, we have to rewrite the text using the multilang tags.

So please, could be possible the combined part feedback not add any text?

Average of ratings: -
In reply to Bernat Martinez

Re: Feedback in Formulas question type

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Hello Bernat,

If you look at the examples of the tutorial and the original examples of Wai Lau, you will see that a basic Formulas question can be extremely simple and that newcomer teachers should not feel overwhelmed as they start to familiarize themselves with the Formulas question.

The "Part combined feedback" will not differ much from the current part feedback that you are using. In fact, the current part feedback will simply be renamed "For any incorrect response", and, if you wish, you can leave the other fields blank (Part general feedback, For any correct response, For any partially correct response).

Text such as "Your answer is not correct" is only proposed as a default for convenience. You can easily change it or simply delete it. It appears by default in the language that your course is set.

Average of ratings: -
In reply to Bernat Martinez

Re: Feedback in Formulas question type

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Hello Bernat,

You raise an important point, which you address in your working document, namely about how to set up a quiz for formative activities or for summative activities. As you mentioned, both the quiz settings and the question settings must be carefully chosen and there are many parameters to set! On the other hand, once you are confident with the settings, these give you the flexibility to deal with many different cases. I take note of your comments because I think there is a gap on this point in the documentation for the Formulas question.

As Jean-Michel mentioned, there will be no change in the Formulas question edit page unless you click on "Show more...", and users should not worry too much about the proposed functionality, i.e. combined feedback for parts. This new feature will be available for more complete feedback that is needed in many cases for better pedagogy (both in formative and summative activities, but especially in formative activities).

Thank you for your invitation. I will follow with great interest the discussion in the training course on building Formulas question.

Average of ratings: -