Time limit per question vs Timed sections

Time limit per question vs Timed sections

by Enrique Castro -
Number of replies: 9
Picture of Core developers Picture of Particularly helpful Moodlers

Hi all,

I'm involved in a project about improving some moodle features for use in Spanish universities. One question raised is the ability to limit answering time in a quiz on a per question basis. In fact, there is a thread not too old,  Time limit per question - implementation guide and some other comments about this topic a least from moodle 1.5, the first version I used.

Analyzing use cases and ways to implement the features that teachers demand I've though about quiz sections. Quiz sections can have a heading and can control de ordering of questions. In addition each section can have a different page break usage.

So, I've realized that it may be simpler for the teacher but ever more powerful and flexible to set partial time limits per section, rather than per single questions.

If questions per se are timed, then that makes mandatory to use sequential single question per page navigation. And interface is cluttered with boxes to specify the time fo each question (as currently the grade).

But with sections for instance, you may have a quiz with 25 MCQ questions and 5 essay questions about two different topics.  Set three section in the quiz, one with de 25 MCQ questions, and establish 1 min per question in the section (just one setting). And page breaks after each question: single question per page.

Or you may have 30 min for the whole section, present the 25 MCQ questions in a single page and let the students to manage their own time.

You may have the essay questions in two pages, one with two questions and a limit of   30 min for both two questions. Ande the other three questions en other section, single question per page and 10 min per question (page) in that section.   

I see that establishing time limits at the section level (been able to set either whole time for the section, or configuring time-per-question and the section time is the sum of the questions) is much more usable and more flexible.

timed questions mockup

We have got funds, so we definitely will develop a modification in moodle 4.x Quiz to include timed questions. We just want to try to develop a modification that could be integrated in the core, if seen as useful and convenient by enough users. 

What I would like to ask to other teachers in other environments how would you like more. Either a time set strictly per single question or this "Timed sections". Just discuss use cases in you schools or Universities. 

Average of ratings: Useful (1)
In reply to Enrique Castro

Re: Time limit per question vs Timed sections

by Enrique Castro -
Picture of Core developers Picture of Particularly helpful Moodlers

Hi again, 

If time limit is set per question then the quiz editing page need to provide a box to set the time. Like the box for actual marks for the question in the quiz.

timed questions

This is quite simple for a 3 questions quiz. But in a 100 questions quiz I see quite possible to add a time where a grade is intended or the other way around. And more difficult to see that a particular question has 10 min, or either 1 min or 1 h: more error prone.

Going the time-per-question route requires to add a "expected answer time" property to each question, in the definition of the question. And then a box to set the actual time in a quiz. As with grades, the time allocated may be different for the same question when used in separate quizzes. 

In addition, this route requires to add settings in the quiz config form to specify the time limit for the whole time. No more a single number, may be the sum of all questions times or other options. And only applicable for sequential one-question-per-page navigation.

Timed sections would not need any modification, even more options, in the quiz config form. Just a validation in quiz editing page to ensure that  the sum of times allocated is shorter  (or equal) than the whole quiz time limit. 

I would really appreciate any comment on the advantages or problems you see in each approach to set partial time limits within a quiz. Or comments on the usefulness or applicability of the very concept of timed questions. 

Average of ratings: -
In reply to Enrique Castro

Re: Time limit per question vs Timed sections

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

I've never understood the pedagogical rationale for wanting to limit the answering time per question. What are the arguments in favour of this feature?

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Time limit per question vs Timed sections

by Enrique Castro -
Picture of Core developers Picture of Particularly helpful Moodlers

I know, I know. I'm very much with you.
But there is a quite large group of teachers that have definite need for this. Here in Spain we even have administrative requirements in Universities. When used moodle in other environments, for some Government examinations, it's a legal issue. 

I just want to compromise. That's the reason I want to put the limit at page, not single questions.  Most teachers want to separate topics (have 30 min for this topic, 45 for that other, with many questions in each topic). Or separate question types. This section (with time) for MCQ questions, this block (with time) for essay questions.

If a teacher, for whatever reason, need to set a time limit per question just present the quiz as single question per page. But setting the limit a page level allows to present a page with N questions and let to student complete those questions in any order, without timing per question just for the whole page.

The timing limit is going to be implemented. There is a clear demand. It will be a reality. Implemented as a strictly per-question timing, it opens doors for not so nice uses.  I prefer the time limit implemented at page level.

Average of ratings: Useful (1)
In reply to Enrique Castro

Re: Time limit per question vs Timed sections

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I am just wondering if there is enough flexibility for you to develop this as a Quiz access rule plugin? That would let you have the feature you want now without wating for it to get integrated into the next version of standard Moodle.

  1. $rule->setup_attempt_page is probably sufficient for you to initialise the timer and other JavaScript on each page.
  2. And, $rule->description() is probably enough for you to show:
    • For teachers, for quizzes set to sequential navigation, a link to a separate screen where the can set up the timing per page. (Which can be stored in your own DB table.)
    • For students, in a quiz with this turned on, a description about what the timing will be.

When we had some slighly weird requirements related to timeing of our oneline exams (break the total time down into separate phases with different names) I was able to do it in an access rule.

It's not one we have shared with the world, becuase it is rather idiocyncratic, but in case it helps you, here is the main part of the code: https://gist.github.com/timhunt/f8c5ddc2a78d1226d249bed17ba4fdfc. One thing to note about this is, while we showed our own timer custom display, we still make the rule automatcally set Moodle's standard time limit, and we left Moodle's standard timer running during the attempt, but hidden, so that Moodle would take care of all the standard functionality related to submitted the quiz when the total time had expired.

Average of ratings: -
In reply to Tim Hunt

Re: Time limit per question vs Timed sections

by Enrique Castro -
Picture of Core developers Picture of Particularly helpful Moodlers
Oooh!!, thanks a a lot, Tim! This is terrific, most helpful!!
I was not aware that access rules could support this behaviour. I tought had  other purposes and capabilities. I've written some custom access rules, but always for checking things before starting the attempt properly. Perhaps that focus on not allowing undesired attempts was fooling me.

We will definitely explore this route. And having a plugin, integration in core is not a crucial issue for use.
Thanks again Tim, your are great!! 
Average of ratings: -
In reply to Enrique Castro

Re: Time limit per question vs Timed sections

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Arguably, this is an abuse of what access rules were intended for, but if it works ...
Average of ratings: -
In reply to Tim Hunt

Re: Time limit per question vs Timed sections

by Enrique Castro -
Picture of Core developers Picture of Particularly helpful Moodlers

If it works, and it's a plugin, not mangling core, it's good enough, IMHO.

Some teachers are quite insistent that they need time-per-question and a general setting for that (expected answering time) in question definition UI and in quiz-editing page. There are some strong opinions. Perhaps we need to discuss again some months from now. But I see an starting point. 

Whatever the code, once built, refactoring as plugin or the reverse, move from plugin to core is a lot easier with working functional  code than just starting from scratch. 

Average of ratings: -
In reply to Enrique Castro

Re: Time limit per question vs Timed sections

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

You have shown the proposed user-interface for teachers. That is a good start.

(Of cousrse, no-one ever complained that the Edit quiz page was too simple and easy to understand, and this proposal makes it more complicated, so we would want to be sure that increasted complixity was worth it.)

To fully understand the proposal, you would also need to show us:

  1. What students will see during the attempt. For example
    • what happens if students move between secitons 2 and 3, doing a bit of work on one, then a bit on the other, and contually swtiching?
    • what happens after student has used up all their time on section 2, but still have not finished section 3.

  2. Anywhere you have time limits, accessibility rules means that you need to be able to overrides those time limits for some students. How do you see that working?

  3. What changes will be required to Moodle's database structure to make this work?
Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Time limit per question vs Timed sections

by Enrique Castro -
Picture of Core developers Picture of Particularly helpful Moodlers

Hi Tim:

I don't want to add even more options to mod_from config form or make the user interface much more complicated.  In my view, initially the teacher just see the Temporize checkbox in Section header(like randomize) , only it ticked JS makes visible the config tools.

2. What happens if students move between sections 2 and 3, doing a bit of work on one, then a bit on the other, and contually swtiching?

Temporalization can only work (and be an option to be set by teacher) if the quiz is configured with navmethod = sequential. Students can't go back and forth.  

2. what happens after student has used up all their time on section 2, but still have not finished section 3. 
What we are planning is that the student see two timers in the current quiz  page (either a single question or N). One timer for the whole quiz (the timelimit for the quiz, if set). One timer for the page/question: how much remaining time I have for this page/question.
If the student answers all questions, she simply click on "Next" and go to net page/question. The timer is reset for the new page (no "time credit" if completed the  previous page in less time than expected).

If the page timer gets to 0, no more time for questions in the page, then all input fields are disabled (not been able to answer) and either next page is loaded automatically, or a message is presented to student informing he must move to next page manually. 

3. Anywhere you have time limits, accessibility rules means that you need to be able to overrides those time limits for some students. How do you see that working?
I must confess that we haven't considered this question in detail. We think that quiz_overrides would be the place. Either a flag to disable timing at all for that student or a factor: 50% more, 200% time for this student.

4. What changes will be required to Moodle's database structure to make this work? 
Using timed sections I think that changes could be confined to a single table quiz_sections. Just adding two or three additional fields after shufflequestions

  • temporize: 0/1 flag
  • timelimit: a duration (may serve as flag if 0 means no timelimit)
  • limitmode: either per_slot or per_page

If time limits are introduced strictly an the question level, then it is mandatory to change the questions table to define an expected answering time (and adding that to the question creation interface) and the quiz_slots table, to add a timelimit to each slot in a quiz (the image in second post in this thread) .

Considering your indication about accessibility,  quiz_overrides would need and additional field to disable/extend partial time limits by a factor.
Average of ratings: Useful (1)