Quiz

So, then I started thinking about how it should work, but it is tricky, so I decided to look what answers our students had actually given in response to numerical questions in the past. Therefore I ran the following query:

SELECT qst.answer, COUNT(1) as num
FROM mdl_question_states qst
JOIN mdl_question q ON qst.question = q.id
WHERE q.qtype = 'numerical' AND qst.answer <> ''
GROUP BY qst.answer
ORDER BY num DESC

It turns out that we have had a total of 189514 responses (because of things like adaptive mode, there there will be several responses stored per question) which group into 6123 distinct responses. (Note that this whole analysis ignores numerical questions that were selected randomly.)

The most popular response is 0, which was recorded 3868 times!

The huge majority of responses are given without using , as a thousand separator. (I am in the UK, remember.)

The first negative answer is -157000, recorded 2619 times.

The first weird entry is 'none', given 1639 times. 'None' scores 514, 'None.' 68 times, 'NONE' 35 times

The first answer with a decimal point is 24.8, given 851 times, so for some reason most Numerical question at the OU have integer answers!

The first answer with a unit is 71%, given 174 times. (71 without a unit given 222 times). % is by far the most common unit entered.

The next most popular unit is £, starting with £2.50 recorded 22 times, but there are plenty more that start £. Of course, Moodle cannot cope with units like that.

'pass' was given 91 times. '?' 74 times, 'xxxxxx' 50 times, 'dunno' 40 times, 'xxxxx' 40 times, 'xxxxxxx' 36 times, '-' 32 times.

'a' was given 63 times. I have no idea what is going on there. That was given in answer to a lot of different questions, and having looked at them there is nothing to lead a student to enter a. (Ah! perhaps the student was just putting in rubbish to see the feedback.)

The first response that uses , as a thousand separator is -157,000 54 times, then 157,000 40 times. (Clear what is going on there!) presumably those ended up being marked wrong, even though the student was right. Oh dear.

'one' was recorded 39 times, some other written out numbers were also given, but rarely.

'completed' 34 times, 'Completed' 28 times.

'i' 32 times - looking at some questions, I don't think that is anything to do with complex numbers.

'nil' 28 times

The first time there is an explicit '+' sign is +100 given 16 times.

Down in the tail, among the rare answers are all sorts of strange things like 'half past two', '2:30' also features. The first rude word, which I will not repeat, was recorded 12 times.

The first time you see a 'real' unit it is among those answers given 12 time. It is 'Angstroms', but the way.

there are a few places where the student has written something like '105000 debit'. I guess seeing that, the teacher could decide to make that a unit with multiplier -1.

Even further down, you start to see people trying to write negative numbers like (1234). Bloody accountants! You also start seeing things like '4.32 million'.

The first time you see a ',' being used as a decimal point by OU students is '1,42' given only 4 times.

I will add, that having looked a some of these questions, a lot of them have the warning

[Note: Do not enter commas or spaces or a £ sign within the number you enter.]

at the top, which I would guess is affecting student behaviour.

OK, so what do I conclude from all that.

1. Supporting ',' as a decimal point is not worth doing among UK students.
2. Supporting ',' as a thousand separator is not essential, but probably is worth it.
3. Stripping spaces from the answer is irrelevant. Probably best not to.
   
4. We should support units like £ in future, that come before the number.
5. We should consider supporting (1234) meaning -1234 in future.

Of course, it is vital to support ',' as a decimal point for languages that need it.


Now, as part of the translation system, each language that Moodle supports defines the correct decimal point and thousand separator characters. Therefore I propose that numerical questions should work like this.

1. Take the student's answer and strip off all leading and trailing whitespace.
2. Remove all occurrences of get_string('thousandssep') that are followed by a digit.
3. Replace get_string('decsep') with '.'.
4. Use a regular expression to separate the number part from the unit, based on the formal spec for PHP floating point numbers.
   
5. Trim any leading of trailing whitespace from the unit.

Does that seem sensible? Can anyone forsee any problems with this approach?

Drat! yes. I can already. At the point where you try to regrade a question, you don't know the current language, so get_string('thousandssep') won't return the right thing. Ah! so I just need to store the user's language when they start the question.