Hay!
How can I discover that students are cheting when are taking the quiz exam?
I discovered, that are at lest two ways, one is:
- and the other is that he close window and after that he can resume from that point
These two options work in safe window with time limit and with no option for resume.
In report is one indicator the time limit if is exeded. Maybe anything else?
Not that one again. It is bogus.
Here is a parallel secnario:
A student is sitting in their college room working on an assignment. The lecturer said the assignment has to be handed in by midnight, and the clock on the wall say 11:00pm. The student does not think that one hour is enough time, so he takes the clock of the wall, and winds it back to 9:00pm. Now he has 3 hours to finish his assignment, right?
What Miha is seeing is that the time limit allowed is exceeded. E.g. the time limit for the quiz is set at 6 minutes and some quizes are shown as beeing completed in 6 min 20 sec or even 7 minutes. As far as I know Moodle should close the quiz at 6 minutes flat.
There are probably only two explanations:
a) the user did everything by the book and there is a bug in Moodle
b) the user somehow cheated (exploited the quiz)
OK, this is one of the parts of the system that is not perfectly secure. Suppose as above that we have a quiz with a time limit of 6 minutes.
And suppose that the student does not cheat. They click start, and answer questions for six minutes, and then the JavaScript timer automatically submits their quiz attempt for the. But then the server is a bit overloaded at the time, so it takes a few seconds to process their submission, so actually their submission is recorded my Moodle as taking place 6:08 minutes after their start time. In this situation, Moodle will give the student the benefit of the doubt, and grade their quiz attempt with full credit. It is not fair to penalise the student becuase the server was slow.
If, however, the submission is received, say, 7:00 minutes after the start time, then the student is awarded 0 marks, but we still record the student's answer. Indeed, we still do all the grading calculations, so we can still show the student the rigth feedback, and so on. It is only the summing of the scores that is changed.
The exact decision of where to draw the line on late submissions is currently be bit of a fudge. It would be nice to come up with a better scheme that more accurately distinguises between server slowness and attempted cheating, but until I manage to think of something cleraly better than what we have now, I just leave this bit of code alone.
Hi Miha,
There are many discussions on this in these forums, and the short answer is you can't stop a determined computer literate student from "cheating" on an online Quiz - on Moodle or elsewhere - unless you have a human being standing behind him (it's usually a him) and watching every click.
Cheers,
Glenys
Well, yes, but. I don't think the computer-literate part of the above is relevant. After all, you can't prevent a determined student from cheating on a written paper handed in on dead trees, unless you have a human standing behind them. That is why we have invigilanted (proctored) exams.
Let's face it. The bottom line is that you cannot teach an unwilling person anything. People can only learn something themselves - at which point a teacher's input is massively helpful.
If you are in a culture where learning is valued, then cheating is an irrelevancy. Assuming the quiz asks interesting questions, so that thinking about them and answering them will help your learning, then why would a student want to cheat. (Of course, if the quiz is a whole lot of boring fact recall that can more easily be looked up on Google, ...)
If on the other hand, all you care about is the test scores, and coming away from the course with a certificate at the end, then cheating is a problem for you, but it is probably not your biggest problem
Of course, I am an idealist who loves learning, and the politicians who have far too much influence over the education system seem to be only interested in test scores, and that is where my argument falls apart.
But, in general the Moodle quiz code is pretty secure. The most likely sources of cheating are nothing to do with the software. They are just the same as for on-paper assignments.
Hi Tim,
That's what I meant to say. Thanks for putting it in a more... literate way.
Cheers,
Glenys
I don't think it's wrong to include a final exam where students are expected to know the answers, so they get grades and a certificate. In all other cases, a quiz must be implemented so that you learn from it. But there are instances, far fewer, where you show what you have learnt.
I'm not a fan of grades and I have given the "you're here to learn and not just to get a certificate" speech to my students when I see some of them miss the point of taking one of my classes.
Once the course is finished and at least a few got the point and studied hard: They must be given the chance to complete one secure quiz where they show what they have learnt. Just one.
I came all the way to this thread to thank Tim for "invigilanted," a word I've never seen or heard before.
Would be better if I had spelt "invigilated" correctly the first time!
Tim, we laughed about that a bit too. I thought it was an interesting mix of "invigilated" and "vigilante". Visions of gun-toting, costumed proctors were dancing around the office...
"How dare you cheat on my exam!" SMACK!!!
You have officially coined a new term.
Invigilante: To proctor, with extreme prejudice.
(Note: I am in no way advocating the use of excessive force to enforce rules. This was just a fun little divergence.)
Hi Buddy,
Most of the time I find the "Useful" rating button just perfect for these forums, but sometimes I wish there was a "LOL" one as well.
Glenys
I first learned the word "invigilator" from the episode of Mr Bean where he tries to cheat on the math (or "maths") exam (probably the funniest thing I've ever seen on the small screen). Rudolph Walker played the invigilator, and he later appeared on The Thin Blue Line.
Anyway, I agree that an invigilator is the only way to prevent "cheating." You can cheat on exams offered in any CMS, not just Moodle. A group in the US published a nice little summary of the issue a few years ago:
http://wcet.wiche.edu/wcet/docs/publications/Briefing_Paper_Feb_2008.pdf
Peter
<offtopic>
defn: Proctor, an English variant of the word procurator, is a person who takes charge of, or acts for, another. The word proctor is frequently used to describe someone who oversees an exam or dormitory.
Invigilate: watch over (students taking an exam, to prevent cheating)
In either case:
defn: Useless: having no beneficial use or incapable of functioning usefully; "a kitchen full of useless gadgets"; "she is useless in an emergency".
The reason:
defn: Real world: In the real world, no-one ever asks you to do something and then refuses to allow you to use the tools needed to do it.
The argument:
defn:Exam: a set of questions or exercises evaluating skill or knowledge; "when the test was stolen the professor had to make a new set of questions"
Is it? What is being tested? Knowledge and skill or memory? In this day of the Internet, can such exams be truly indicative of real skill or the ability to regurgitate information? Read Baroness Susan Greenfield - the pocket rocket has a really interesting view - met her once and she is really full of energy and ideas.
</offtopic>
Hi,Tim.
"If you are in a culture where learning is valued ... " - it's a nice stuff, but (as most of the nice stuff) a bit too nice for real world ;)
What should we do in a culture where a certificate is valued, but knowledge much less so (especially among the youth)? Or, even worse, not even the sertificate is a real matter, but certain very important previlegies you get for the learning time, and that could be taken away if you don't complete you learning in time? Where majority of the student's motivited by not getting fired? It's sad but not exaggerated actually.
Oleg, you're right about "If you are in a culture where learning is valued ... "
In the real world education is not always valued. You get teachers for whom teaching is just a job. They do it just for the money. So they'll employ the easiest and quickest methods to finish the syllabus so that they get paid. Often those methods don't give the children the knowledge that is intended.
One of the things that is often lacking in those environments is propper student supervision. And in that environment some students will try anything to pass.
Another problem can be a shortage of good teachers. Many good teachers may have left the profession because they can't earn a decent living as teachers. Those that remain may not have the skills to properly supervise the students and the students get away with a lot of nonsense that wouldn't be tolerated in a more "mature" environment.
That's a very big reality here at the tip of Africa... But let's not get into politics!
In the end cheating is a social problem, not one that can be fixed in Moodle. (But Moodle can make it harder!)
Greatly appreciate the statement, "If you are in a culture where learning is valued, then cheating is an irrelevancy. Assuming the quiz asks interesting questions, so that thinking about them and answering them will help your learning, then why would a student want to cheat." IMO one of western culture's greatest deficiencies is the propensity for believing one way and acting another. Any time you get the chance, remind the audience that the purpose of learning is learning.
The assertion that the purpose of learning is learning is, IMO, as meaningless as the assertion that the purpose of drinking is drinking. Even a so called 'intrinsic value' serves some purpose.
I have a simpler (I think) theory about making cheating irrelevant. Nullify the concept by stating that anything goes and make the questions such that any type of attempt to solve them will require substantial work. The outcome is twofold. The student can't cheat simply because whatever the student does is not considered cheating. The student will learn something simply because we always learn something when we work.
The only problem with this theory is that it poses some interesting questions to the instructor who is supposed to construct "interesting questions" for the students. And there remains the question, how many instructors are willing to do what they expect their students to do, that is, work hard and not cheat?
Just to read Joseph's always interesting comments is often learning something new of itself!!! Again, he has made me think a little and after reading Alfie Kohn's article, i think I have a new dimension on the issue of cheating. it comes in the first paragraph actually - the phrase "unethical behavior" does it for me.
Where does this unethical behaviour come from? How did we allow it to rise? How can we deal with it? Not the cheating, we are never going to win that one, but the behaviour? Is there no way in which we can instill into children a sense of ethics and values that are real and meaningful?
The other side is allowing the kids to take risks with their learning. Reduce the importance of getting a grade in favour of showing that the student understands something of human nature, or human spirit or the wider world around them. Who cares about Pythagoras these days? Yet such a simple mathematical theorem is fundamental to our technology (believe it or not). A more important consideration would be where do we use Pythagoras' theorem today, not the actual theorem would be one example of allowing Students to take a risk in their learning.
In its most simplistic form, properly deal with these issues, and I suspect cheating will become a thing of the past.
These discussions are another fabulous example of how technology and pedagogy intersect when using Moodle and on the Moodle.org forums.
What begins as a technical question because an analysis of education itself.
PS-
I agree that we shouldn't really put too much value on a student's test score. If you really want a student to prove they know something, have them create something or apply knowledge into a new area. Tests are great learning activities (the brain learns better from taking a test than from reviewing material) and learning discrete facts is still very useful in the age of Google. However, we need students to be assessed in more complex skills and so we shouldn't be giving a certificate or degree to anyone based on test scores and any school (or government agency) still doing that should change. I am hopefully pushing my school towards portfolio based assessment through our experimentation with Mahara.
Thanks again for being a great community.
mm I reckon I have read this article from Roediger and Karpicke before, Colin, and it struck me then as it does now that this is one of those examples of the recursive thinking referred to above. I have been intrigued with the idea of developing memory, as opposed to using memory, as being an integral and important part of the educative process. I ran into Sherlock Holmes and Agatha Christie, as a child, then discovered the concept of a human memory bank in Frank Herbert's "Dune", racial memories in any number of books and stories a startling "future memory" in "Tomorrow's Children", all followed up with Harris' "Hannibal" and now, Elisabeth Salander, but then reality creeps in and some people do actually recall lots of things with a trained memory. If we could improve people's ability to recall by simple exercises and tests and then encourage people to recall the promises made by politicians and business leaders, perhaps things would get better for everyone.
Hi,
I love these discussions about tests, cheating and learning. I once posted a question on another forum along the lines of, "How would assessment change if everyone had total recall?" (i.e. photographic memory).
I suspect the majority of tests given to students only exercise the lower order thinking skills of Bloom's Revised Taxonomy of learning - Remembering, Understanding and Applying. Why don't we have more open-book, collaborative and task-based assessments?
Portfolios are a great start, they've been used in the arts and creative professions for centuries and it's surprising that we haven't seen more widespread use of them before now.
Just my €0.02!
I have tried to use task based and open book assessments, much to the disgust of some of my colleagues, I suspect. What I have also been trying to do is to develop a range of tasks requiring an analysis and/or synthesis and at Middle School this is incredibly difficult in some subjects. Well, OK, it is incredibly difficult to get Middle School students in this country to look past lunch time let alone anything else. I have not had a lot of success yet, and that is, of itself something of a worry. It is easy to blame the students, but the tasks set may just be beyond them or the rich questions given are too broad for them to grasp what it is being asked for. The main reason for this? I suggest that this is far more indicative of their underlying learning rather than their ability to recall and regurgitate.
I love your comparison of what teachers expect students to do with what teachers might be willing to do themselves.
To state something such as, "the purpose of learning is learning" is similar to saying, "the main thing is to keep the main thing the main thing". Sorry if you found it meaningless.
How do you do that when you need to test something that only has one answer, e.g.:
1 + 1 = 2
How else can you test the students knowledge of the above concept without asking him for the answer? In this case, you want to test whether the student can do basic addition. How do you test his knowledge of addition in an "interesting" way without asking him to provide the correct answer i.e. 2?
I suppose that asking him for the answer (2) might not be the right approach. You want to test whether he can do basic addition. The actual answer to 1 + 1 doesn't really matter if the student can demonstrate how to work out the answer, which is what you really want to test. But how else would you test basic addition skills without getting the student to do addition, i.e. adding 1 and 1?
I'm not a teacher so I'm allowed to ask such a daft question! ;->
I'm just the sort of perverse person who would ask my students, "When isn't 1 + 1 = 2?"
I suspect that question might actually provide evidence that students know addition, among many other things.
Well, why are you asking that question?
Yes, basic mental arithmetic, like spelling, is an important skill that you can really only learn by practice. The Moodle quiz can be set up to let students do as much practice as they want. The question is, how you give them enough of an incentive so that they actually do it, and do it without just typing all the answers into Google.
Well, to rule out Google, you could use quite a stringent time limit. After some practice, doing it in your head is much faster than typing it into Google.
Another option is to use Moodle 90% of the time, but on occasion to the test in class, on paper and pencil. Than student will quickly work out that the only way to do well in the in-class tests is to have done enough practice on the other tests.
Also, try to make the Moodle tests like a fun game, not like a boring serious test. After all, some people pay good money for 'brain training' 'games' for games consoles that are just mental arithmetic. That just shows that exercising your brain can and should be fun. So perhaps don't use Moodle quiz at all. Find some free Flash game that tests basic arithmetic (and which does not have undesirable adverts) and use that instead.
On a related note, I read an interesting paper recently. The authors had constructed a computer program to teach long-hand addition and subtraction like 6523 - 1280. What made this clever was that the system tried to analyse the mistakes the student made, and so diagnose what systematic mistakes they were making in their calculations. For example, perhaps all their mistakes could be explained because they did something weird every time they tried to carry one. The paper found that the majority of student errors were systematic, rather than being careless mistakes. The system could often predict what wrong answer the student would give to a problem in advance.
On the one hand, this struck me as terribly clever. On the other hand it completely misses the point about what maths teaching should be. If students are taught long-hand subtraction as an algorithm to follow, with no justification, then of course they can make all sorts of strange errors if they get one step wrong. On the other hand, if you teach them to really understand what they are doing - what subtraction means - then they should not be able to have weird bugs in their subtraction algorithms.
As it happens, this paper was from 1978, so thanks to calculators and Google, that sort of maths teaching is now an anachronism, but the computing was, if anything, ahead of its time. The paper is Brown and Burton "Diagnostic Models for Procedural Bugs in Basic Mathematical Skills" Cognitive Science Vol 2, 1978. Sadly only the abstract is available online. I had to go to the OU library and take a copy on dead trees.
Another consideration when it comes to testing basic skills is situated learning ( http://tip.psychology.org/lave.html ) which baisically says that knowledge and skills are often locked into the context in which they are learned. An example is that if you give a typical school style math test to students they show a certain level of skill with arithmetic, but then when asked to use those skills in science class or in a real world setting they often perform at a lower level. The process is also true of people who show arithmetic skill in real world aplications. In research done on unschooled apprentice tailors, the tailors were able to solve math problems involving lengths of cloth and thread, but could not solve those same problems presented in a typical abstract school math test (the reverse was true for their counterparts in school). Housewives (and I am sure househusbands as well) have been shown to do complex math when shopping that they cannot perform on a math test.
So, I am more and more under the belief that you shouldn't teach and test basic skills in isolation in the belief that students will then be better prepared to apply those skills to real situations (which is one of the main goals of teaching students skills like arithmetic). You should "test" students ability to do arithmetic by giving them complex questions and situations (What is the best cell phone plan for a family?) that require them to use arithmetic (as well as critical thinking, research, and communication skills).
As they say the proof of the pudding is in the eating (not the testing).
The other side of that coin, Colin, is the really serious issue of "transferable expertise". Some of my colleagues consider me to be a "Moodle expert", a nonsense of course, they ask me questions and I provide simple direct answers. We have discussions on Moodle processes or Moodle admin decisions and I am asked for an opinion, which, being very opinionated, I oblige them, but that does not make me an "expert", just opinionated.
I am also teaching multi-media, applications, graphic design and maths, so therefore I am a "computer expert". I get asked all sorts of computer type questions and I am expected to provide cogent, accurate answers to highly technical questions that lie completely outside my limited areas of understanding. I am becoming increasingly amused by the looks of sheer disappointment when I say such things as, "Sorry, no idea." or "Don't know, you need an expert to answer that." I am constantly threatened by the perverse notion that I should make some goobledy-gook up just to tell them they cannot do anything because the sun is up or they are not standing on their left foot. It seems almost a sin to give up such opportunities for mischief, I can tell you.
The disturbing part of this is not the perversity of my sense of humour, it is the expectation that I can solve all those networking problems or server issues or some other problem, I am a computer "expert" so I can do this. The context of application of skill is important, but also it seems is the perception of that skill is as important, to the beholder. I am sure that we have have all run into this before, both as the beholder and the "expert". My biggest concern is that I succumb to the desire of a solution and provide some answer to a question that is totally inadequate. If I get it wrong here, then I can learn something, but if I get it wrong in my workplace, my credibility as an "expert" is questioned. It wont matter that I am not knowledgeable in a given area, I got it wrong so I am not an "expert", which means I am not an "expert" in any area of computer studies. This is my major concern about "transferable expertise", it is not really transferable, yet so many people seem to think it is.
Well-said, Colin.
Are you sure you test basic addition skills? For all you know, the student may have been familiar with the otherwise meaningless pure form '1 + 1 = 2' and completed correctly the partial '1 + 1 = ' to the complete form, in which case you actually tested the student's form completion skills.
Suppose then that you want to prevent the application of form completion skills by presenting the student with enough arithmetical addition samples (from your point of view) such that the student is unfamiliar with the form of at least one of the samples. How can you tell whether or not the student will be forced to apply addition skills rather than form completion skills? If the student answers that new sample correctly you may conclude that since he/she couldn't have done that by form completion he/she must have done that by addition. Perhaps. Of course it could be a lucky guess. So you need to sample more to exclude that option as well as a bad day (for incorrect answers) and other factors.
So, whatever you do you are faced with interesting questions and hard work. The point is that the harder you work to solve these questions, the more opportunities the student gets to develop and practice whatever skills he/she employs in the attempt to answer correctly and collect the points, whether or not you really know what these skills are and whether or not you actually managed to answer those interesting questions. Perhaps we just don't need to assume that we know what skills are or should be employed for solving the problems we present our students with and focus on creating enough well defined problems to practice on.
1 + 1 = 2
but also
1 + 1 = 2
and these are two different instances of the same addition and bit less trivially of the same form completion (which may mean that being able to solve it by form completion is actually a bit more advanced to solving it by addition).
Tim,
I haven't looked that paper up, but I'll bet it is an example of an "expert system." They know what the typical mistakes are, and they program for them.
Itamar,
My test isn't for addition skills. It is for an *explanation* of addition skills. And, the idea is that, in explaining, the student spontaneously explains what the grown-ups need to learn
The discussion of test for addition skills was a reply to Luis. To you I offered in another post evidence that I know addition (among other things).
Actually, no. Their system implemented addition as an algorithm consisting of a number of nested subroutines. Each subroutines had a correct implementation and several buggy alternative implementations. Therefore, the program could simultate performing addition with various combinations of bugs, and could search thought the possibilties to find which bug or bugs explained the student's responses.
Nifty!
"Each subroutines had a correct implementation and several buggy alternative implementations. Therefore, the program [...] could search thought the possibilties to find which bug or bugs explained the student's responses."
Simulating the student's wrong answers? Sounds like artificial intelligence in the reverse... artificial stupidity, maybe?
Here we go again with what someone called this forum's "favorite topic" and what I have called "an obsession with cheating". A moodle.org forum search brings no less than 35 pages, that's 350 messages with the word "cheating" in them! Apart from the technical aspects and the diverse educational contexts, it seems to me that the notion of cheating is a cultural thing.
Many arguments and counter-arguments have been given in this forum, including a useful reference provided by Tim to Alfie Kohn's Who’s Cheating Whom? article.
I share with many other participants in this forum the view that - whatever "secure" features are included in Moodle's Quiz activity (or any other computer-based or paper-based testing activities), nothing will deter a student intent on cheating from doing so. If an educational system is such that it "provokes" a cheating attitude, then there is something wrong with that system, not with the students. Finally, I would like to quote from the conclusion of Alfie Kohn's article:
Outraged condemnations of cheating [...] may turn out to have more to do with power than with either ethics or pedagogy.
Joseph
