Mastery-Type Lesson Flow?

Mastery-Type Lesson Flow?

by Chana O'Leary -
Number of replies: 13

Those of you familiar with Mastery Learning (Khan Academy is one example) - know that it is predicated on mastery of one topic before allowing moving on to the next (more advanced) topic. 

So far in Moodle, I have not found a way of enforcing "mastery" of a topic. Apparently, after 10 tries - a question is marked "wrong" and the student moves on! When they obviously don't understand the first concept - how are they going to understand the second, advanced topic??? It creates "swiss-cheese" learning to just fail them and then allow them to continue...eventually, it will come back to bite the student.

I would love a scenario where a student would be given unlimited opportunities to attempt a task - and only allowed to move on after 10 successful attempts at different problems. In my case, I am teaching ESL - I would want a student to successfully complete a task (different questions) before allowing them to continue to the next unit or level.

Is this possible in Moodle? I am fairly new and perhaps missing something obvious...?

Thanks very much for any and all assistance in advance!

Average of ratings: -
In reply to Chana O'Leary

Re: Mastery-Type Lesson Flow?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Well, there are two issues here:

First, interacting with a single question. There are several options for this, which you select using the setting "How questions behave". "Adaptive" behaviour has no limit on the number of tries that students get at a question. "Interactive with multiple tries" behaviour has a limited number of tries. The number of tries is one more than the number of hints you add to the question. Which you use is your choice as a teacher. Also, how many marks are deducted for each try.

Second, there is progression between questions. For example, in Kahn academy there is a simple rule that you have to get 5 questions right in a row to finish. In Moodle, there is just the quiz, where the student attempts a set number of questions. (Well, leaving aside add-ons like https://moodle.org/plugins/view.php?plugin=mod_qpractice.)

Coming back to the question of which behaviour to use, the choice is an iterating one. Questions in a quiz do not necessarily follow one from the previous. They may be about different things, in which case it is better if we don't force students to get stuck on one question. It may be better for them to move on, and return to the question they are stuck on later. Also, some students may just get frustrated if they cannot get past a question because they do not understand it at all. As I say, it is down the the teacher, what they are teaching, and how they are trying to teach it.

Average of ratings: Useful (4)
In reply to Chana O'Leary

Re: Mastery-Type Lesson Flow?

by Floyd Saner -

Chana, 

There are several ways to accomplish what you desire:

1) Look at the Moodle Lesson activity.  You can build adaptive learning lessons with loops backward/forward based on quiz results.

2) Use conditional activities so students must achieve a certain grade on a quiz before subsequent material becomes accessible.

3) You can create quizzes that present a different set of questions (randomly selected from a pool of questions) each time a quiz is attempted.

Your post does raise other questions for me. What is mastery?  How do you determine that?  Might it defeat or discourage learning if a student is forced into a fixed sequence of linear learning?  

I am not a linear learner.  I often jump around in material - go back to review something when I see how it is used in subsequent situations - and react quite strongly against being forced into a particular learning sequence or style. In spite of this, I managed to complete an advanced degree in a subject area that is often viewed as very structured and linear - computer science. (although my dissertation involved the application of fuzzy reasoning to software development smile )

I firmly believe in learning activities and assessments that are aligned with outcomes.  However, I also believe students should be given freedom to engage with content and learn in a variety of ways.

Floyd

Average of ratings: Useful (1)
In reply to Floyd Saner

Re: Mastery-Type Lesson Flow?

by dawn alderson -

Hi Chana, Tim and Floyd pipped me to the post here. Agree on all fronts with their helpful advice...although I did assume it was the quiz you were referring to, twasn't stated, the word 'question' gave it away smile

Am inclined to add, that if a student doesn't get something after ten attempts, is it worth changing tac? You know that saying...more than one way to stuff a mushroom! wink

cheers,

Dawn 

Average of ratings: -
In reply to Chana O'Leary

Re: Mastery-Type Lesson Flow?

by Enrique Caliz -

Hi, I have been thinking about this for some time now. The "mastery"  functionality offered by Khan Academy would be a great addition to Moodle. Does anyone here know if anybody is working on this?

Average of ratings: Useful (1)
In reply to Enrique Caliz

Re: Mastery-Type Lesson Flow?

by Matt Bury -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Hi Enrique,

Personally, I wouldn't call what they do on KhanAcademy mastery learning in the strict sense of the term.

One of the core components of mastery learning, as I understand it, is formative assessment; something that automated grading and feedback systems aren't particularly well suited to. A computer grading system can tell you if your response "ticks the right boxes" (or fits an algorithm in more sophisticated grading systems) but they're very poor at giving targeted, specific, meaningful feedback (You know, the stuff that teachers spend many hours on when they're marking students' work). Computers cannot understand learners' intentions, i.e. what they intended their response to mean, and even the most sophisticated AI systems still fall flat on pragmatic (as opposed to semantic) meaning.

Here's some examples of questions that any human can answer with relative ease but that completely stump the most advanced computers, called Winograd Schemas: http://www.cs.nyu.edu/davise/papers/WS.html

In short, whatever the subject may be, if the grader/feedback giver cannot understand what the learner intended to say, how can it understand what would be the most appropriate feedback to give? How could it provide a reasonable, consistent, meaningful, and appropriate degree of instructional scaffolding to learners to help them to acquire mastery of a given topic or skill?

So, I'm arguing that an effective mastery learning curriculum requires skilled, experienced teaching practitioners - or in some cases, trained, experienced teaching assistants, or even experienced, trained learners themselves (as demonstrated in peer-review writing programmes) - to provide effective formative assessment/instructional scaffolding to help learners. Otherwise, how would it be any different from trial and error?

This would be more in line with social constructivist approaches to learning and teaching. ( http://docs.moodle.org/26/en/Pedagogy )

Average of ratings: Useful (2)
In reply to Matt Bury

Re: Mastery-Type Lesson Flow?

by dawn alderson -

Matt, hi

AGREE.  And I enjoyed reading the research links, thank you.  In addition, the programme Mastery learning is informed by research that involves RCTs or RCEs (random controlled trials/experiments).  Basically, I am talking about experimental conditions with control groups.  Measuring human activity in order to generalise, can be very tricky-not one brain is exactly the same as another and not all outcomes/behaviour/action can be predicted on that premise.  So, achievement is going to look very different across a group, anyway, control or not.  However, to measure attainment, then I think there may be some fruitful purposes  for this type of research process. I COULD BE WRONG. But I suspect that is why the Khan academy still exist.

cheers,

Dawn wink   

Average of ratings: -
In reply to Matt Bury

Re: Mastery-Type Lesson Flow?

by Itamar Tzadok -
So, I'm arguing that an effective mastery learning curriculum requires skilled, experienced teaching practitioners - or in some cases, trained, experienced teaching assistants, or even experienced, trained learners themselves (as demonstrated in peer-review writing programmes) - to provide effective formative assessment/instructional scaffolding to help learners. Otherwise, how would it be any different from trial and error?

A possible problem with this argument is that it does not specify the criterion for being a skilled, experienced teaching practitioner and thus it can be charged as falling into social conventionalism rather than social constructivism. The question always remains who assesses the masters who assess the apprentices.

Expert domains are full with Winograd Schemas which non-expert humans fail miserably to disambiguate until they have been sufficiently initiated which in turn is determined by the ability to pass a final assessment by guessing correctly 70-80% of the distinctions instructed by the expert. The real world domain is one domain that people experience on a daily basis from day one. Given that more than 70% of the news items that cover demonstrations show violence on part of the demonstrators, and that there are significantly less reports of violence by councilpersons in council meetings, people are likely (if not conditioned) to guess that councilpersons fear violence and demonstrators advocate violence. Here the "expert" is simply mass media.

Both computers and humans can at best guess the intention of their interlocutor, be that a computer or a human. This is no less evident in the case of humans, considering the fact that humans are easily inclined to think that computers express feelings and thoughts when the appearance of these computers is made to look human and generate human gestures.

smile

 

Average of ratings: -
In reply to Itamar Tzadok

Re: Mastery-Type Lesson Flow?

by dawn alderson -

Itamar, hi

I am getting bits of this.  Would it be possible to break down the points....like I am a six year old smile...seriously, can you unpick it a bit-I am struggling to understand mixed what is your point?

cheers,

Dawn

Average of ratings: Useful (1)
In reply to dawn alderson

Re: Mastery-Type Lesson Flow?

by Itamar Tzadok -

The point is that the answer to Matt's question 

Otherwise, how would it be any different from trial and error?

is that it is no different from trial and error because learning is all about trial and error. The crux of the matter is that where there is a possibility of error, there must be a criterion to determine when a trial ends with error. The problem with the convention of skilled, experienced teaching practitioners is that it allows for not providing a clear consistent criterion on grounds that the expert teacher can just tell what's correct and what's an error by virtue of his/her experience and expertise. The problem with that is not that it's necessarily false but that it can be easily abused and that it is abused. If you take the time to build a well defined problem domain for the learning, automated assessment is likely to be more effective than human assessment. But who has the time?

smile

Average of ratings: -
In reply to Itamar Tzadok

Re: Mastery-Type Lesson Flow?

by Matt Bury -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Re: "it is no different from trial and error because learning is all about trial and error" -- This hypothesis of learning is closer to the behaviourist view, encompassing the concept of operant conditioning, i.e. that we learn by trial and error and receiving positive and negative feedback or, in the parlance of behaviourism, reinforcement (Edward Thorndike, John B. Watson, B. F. Skinner, et al).

Perhaps a relevant feature of Lev Vygotsky's view of social constructivism in this instance is his view that we learn by "making sense" of what we observe in the world around us, that we construct our own knowledge and understandings through observing and interacting with concrete objects, abstract concepts, and other people, all the time actively forming hypotheses of what things mean and how they work, and testing and exploring them. This is a very much more active role for the learner which affords her/him agency and autonomy of thought and to think deductively, inductively, and abductively, to construct categories, analogies, and metaphors, and to master her/his own understanding ("sense") of her/his environment rather than to simply react and adapt to it as the behaviourists supposed.

Average of ratings: -
In reply to Itamar Tzadok

Re: Mastery-Type Lesson Flow?

by Matt Bury -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Hi Itamar,

Re: "it does not specify the criterion for being a skilled, experienced teaching practitioner" -- Yes, this always has been and more than likely always will be an issue in learning and teaching practice. In practice, when we compare assessments from standardised tests (i.e. those that typically can be graded by computers, e.g. SATs and 11+) vs. opinions of teachers for learners' future academic success, teachers turn out to be at least as good and frequently better.

Testing and assessment organisations also tend to spend a lot of their time helping/requiring their human assessors to develop inter-rater reliability, i.e. that all assessors consistently arrive at the same grades/conclusions for any given candidate's work/performances/submissions. However, this is for tests and summative assessment, not formative.

The example of Winograd Schemas was to illustrate the types of reasoning that computer AI typically fails at rather than any of the specific questions; a straight-forward induction of who's doing what to whom appears to be an insurmountable challenge to computers. A simpler, less accurate analogy would be to ask, how do you teach a computer common sense? How do you teach a computer to understand what humans mean when they speak and write? That is still in the realms of science fiction and that is the kind of reasoning that seems to matter most in education, at least for preparing learners for higher education, public life, work, and citizenship.

Again, I'm arguing that a requirement for effective mastery learning is for who or whatever is giving the feedback to be able to discern what a learner is failing at and/or misunderstanding something and why. Summative assessment tells the learner the former, i.e. the "what", but not the latter, i.e. the "why." The "why" is the part that requires understanding, interpretation, and "reading" the intentions of others, and may require ("diagnostic") conversation with the learner, so that the appropriate feedback can be provided.

Average of ratings: -
In reply to Matt Bury

Re: Mastery-Type Lesson Flow?

by dawn alderson -

Hi Itamar and Matt,

 Could I be so bold and ask if you could apply your argument with an example in Moodle?  I sense a tension in the interpretations here surrouding behaviourism and SC, but it would be really helpful if I could read about what you understand in terms of the limitatons, or as Matt put it the challenge inherent in the claims surrounding the measurement of attainment here 'for practice'. Does Moodle not offer oppts to measure attainment, some would argue very effectively- me included, in some of the ways described in this thread?  I am ever so curious to hear about this.

thanks in advance smile

Dawn 

Average of ratings: -