Question Creation as A Student Activity

Question Creation as A Student Activity

by Jamie Pratt -
Number of replies: 16
Peter Ruthven-Stuart and I are embarking on a project we are naming the "SelfStudy" project, which will involve producing several modules

Question Creation as A Student Activity

The first of these modules will be an activity module for students to create their own questions. This module and Peter's research seek to do two things :
  • to encourage using question creation as a learning activity, to get the students reflecting on their learning.
  • after review by the teacher these questions can then be included in their question banks for use later in a course and / or in a subsequent course.
The 'Question Creation as A Student Activity' module, will look like this :

Teacher Interface - activity set up

We want teachers to be able to :
  • set up an activity where they specify :
    • what question types a student is expected to set up (multiple choice, true/false, short answer, etc)
    • how many questions overall a student is expected to set up (minimum ammount).
    • and / or a teacher will be able to set minimum numbers of questions of each allowed question type the student should set up.
    • students will be automatically assigned a grade based on how many of the minimum amount of questions they have set up
    • a teacher can decide what percentage of the grade the student is awarded automatically assigned and what percentage is calculated by the grades for each question assigned by a teacher.
  • A teacher decides what question bank and category questions should be created in.
Student Interface

For our question creation student activity we need to be able to :
  • let students use the question creation interface to save questions in a question bank
  • keep student generated questions seperate to teacher generated questions.
  • tell the student how many questions overall and of the required question type they have created in the question bank / category the teacher specified.
  • provide links directly to the question creation forms for the required question types - question module allows us to supply a parameter telling the form script where to return to after successfully saving a question.
  • show grades and comments from teacher.
Teacher Interface - Grading / Commenting on Created Questions

Then a teacher can grade and comment on questions that a student has created. Grades will also be automatically assigned as per the settings of the teacher when he set up the activity. Created questions might appear in a pop-up preview window or inline in the page.

Proposed Solution

One challenge of this project that could be solved in several ways is the question of how to differentiate student created questions from teacher created questions. Tim Hunt (the Quiz Module maintainer) indicated he was keen on turning the question bank into an activity module and that he thought this would help the question bank fit better into the Moodle architecture. And we thought that this would be a good solution for us. Having a question bank as an activity module would allow us to :
  • have several question banks in one course
  • each question bank can have different permissions of who can write questions.
  • we'd like to add a new capability to the question bank so we have seperate capabilities for user can view his own questions / user can view other users questions. If you can view a question you can also create a copy / new version of that question.
  • we'd add a new drop down box to the question creation / editing form for those who have the capability to save questions in more than one question bank (wouldn't appear for student if there is only one q bank in a course allowing students to save questions).
Activity Module / Assignment Type Plug In

We felt that although we could put the activity set up and grading interface into the question bank code it was probably better to seperate it from the question bank code. We envisage an assignment type plugin or possibly a new activity module plug in being used. Questions would all be saved in a question bank and the question creation forms of the question module would be used (ie current question/question.php). The activity module would query the question bank to see how many questions a user had created in the specified categories / sub categories.

I'd like to work on converting the question bank into an activity module if it is agreed that this is the direction that the question bank should move in. Tim would you be interested in taking a look at the code and possibly offering some guidance and would there be any interest in including such code in Moodle 1.9? I'd start working on this in March.

Question Bank as An Activity Module or An Alternative?

I'll follow this post after a few days with a detailed plan for converting the question bank to an activity module or do others think an alternative scheme would be better? eg. :
  • have within question banks the ability to filter questions according to the creator of the question and or the role of the creator of the question. So we could allow students to create questions but keep them seperate from teacher created questions by a filter in the question selection page controlled by a drop down box. Teachers could save questions as their own questions so that they appear as a teacher created question. We would need a new capability to allow/disallow people to view their own questions.
  • have a capability given to the teacher roles of 'approving' a question and having a filter controlled by a drop down box to only display approved questions / all questions.
  • or have all students only able to save questions in sub categories of a category they've been given the capability to save questions to. Ie. make new contexts for capabilities depending on the category / parent category.
Average of ratings: -
In reply to Jamie Pratt

Re: Question Creation as A Student Activity

by Pierre Pichet -
This is a very interesting project.
As you noted, it will imply that the visibility of the questions should be well controlled.
I am actually working on an interface to control the publishing of the category questions down to the course level.
Your work imply that it should also be controlled down to the student level or effectively to the user level given their roles.
I will be happy to join you on this work given however that I am more a full time teacher and programm director than a moodle programmer, the last status is mostly under my wife supervisionsmile
A first example using the actual roles/assign.php and roles/assign.html as an example to code it.
example
Average of ratings: -
In reply to Jamie Pratt

Re: Question Creation as A Student Activity

by Pierre Pichet -

As a mather of fact, among all the possible problems there will be the fact of who is the first author ("plagiarism") if these new questions are part of the grade of a course.

So, there should be an additional field in question object giving at least the first author ( user id) and a search feature to trace similar questions (the authors id could be added each time there is a copy of the question) etc..

Given the complexity of the project, I suggest that it should be more fully described in the developper docs and, if necessary, different postings on the quiz forum address each peculiar problem.

Pierre

Average of ratings: -
In reply to Jamie Pratt

Re: Question Creation as A Student Activity

by Peter Ruthven-Stuart -
Picture of Plugin developers
Hello,

Thank you to Jamie for introducing this project to the Quiz module forum.

My original conception of this project was as a single module that would allow students to make their own quizzes for self-study purposes. However, due to the underlying complexities of this goal we have had to break the project into a number of phases, which will each feature the creation of a module or the development of a new feature within an existing module, or even possibly the adaptation of exiting features; e.g. the 'question bank'. So, what started out as the SelfStudy Module has now been renamed the 'SelfStudy Project'.

Jamie has explained some of the technical details behind the SelfStudy Project, so I'll take the opportunity to explain our plans from a Moodling teacher's stand point. I explained my motivation for developing self study features in Moodle in a posting in the 'general developer forum'. To summarize, as a teacher, I believe it is important to promote 'Autonomous Learning' amongst students, or at least encourage them to take more responsibility for their learning. However, one deficiency of Moodle is that the activities in Moodle courses depend upon the teacher to first set up and then facilitate the activity. Now, it is true, as has been pointed out by Martin Dougiamas, that the admin of any Moodle site can easily give courses to students in which they would be free to make their own quizzes, and any other activities. It is also possible to have students use Hot Potatoes to create their own quizzes, which could then be uploaded to a Moodle Forum to share with their peers. However, having had my students do both of these things it is clear to me that most students need some guidance, and even cajoling, to discover how to take more responsibility for their learning. In other words, just giving them the tools (e.g. editing access to a moodle course, or an authoring tool like HotPototes) is insufficient. What is needed, is the ability to integrate this self study habit into students' main online courses. From a Moodle perspective, this means making it possible for a teacher to monitor self-study activities from within a regular Moodle course - i.e. a course in which a teacher has various learning activities, some of which will be activities that foster autonomous learning. This is thus the purpose of the SelfStudy Project.

At the moment I envisage 3 phases for this project, the first of which we are now embarked on. The goals of the three phases are:

Phase 1: "Question Creation"

  • Create a stand alone module, or an addition to the Assignment module activity, that makes it possible for students to create questions
    • note, when writing about question creation, I include the writing of feedback, for both correct and incorrect answers (the distractors)
  • Make it possible for teachers to evaluate these questions, which could be done automatically and manually:
    • automatic grading: a simple grade based on the number of questions created by a student divided by the number required. This number will be sent to the gradebook
    • manual grading: allow teachers to individually grade questions and feedback, again this grade will be sent to the gradebook
  • Make it possible for the teacher to import these questions into regular teacher-created quizzes. This could be done in one of two ways:
    • manually: the GIFT import feature; questions could be saved as a GIFT file, then 'imported' back in by the teacher to another category
    • automatically: all student created questions designated as 'good' could be automatically moved or copied to a certain category
      • once in this 'good question' category, the questions would no longer be editable by the original creator, although a record of who made the questions would be kept (i.e. the original author)
I envisage the educational advantages of Phase 1 as follows:
  • It could be used to encourage students to reflect on a body of knowledge (a course / a unit / a topic, etc.) that they have studied.
  • The creation of questions, which includes writing good distractors and feedback, will itself be a good learning experience. As question creators, students will take on the role of teacher, and will therefore have to make their knowledge explicit, which is a higher level cognitive task than answering questions
  • The writing of feedback in particular will require students to explain a topic in more detail
  • The knowledge that students' questions will appear in quizzes will be an incentive for students to write many good quality questions.
  • It will be a good way to help individual teachers to increase the size of their question bank. Of course, with lower level learners the teacher would have to spend more time correcting and rewriting questions.
  • It will allow teachers to discover what students perceive as important in a certain body of knowledge.
  • Hither to, I've been assuming that the students create questions to which they already know the answers, since they also have to write the answers, distractors and feedback. However, another possible scenario is that students could be asked to write genuine questions that they have about a topic they are about to study. Here they would be asked to just write the questions. The teachers could then include the questions in quizzes having added correct answers and feedback, or students could add the answers as they discover them.
    • Another alternative to this would be to have the teacher provide just the questions, and have the students write the answers, distractors and feedback.

Phase 2: "Quiz Creation"

  • Create a module that will allow students to create their own quizzes, consisting of self-made, peer-made, and teacher made questions.
  • The peer and self made questions would be only those that have been 'passed' by the teacher, i.e. those transferred to the 'good questions' category.
  • Ideally, students will also be allowed to share quizzes with their peers. So, for example, there could be a link on the course page, called "Shared Quizzes". If a student (or teacher) had indicated that a quiz they had made could be tried by other students, a link to the quiz would appear on the "Shared Quizzes" page. Such a feature would encourage cooperative learning.
  • Another possible feature of this phase could be to allow students to work on the creation of quizzes in groups, in a similar way that students can work together on projects in the Project Module.

Phase 3: Generation of On-the-fly Quizzes from Smart Question Lists

This final phase will not involve the creation of a new module, but will require a number of changes to the question bank architecture, in particular the 'categories feature'.
If successful, these changes will allow both students and teachers to create quizzes consisting of questions that fit certain criteria; i.e. customized quizzes. At the moment, it is only possible for a question to belong to one category, or sub category. For example if there is a question category called 'Unit 1', and within that a sub category called 'verbs', a question that belongs to the sub category 'verbs' in category 'Unit 1' can not also belong to the sub category of the unit 2 category also called 'verbs'. The solution is to replace the category feature with a tag system so that questions could have multiple tags or attributes. There could be two kinds of tags:
  • automatically generated tags:
    • transient tags (because they change): difficulty, number of attempts, known or unknown, etc.
    • fixed tags: date made, original author, grade given for question, question types (multiple choice, short answer ...), etc.
  • manual tags:
    • these tags would be like the current 'categories' with the difference that questions could have multiple tags, for example; unit number, parts of speech, textbook name, etc.
From teachers' point of view, they would set up a 'customized' quiz with the regular quiz module, and instead of adding individual 'fixed' question or perhaps random questions, they would be able to add questions that fit certain criteria by using the new tag system. For example quizzes could include questions that match one, some or all of the following criteria:
  • questions that students trying the quiz have never got right before; these questions would be specific to individual students
  • the 5 questions that the class found hardest; these would be the same for all students in the same course (duplication with the above criteria may be a problem)
  • only questions from a certain unit or topic AND that match another another tag
From students' point of view, they could use the new module described in phase 2 to make a custom quiz. Like their teacher, they would also be able to pick questions from the Question Bank using the new tagging system as search criteria. In this way, they could generate customized, on the fly quizzes to suit their immediate learning needs.


To conclude, each phase of this SelfStudy Project is designed to engender a constructivist approach, and foster both cooperative and autonomous learning habits amongst students. Since this is a work in progress, we would very much welcome any comments about this project that address either the technical aspects or 'Moodling teacher' (educational) issues. Also if you have any questions about how these various features might work in a Moodle course, or indeed any ideas about how you might implement the features, please let us know.
Average of ratings: -
In reply to Peter Ruthven-Stuart

Re: Question Creation as A Student Activity

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
An activity for students to create quiz questions in a guided, structured way is an absolutely terrific idea, as long as the questions themselves are stored in the existing core question tables (even if we need to extend those to cope with it).

See the wider discussion about Questions as a core feature here:

http://moodle.org/mod/forum/discuss.php?d=67006
Average of ratings: -
In reply to Martin Dougiamas

Re: Question Creation as A Student Activity

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Martin, this 'Question Creation as A Student Activity' idea is exatly why Jamie starting thinking about changing the question bank. It is a collaboration between him and Peter.
Average of ratings: -
In reply to Tim Hunt

Re: Question Creation as A Student Activity

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Sure I know, I was just linking to the discussion for anyone who might be reading this one. smile
Average of ratings: -
In reply to Martin Dougiamas

Re: Question Creation as A Student Activity

by Jamie Pratt -
Very glad to hear that our idea has caught your interest Martin! I think it is a great idea (mainly Peter's).
Average of ratings: -
In reply to Martin Dougiamas

Re: Question Creation as A Student Activity

by Peter Ruthven-Stuart -
Picture of Plugin developers
Hello Martin,

Wonderful that you like this 'question creation activity'.

I can see that my initial idea for a SelfStudy module has taken on a life of its own, and has perhaps opened up a can of worms re the question bank. Although, it seems that changes to the way different modules interact with the question bank were already in the pipeline.

As a teacher, my main concern it to create something in Moodle that allows students to take more responsibility for ther learning, while at the same time allowing teachers to guide this learning. So to that extent, I'm not too bothered about the underlying mechanics of the actvity - as long as it works. That said, I realise that this kind of development could be disruptive to the core Moodle code, which is something I definitely want to avoid. Also, at some point in the future, I'd very much like the various features of the SelfStudy project to become standard Moodle features. Yet another reason that it not only conforms to the moodle code, but also enhances it.

So any advice you have for the people working on this (Jamie Pratt et al) will be much appreciated.
Average of ratings: -
In reply to Peter Ruthven-Stuart

Re: Question Creation as A Student Activity

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Peter,

If you have opened a can or worms, then it was a can of worms that I had pushed into a corner and tried to ignore, and which was getting a bit old and starting to smell. So I am really glad that you persuaded Jamie to clean up the mess. It badly needed doing.*

Students taking responsibility for their own learning, and peer learning, are very much at the heart of Moodle's educational Philosophy: http://docs.moodle.org/en/Pedagogy. And on the technical side, Jamie's changes are making the question bank less disruptive to Moodle core code.

And the thread you like to in the General developer forum was a very productive discussion. A great example of students (of the Moodle code) guiding each other's learning.


(* did I push this metaphor too far? I hope so!)
Average of ratings: -
In reply to Jamie Pratt

Re: Question Creation as A Student Activity

by Jamie Pratt -

These are the latest specifications for the 'Question Creation as A Student Activity' module for those who are interested.


Teacher Interface - activity setup

We want teachers to be able to set up an activity where they specify :

  • what question types a student is expected to set up (multiple choice, true/false, short answer, etc)
  • how many questions overall a student is expected to set up (minimum ammount).
  • and / or a teacher will be able to set minimum numbers of questions of each allowed question type the student should set up.
  • students will be automatically assigned a grade based on how many of the minimum amount of questions they have set up
  • a teacher can decide what percentage of the grade the student is awarded automatically assigned and what percentage is calculated by the grades for each question assigned by a teacher.
  • decide what question bank and category questions should be created in.
  • sets start and end time of activity
  • Group mode: No groups / Separate groups
  • Specify not only the total grade for the activity(automatic and manual), but also indicate how many points will be givenfor each question

Student Interface

For our question creation student activity we need to be able to:

  • let students use the question creation interface to save questions in a question bank
  • keep student generated questions seperate to teacher generated questions.
  • tell the student how many questions overall and of the required question type they have created in the question bank / category the teacher specified.
  • provide links directly to the question creation forms for the required question types - question module allows us to supply a parameter telling the form script where to return to after successfully saving a question.
  • show grades and comments from teacher.

Teacher Interface - Grading / Commenting on Created Questions

  • Then a teacher can grade and comment on questions that a student has created.
  • Grades will also be automatically assigned as per the settings of the teacher when he set up the activity.
  • Created questions might appear in a pop-up preview window or inline in the page.
  • Teacher will be able to select approved questions (based on grades) and make a copy of them in another question bank.
Average of ratings: -
In reply to Jamie Pratt

Re: Question Creation as A Student Activity

by Pierre Pichet -

How about letting the student build a kind of lesson or quiz (without access to the grading?) with their questions.

This could simplify the code because the questiontype code could stay where it is now .

Pierre

 

Average of ratings: -
In reply to Pierre Pichet

Re: Question Creation as A Student Activity

by Jamie Pratt -
The question type code will stay where it is. We will use the question bank for the Question Creation as A Student Activity activity in the same way that the quiz uses the question bank. The question bank is set up in such a way that any module can display the question creation interface you see in the quiz module.


Average of ratings: -
In reply to Jamie Pratt

Re: Question Creation as A Student Activity

by Pierre Pichet -

I happy that it stays as it is.

Having a look to the lesson, I was surprised the way they handle questions creating their own questiontypes. I understand that they need special fields to control the flow. I will look in the past forums why they were not integrated with the other questions.

Pierre

Average of ratings: -
In reply to Pierre Pichet

Re: Question Creation as A Student Activity

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
One part of the plan of moving the question types out of the quiz was that they could then be shared by the lesson. The only reason this has not happened is lack of time.
Average of ratings: -
In reply to Tim Hunt

Re: Question Creation as A Student Activity

by Pierre Pichet -
Looking at lessons I just realize that lesson also use import of different format.php.
I wil look closer given our other discussion on merging questiontype and format.
Pierre

Average of ratings: -
In reply to Jamie Pratt

Re: Question Creation as A Student Activity

by Peter Ruthven-Stuart -
Picture of Plugin developers
Hello Jamie,

This all looks good to me.

Just a few comments and ideas:

Teacher interface - activity setup
  • Group mode: No groups / Separate groups / Visible Groups
Also, would it be possible to allow the teacher to give all students some questions, and (especially in the case of multiple choice questions), require them to write the answers and distractors. This feature would be especially useful for 'lower-level students, and for students that were new to this kind of activity. It would get the ball rolling. Ideally, the teacher could write, for example, 20 questions, and then have them randomly distributed amongst students. Of course the same question could be given to more than one student.

Student Interface

If it's possible for teachers to 'seed' the students question bank, then in their interface they would see that some questions were already available to them to edit.

Teacher Interface - Grading

In the Workshop module it is possible for teachers to save frequently used comments - very useful since students often make similar errors. Could such a feature be possible here?

It would be good if there was an option to automate the copying of approved questions to another question bank. i.e. once a question goes above a certain 'approval threshold' (i.e. a certain score) it will automatically be moved to a list of good questions. Also, it would be good if the teacher could also export the questions as a GIFT file - for question editing and sharing with other teachers.

I hope these requests are not too unrealistic thoughtful.
Average of ratings: -