Contributing to Moodle for MSc project

Contributing to Moodle for MSc project

by Peter McArthur -
Number of replies: 19

I'd like to introduce myself, and announce my interest in contributing to Moodle for my MSc final project.

I'm an English as a Foreign Language teacher by profession, and a computer nerd by inclination. I'm bringing these strands together by studying part-time for an MSc in e-Learning Technology.

My final project starts soon and, if it's feasible, I'm very interested in taking this opportunity to add something Moodle.

Anything I do will have to be grounded in pedagogical theory / research. Tim Hunt's idea for a self-assessment activity[1] looks very interesting, because even an unambitious implementation might be of some value, and could open the door to all kinds of pedagogically interesting future extensions.

I've looked at the list of suggested projects for new developers, but I'd be very interested to hear about other suggestions. And I'd be especially interested to talk to anyone who could spare the time to mentor me through the implementation stage of my project.

By the way, work and family obligations mean I'm ineligible for the Google Summer of Code.

----

If I post a URL, my message gets blocked as spam, but I hope you can make sense of this:

[1] tjhunt DOT blogspot DOT jp/2011/09/what-i-want-to-build-next.html

Average of ratings: -
In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Mary Cooch -
Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Hello and welcomesmile You probably tried to add a link which is why your post was marked as spam but you should be fine in future. I don't have any specific suggestions to make (will leave that to the developers) but as an MFL teacher by qualification and wannabe computer geek by hobby I shall be interested to follow your progress, and particularly on the pedagogical side.

In reply to Mary Cooch

Re: Contributing to Moodle for MSc project

by Peter McArthur -

Thank you for the warm welcome. I'm looking forward to working with you.

In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That all sounds good, except that someone else is going to be working on building that activity as part of this year's Google Summer of Code.

So, it might be good to find something slightly different to work on.

One other idea, is the idea of being able to embed questions anywhere, for example in the middle of a Moodle Page resource, or a label. This would need to be done as a Moodle filter (http://docs.moodle.org/dev/Filters ). The idea is you would have some sort of token that people could type, like {{question:123}}. The filter would replace that with an iframe, and in the iframe would be a script that lets the student interact with the question (a bit like what appears in the pop-up when a teacher previews a question in the question bank, but without any extra UI).

The trouble with that is that it is just a technical thing. All the pedagogy/theory would be in how teachers chose to use it.

 

Another idea, where some pedagogic theory would be required, but I have no idea what, would be some sort of report on forums. There must be scope for a report that analyses who has replied to whom in a forum, and computes something interesting about that. I am sure there is a literature about that, but as I say, I am not familiar with it. 

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Contributing to Moodle for MSc project

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I like Tims idea. My wish list includes a (Moodle) lesson like activity that uses the question types from the question activity.  The idea being that the path through a quiz and the questions displayed would depend on previous answers.

If someone worked on Tims idea of embedding questions anywhere and documented it well it would probably help with ideas for such an activity.

In reply to Peter McArthur

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Gareth J Barnard -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers

Dear Peter,

Having completed an MSc project in 2011, I would recommend:

Keep the implementation small and achievable in the time that you have, but big enough to justify your project proposal.  Something like a block or an enhancement to an existing component.

You can replicate other existing investigations / work at MSc because the project is about learning how to do and justify research.  Only PHd's have to be globally unique.  Double check what you will do with your tutor before getting too involved.  Avoid users under 18 as that leads to lots of paperwork.

Be actually interested in the research, pick an area that you find enjoyable such that you will keep going when things get tricky.

Idea:

As an eLearning MSc I would have thought you could look at the user interaction with Moodle as a possibility as I suspect quite a bit has already been written about pedagogical theory in classrooms / virtual tools.  Perhaps be more Moodle specific on how users engage with the software.

There is lots of current talk about the Bootstrap theme and 'Mobile first' in the forums, so perhaps now is a good time to look at the user interface in a mobile learning environment.  You could create / adapt a theme, try it with adult users and compare / contrast with other themes.

Cheers,

Gareth

Average of ratings: Useful (1)
In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Peter McArthur -

Thank you for all your thoughtful responses. It gives me confidence that Moodle is the right way to go. smile My initial responses to your suggestions are below but, when the time comes, I'll discuss them all with my supervisor.

Thank you, once again.

Peter

Tim: I thought someone on the Google Summer of Code might take up the self-assessment idea first! In any case, given the scope of the self-assessment project (you estimated "several months"), it's probably more suitable for the GSoC than for me. I'd only have enough time to gather the requirements and lay the foundations, which would be sufficient for an MSc project, but probably not what the Moodle project wants.

What do you think of the complexity of the embedded question idea? (I'm not sure whether you're the best or the worst person to ask, given your intimacy with the question engine.) It's certainly pedagogically justifiable: embedded self-test questions improve recall and meta-cognition.

I like the forum report idea, if there's demand for it. Technically, it doesn't sound overly ambitious, and a group of Moodle-using tutors who are willing to test it and give feedback could really bring the project to life. The only risk I can foresee is the problem of gaining ethical approval for what could be considered a surveillance system.

Alan: Despite my earlier remark about "bringing these strands together", I'm wary of connecting my work as an EFL teacher to my MSc project. I work in a junior high school in Japan so, if nothing else, it would be a nightmare to obtain bureaucratic approval from both the school and my university, and the risk of being turned down by the school would be very high. Also, most of my training as an English teacher is practical, rather than academic, so I'm likely to get bogged down in the literature review if I start a project that is focussed on language teaching.

Gareth: Thank you for the MSc advice - all stuff that bears repeating. I agree that usability and user interaction are an area where academia can make a big contribution to FOSS projects. I'm concerned that my circumstances may get in the way of running a formal usability study, but I'll discuss it with my supervisor and see if we can come up with something workable.

 

In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

The embedded question filter should be quite easy. Filters are one of the easiest sorts of plugins to create. They basically just do a search and replace on HTML before it is displayed. In this case, the search-and-replace needs to replace some reference to the question with a iframe. The iframe needs to link to another script, which as I said before, is really just a cut-down version of question/preview.php.

So, getting something working is not too big a task, although you will have to learn about certain parts of Moodle. The hard part is stopping it working all the time wink For example, if anyone can type something like {{question:123}} anywhere, to see the question with id 123, then it won't take long for some enterprising students to use that to get at the end of course exam questions before they are supposed to.

It is this problem of making it secure that is the main reason I have not already implemented this myself. (It has been sitting about half-way down our todo list at the OU for some time.) For an MSc project, I don't see that as a problem. You can build something that works, then you can write about the pedagogy, then if time permits you can work on solving the security issue, and depending on how much time and inspiration you have, you can either solve it, or just write about the issues. It is bound to be a trade-off between greater security, and more hassle for teachers wanting to use it. Anyway, the difficult bit comes after you have something that basically works.

 

I don't think the forum thing raises many ethical issues. It is simply re-organising information that is already available in the forum UI in a more convenient way for teachers (and possibly students) to understand. It may raise some more interesting pedagogical issues, relating to trust. Used well, it could help students become more self-aware, and hence more effective, in their use of online forums. Used heavy-handedly, it could make the students feel that they are being subject to surveillance, as you say, which might cause them to stop communicating altogether, and destroy the trust you need in a classroom. However, I don't think that is something that will really be affected by the technology, just by how the people use it.

In reply to Tim Hunt

Re: Contributing to Moodle for MSc project

by Peter McArthur -
You can build something that works, then ... if time permits you can work on solving the security issue...

That sounds ideal from where I'm at now. A modest core objective, plus a clear direction to expand the scope. Am I right in thinking that a mechanism to control the visibility of questions might come in useful in other areas?

In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Well, there are already lots of mechainsms for controlling who can access which question. The problem is, you really need to control which user can add a reference to a given question, but the filter runs at the time a different user views the reference.

In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Peter McArthur -

Dear Moodle people,

Sorry that I've gone quiet for a while. Please be assured that I have not lost interest in contributing to Moodle for my MSc project. I've given all of your suggestions very careful consideration. I'm going to have to commit to a project soon, so I'd like to discuss where I'm at with my ideas. I'd also like some feedback on one more idea I've had.

Thanks for your time and consideration.

The ideas suggested by other people were:

1. A dashboard for monitoring forum activity

I really like the idea, but I can't see a way to make something of sufficient scope for an MSc project without real world testing. And real world testing could lead to serious ethical problems. If you can see a way out of this dilemma, I'm all ears!

Tim seemed surprised about the ethical problems, so I'll explain in more detail. The university holds me to near-medical standards of research ethics. In my nightmare scenario, the ethics committee will say something like:

The mere fact of its usage suggests that the dashboard does more than just repackage available data.
We consider the students being observed by the dashboard to be indirect participants in the study, and their voluntary, informed consent will be required.
Students are sometimes required to use course forums for assignments. The power tutors wield over students might lead some students to conclude that it would be in their best interests to participate in the study, against their wishes.
We conclude that it is impossible to guarantee that all participation will be voluntary. Ethical approval denied.

This possibility is a risk I can't afford. So, unless someone can suggest a Plan B that will get me a good MSc project even if the committee says "no", I'm going to have to give up on this idea.

2. The question filter

I also like this idea, but I'd like to know more about the scope of the security problem before I commit to it.

I'm ignorant about Moodle's architecture, so please, please tell me more about this. What kinds of changes will I need to make? Will it be enough just to make the filter smarter, or will I have to modify other parts of the system, too? Will it require a Moodle fork? What about plug-ins?

3. Mobile usability study

I wasn't keen on this idea at first, but the more I think about it the more I warm to it. It could certainly be a great learning experience for me. However, the practical problems remain daunting, and I have no experience in these matters.

It looks too risky to attempt, but I want to discuss the idea with some usability experts here in Japan before I dismiss it entirely.

4. (my new idea) Adding a programming question type

Inspired by this discussion:

How can I include a Python interpreter written in Javascript in Moodle?

and by my experience of being obliged to take an appallingly designed multiple choice exam that was supposed to assess my competence in Java.

We can assess somebody's competence as a programmer by asking them to write a function or class that meets certain specifications. We can check the accuracy of the results using automated unit tests. There are various ways we can implement this.

We can run the unit tests:

1. in the browser, natively (if it's Javascript)

2. in the browser, through an emulator (e.g. brython, js.js)

3. in a browser plug-in (e.g. JRuby, JPython, Rhino and Clojure can all run in the Java Virtual Machine)

4. on the server (this one has already been done: Virtual Programming Lab for Moodle)

5. some combination of the above. (E.g. student checks his implementation in the browser, and then uploads it for official verification. Or the unit tests are on the server, the implementation is on the browser, and a web service connects the two.)

The student can edit the code in:

1. their preferred IDE.

2. a code editor embedded in the browser.

3. a code editor running on the server, via a virtual terminal embedded in the browser.

4. some combination of the above.

My proposal is to implements some portion of the above, and test the various permutations for suitability in various domains. (E.g. an in browser plug-in might be good for quick self-assessments, but server side assessment would probably be a must for high-stakes exams.)

Average of ratings: Useful (1)
In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

2. The question filter

The security issue is not that serious.

The bad thing that might happen is revealing a question that students should not see. There are worse things that can happen in Moodle, for example, revealing user's personal information and data-loss.

My proposed solution would be this:

  1. Create a new page where teachers can go and enter / select a question id.
  2. At this point, we can check the teacher's log-in, and that they are permitted to use this question. (The same check that is done when a teacher tries to add a question to a quiz.)
  3. The system then generates the necessary embed code for the teacher to copy-and-paste. This includes some sort of secure hash of the question id and a secret salt.
  4. The filter should check the hash before displaying the question.

Thus, only teachers with the right permissions can generate the code necessary to display a particular question.

All this can be done within a Moodle filter plug-in. No core changes needed.

4. (my new idea) Adding a programming question type

This is a very cool idea. It has acutally been done before in Moodle 1.9: http://docs.moodle.org/19/en/Junit_question_type. That was Java, executed server-side. However, that work was never update to Moodle 2.x.

As you say, many variations are possible. My feeling is that this is more work that the filter idea:

  • Qtypes are more complex than filters
  • Qtypes, and this idea, is more functionality to implement than the filter idea.
  • This idea involves incoporating some quite complex third-part code into a Moodle plug-in.

None of that is to say that this is too big for an MSc project, if you are up for a challenge.

There is also a security concern here: when I was listing the worst sorts of security vulnerabilies above, I forgot the worst one: "Execute arbitrary code". Which is exactly what you are proposing here wink

But, as I say, this is a really good idea. It is a proper use of computers for automated assessment.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Contributing to Moodle for MSc project

by Peter McArthur -

2. The question filter

That's a really neat solution to the security problem. I'm kicking myself for not thinking of it myself. Unfortunately, it also means the question filter idea is no longer of sufficient scope for an MSc project.

So that leaves us with...

4. Adding a programming question type

I'm glad you like this idea, because my course tutor also favours it, and I've already started putting together the initial proposal. And thanks for digging up the JUnit link.

The security issues can, of course, be fixed with sandboxing, but that's kind of a worry for me. My server admin skills amount to little more than googling for how-to guides, and I've never worked with sandboxes before. Well ... there's a first time for everything!

I'll start a new thread for my project after I've finished my preliminary proposal.

I really appreciate your input, Tim. I'd have probably taken a lot longer to reach this point without your help.

Peter

 

In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Peter, have you thought of the idea of reporting/blocks on who has NOT done stuff. For example in classroom situations I can easily see who has posted to a discussion forum, but I frequently want a quick and easy way of seeing who has not posted (and so try to make the number go down). In the gradebook I would love to be able to quickly and easily see who has not submitted an assignmnet etc.....

In reply to Marcus Green

Re: Contributing to Moodle for MSc project

by Peter McArthur -

That is, indeed, exactly the kind of thing I was thinking about for the forum dashboard. And I really like your idea about extending the idea to other forms of monitoring. If we take this idea to its logical conclusion, I can imagine using data mining techniques to look for early warning signs about problem students.

But ...

I've decided to back off from this idea. There is too great a risk that this project will become mired in restrictions imposed by the ethics committee.

In reply to Peter McArthur

Re: Contributing to Moodle for MSc project

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

It is really no problem to help someone who is proposing to do cool/useful stuff for Moodle.

As well as the Junit question type, here are some more helpful links:

  • The Virtual Programming Lab for Moodle http://vpl.dis.ulpgc.es/ Looks like they have some approaches to the security issues. I think that what they are doing (programming assignments) is sufficiently different from what you are proposing (single programming questions as part of a quiz) that this is not a problem for you. There is plenty of work to do building a question type even if you can use some of their lower-level infrastructure.
  • There is lots more earlier work by other groups, e.g. https://www.google.co.uk/search?q=nottingham+automatic+grading+computer+code. Again, this is not a problem for an MSc project. (Thanks to my colleage Phil Butcher for telling me about this work.)
  • The STACK question type. https://github.com/maths/moodle-qtype_stack. This is not exactly executing computer code entered by the student, but it is processing computer algebra syntax by connecting to another system (Maxima). There may be something to learn from the connection classes in https://github.com/maths/moodle-qtype_stack/tree/master/stack/cas. Also, the white-list / black-list approach in the casstring class.
In reply to Tim Hunt

Re: Contributing to Moodle for MSc project

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

The VPL system and the STACK question types are awesome but they do require more than the basic W/LAMP software. This is a limitation for many people, including me and I am very interested in questions that test programming and maths.

I have been looking around for (very) affordable VPL hosting that would support Maxima and Gnuplot and so the STACK question type.

In reply to Tim Hunt

Re: Contributing to Moodle for MSc project

by Paul Nicholls -

Hi Peter,

In addition to the various plugins/work that's already been listed, one of our Computer Science lecturers has developed (and been using) a system called CodeRunner (consisting of a question type and a question behaviour), which runs test cases against students' code.  It started life as PyCode, which, as the name suggests, was purely for Python - though "currently Python (versions 2 and 3), C, Java and Matlab are all supported, but the architecture is sufficiently general to accommodate additional languages with very little extra code."

Please see the link above for more information - including more background on CodeRunner, installation/configuration instructions and even suggestions on how it should be used.

 

I find the idea of client-side code execution interesting - it opens the door to giving feedback even closer to real-time (the overhead of constantly executing the code would likely prevent it from actually being real-time), and without requiring ridiculous amounts of server capacity.  As you suggested in an earlier post, it could be used to give the students quick feedback (without putting extra load on the server), while relying on something like CodeRunner to provide "official" results server-side (to prevent students from fudging their results).  Naturally, you'd need to make very sure that there weren't any implementation differences between the client-side and server-side checks which could result in code which passes the client-side checks but fails on the server...

 

-Paul

In reply to Paul Nicholls

Re: Contributing to Moodle for MSc project

by Peter McArthur -

Awesome! Server-side sandboxing is the part I was most worried about / least qualified to do. big grin

I'm just finishing the first draft of my project proposal now.