Using H5P for high-risk assessment

Using H5P for high-risk assessment

by Dominique Bauer -
Number of replies: 14
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Hello,

This thread is the continuation of https://moodle.org/mod/forum/discuss.php?d=412988 and https://moodle.org/mod/forum/discuss.php?d=438964.

I might be wrong, but I think the correct answers are definitely not in the jsonContent. But who am I to argue against senior developers? Maybe they are encoded in the subContentId, for example "638ddbbf-a920-4aef-b27b-f08f37ba0715", which would be difficult to decipher.

Could someone with authority who really knows the facts answer the following questions.


With H5P, the correct answers can be revealed from the browser:
(select one)

○ True
○ False


If true:
(select one)

□ The answers can be revealed easily. Please demonstrate in an undoubtful manner.
□ The answers are encoded and can not be revealed easily.

Average of ratings: Useful (2)
In reply to Dominique Bauer

Re: Using H5P for high-risk assessment

by Oliver Tacke -
Hi!

1) True.

2) The answers can be revealed easily. Please demonstrate in an undoubtful manner.

One could check the `H5PIntegration` object (as demonstrated at https://campus.oercamp.de/lessons/sollte-ich-h5p-fuer-pruefungen-benutzen/), one could find the same information in the page source inside the script that's used to pass the values from the server to the browser, one could check `H5P.instances` and check the actual instance, one could have a look in the source using the browser's development tools, set breakpoints and even modify things, ...

Best,
Oliver
Average of ratings: Useful (8)
In reply to Oliver Tacke

Re: Using H5P for high-risk assessment

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Hello Oliver,

Thank you for your reply.

I found H5P instances in attached JS files... that didn't tell me anything. For the record, IMHO I would say that finding and understanding what the correct answer is from H5P instances in a script is probably doable for savvy programmers, but not doable for 99.9999% of Moodle users.
Average of ratings: Useful (1)
In reply to Dominique Bauer

Re: Using H5P for high-risk assessment

by Oliver Tacke -
Hi!

I am not talking about attached JavaScript files. An instance is the representation of the content type in the browser's memory and can easily be accessed.

I agree that this is one of the "harder" things, but anyone can type `H5PIntegration.contents` into the console and find his/her way to the parameters (`jsonContent`) as I demonstrated in the video. If just one person with very average coding skills required creates a browser plugin to make this even more convenient and a little better arranged ...

If that's "easy enough" to rule out "high-risk assessment" is up for you to decide, but please feel having been informed.
Average of ratings: Useful (4)
In reply to Oliver Tacke

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Deleted user

Re: Using H5P for high-risk assessment

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
...please feel having been informed.
Thank you for the transparency.

To be honest, I didn't believe it when I was told you could see the answer. I also didn't think it could be easy to do.

Knowing it's the opposite, unfortunately I will not use H5P questions in my quizzes because they are used to assess my students in a university engineering course, and I do not want anyone to think that they could cheat that way.
Average of ratings: Useful (2)
In reply to Dominique Bauer

Re: Using H5P for high-risk assessment

by Oliver Tacke -

Knowing it's the opposite, unfortunately I will not use H5P questions in my quizzes because they are used to assess my students in a university engineering course, and I do not want anyone to think that they could cheat that way.

That's fair judgement I'd say and what I'd recommend - not that I am a fan of exams the way they are today in the first place, but that's what H5P is not really suitable for. It's no silver bullet (like any other tool isn't one).

Average of ratings: -
In reply to Dominique Bauer

Re: Using H5P for high-risk assessment

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
How high is the "high-risk" here? To my knowledge H5P quizzes are not even tied to the Safe Exam Browser.
Average of ratings: -
In reply to Deleted user

Re: Using H5P for high-risk assessment

by Oliver Tacke -

Where are the other knowledgeable programmers? and why aren't they helping to set the record straight?

H5P was not designed for assessments (only). It should be (and is) usable for all kinds of contents and content for teaching and learning also goes way beyond the notion of assessments. But if high-stakes assessments are that crucial (I am not a teacher, I cannot judge), then choosing client-side evaluation was clearly a design flaw. I don't know the background however, so I can't tell what the prerequisites were.

Server-side evaluation is on the roadmap of H5P Group, but I don't thing this is going to hit the shelves soon, because it would mean quite some work to redesign the core of H5P and also existing content types.

So, the answer to "Where are the other knowledgeable programmers?" most likely is at "H5P Group". There are not too many H5P developers outside of the H5P core team, at least as far as I now of. I also don't believe that the moodle team is going to invest resources.

And my answer to "why aren't they helping to set the record straight" would be manifold: lack of documentation (for external contributors), lack of interest (the moodle team included H5P in core even though they knew it's using client-evaluation when they started, and they probably don't want to invest resources in "competition" that would make it easier to get server-side evaluation into other platforms, too), lack of funding (the number of developers who would work on this in their free is tiny), complexity of the project (many technical aspects, possibly even coordinate developers inside and outside of the H5P core team), ...

Average of ratings: Useful (4)
In reply to Oliver Tacke

Re: Using H5P for high-risk assessment

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I am far more interested in learning than assessment and H5P is a wonderful tool for learning. Once the learning is done Moodle Quiz is an excellent tool for assessment (especially with the cool 3rd party question types like Gapfill, Wordselect and Formulas). However people should be aware of the key differences.
Average of ratings: Useful (6)
In reply to Oliver Tacke

Re: Using H5P for high-risk assessment

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
In education, it is more and more about learning, for example with flipped classrooms. The harsh reality is that as long as there are report cards or grade transcripts and, for example, minimum levels of achievement for access to higher education, strict assessments will be necessary. So there is room for both types of questions. However, as Marcus said, people should be aware of the differences, which many probably aren't. As for me, I know now. I particularly thank Oliver for his openness and his explanations.
Average of ratings: Useful (4)
In reply to Dominique Bauer

Re: Using H5P for high-risk assessment

by Przemek Kaszubski -
Picture of Particularly helpful Moodlers Picture of Testers
I think this discussion should be PINNED !
Many thanks !
Average of ratings: Useful (4)
In reply to Przemek Kaszubski

Re: Using H5P for high-risk assessment

by Klaus Steitz -
Picture of Plugin developers Picture of Testers
Hello together,

thank you for this enlightening discussion and demonstrating the possibilities. Apparently it is question type dependent. It seems that with some H5P types picking the answers is not possible or very difficult? For the fill in the gap question type demonstrated in the video, I was also able to recreate/search it out in Moodle ("h5player (embed.php) -> h5p-iframe-[idnr]"), only in a different place than in the Drupal example (https://apps.zum.de/apps/15985). But I did not find it for multiple choice in Moodle (using Moodle 3.11.15).

Is it possible to say that the correct answers are findable for each H5P type, but it's just a matter of effort and knowing the right place?

Thank you very much for a classification!

kind regards
Klaus
Average of ratings: -
In reply to Klaus Steitz

Re: Using H5P for high-risk assessment

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
The issue is not the effort required to work out how to do it, it is more that if the stakes are high and one person somewhere can work out how to do it, that knowledge will quickly spread around the globe. if the stakes are high and something is difficult, someone somewhere will probably automate it, e.g. a greasemonkey/tampermonkey script.
Average of ratings: Useful (1)
In reply to Klaus Steitz

Re: Using H5P for high-risk assessment

by Oliver Tacke -

Hi!

H5P editors fill out a form that is structured by the semantics.json file (even if there's a visual editor used as in Course Presentation, in the end one fills out the form). That semantics.json file can easily be understood, in particular if you look at the rendered editor next to it. It's quite obvious that for "Multiple Choice" the answers field holds the answer options including the text and the correct state. The structure will vary from content type to content type, but the principle is the same for all of them.

The H5P Core transfers all the parameters that were entered by the editor from the server side to the iframe on the client side by passing a JSON object that follows the same structure as semantics.json. You could retrieve that information as part of the `window.H5PIntegration` object as `window.H5PIntegration.contents[cid-<contentId>].jsonContent` directly from memory. Just try running

for (id in window.H5PIntegration.contents) { console.log(JSON.parse(window.H5PIntegration.contents[id].jsonContent)); } // no error handling

in the development console inside either the H5P iframe's context or its parent's context and explore the output. You could even find the answers without knowing about the semantics.json file ...

You could do the same programmatically as well, of course.

Best,
Oliver

Average of ratings: Useful (1)