SCORM Module setting status not sent by package

SCORM Module setting status not sent by package

by Ian Ross -
Number of replies: 14

Hello,

I have a SCORM 1.2 package running on Moodle 2.5.2.

Previously the package has worked fine - it has no force new attempt or compatibility settings enabled. However, Moodle seems to have decided that it now wants to set the status as failed, no matter what happens in the module, when it is exited.

I have opened attempts and sent statuses back and checked them and it is changing between passed/failed/complete/incomplete no problem as the package runs. This is fine until I exit, then the status changes to failed but the package isn't setting this value. It is setting incomplete.

Here is an outline of the API log, where you can see the statuses I am sending and the final closing logs (sent by closing the module in the right way). I don't see failed here anywhere so it's very strange that this is how Moodle is recording it in the DB.

I should note that the time in the module is updating so info is being pushed back in to the DB.

SCORM API Activity Log

Tue, 08 Oct 2013 08:42:14 GMT: Moodle SCORM 1.2 API Loaded, Activity: Principle 1, SCO: jCloud_417
Tue, 08 Oct 2013 08:42:19 GMT: LMSInitialize("", "") => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.student_id") - admin => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.student_name") - Ross, Ian => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.lesson_location") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.credit") - credit => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.lesson_status") - not attempted => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.entry") - ab-initio => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.score.raw") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.score.min") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.score.max") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.total_time") - 00:00:00 => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.core.lesson_mode") - normal => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.suspend_data") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.launch_data") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.comments") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.comments_from_lms") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.student_data.mastery_score") - 70 => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.student_data.max_time_allowed") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.student_data.time_limit_action") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.student_preference.audio") - 0 => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.student_preference.language") - => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.student_preference.speed") - 0 => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.student_preference.text") - 0 => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.objectives._count") - 0 => 0
Tue, 08 Oct 2013 08:42:19 GMT: LMSGetValue("cmi.interactions._count") - 0 => 0
Tue, 08 Oct 2013 08:42:33 GMT: LMSSetValue("cmi.core.lesson_status", "incomplete") => 0
Tue, 08 Oct 2013 08:42:33 GMT: LMSGetErrorString("0", "No error") => 0
Tue, 08 Oct 2013 08:42:37 GMT: Commit("", "") => 0
Tue, 08 Oct 2013 08:42:37 GMT: LMSCommit("AJAXResult", "true") => 0
Tue, 08 Oct 2013 08:42:37 GMT: LMSCommit("result", "true") => 0
Tue, 08 Oct 2013 08:42:37 GMT: LMSCommit("errorCode", "0") => 0
Tue, 08 Oct 2013 08:43:46 GMT: LMSSetValue("cmi.core.lesson_status", "failes") => 405
Tue, 08 Oct 2013 08:43:46 GMT: LMSGetErrorString("405", "Incorrect data type") => 0
Tue, 08 Oct 2013 08:43:51 GMT: LMSSetValue("cmi.core.lesson_status", "failed") => 0
Tue, 08 Oct 2013 08:43:51 GMT: LMSGetErrorString("0", "No error") => 0
Tue, 08 Oct 2013 08:43:55 GMT: Commit("", "") => 0
Tue, 08 Oct 2013 08:43:55 GMT: LMSCommit("AJAXResult", "true") => 0
Tue, 08 Oct 2013 08:43:55 GMT: LMSCommit("result", "true") => 0
Tue, 08 Oct 2013 08:43:55 GMT: LMSCommit("errorCode", "0") => 0
Tue, 08 Oct 2013 08:44:09 GMT: LMSSetValue("cmi.core.lesson_status", "incomplete") => 0
Tue, 08 Oct 2013 08:44:09 GMT: LMSGetErrorString("0", "No error") => 0
Tue, 08 Oct 2013 08:44:13 GMT: Commit("", "") => 0
Tue, 08 Oct 2013 08:44:13 GMT: LMSCommit("AJAXResult", "true") => 0
Tue, 08 Oct 2013 08:44:13 GMT: LMSCommit("result", "true") => 0
Tue, 08 Oct 2013 08:44:13 GMT: LMSCommit("errorCode", "0") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.core.lesson_location", "0") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.core.lesson_status", "incomplete") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.core.score.raw", "0") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.core.score.min", "0") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.core.score.max", "0") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.core.exit", "") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.core.session_time", "00:05:05") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.suspend_data", "0^1^2^3^4^5^6^7^8^9^10^11^12^13^14^15^16^17^18~15,0^16,0~~0~1^0^0^0^0^0^0^0^0^0^0^0^0^0^0^0^0^0^0~0~0~") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.comments", "") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.student_preference.audio", "0") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.student_preference.language", "") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.student_preference.speed", "0") => 0
Tue, 08 Oct 2013 08:47:27 GMT: LMSSetValue("cmi.student_preference.text", "0") => 0
Tue, 08 Oct 2013 08:47:28 GMT: Commit("", "") => 0
Tue, 08 Oct 2013 08:47:28 GMT: LMSCommit("AJAXResult", "true") => 0
Tue, 08 Oct 2013 08:47:28 GMT: LMSCommit("result", "true") => 0
Tue, 08 Oct 2013 08:47:28 GMT: LMSCommit("errorCode", "0") => 0
Tue, 08 Oct 2013 08:47:28 GMT: LMSFinish("AJAXResult", "true") => 0
Tue, 08 Oct 2013 08:47:28 GMT: LMSFinish("result", "true") => 0
Tue, 08 Oct 2013 08:47:28 GMT: LMSFinish("", "") => 0
Average of ratings: -
In reply to Ian Ross

Re: SCORM Module setting status not sent by package

by Amy Groshek -
In your API log, lesson_status *is* actually set to failed at one point:

Tue, 08 Oct 2013 08:43:51 GMT: LMSSetValue("cmi.core.lesson_status", "failed") => 0


If you can provide a screenshot of your scorm settings, and the scorm package, someone else might be able to have a look for you.



Average of ratings: -
In reply to Amy Groshek

Re: SCORM Module setting status not sent by package

by Ian Ross -

Hello, Sorry I should have explained that the bits I have highlighted in blue are where I am setting the status using the API tool. It is taking these updates whilst the package is open so I can move it between states. It is when the package closes, it doesn't keep the status as the one the package last set.

Some digging our side seems to show that it is being set in the JS.

Settings attached.

I can't attach the SCORM as it won't work unless I know the URL launching it.

Attachment Screen Shot 2013-10-08 at 16.39.03.png
Average of ratings: -
In reply to Ian Ross

Re: SCORM Module setting status not sent by package

by Matteo Scaramuccia -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Ian,
check in your imsmanifest.xml file for Mastery Score settings: LMS is responsible to force/overwrite the lesson_status WHEN a mastery score has been set by the author and a score has been set by the content BUT that should happen only when the lesson_status is completed.

To verify if there is a bug in the Moodle SCORM module, we need to create a package replicating this issue: that's the reason why we need to "understand" your package, if we can't privately have it.

HTH,
Matteo

Average of ratings: -
In reply to Matteo Scaramuccia

Re: SCORM Module setting status not sent by package

by Ian Ross -

Hey Matteo,This was it! I think we have found a bug.

The course has a mastery of 70 and was passing back a raw score of 0 and a status of incomplete but Moodle was changing this to failed.

I checked this by removing the mastery score from the manifest and it worked as expected.

Clearly (if I understand you right) this is not the correct behavior as the course should be able to suspend itself half way (which could potentially be a failing score thus far) with a failing score and an incomplete marker. If it does not then it has forced an attempt closed and the user cannot resume.

This would be useful on the 'force complete' or 'force new attempt' but is not right in normal behavior inside a SCORM.

More than happy to send the test package in pvt solong as you tell me the launch url so I can allow access.

Let me know if I have gotten it completely wrong but I think this makes sense and is how it was working until I upgraded a few weeks ago.

I really like that SCORM is being tightened up tho and there are safeguards being put in place for rubbish packages. It's been much much better.

Average of ratings: -
In reply to Ian Ross

Re: SCORM Module setting status not sent by package

by Matteo Scaramuccia -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Ian,
I think I'm fine with the information you provided about Mastery Score: starting to review the code and recalling the past history. Indeed I need to recap in my mind the Moodle FAQ I've quoted - and wrongly reported in my previous post -, the SCORM Specification and some thoughts done in the past.

Let's start with MDL-21305 when completed was removed from the conditional branch by... me.

The SCORM 1.2 specs tell something slightly but significantly different from my previous post - apologies, I did a quick reply early in my morning - : see http://www.adlnet.gov/resources/SCORM-1-2-Specification?type=technical_documentation.

There, in SCORM_1.2_RunTimeEnv.pdf:

  1. at page 3-25, you'll find that the Mastery Score logics should be triggered when the lesson_status is equal to completed ... but only when the lesson_status is left to not_attempted by the SCO, when the LMS must move it to completed;
  2. at page 3-43 the specs define the rule to overwrite the lesson_status when a valid score(.raw) has been provided by the SCO and the package contains a valid mastery score.

That being said:

  • Moodle has no bugs here;
  • Docs trivial cleanup: FAQ must be reviewed in order to change the link to the conformance requirement specs, now dead (ADL changes their site "frequently"). @Dan, @Amy: shall we rewrite the FAQ to point to the RunTime Environment spec or to point just to a copy of the Conformance Requirements document, outside their site e.g. http://scorm12.com/SCORM1.2/SCORM_1.2_ConformanceReq.pdf - besides page number is 2-19 or the 39th, out of 155 -.  The 1.2 ADL Test Suite setup (CTS1_2_7STSetup.exe, http://www.adlnet.org/resources/adl-conformance-test-suite-verion-1_2_7?type=software_downloads) does not contain the Conformance Requirement specs. When we'll agree on what change, I'll do it;
  • You need to remove the Mastery Score info from your package if you don't want your LMS - not Moodle but ANY LMS - to overwrite the SCO status when the SCO closes the communication - calling LMSFinish() -  leaving incomplete for the lesson_status value.

If you look at the time when MDL-21305 and MDL-21306 have been implemented, your package worked as you'd expect in a Moodle version before 1.9.10. Isn't it?

HTH,
Matteo

Average of ratings: -
In reply to Matteo Scaramuccia

Re: SCORM Module setting status not sent by package

by Ian Ross -

Hey Matteo,

Thanks for this. The only thing that I can really say is that this behavior wasn't happening in 2.4 but is happening in 2.5. Beyond the usual upgrades, nothing special has been done to flip this behavior.

I see your point re the SCORM modules and the mastery score but I still think there is a fault in SCORM.

Why would Moodle force an attempt to failed when someone is half way through an assessment and wants to exit and then resume - without force completed/new attempt being switched on?

The documentation you refer to has the behavior for Pass/Fail when the status is sent to complete but Moodle is doing this when the status is incomplete. It has to be possible to suspend a package with a failing score and resume it. This was certainly available previously.

So I guess my question for confirmation is - Should a SCORM in Moodle be able to have a failing (raw) mastery score and a status of incomplete? 

If the answer is yes, then there is a bug, certainly in mine.

Can you point me to the right page and line where this logic is worked out? I can have my developer look at it in that case and put some heads on trying to figure out where this might be. I want to help out and not just pose problems.

Attachment Screen Shot 2013-10-11 at 12.50.41.png
Attachment Screen Shot 2013-10-11 at 12.57.13.png
Average of ratings: -
In reply to Ian Ross

Re: SCORM Module setting status not sent by package

by Matteo Scaramuccia -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Ian,
interesting, could you provide the exact 2.4 version you were using? In my spare time I could try to replicate the fact that in 2.4 it worked in a different way from the current 2.5.2+: it's worth trying to do some more investigations to find out a possible regression. We need also the list of your SCORM settings to identify if a combination of those settings is the root reason for your issue e.g. providing the vertical output of the activity settings via your DB e.g. in MySQL: SELECT * FROM `mdl_scorm` WHERE `id`=<id_of_your_affected_SCORM_activity>\G;.

About SCORM (not Moodle): the bug here is to provide a score, keeping the status incomplete: if you're writing a score, certainly you mean the session to be taken for credit. Isn't it? Besides the specs says a slightly different thing: an LMS should move from completed when it itself, the LMS, has moved to the completed status i.e. being the tracking session closed without setting, the content, a status i.e. being the SCO status equal to not_attempted... which means, in other words, that the Mastery Score logics applies even when the LMS should move the status from not_attempted to completed.

About Moodle SCORM code: it's quite easy to add the condition, I mentioned in my above post, back - though I'm interested in understanding the possible regression: as said above there was no changes in that SCORM logic since 1.9.10 - : look at https://github.com/moodle/moodle/blob/MOODLE_25_STABLE/mod/scorm/datamodels/scorm_12.js.php#L651, there you'll find the conditions under which the Mastery Score logics should run, again, regardless possible regressions in other areas when you moved from 2.4 to 2.5.

HTH,
Matteo

Average of ratings: -
In reply to Matteo Scaramuccia

Re: SCORM Module setting status not sent by package

by Ian Ross -

Hey Matteo,

Sure - I actually still have these running so it's easy to recreate. I can even point it down to a difference between 2.5 and 2.5.2.

I have a site on Moodle 2.5 (Build: 20130514) which can run the SCORM fine. User can resume and suspend the exact same SCORM as the one having issues in 2.5.2. I have attached the API output, SCORM settings and SCORM source in to Google Drive for you -  in the Forum Post Files.

Tracking back a little, I know there have been SCORM issues logging under the wrong SCOid (in 2.4) so I know that things have been happening, even if not exactly what files they are happening in.

This is definitely something inside of the Moodle and upgrade to 2.5.2.

Going to use that info to dig in a little here too. Let me know if you need more files.

Average of ratings: -
In reply to Ian Ross

Re: SCORM Module setting status not sent by package

by Matteo Scaramuccia -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Ian,
here is the "culprit": MDL-39363, https://github.com/moodle/moodle/commit/d8d0b8b39001e6cd07737680c4f97ab0e64322a6.

In your API log file the SCO sets a score equal to 0: before applying the fix above 0 was incorrectly not taking into account by the Mastery Score logics. Try by yourself by removing the doubled equal sign.

Next step?
Try to investigate the didactic reasons for setting - twice - a score equal to 0 when the status is still incomplete: any test has been submitted at that time? No, I'm guessing wink... and that could be IMHO a good starting point to solve your issue.

Keep also in consideration the need for having a Mastery Score in the manifest, triggering such a correct behaviour.

HTH,
Matteo

Average of ratings: -
In reply to Matteo Scaramuccia

Re: SCORM Module setting status not sent by package

by Ian Ross -

Hey Matteo,

That's excellent news. I actually found the same with our developer last night and we tried the change and it didn't work. Turns out we missed the '!' so this is great news for me. Three heads are better than two, it seems.

I have tested with the ! and it works!!!

Is this something that should be pushed back up to Moodle core?

Thank you so much for your help here.I really appreciate it.

I'm now going to make some changes my side as those SCORMS shouldn't have that mastery score anyhoo.

Average of ratings: -
In reply to Ian Ross

Re: SCORM Module setting status not sent by package

by Matteo Scaramuccia -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Ian,
unfortunately that change is a fix and not a regression, even if the fixed behaviour appears as a regression when you run your SCORM package.
We'll just leave the amount of info about your issue here in the Community, for future reference.

Next steps are all at your side, by reviewing your SCORM package to:

a. remove the Mastery Score for that SCO (it seems the selected choice);
b. change the SCO logics to allow setting a score when a score is really submitted by the actions performed by the user e.g. when doing some tests.

HTH,
Matteo

Average of ratings: -
In reply to Matteo Scaramuccia

Re: SCORM Module setting status not sent by package

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

Just following up here to back-up Matteo - Moodle is operating correctly in this case and following the SCORM spec - we won't be adding any code to revert or adjust this patch in Moodle - you will need to fix your SCORM packages to conform to the SCORM spec.

Average of ratings: -
In reply to Matteo Scaramuccia

Re: SCORM Module setting status not sent by package

by Mark Melia -

Hi Matteo,

I am having the same issue I think and what you are saying makes sense to me. One quick question regarding the following:

  • at page 3-25, you'll find that the Mastery Score logics should be triggered when the lesson_status is equal to completed ... but only when thelesson_status is left to not_attempted by the SCO, when the LMS must move it to completed;

I can see the aspect which outlines that the mastery score logic should only kick in when lesson_status = completed but I dont see anything mentioned regarding lesson_status and not_attempted in the spec. Could you point me to it?

Many thanks,
Mark

Average of ratings: -
In reply to Mark Melia

Re: SCORM Module setting status not sent by package

by Matteo Scaramuccia -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Mark,
find below the related quote, still in page 3-25, just the lines above the statement quoted few posts above:

LMS behavior:

  • Initialization: If it is the student's first attempt at the SCO the lesson_status is set to not attempted. The LMS is responsible for setting the initial value to "not attempted".
    • Additional Behavior Requirements: If a SCO sets the cmi.code.lessons_status then there is no problem. However, the SCORM does not force the SCO to set the cmi.core.lesson_status. There is some additional requirements that must be adhered to successfully handle these cases:
      • Upon initial launch the LMS should set the cmi.core.lesson_status to "not attempted".
      • Upon receiving the LMSFinish() call or the user navigates away, the LMS should set the cmi.core.lessons_status for the SCO to "completed".
      • After setting the cmi.core.lesson_status to "completed", ...

HTH,
Matteo

Average of ratings: -