Performance and report issues with a SCORM

Performance and report issues with a SCORM

by Alistair Cooper -
Number of replies: 6

Hi all,

We're piloting some eLearning packages at the moment, developed in Storyline 3 and published as SCORM 1.2, and are experiencing a few issues with the package stalling and not always capturing users responses in the report. I've posted this on the storyline forums, but  Dan rightfully suggested posting here too.

 

We've set up the course as per the instructions in this blog post - basically there is a page with a whole series of questions on it, and the response of each question sets a value to a separate variable (see attachment 'question screen screencap'). There is then a quiz slide for each variable, hidden from the user by a 'loading' image. The results slide then pulls through each of these quiz slides to the (interactions) report. There are 5 packages.

But we're having the following issues:

  1. When we pull off the reports, there are some oddities:
    1. The main one - users have completed the module, but not all their marks are being captured;
    2. In the status column, as well as completed/incomplete, some users have the entry '50';
    3. Some user's responses for a quiz have been captured twice - ie, on the report, there are extra columns at the end for just this user, which restate the results for some of their questions.
    4. One user is showing as complete, but no responses are captured for them.
  2. A few users find that the elearning stalls when going through the 'loading' screens (that cover the quiz slides that the variables are reporting to). If they close it down, reopen and resume from where they left off, it appears to continue from where they left off and take them through to the final screen. This 'stalling' is MUCH more common on the first run through a package. On subsequent run throughs it's rare for it to occur.
  3. One user has reported that the audio drops off for some of the videos for stretches of about 10 seconds.

 

Each module contains video files - we wondered if file size was one of the problems, but they are all about the same size, in terms of video content (about 55mb each). However the two modules where the issues seem to occur more frequently have a larger number of questions/variables and therefore quiz slides - about an additional 15 in total (a total of around 40 quiz slides). This makes us wonder whether this is overtaxing the connection with the moodle in some way?

Moodle wise, we're running version 3.1, hosted on our own virtual server.Any ideas? Thanks indeed for any help!


Attachment question_screen_screencap.PNG
Average of ratings: -
In reply to Alistair Cooper

Re: Performance and report issues with a SCORM

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

you mention that you are using a self-hosted site - SCORM is pretty sensitive to performance issues - a single page load results in a lot of calls to your server to return each file. On the first run through the package it may be caching the files in your users browser, so the 2nd run may be causing a smaller load on your server and making it run a bit faster for the user.

I would suggest that you take a look at some of the documentation around performance tuning of your Moodle server or engaging a Moodle Partner to help you tune your server - they may also be able to help diagnose some of your issues a bit easier.

Average of ratings: -
In reply to Dan Marsden

Re: Performance and report issues with a SCORM

by Alistair Cooper -

Thanks Dan, that's really useful. We'll investigate with the server team and see what we can come up with. Thanks for the moodle partner link too.

Average of ratings: -
In reply to Alistair Cooper

Re: Performance and report issues with a SCORM

by Silvia Pinheiro -
Hi Alistair,

Were you able to understand the reason behind the issues on the SCORM reports?


The SCORM packages we use were created with HTML5 and they were reporting everything we needed to build a custom report on Moodle.

At the moment, we are testing the new packages built for the third course and I noticed some missing results on the SCORM reports. I mentioned this to my project manager and she told me she had noticed some of this on the previous year. I tried to find the same issues but they don't seem to occur as much as now...She also says that she noticed this after we upgraded the Moodle version from 2.8 to 3.1

I've been trying to understand if it is a Moodle performance issue or the SCORM package. I uploaded two different packages on scorm cloud and they seem to be saving all the interactions and score as expected.

Here is one of the logs obtained from the cloud:

+ [10:13:59.527] LMSSetValue('cmi.interactions.19.id', '20') returned 'true' in 0 seconds
+ [10:13:59.527] LMSSetValue('cmi.interactions.19.result', 'wrong') returned 'true' in 0 seconds
         [10:13:59.527] CheckForSetValueError (cmi.interactions.19.result, wrong, cmi.interactions.n.result, 19, )
         [10:13:59.527] Element is: interactions.result
         [10:13:59.527] Call is error free.
         [10:13:59.527] StoreValue (cmi.interactions.19.result, wrong, cmi.interactions.n.result, 19, )
         [10:13:59.527] Element is: interactions.result
+ [10:13:59.527] LMSCommit('') returned 'true' in 0 seconds
+ [10:13:59.527] LMSSetValue('cmi.interactions.19.student_response', '0') returned 'true' in 0 seconds
+ [10:13:59.527] LMSCommit('') returned 'true' in 0 seconds

Here is one of the logs obtained from the Moodle API debug tool:

scorm bug report commit interaction

This would match to the missing record on the database. 

I don't have access, at the moment, to the apache logs, but hopefully I can see them during this week.


I would like to know if there is anything else I should test on our end? 


Thanks

Average of ratings: -
In reply to Silvia Pinheiro

Re: Performance and report issues with a SCORM

by Alistair Cooper -

Hi Silvia,


For us it turned out to be a database connection issue. Basically the connection between moodle and the database was dropping out every now and again, and because scorm packages 'ping' back and forth between moodle and the database multiple times, when any of those pings failed in their connection, various hanging behaviour would appear. Specifically, user's results not getting captured.

The below is from our database engineer:

"...the only reference to anyone having any trouble relating to intermittently failing on connection to a SQL Server that I can find is this email exchange which dates back to 2004 https://bugs.php.net/bug.php?id=29074. If you follow this all the way through you will see that there was no resolution and the originator put it down to a corrupt install of either php, apache, ubuntu or all three. He eventually retired the server and rebuilt with newer components and the problem went away."


His interim solution was to build a loop into the (moodle? database? bit beyond me, sorry) code so that if the connection fails, it will wait for 1/10th second and then try again, up to a maximum of 10 attempts after which it will assume the database server really is down. ie, if the database connection failed on one of the scorm 'pings', rather than contue on without capturing that data, it would try again, up to 10 times. That's solved it for us.


Hope that's of some use!

Average of ratings: -
In reply to Alistair Cooper

Re: Performance and report issues with a SCORM

by Silvia Pinheiro -

Good that it worked for you. I will investigate with the server team. Thanks Alistair. 

Average of ratings: -