Some Hotpot quizzes crashing Apache

Some Hotpot quizzes crashing Apache

by Trevar Pearce -
Number of replies: 5
Hi there,

Just wondering if you might have some insight into a problem we're having with Hotpot. We are finding that since an upgrade to Moodle 2.9.5+ and Hotpot build 2016011287 (we have also had this problem on the most recent version of the plugin), we are seeing Apache crash when certain Hotpot quizzes are attempted. We have narrowed it down to quizzes which have at least one question that has at least one question text element or feedback element with 105 or more characters in it (at 104 or fewer characters, there is no problem). The Apache error log is in this case not all that helpful, but there are clues that lead us to believe that it is a stack overflow issue.

We are running Apache 2.4.18 and PHP 5.6.19 on a Windows 2012 R2 server. Database is MySQL 5.6.

I have attached one of the problematic quizzes.

Any thoughts
?

Trevar
Average of ratings: -
In reply to Trevar Pearce

Re: Some Hotpot quizzes crashing Apache

by Gordon Bateson -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Trevar,
thanks for supplying a sample file to test on.

It seems to run OK on the Moodle 2.9 (MacOSX, Apache 2.2.29, MySQL 5.5) site on my research server:

We need to find out why Apache on your server is choking. 

When you say, "Apache crash", do you mean that Apache comes to a stop and has to be restarted? What do the error logs reveal? What do you mean by "clues that lead us to believe that it is a stack overflow issue"?

We have narrowed it down to
quizzes which have at least one question
that has at least one question text element
or feedback element with 105 or more characters in it
(at 104 or fewer characters, there is no problem)

If this is true, then it should be possible to reduce the sample jqz to just one question that breaks your server. Please could you make that file, confirm that it breaks you server, and attach it to your next post to this forum.

cheers
Gordon

In reply to Gordon Bateson

Re: Some Hotpot quizzes crashing Apache

by Trevar Pearce -

Hi Gordon,

Sorry for the delayed response. I've attached the quiz edited down to one offending question. It breaks on our server but, as with the full version, not on your test server.

The Apache error log shows:

[Tue May 17 09:19:30.420877 2016] [mpm_winnt:notice] [pid 1240:tid 620] AH00428: Parent: child process 4984 exited with status 3221225725 -- Restarting.

So Apache crashes and auto-restarts immediately. A Google search on that status code returns some results - all older, unfortunately - that indicate a buffer or stack overflow issue is occurring. Most of the results show that it's an Apache/PHP on Windows issue. Some suggest increasing the thread stack size, but that setting (again, for Windows users) has been removed from newer versions of Apache.

I have since tried restoring to an older version of Hotpot (2015021162), and the problem doesn't occur using any of the problematic quiz files, including the one attached here.

Trevar

In reply to Trevar Pearce

Re: Some Hotpot quizzes crashing Apache

by Gordon Bateson -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Thank you Trevar, for the simplified test file, and the extra information regarding error messages.

I would guess that the crash occurs in the "mod/hotpot/submit.php" file, which is the script that receives and processes data from a student's attempt.

Could you try removing the following lines (around line 96-97 in "mod/hotpot/submit.php") and let me know if that alleviates the problem:

  • $completion = new completion_info($course);
  • $hotpot->update_completion_state($completion);

Gordon

P.S. If you could let me have access to a server that has these issues, I would be interested to come and investigate.


In reply to Trevar Pearce

Re: Some Hotpot quizzes crashing Apache

by Ryan Smith -

Trevar,

These errors happen all the time with Moodle while running PHP as an Apache module on Windows. There are unresolved bugs that have been around for years using the module method. The errors increase as you add any sort of user load to the server. 

You can resolve them by switching to mod_fcgid. mod_fcgid is very stable with Moodle on WAMP.

I'm running Apache 2.4.20 with PHP 7.0.6 (NTS) and mod_fcgid 2.3.9. No crashes and around 500 users logged-in at once.

Average of ratings: Useful (1)
In reply to Ryan Smith

Re: Some Hotpot quizzes crashing Apache

by Trevar Pearce -

I had my attention diverted from this for a while but have now tried out the mod_fcgid module in Apache. It has resolved the problem! Thanks to both of you for your assistance.

Trevar

Average of ratings: Useful (1)