Database problem with 1.4 upgrade

Database problem with 1.4 upgrade

by Zbigniew Fiedorowicz -
Number of replies: 11
I just upgraded from Moodle 1.2 to Moodle 1.4. I noticed that the upgrade broke the display of site logs, i.e. that logs for the course site (courseid=1) stopped displaying, whereas logs for individual courses are still displaying correctly. After some debugging, I traced this to a failure of a database call. Here is the debugging code I place in course/log.php: 

$select = "username <> 'guest' AND deleted = 0 AND confirmed = '1' AND id NOT IN (1,2,5,6,1878,3605)";
$sort = "ORDER BY lastaccess DESC"; 
$limit = "LIMIT 0,99999";
$fields = "*";
$count = count(get_records_select("user", "$select $sort $limit", '', $fields));
error($count);

I find that the get_records_select call hangs. If I replace the 99999 in $limit by 1403, the get_records_select call succeeds, whereas with 1404 it fails. If I set $sort="", then it succeeds with 1408 and fails with 1409.

In Moodle version 1.2, the get_records_select call succeeds with 2460 and fails with 2461. I guess that this is some kind of configuration problem with MySQL/php/adodb? Any suggestions?

TIA,
Zig

Average of ratings: -
In reply to Zbigniew Fiedorowicz

Re: Database problem with 1.4 upgrade

by Zbigniew Fiedorowicz -
Just to forestall the obvious explanation, let me note that I used phpMyAdmin to check/repair the Moodle user table. I didn't get any error messages, and the problem persists.  Also I upgraded by making a copy of the version 1.2 database, and still have the original database running along with the previous Moodle version 1.2 installation.
Average of ratings: -
In reply to Zbigniew Fiedorowicz

Re: Database problem with 1.4 upgrade

by Zbigniew Fiedorowicz -
OK, I figured it out. It is a PHP memory problem. I didn't see this at first, since I had set 
memory_limit = 64M
in my php.ini, which I figured was more than enough for any purpose. Unfortunately I didn't notice the following line in lib/setup.php:
@ini_set('memory_limit' , '16M');
I found that to get site logs to work on my site, I had to replace 16M here by at least 48M. Also note that I did not get any out of memory error messages, even with the Moodle debug switch set to on.
Average of ratings: -
In reply to Zbigniew Fiedorowicz

Re: Database problem with 1.4 upgrade

by Matt (M) -
Why does Moodle set its own memory-usage limit (do I understand this correctly)? Is this a "play nice with system resources" attempt? Are there lots of memory leaks with this software (seems tough to do if it's all in php code...but I'm not a php programmer)?

This seems to be asking for trouble. Given what I know now, I'd rather that Moodle, by default, simply warn it's admin(s)--probably via email as well as a 'message warnings' admin-only web area--if it's hogging memory beyond a reasonable level (something that's far beyond 16MB...which is quite low for systems nowadays)...but maybe I don't have sufficient knowledge/background on what's going on in this scenario?

-Matt
Average of ratings: -
In reply to Matt (M)

Re: Database problem with 1.4 upgrade

by Zbigniew Fiedorowicz -
The comment next to that line of code reads:
/// Increase memory limits if possible
I guess Moodle expects that memory_limit might be set to 8M and is trying to increase it to 16M, which used to be enough for versions < 1.4.

Average of ratings: -
In reply to Zbigniew Fiedorowicz

Re: Database problem with 1.4 upgrade

by Matt (M) -
Oh, I then take this to mean the "incremental memory size" of a request for *increase* in memory from the OS/system by the Moodle application.

If so, this makes a lot more sense to me. Thakns for the clarification (if this is indeed the case).

-Matt
Average of ratings: -
In reply to Matt (M)

Re: Database problem with 1.4 upgrade

by Zbigniew Fiedorowicz -
Well, not exactly.  If you have memory_limit set to something higher than 16M in php.ini, then this line of code will result in a decrease of memory available to Moodle.
Average of ratings: -
In reply to Zbigniew Fiedorowicz

Re: Database problem with 1.4 upgrade

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
That is correct: until recently the default PHP limit of 8Mb was only occasionally being broken, so I assumed 16M would cover it. In fact 16M works fine for me on all my hosted sites (including Moodle.org which has a lot of activity).

Why are your sites consuming so much memory? That is the main issue here. Something is not right there.

Are you using non-standard modules that are not careful about memory use? Could all of you having this problem please list the extra modules or blocks you are using?

I'll fix the ini_set in lib/setup.php to set it to 64Mb for now but we really must trace down the problem for those sites that are having it.
Average of ratings: -
In reply to Martin Dougiamas

Re: Database problem with 1.4 upgrade

by Tim Allen -

Well, I had the standard modules for 1.4 installed, along with questionnaire, scheduler, exercise and book.  I even deleted scorm and attendance (why is that included in the standard distribution?), and was still having problems.

I am also running phpMyAdmin on the server.

Tim.

Average of ratings: -
In reply to Tim Allen

Re: Database problem with 1.4 upgrade

by Matt (M) -
For what it's worth, I was confused about Moodle/php memory limit stuff, until I stumbled upon this "memory limit" glossary reference:

http://moodle.org/mod/glossary/showentry.php?courseid=5&concept=Errors+about+%22allowed+memory+size%22+being+exhausted

This answered my questions and cleared up my confusion in this area.

This reference was not showing up in many cases (like this thread) because it looks like everyone was using the "memory_limit" text in their description...with the underscore.

Might it help future conversations to link this glossary entry to "memory_limit" as well? Where do I submit a request for this? The doc project?

-Matt
Average of ratings: -
In reply to Martin Dougiamas

Re: Database problem with 1.4 upgrade

by Sushil Bajpai -

I was reading the release notes pertaining to php accelerator http://www.php-accelerator.co.uk/releases/1.3.3/release_notes

and there is mention of a bug in PHP memory handling

"A bug in PHP memory handling causes PHP builds with --enable-memory-limit to believe that memory is already allocated at the start of each request, and for the amount to increase at each request.

The bug exists in PHP 4.2.1, 4.2.2, and probably some or all other releases.

Release 1.3.3 incorporates an experimental workaround for this PHP bug and
can be enabled with by setting the php.ini entry
enable_php_memory_bug_workaround = 1"

Further in http://www.php-accelerator.co.uk/releases/1.3.3/CONFIGURATION

they write

"; Enable a workaround for a bug in PHP that can cause PHP to incorrectly
; report that a script has run out of memory. Defaults to 0, i.e. disabled.
; Set to on or 1 to enable.
; NOTE: This is only relevant is PHP was built with --enable-memory-limit

phpa.enable_php_memory_bug_workaround = 0"

Could this be the reason that some sites are/were having problems?


Average of ratings: -