General developer forum

General Developer Meeting, 07:00 UTC, Tue 26 Feb

Picture of sam marshall
Re: General Developer Meeting, 07:00 UTC, Tue 26 Feb
Core developersParticularly helpful MoodlersPlugin developersTesters

I've asked Michael if I can explain my logging-to-file proposal at the meeting. Trying to gather support for it. smile

On the down side, if he says yes, I will have to get out of bed in time for it!


Average of ratings: -
Martin Langhoff - Sailing
Re: General Developer Meeting, 07:00 UTC, Tue 26 Feb
Hi Sam,

I have an alternative approach that might work to relieve the pressure on mdl_log, while keeping things simple and JOINable.

My worry is that cost of replacing JOINs with get_logs_blah() for pervasive stuff (ie:recent activity block, stats) will be significant.

So my proposed approach is to log it to a file... then batch-INSERT it to mdl_log (from cron for instance). Batched INSERTs are a lot cheaper than an INSERT per row -- that holds true across all the RDBMSs Moodle supports these days.

This can be drafted and perf numbers obtained with relatively modest effort...
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: General Developer Meeting, 07:00 UTC, Tue 26 Feb
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

Martin, we (the OU) did that in our Moodle 1.9 site.

In Moodle 2.x we have achieved a similar speed-up with a sneaky trick. mdl_log is a view that aggregates a small table where write are redirected, and a bit table that is basically read-only. The advantage of doing it this way is that it is transparent to Moodle, so no core code changes.

Even so, a very large mdl_log table still starts to cause performance problems. Talk to anyone who hosts large Moodle sites. The only option is to regularly throw away most of the log data (or copy it to somewhere else if you really want to keep it) but then it is really no longer joinable, and anyway we want to keep this data for analytics and research. 

Average of ratings: -
Dan at desk in Moodle HQ, Perth
Re: General Developer Meeting, 07:00 UTC, Tue 26 Feb
Core developersMoodle Course Creator Certificate holdersMoodle HQParticularly helpful MoodlersPlugin developersTesters

There is a forum thread on the Future major features forum that these comments might be better on. smile

Average of ratings: -