General developer forum

The API is all of accesslib.php as far as I'm concerned.

Martin -- can we avoid taking that position? Only a few of those functions are ever used outside of accesslib and a few upgrade and special scripts. If we can segregate the "public" part of accesslib vs the "private" part, I have a chance of reworking things to run faster.

Why do you want to make it all public?

The checking is not insane ...

Well... it is not insane for data coming from the end user. But it is for functions that are internal, where the caller is supposed to have done the homework first. And in this case, the checks are very expensive.

Yes, role assign/unassign is slow - it was expected to be slow.

Hmmmm. Actually it isn't expected to be slow on 1.4/1.5/1.6 it was really fast! And that's what all the large installations were counting on.

One of the expensive operations is forum subscription init/cleanup.

Thanks for the hint - I'll look into it - I'm sure there's a way to do it in pure SQL over many enrolments.

One of the expensive operations is forum subscription init/cleanup.

Ok - I've given this a bit of a read. My initial notes:

We have a problem in course and category moving. It used to be really lightweight as categories had no bearing on Moodle behaviour. Now one of our main challenges should be moving categories cheaply because there is a ton of work that needs to happen...

Except that we aren't doing it... Do forum default subscriptions still work (if you are a student category-wide and courses are moved)? Is there a way forum is catching those users somehow?

I am not too partial to implementing these -- as I don't see any easy way we can support category-level enrolments in a scalable fashion. Personally, I am more inclined to discourage category-level enrolments and add an option to switch them off. At least until we find a scalable way of dealing with them. In the meantime, it is cheap to optimise for the case where there are none.

Questions:

• If listing course participants was very cheap - could we drop these forum_assign_role() calls, and use an "unsub" table to keep track of who's explicitly unsubbed from the default forum subscriptions?
• If that works, and noone cries, can we drop the "call all modules" part of assign_role()? }-)
• Or -- can I change the semantics...?

In terms of changing semantics, I am thinking of either changing the way we call it, or supporting an alternative $mod_assign_role_batch(). The idea is that is will be callable from any and all of

• cron
• at each role assignment
• at the end of batch role assignments

The module keeps track of the timestamp of the last role assignment processed and will handle a single assignment or a thousand with a constant number of DB queries. We might need some extra thinking for role unassignments (maybe log them temporarily so we can execute the unenrolment actions...)

And I want to say... I'm sorry! -- I know all of this is a bit annoying, and it is hard work... But we must switch to an approach to abstractions that plays better with the DB than making db queries inside code loops.

We have to move all the SQL out of the loops, gather a ton of data using JOINs and use the loops (walking those evil recordsets!) to process it. At least that's the only approach I know that will help us scale the way 1.4/1.5 scale and further.

If we move towards such constant-db-queries model in the core code, then Moodle's scalability bottleneck shifts to the DB, and the current crop of DBs are really blazing fast. And they all have a bagful of tricks we can use to make things even faster...