## General developer forum

### (Dreaded) Hooks in Moodle :)

This discussion has been locked because a year has elapsed since the last post. Please start a new discussion topic.

For at least the last 7 years people have been discussing adding a hook system to Moodle. Following similar systems to drupal and mediawiki. Most recently, Marina and Petr championed the hooks spec:

It's fair to say I've been very dismissive of this proposal, my main arguments against it:

• I don't think the 'lack of a hook system' causes problems. I think its the lack of hooks (whether 'hooks', or existing callbacks etc). I don't think that introducing the hook system will solve that lack of hooks. I don't see how we get to a useful hook system without extensive hooks landing too.
• Moodle was not built this way, the architecture is quite opposite to (say) drupal - for many developers, this is a plus  because it makes the code easier to understand with less indirection going on.
• It begins to muddy the potential new API design - do we use hooks or a traditional Moodle OO/module callback approach
• The potential for converting existing callbacks to hooks is just another thing which puts a burden on third party developers, so I wouldn't love to see that sort of change for changes sake
• A little bit of the viewpoint expressed by Tim in his old blog post - that the ability to hack the sourcecode is a solution for many institutions.

While I mostly stand behind my previous concerns, as time goes on and people mention the hooks system at any available opportunity, it wears me down and I consider if its something we should add to ensure it's not the 'lack of the system' which is constraining us.

There are some use cases where I am starting to become attracted to the idea. It comes down to the times when some sites want to change some fundamental functionality, it's too niche to expose everyone to setting and too small to invent a new plugin type (particularly larger Moodle deployments). These sort of scenarios seem to end up with us ever increasing our settings, or complex design considerations of how to support everyone with the new system and not simplifying. A current example which has sparked my thinking off in this is

I think in my vision of this, the hooks would not be used in our API design (it feels too counter to that). The hooks would start small in number and grow organically as real deployments find a need to hook in places. So rather than what happens now - where institutions need to either hack core or spend a great deal of time make their development sufficiently generic and considered enough for all cases, they could put the smaller investment of introducing a hooking point. In summary, i'm arguing against my first point - that having the system is worthwhile.

Thoughts?

Dan

Average of ratings: Useful (2)
Re: (Dreaded) Hooks in Moodle :)

I completely agree that the issue is that there are just not enough hooks in the right places. As a partner we churn through lots of custom plugins and our #1 focus is on not touching core if we can avoid it. Even if I don't push it upstream I prefer to add a new hook and then use it in a plugin rather than intermingle new code directly in core. There is also a lot of vagueness in the various discussions and trackers around whether the word 'hook' only applies to the proposal above, or any of the existing callback or older legacy api's. In my mind any and all of them are hooks, it's a generic term.

The thing that I've found the most challenging about MDL-28030 was that almost all of the discussion has been around which hook architecture we should use, when all I care is that the hook, or the callback or whatever is simple there and usable and stable.  I am far less concerned with *how* 3rd party code is called rather than whether it simply *can* be called without a core hack.

I'd prefer it if there was just a clear set of dev guidelines / best practice for adding new hooks (or whatever we call them), ie when to use get_plugins_with_function() vs when to make a new $CFG item. Because there is so much legacy code already mixed together it's easy to model new code based on existing code only to get negative feedback in peer review, so it would be great to have a wiki page with some pointers to examples to mimic and counter examples to avoid that a reviewer could put in the feedback and then the whole conversation would be over in hours instead of months. Spending literally 4 months discussing *how to call a function* is a massive waste of everyone's time. Hopefully next time will be better Average of ratings: Useful (2) Re: (Dreaded) Hooks in Moodle :) Thanks Dan for raising this again. I am personally a huge fan of hooks, I've been working with drupal before I joined moodle and I saw that hooks do not cause chaos but rather make systems very flexible but only flexible when it's needed. I've been told many times that moodle has a different way of evolving [by introducing uncountable number of settings]. At the same time I was the one who was collecting and processing the "wishtree leaves" from Moodlemoots and I saw that that the huge number of requests were to "create Moodle lite", "simplify interface", etc. Developers add more and more settings. Users hate these settings. When doing my triage or assigning peer reviews nowdays I often reject issues with proper patches that ask to create new settings or otherwise overcomplicate UI unless they demonstrate community support (as votes on the issues and/or forum discussions). I was the one who originally created the issue MDL-44078, and Petr shared his vision of the hooks API. I'm not saying this particular API is the only way to go forward with hooks. Actually we now cache the callbacks list and it is very "cheap" performance-wise to ask which plugins implement one or another callback. Using autoloading would be better imho but it's not the requirement. Hooks will simplify the UI, especially for administrators or teachers - please, keep this in mind. Average of ratings: - Re: (Dreaded) Hooks in Moodle :) What can you do with a hook that you can't do with a callback? I am not against an improved "hook" system though - but I still (strongly) think it should be based on interfaces and not on "magic class name voodoo". Average of ratings: - Re: (Dreaded) Hooks in Moodle :) Surely callback is a type of hook. Existing callback system is very counter-intuitive in moodle - plugin developers do not know what they can implement, there is no way to validate the arguments. Also it is very difficult to implement two-stage process callbacks - for example hooking into course reset form and actually doing course reset - this has to be two callbacks and such design is very difficult to follow. Average of ratings: Useful (1) Re: (Dreaded) Hooks in Moodle :) Marina, can you elaborate on the two-stage callback issue? What existing hooks/callbacks would you implement to do that? Average of ratings: - Re: (Dreaded) Hooks in Moodle :) Hi Brendan, among existing implementation I know two: 1. Course reset consists of callbacks reset_course_form_definition, reset_userdata and reset_course_form_defaults (there are even three and not two like I thought). So each module that wants to hook into course reset form has to implement three callbacks. 2. Recent activity - get_recent_mod_activity, print_recent_mod_activity and print_recent_activity. This is however an example of very bad design and this api desperately needs changing. In the future if/when we allow callbacks for the other forms - it is not possible to implement hooking into forms with only one callback - there are form definition, form validation and form processing stages. Most of other callbacks only need one-function interfaces. Average of ratings: Useful (1) Re: (Dreaded) Hooks in Moodle :) At this point, I'm neither for nor against hooks. There have been many times in the past when hooks would have helped immensely, but having worked my way through Drupal code, I know that they can make problem solving a nightmare. I think what is more important is to clearly define what we want to solve with hooks (or another solution). I know you are saying that we should just have more hooks rather than worrying about the solution, but that sounds like we'll just keep adding technical debt. Currently, we have several "systems" built into Moodle to provide solutions for some of the problems hooks would solve. The ones I can think of, are: 1. Class method overriding. 2. Events and event handlers. 3. config.php overrides (not all "hooks", but provide similar solutions) - there are many, e.g: 1.$PAGE replacements - $CFG->moodlepageclass,$CFG->moodlepageclassfile, $CFG->blockmanagerclass,$CFG->blockmanagerclassfile.
2. Memcache settings and replacements.
3. Custom script replacements - $CFG->customscripts. 4.$CFG->themedir.
5. \$CFG->altcacheconfigpath.
6. ...
4. Any callbacks that currently exist.
5. Renderers?

Many (all?) of these would not necessarily be considered a pure "hook" solution, but they definitely are there to solve similar problems. If we brought in more hooks and a new hook system, should it replace these? I would think, it "should" replace "customscripts" as a minimum.

Looking forward to this discussion...

mike

Average of ratings: Useful (1)
Re: (Dreaded) Hooks in Moodle :)
Places where we need to allow plugins to "hook" into the core functionality:

• Any place where we currently use callbacks
• course reset
• additional fields for the courses (hook into course edit form)
• allow plugin to make clean URLs by injecting into moodle_url
• course overview
• recent activity
• user profile
• user preference page
• cohort page (lots of plugins want to hook into this page)

I can go on forever

Average of ratings: Useful (1)
Re: (Dreaded) Hooks in Moodle :)

Regardless of whether we go with hooks spec as it is now, these two sections have good summaries of current solutions and use cases:

https://docs.moodle.org/dev/Hooks_spec#Current_solutions

https://docs.moodle.org/dev/Hooks_spec#Use_cases

Also in this comment I summarised all callbacks that existed in Moodle as of April 2014.

Average of ratings: Useful (1)
Re: (Dreaded) Hooks in Moodle :)
If we brought in more hooks and a new hook system, should it replace these?
Thanks for reminding me that to do something like that is one of the strongest arguments against doing this for me - I don't think the use case is strong enough to radically rethink the api structure of the entirety of Moodle - I'd much rather we spent that time on things with much more directly impacting user-facing changes.

There are some people in the discussion who think that our existing approaches are inadequate/suboptimal/ need replacement, I am not one of those people.

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)
##### So if I don't think existing approaches are inadequate why did I reignite this fire?

• (Correct me if i'm wrong) I think there is a case which we don't cover in our existing 'plugin points' well - where we want to allow a plugin to impact things outside of the scope of their plugin and affect moodle globally. e.g. the start of script/before output cases/ url rewriting. In these cases - we could put a global callback in place a bit like we do for navigation hooks.  But putting my admin hat on, it strikes me that we'd actually want to give the admin control over which plugins can do this - I might not trust a random block_clock to impact on every page or url, so a better managed hook system is desirable. But... am I actually talking about a permission system for plugins here..?
• The ability to add well described quick 'avoid hacking core' plugin points all around Moodle without too much overhead. Say, add a hook to show something extra in the course summary. What this is really competing against is - people hacking core to do it, or extending it to become a real plugin point. The 'hacking core' solution sounds bad and limits the ability of people to share their solutions, but then the burden of making a supported, backwards/compatible 'hook' is not trivial and adding a real plugin point is a viable solution - I was going to use mod_assign submission form or mod_quiz submission form as examples - but I am pretty sure they already have plugin points which can allow this sort of thing.

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

"add a hook to show something extra in the course summary"

We have had that since Moodle 2.0. It is called overriding renderers.

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

I would say that there are several separate parts to this, some of which are more clear-cut than others:

##### A) A way for Moodle core to call callbacks in different plugins

In the past, the convention for doing this is different functions in lib.php of the plugin, with a set name, like forum_add_instance, or quiz_extend_navigation.

The suggestion is, e.g. in the Securely passing callbacks from client side code thread, that we start using that with methods on an auto-loaded class. I think that is just obviously a better suggestion. It groups related things together in a more structured way that just dumping everything in lib.php. It also lets you have a common base class to put some of the common code that would help most implementations.

I think that this is just obviously better than what we did in the past, while also being quite similar, so it is not some huge strange new thing for Moodle developers to learn. It is a natural evolution of what we did in the past. In fact, I think in some places we already do it. E.g. mod_form.php form definition class in each mod plugin, implementing the definition() and validation() callbacks.

##### B) A discussion about the different ways we might use callbacks

For example, until now Moodle has had specific types of plugins for modifying specific bits of functionality. E.g. if you want to 'hook' format_text, then you have to implement a filter plugin.

As Dan says, this is quite good for developers: it makes it clear what the purpose of each extension point is, and what you have to do to implement it.

Also, it is good for trouble-shooting: when something is messing up display of user input, you only really need to look at the filters you have enabled, you don't have to consider every possible plugin.

There are some callbacks that are offered to all plugins. E.g. the settings.php file (which does not follow the standard pattern of a function in lib.php) or the ..._extend navigation callbacks (although that is really a set of similar callbacks, since different types of plugin automatically get added to the navigation in different places).

Now, we could implement a hook "add_random_extra_stuff_to_the_course_settings_form" and let any type of Moodle plugin implement that, but it potentially makes a big mess. At the moment, the way you can do that sort of thing is with a course format plugin, but course formats are a particular sort of thing. Do we really want the potential for spaghetti that opens up? (We might do. I should not use emotive terms like spaghetti.)

Another example is in the quiz, where I fairly recently implemented an add_random_extra_stuff_to_the_quiz_settings_form_and_let_that_affect_how_the_quiz_works sub-plugin type: That is better know as https://docs.moodle.org/dev/Quiz_access_rules, and it seems to work quite well. (I was able to design that with lots of examples of things that were already in Moodle core. Moving lots of things like time limits to sub-plugins cleaned up the code a lot.) Anyway, the point is there, I was able to do things the 'Moodle way' and add a new plugin type for a particular type of hook-in point.

##### Summary

There are two separate issues here:

I think we should be able to agree quite quickly that using auto-loaded classes rather than functions in lib.php is a better way to do callbacks in future. (But we should be slow to change existing callbacks and break backwards compatibility.)

We should be more hesitant about changing the principle of specific plugin types for specific extension points in Moodle, but it is worth continuing the discussions.

Average of ratings: Useful (1)
Re: (Dreaded) Hooks in Moodle :)
I have a simple suggestion for moving existing callbacks into autoloaded locations. Even though I was a co-author of https://docs.moodle.org/dev/Hooks_spec I don't fully agree with it. It was mostly Petr's design and it may be a bit overcomplicated. My initial title for MDL-44078 was "Proposal: API standard in Moodle that uses autoloading" and this is all I wanted, I added "(hooks)" to the title later when all discussion became around hooks.

So, here is my suggestion:

We create interfaces for callbacks as lib/classes/callbackbase/callbackname.php (interface \core\callbackbase\callbackname)

Any plugin (or core component) can EITHER

• define class implementing this interface that must be located in its classes/callback/ folder, for example \mod_foobar\callback\classname implements \core\callbackbase\callbackname (alternatively mod_foobar_callback_classname)
• define function mod_foobar_callbackname() it it's lib.php with the same arguments as the function in the interface.

We will add interfaces for some/all existing callbacks allowing the function alternative. New interfaces that we add in the future will not allow the function alternative.

Plus obviously some helper functions in core_component that will allow to find/call these callbacks (in all plugins / in all plugins of particular plugintype / in the particular plugin).

What will we achieve with this approach:

• Full backward-compatibility with existing callbacks
• All possible callbacks will be self-documented in phpdocs of the interfaces and they can be easily located in /lib/classes/callbackbase/
• All callbacks implementations by plugins must be located in their classes/callback/ folder, therefore it is easy to find which callbacks are implemented by particular plugin. Any IDE will immediately find you all implementations of a particular interface. Both these actions are not so trivial with the current callbacks naming.
• Callbacks implementations do not need to have the same class name as the interface name - it allows to have one class implementing several interfaces if plugin dev finds that they are related. I realise that it will also allow one plugin to have several classes implementing the same interface - in the cases where the caller requires to have no more than one implementation per plugin we will check it and show dev warning.
• Caching of callback implementations - we already cache autoloaded classes list and plugins implementing functions callbacks, therefore searching for the callback implementations will be super easy.

I liked very much the design we came up with in https://moodle.org/mod/forum/discuss.php?d=326884 and even implemented it in my MDL-51802 (here), however I changed it now to the callbacks because I agree that we must agree on the api first.

Thoughts?

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

Was just talking to Fred, he wants to see a clear distinction between callbacks and hooks. According to him, callback is some function that we call in a particular plugin, if we get to the point where we call it we expect that it must exist; hook allows all plugins to hook into some place of code, each plugin may decide to implement it or not.

There is no separation currently in Moodle and they all are called callbacks. It may be a good idea to distinguish, in which case my proposal above just splits into interfaces under "callbackbase" and "hookbase" and implementations under "callback" and "hook".

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

I just want to point out quickly that these concepts have existed in software for a long time and there is a distinction between the two ideas. We should be careful coming up with a moodle-specific definition. There are even separate wikipedia pages for each: hooks, callbacks.

Average of ratings: Useful (1)
Re: (Dreaded) Hooks in Moodle :)

There is one part of this that proposal that I not sure of:

• Are callbacks some amazing special thing that must be hidden in a separate namespace \mod_foobar\callback inside each plugin?
• Or are they just a natural part of some APIs? So the callbacks related to the inplace_editable should be in \mod_foobar\output and, suppose we replaced the old mod_foobar_pluginfile callback that would go \mod_foobar\file, etc.

I prefer the second of these. I think that it is more important that each API is a logically self-contained thing. It is less important that you can see in one folder a list of all the callbacks that you could possibly override if you wanted to do a million different things in your plugin.

If you do want an easy way to find all callbacks, just make a new empty interface (like renderable) and have all callbacks extend it.

Also, I an not a big fan of callbackbase as a namespace name.

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

I don't have an answer to the overall problem. Definitely some hooks would be useful, some key examples:

• At start of any script (i.e. end of setup.php before page code runs)
• Before output (i.e. before outputting header)

What I also wanted to say though is two things:

1) I think there's a bit of a distinction between plugins and hooks. Basically for things like filters, this is a thing that can often stand alone. For hooks, it's usually something like 'well I've built a block that does x, but in addition to that I want it to display a message at the top of every page while it's turned on'..

2) I wonder if some of these problems (but not all of them) might be resolved by rearranging plugin structure. For example let's say that all plugins (of any type) go in a root /plugins folder, with something in their version.php to define what type of folder - and let's say that within any plugin you could have another /plugins folder that contains child plugins - no particular link just they're packaged and installed together. (So you could have, for instance, a module like 'glossary' that contains within it a filter.) That would actually be a relatively minor change and would help improve some of the things that are wrong with the plugin system - but not all of the things that hooks are needed for.

--sam

Average of ratings: Useful (1)
Re: (Dreaded) Hooks in Moodle :)
Just commenting, FWIW, to say that Drupal is moving away from hooks. It's an old way of doing things, designed to accommodate old versions of PHP. There are much "better" approaches now that PHP has grown a bit.

As far as terminology is concerned, I think generally the word "callback", in the context of PHP and JavaScript, is associated with first-class functions; whereas a hook is usually associated with intercepting execution. They are similar (and probably used interchangeably sometimes) but slightly different.

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

They are NOT moving away from hooks, they are changing their hook api. And this is exactly what we want to do with our callbacks too

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

They are definitely moving away from what I would consider a hook in favour of a more modern approach (even if they continue to refer to it as "hooks"). In an earlier post you talk about having used Drupal hooks in the past, so I think you must have been talking about hooks that involve something like what Damyon referred to as "magical class name voodoo" (Drupal 6 or 7 style). My point was supposed to be that it's probably unwise to use Drupal as an argument for hooks when they are currently in the process of drastically changing the way "hooks" are handled. Apologies if I've misunderstood what you mean.

At the moment it is a bit hard for me to comment on what is best for moodle, or even on the current state of affairs, but I am keen to share opinions on programming practices. I can say that I see things done in unusual (to me) ways, and sometimes terminology is used differently to how I expect. In this particular case, I think we need to be clear on the distinction between "hook" and "callback" because I see both terms being thrown around and as an inexperienced moodle dev I'm not 100% sure what is being said.

To contribute something more on topic, in regards to interfaces and function/method names, I am hugely in favour of interfaces, I think we should push for that.

Average of ratings: Useful (2)
Re: (Dreaded) Hooks in Moodle :)
Hello all,

we have spent an hour discussing this in the HQ scrum today. I summarised the discussion on the issue MDL-44078

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

Hi,

There is a significant patch ready to be integrated over in MDL-44078 as well as a proposed overview of it all for Moodle docs.

Average of ratings: Useful (1)
Re: (Dreaded) Hooks in Moodle :)

Hi Dan, thanks for bringing this to our attention. It's good to have major changes like this pointed out in the forum (I'd been aware of the discussion for a while but was surprised to see a patch in integration review already!)

As a plugin developer, I'll be very happy to see the end of the current "specially-named global function" callback system and the new callback API looks really sound. I'm not really sold on the separate "hooks" API, although I can see it'll prevent core developers having to write a bunch of boilerplate in order to put an extension point into code, and there has clearly been a lot of discussion about it in HQ.

I have to say, though, I'd hate to see this integrated as-is without the nomenclature issue being sorted out. Quite a few people have pointed out that it's basically redefining "hook" and "callback" from their well-defined meanings in software engineering, and I don't really understand why this hasn't been taken on board. I certainly found the whole discussion,  documentation and code much harder to follow than it should have been because of this, and wouldn't relish the prospect of trying to explain it to a junior developer who will have learned the same meaning for these terms.

I'm certainly not saying we always need to strictly follow standard terminology, and I don't think this would be an issue if there were only the callback API - the confusion comes from the fact that there are two distinct systems named with words that for most trained software developers are two sides of the same coin. Could I propose getting rid of the word "hook" altogether and replacing it with something else (e.g. "hack" or "extension")?

Average of ratings: Useful (2)
Re: (Dreaded) Hooks in Moodle :)

I am new to moodle and come from using drupal. I'm quite alarmed that our team hacks the code if they want to change functionality. How can you quickly and easily update contributed code if you have modified it? I could be wrong that this is a problem, however, because I'm new to working with moodle. But it certainly seems like bad practice to me to hack contributed plugins rather than use a hook system.

Average of ratings: -
Re: (Dreaded) Hooks in Moodle :)

Hi everybody,  it's been years since I've come into this development forum.  But I thought I'd chime in here because I have a lot of experience with one of the open source projects that Dan mentioned (Drupal, 10 years exp):

The hook system in Drupal has served it well in the past 13 or so years, but it's not a system that Drupal developers seek to expand at this point.  If you like nerd words you can think of a hook system as a Global Event Dispatcher that allows custom code to register code that will execute when a specific event is executed.  You can write custom code that says, "Find anything that implements this hook, and let it run now".

Since the release of Drupal 8 last November, Drupal has included within it's core platform an actual Event Dispatcher.  As a result many of the new "hook" that plugin developers would introduce are created as a events / event listeners / even dispatchers.

If you're looking to introduce a hook system into Moodle, I would recommend looking at Event Dispatchers instead.  For several reasons:

1. It's a solved problem: There is a good chance you could look around to existing PHP Frameworks (Drupal uses Symfony2) to reuse their Event Dispatcher.  You could just drop a library in (as long as the rest of the code can work with composer / autoloading code).  I hear that someone made it possible to get Moodle installed with Composer, so you might already have that level of support.
2. 3rd Party code means less work for you (sort of): EVENTUALLY it will mean less work for you.  But, as Drupal discovered, having the ability to load a library of code to do a critical part of your platform will kick off a lot of work modernize your code (meaning working with a service container / Dependency Injection and finding ways to make your own code into libraries that code be independently loaded, etc.)  But once you do that you'll have a mechanism to update your platform code and its dependencies (which is basically Composer)
3. It eventually causes you to participate in the development of 3rd party code so that it can work better with your own code.  Drupal is an excellent example of this.  Drupal initially wanted to bring in libraries to provide a Web Service out of the box, eventually that meat adopting sever libraries from the Symfony Framework and we contributed a lot of code to those libraries to ensure we could use them.

Drupal also worked with CKEditor to ensure we could deliver a text editor out of the box.

It ultimately took about 4.5 years for Drupal to rewrite what we needed to rewrite in order to use all the Symfony libraries we wanted to use but that doesn't have to be your story.  If you just start by loading in an event dispatcher, then initializing / using it as you need to you'll start the project down the road of adopting modern PHP practices and libraries.

Edit: sorry for the wall of text, but here's an excellent book that basically describes the process Drupal has gone through be able to use 3rd party code effectively:

Modernizing Legacy Applications in PHP: https://leanpub.com/mlaphp

Average of ratings: Useful (8)