Moodle in English: My Total developer rant with the Events API

Ok let's start this off with a full and total rant, The new Events API sucks terribly.

I am ripping my hair out dealing with the total and utterly uselessness of the event API triggers putting in simple replacements of add_to_log calls. Anyone thought that replacing an add_to_log one of two LOC call with:

Create a specifically named class
In a specifically named directory two levels deep
with default required classes
without any proper data handling, but relying on "other" for much data handling, which:
- Does not correctly evaluate values, but due to the construct requires already assigning them to variables before passing it
providing no standard handling of legacy add_to_log replacement, but requires manual adding of these calls
requiring at least 20 LOC to achieve a simple "view" call
being the most counter intuitive thing I have ever encountered in MY ENTIRE LIFE

Should be considered to be a rabiate puppy. Cute at first, but eventually have to be taken behind a shed and shot.

The Events API from a developer point of view appears to be the worst invention since Mustard Gas.

Average of ratings: Useful (1)

Re: My Total developer rant with the Events API

by Conn Warwicker - Wednesday, 27 August 2014, 6:47 PM

If that's the most counter intuitive thing you've encountered, I'd recommend not looking too much deeper into any of the Moodle code.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by David Bezemer - Wednesday, 27 August 2014, 7:00 PM

I have and the rabbit hole goes deep.

Average of ratings: -

Re: My Total developer rant with the Events API

by Davo Smith - Wednesday, 27 August 2014, 9:16 PM

Now you've had your rant, would you like to take a look at the documentation for the events API ( https://docs.moodle.org/dev/Event_2 ), particularly the reasons why the new events system was needed, then give a step-by-step explanation of exactly how you would have designed it to work and/or specific ways that you think it could be improved?

Yes it is more code, but it achieves much more than the old system was able to do, makes it much clearer to other developers what events/log entries your plugin is producing and, after not very much time spent using it, isn't particularly difficult to write code for.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by David Bezemer - Wednesday, 27 August 2014, 9:55 PM

It might not be terribly difficult to write event calls that use standard events (like view resource), but try adding event calls for grading events. There are so many undocumented "features" in undocumented functions that return undocumented results.

So ways to improve:

Define default events for default events (view, grade, error, update, delete, etc.)
Do not tie these events and event declarations to specific plugins. Grading an assignment is in no way different from grading a quiz for logging purposes.
Do the above to make it usable with all types of resources in a coherent way (no different calls for mod and block)
Do not require redeclaration of all functions before you can use them (duplication is the worst sin in MVC patterns)
Reduce the LOC needed for declaration of events by following #1 by using default events
REMOVE the dependency on "other" for almost half of the events, like SERIOUSLY????
Provide usable examples of events to be created for real plugins, not some loosy-goosy partial examples that actually cover different plugins, but look like they are part of the chain
Create vocabulary for events, so not every plugin has to re-declare vocabulary (which also means that all of the events in Moodle are currently not multilanguage, but english only, just look at all the hardcoded strings in mod_assign)
If you want to make Events 2 actually usable, add hooks support as default
Simplify the event handling and requirements, it's not a spaceship, they are log messages
Provide proper error handling for events, breaking debugging output because a not yet fired event is invalid is retarded
Class autoloading would be useful if it actually auto-loads, not requiring increasing versions to trigger upgrade for every single change or addition

Also, I have read the documentation, which is a large part of the contribution to the frustration.

Simply looking at "Why was a new events system needed?" triggers an allergic reaction with me:

Log messages could contain much more advanced information than events can, making me seriously question that nobody spotted a contradiction between 1 and 2
Reason for duplication was simple: logs are different than events, took a shortcut to DB (awesome performance) and allowed arbitrary developer aimed log data to be added. Also defending code duplication and instead requiring mountains of code to be written to achieve the same seems backwards to me
It is clearly built with plans in mind that were never realized. The spaceship events are there, but simple hooks support isn't, and you cannot trivially create an observer for any event as it needs to be tightly namespaced. Looking up whether a namespace is declared/exists is a terrible usage of CPU cycles and class autoloading.

Also: if you look at those docs you can see they consist of half finished items like "This next two paragraphs need to be replaced with results."

I for one have simply reverted to using add_to_log() for anything except users viewing resources, as that is the only event that seems to have a clearly defined default event firing. Do I care that debugging will show deprecation messages? Not a slightest bit, as performance of add_to_log is a multitude better than any event.

TL:DR: Events 2 is as half-baked as anything gets, prematurely released, badly documented. Typical Moodle.

Average of ratings: -

Re: My Total developer rant with the Events API

by David Mudrák - Thursday, 28 August 2014, 3:20 AM

Thanks David for sharing your feedback. It's good to hear from plugin developers on the results of the core teams work. I'll make sure to pass this to get response and make steps to improve things further. I believe there is a space for simplification of these APIs - and detailed proposals and suggestions like this may help. Please let's try to keep this conversation in a constructive spirit. Blindly casting blame-balls on all sides won't help anybody. Thanks.

Average of ratings: Useful (1)

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 29 August 2014, 9:34 PM

David,

exactly why the topic title came with a warning (rant ahead).

I think the biggest improvement for the Events API would be defining standard events like course_module_viewed that can be triggered without the need of (re)writing entire classes.
Moodle largely consists of default actions inside all modules, and I think this would make the most sense to implement.

Also, please add a specific log/exception event and make sure events do not rely on 'other', as that is the most frustrating thing to find out (other is required for some fields, very counter intuitive).

Average of ratings: -

Re: My Total developer rant with the Events API

by Darko Miletić - Wednesday, 27 August 2014, 10:26 PM

You are not the only one who feels that way.

https://moodle.org/mod/forum/discuss.php?d=256457#p1137708

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Wednesday, 27 August 2014, 10:45 PM

I had my suspicions. I would not say it is over engineered tho. A solution like monolog (https://github.com/Seldaek/monolog) is over engineered, and yet does a fantastic job at providing a logging framework that supports watchdog functions/triggers.

Other good examples are for instance the Drupal watchdog, superbly easy logging and support hooks out of the box.

But judging by the typical response the Events API suffers from the NIH syndrome just as much as Atto, forums, webservices, and in general the non-MVC framework

Average of ratings: -

Re: My Total developer rant with the Events API

by Valery Fremaux - Thursday, 28 August 2014, 4:32 AM

Actually Event API was not the only change that raised such headakes... i remember the first time i tried to complete a full handling of files in the new moodle file system !

Each time an implementation is restructured causes such heatings in the developping community with almost the same queries :

Why did they do it so complex and in appareance overstructured ?

After a while you get a bit more trained, and seing how many common errors you got in the past on unchecked situation are now avoided helps you understand the underlying structural benefits. Now i understand (yet still grumbling a bit) the large code overhead files are needing now.

Moodle gets raising to industry level in it code, though not any more "trivially pragmatic", but systemically strong. I suffer just like you do trying to reinstall the logging features in my 70 plugins and am far from result ! So if core team has an enlightening idea to give us some more compact codecuts to get minimalistic implementation, this will be much appreciated.

If you want to fight your neurones against moodle complexity : look a bit deeper the backup engine, or the question/behaviour engine... it's really joyfull !

One just tricky consquence of some geeky trends of our core gurus is an increasing difficulty to deal with our customers for development budget support as the API complexity requires much more time to get things done in a way we cannot even explain the first word to our stakeholder. This is an economical consequence of immoderately adding more patterns and deep structure, while external features and user experience do not seem being drastically a revolution (by now -- there are revolutionary changes indeed, but only perceptible to skilled users that have followed all the path).

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by David Bezemer - Thursday, 28 August 2014, 6:14 PM

Valery,

I develop plugins for many other platforms as well, Kaltura, Joomla, Drupal, Wordpress.
None of them have a complex system for logging, and that has a clear reason: logging is supposed to be simple, fast and efficient.

Where I do understand that Moodle develops, I disagree with your idea that they are raising to industry standard. Lacking proper MVC patterns, having mountains of hardcoded strings, checks and paths in the code is in no way good practice, or even near industry standard.

What instead seems to be necessary is not more structured features in Moodle, but rewriting and optimizing the core essentials. A page load requiring over 600 files, taking 0.5s of compiling/interpreting time (with OPcode enabled) are a clear sign of bad design.

If you realize that just the Moodle libs are bigger than Joomla, Drupal, Wordpress as a whole, you know that there is something clearly going in the wrong direction. Also the growth of these libraries is at a staggering rate of a few Mb per release, this means thousands of lines of code.

As to the guru-ness of the core developers, I stated before that they seem to have a strong case of the NIH syndrome, and I stand by my words.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by Jose Barrera - Thursday, 4 September 2014, 10:30 PM

Hi David,

I agree with you so strongly that I registered just to reply to you and add my voice of support.

Indeed Moodle is way behind industry standards. What I think Moodle needs is a methodical refactoring that will take it from being a legacy application to being a modern framework.

Such methodical refactoring would involve, at the very least, the following steps:

1. Moving from the include-oriented architecture to an object oriented architecture (this involves the use of classes and autoloading).
2. Completely obliterating the use of globals from the code base.
3. Introduce dependency injection
4. Add unit tests for all the new classes added in step 1.
5. Introduce an ORM system
6. Implement an MVC architecture
7. Introduce a routing system instead of relying on an architecture of direct access to php scripts; in other words, decouple URL paths from file paths.

There are perhaps many more improvements that can be suggested but these are the ones, off the top of my head, that I think are more needed and that will lay a solid foundation to be able to bring the Moodle code base up to current industry standards.

What every developer at Moodle HQs should have is a copy of the book Modernizing Legacy Applications in PHP by Paul M. Jones to read in their spare time; it basically outlines a very sensible path to modernise a legacy application like Moodle whilst keeping it functional at all times (no massive rewrites from scratch required or even prescribed).

Just my two cents.

José

Average of ratings: Useful (5)

Re: My Total developer rant with the Events API

by Darko Miletić - Thursday, 4 September 2014, 10:59 PM

5. Introduce an ORM system

I hope this will never happen. ORM is well known performance killer, or if it is introduced let it be optional not forced upon all developers.

Average of ratings: -

Re: My Total developer rant with the Events API

by Rex Lorenzo - Friday, 5 September 2014, 10:50 PM

There already is an database abstraction layer in Moodle, the DML libraries (https://docs.moodle.org/dev/Data_manipulation_API). But they are useless once you need to join tables. So you end up resorting to direct SQL that might not be compatible across different databases.

I believe that ORM is suppose to further that abstraction from the actual implementation. The balance between usability and cross database support could outweigh the performance hit.

I have seen some very cool and fun ORM features in Active Record when I was running a Ruby on Rails app.

It should be useful to note that the Symphony project uses an ORM called Doctrine (http://en.wikipedia.org/wiki/Doctrine_(PHP)).

Average of ratings: -

Re: My Total developer rant with the Events API

by Marcus Green - Friday, 5 September 2014, 10:57 PM

How much of a problem is creating table joins that work across databases?,

Average of ratings: Useful (1)

Re: My Total developer rant with the Events API

by Rex Lorenzo - Saturday, 6 September 2014, 12:40 AM

It's not that it is a problem, it is that you are back to SQL land when previously that was abstracted away with function calls. Eh, it's just that an ORM is a "nice to have" and fun to use.

Average of ratings: -

Re: My Total developer rant with the Events API

by Tim Hunt - Friday, 5 September 2014, 11:47 PM

Two things worth reading:

A more balanced view is

http://martinfowler.com/bliki/OrmHate.html

which includes this quote when taking about approaches of just using SQL in your application like Moodle does.

"They work very well for applications where you're just pushing data to the screen and back, or for applications where your logic is well expressed in terms of SQL queries. Some problems are well suited for this approach, so if you can do this, you should. But its flaw is that often you can't."

Most of the time we are "just pushing data to the screen and back, or for applications where your logic is well expressed in terms of SQL queries".

Average of ratings: -

Re: My Total developer rant with the Events API

by Tim Hunt - Friday, 5 September 2014, 1:37 AM

In addition to Darko's point,

1. and 4. are already work in progress.

2. in an interesting point. Moodle mostly uses globals to implement the Thread scoped registry pattern, which is not necessarily bad http://tjhunt.blogspot.co.uk/2009/04/php-global-variables-are-not.html

3. Dependency injection, like ORM, is just one possible pattern to use, and it can just serve to obfuscate things. I am not convinced it is right for Moodle.

6. Large parts of the Moodle architecture are already MVC. If you don't reacognise that, perhaps you stil lhave something to learn.

7. Why?

You use a bunch of emotive, but basically meaningless words in your post. E.g. 'industry standard', 'modern' without any justification. One could equally describe some of the things you advocate as "trendy" or "flavour of the month". It does not actually help the debate.

Also, remember the very wise words of the Perl motto: TIMTOWTDI.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 5 September 2014, 6:20 AM

Tim,

I will take a lot of things you say for granted based on your credentials, but Moodle has no MVC model at all. Any form of MVC separates Model, View and Controller, and by the piles of HTML (or htmlwriter) in all libs in Moodle I can safely say that MVC in Moodle does not exist.

Where I do agree is that dependency injection is a bad thing with the current state of Moodle, as it would require a considerable performance hit to scan for these dependencies. However, if in the future a fully refactored, correctly implemented MVC pattern is introduced I hope there will at least be considered to use things like classmaps (which ZF2 can render into a static classmap, ideally to be combined with the MUC) so all instances are bootstrapped and can use functions within models and controllers without the need to manually include them every time (which is a big performance hit as well due to disk access hits).

In response to David's post after you, if plugging the holes is the way forward then I am afraid that I do not understand the whole concept of changing things anyways. Most major versions break something (deprecation or replacement) which means contributed code has to be altered anyways. So why not change bigger things and improve the foundation instead of fixing the hinges on the windows?

I think it should be a big telltale sign that all major Moodle partners have written their own frameworks that implement correct MVC patterns to overcome problems with Moodle, and that way extend Moodle in ways that would have been simple to achieve if Moodle core was better built instead of constantly patched up. In this day and age it is evolve or die out, and the current way I am afraid will only lead to extinction.

Average of ratings: -

Re: My Total developer rant with the Events API

by Tim Hunt - Friday, 5 September 2014, 6:43 AM

To take just one example:

Controller: https://github.com/moodle/moodle/blob/master/mod/quiz/attempt.php

Model: https://github.com/moodle/moodle/blob/master/mod/quiz/attemptlib.php#L417 and related classes.

View: https://github.com/moodle/moodle/blob/master/mod/quiz/renderer.php#L407

Certainly, not all Moodle code has been tidied up as much as this, but it all could be. It is just a question of: is that the most important development task? Mostly no, because screwing around behind the scenes might make developers happy, but it does not heap teachers or learners.

Most new features are being implemented with a reasonable separation between the M, V and C bits.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 5 September 2014, 7:15 AM

Arguably this area is the most approaching MVC, but bear with me:

What is all the output doing in a controller? set_title set_heading etc. these are not user initiated actions and therefor do not belong in a controller.
If I am not mistaken then quiz_attempt is a singleton
quiz_review_nav_panel still outputs html
The view still gets parsed into the big outputlib mountain, in itself it is well designed
I am missing __construct __set etc. in many of these classes, which means you cannot use $this-> easily.

Don't get me wrong, the code is well written, very well documented, and there is nothing wrong with it in itself, but it is not an example of an MVC design pattern.

Average of ratings: -

Re: My Total developer rant with the Events API

by Tim Hunt - Friday, 5 September 2014, 4:33 PM

Is that output or is that setting up properties of an object (the current page) ready to be passed ot the template/renderer?
You are mistaken if you think that.
Yes. It was extremely hard to separate the view and the logic here, and I failed. This code needs more work when I get a round tuit.
I'm not sure I follow this.
I also don't follow this. MVC is a pattern. It can be implemented in many ways. __construct __set are specific syntactic sugar, which may, or may not have a place in a given implementation. There absence means nothing.

We have to deal with reality here. There is a lot of Moodle code that does useful things for teachers, students and admins, even while offending the sensibilities of (some) trained software developers. We do want to make things better for developers behind the scenes, and if you compare what we have today with something like Moodle 1.6 (the earliest version I worked on) we have come a long way. However (Moodle 2.0) aside. We have tried to do that without breaking backwards compatibility too much, and without going away for years during which we don't deliver a new version.

Also, if you can be bothered to learn the Moodle framework, it is actually pretty nice, and you can get work done efficiently, even thought it doesn't match your preconceptions about how code should be written.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by David Mudrák - Friday, 5 September 2014, 4:18 AM

If only our world was that easy that "such methodical refactoring" was even possible.

Moodle is not a single site project where agile developers can try new and new patterns with every new client (often hosted in an environment under total control). It's not like "we will use the new version of the framework X for this project because it has this and this feature". If the HQ team was refactoring things over and over again as they become available and popular, there would not be any resources left for any further development. And what's the point for actual teachers and students who use Moodle to have it without a single global variable, if all it really does is still the same as f9903ed.

And even if Moodle HQ itself had enough resources to do such a step, will we just throw away all the contributed plugins? Even minor changes in APIs take ages to get adopted by our community contributors, and I fully understand that.

I'm all for good and quality code design and implementation. But during the years with Moodle, I also learnt there are other valuable aspects of the software, some of them even more important than the code itself.

Becoming a Moodle developer is like buying a big but old house with lots of rooms and moving into it with whole your family (which has millions of members). You must fix apparently broken things (like holes in the roof to stop water leaking in) and eventually renovate some parts of it. But at the end, you will just repaint the walls in the rooms you spend most of the time. Somehow, your family members demand rooms that seem to work right now (e.g. they have a bed to sleep on, or a bathroom). Many of them are not interested in how the water in the bath gets warm. There is certain border where you just must admit things were built this and this way ages ago and today we must live with it. What you call for is like destroying the house and starting completely again from the ground (or rather a bit deeper) just to have the new house full of modern technology. You can easily find yourself rebuilding the first floor over and over again as new bricks are coming to the market. By that time, you will probably live alone in that superiorly designed, just not yet finished, building.

Average of ratings: Useful (5)

Re: My Total developer rant with the Events API

by David Mudrák - Friday, 5 September 2014, 5:31 AM

Disclaimer: I admit I haven't read the book. Chances are it really contains good advices but I'm generally very sceptic when seeing the list of your suggestions and can't imagine to make them happen with "no massive rewrites from scratch required or even prescribed".

Average of ratings: -

Re: My Total developer rant with the Events API

by Valery Fremaux - Friday, 5 September 2014, 6:11 AM

Thank you david for those wise words....

Never forget that Moodle would never have reach this level without the involvement of the community, most of which do not even know what "design patterns" mean.

Now Moodle is big enough to attract very skilled and educated developers, This is nice for us, as a proof of success..,

About trends to "all object oriented", would you ever would like to risk getting moodle so tricky than Zend framework... Object is great at middleware level, less in entry points : Most of screens that just say :

$module = new moduleclass()

echo $module->view();

Is not IMHO such a good practice (unless you really implement renderers properly to get all GUI overridable).

Too may object and too granular design kill the overral and quick undestandability of a structure.

About MVC i rather agree with David, I tried in the past to orient some of my contribs with real MVC pattern, using explicit .controller.php separation. I felt a nice experience with it, but i went back on practices to be more "in the core culture" and let other moodle developpers to analyse my code... Actually i do not think moodle has so many parts where clearly explicits a real MVC pattern. Forms seem to be, but is not so mucjh...

Average of ratings: Useful (1)

Re: My Total developer rant with the Events API

by David Mudrák - Thursday, 28 August 2014, 3:58 AM

For the reference, this thread is a follow-up on https://moodle.org/mod/forum/discuss.php?d=263614#p1156663

Average of ratings: -

Re: My Total developer rant with the Events API

by Ankit Agarwal - Monday, 1 September 2014, 3:51 PM

Hi David,

Thanks for your feedback. In this post I will try to address your concerns regarding the events/logging api.

Let me start by stressing the fact that the new api is not just "replacing old log calls". We are doing way more than that in the new apis.

# First, let us consider the logs. In Moodle, knowing "something has happened" is not enough. The old logging interface was fine for that. What we wanted was rich and more importantly consistent logs. We need to know exactly which modules are responsible, we need to know the users involved, we need to know a ton of other similar stuff. We want the logs to evolve more than simple tracking and debugging tool. We plan on building numerous analytic reports based on the logs, hence meta data and consistency is an important part of the whole system. We had free flow data in the old system. This led to numerous inconsistency, for example, the "info" field. If you go through entries in plugins db and find the log calls, you will see the random things that has been put in there, starting from orphan ids, hard-coded language strings etc. These requirements called for making the whole api stricter.

# Secondly, logs are not the whole spaceship here. Events are closely bound with logs. While designing the system we did consider the alternative to keep them separate. Keeping them separate would be asking developers to write more code than present with an added cost of performance. It simply seemed pointless to keep them separate when it was quite obvious that anything that is worth logging should be an event as well. So the work involved to "replace a log", also gives you the added benefit of triggering events which will allow other plugins to interact better with your code.

#Third, we are aware that hooks are needed in Moodle. We already have a proposal in place (see MDL-44078). But we have limited resources, hence unfortunately the hooks project had to be delayed.

Now lets look into more specific concerns that you raised regarding creating an event,

       Create a specifically named class
       In a specifically named directory two levels deep

These are Moodle standards and not specific to Events api. For example, scheduled tasks needs to be placed in classes/task. Personally I don't see any reason why this can cause confusion, on the contrary, I would say this makes things more clear and granular.

        with default required classes

Again not sure, why this complicates things. I see this as a common design pattern. The default class handles tons of things for you, and all you need to do is to extend it. Is it really that complex to extend a class?

       without any proper data handling, but relying on "other" for much data handling, which:
       Does not correctly evaluate values, but due to the construct requires already assigning them to variables before passing it

Have you tried creating an event with in correct base properties? For example - conflicting course-id and context ? or leaving out object-id, while specifying an object-table? If you did, you would know there are quite a few validations already in place. What we leave up to developer is to validate the "other" filed by themselves, as this is a free flow field and only the developer knows what exists in this field. Still there was another proposal to make these fields strictly validated (MDL-45108), but we decided to delay that, as that would add more complexities to the api. Feel free to suggest alternatives on the issue.

      providing no standard handling of legacy add_to_log replacement, but requires manual adding of these calls

Have you seen this doc https://docs.moodle.org/dev/Migrating_logging_calls_in_plugins ?

     requiring at least 20 LOC to achieve a simple "view" call

As I mentioned earlier, it is not a simple log entry to say something has been viewed, it has tons of other things associated.

     being the most counter intuitive thing I have ever encountered in MY ENTIRE LIFE

Well, Spaceships are pretty rare.

Let's move on to more specifics
So ways to improve:

      Define default events for default events (view, grade, error, update, delete, etc.)
      Do not tie these events and event declarations to specific plugins. Grading an assignment is in no way different from grading a quiz for logging purposes.
      Do the above to make it usable with all types of resources in a coherent way (no different calls for mod and block)
      Reduce the LOC needed for declaration of events by following #1 by using default events

We have already created default abstract classes for events that made sense(For example course_module_viewed). Creating default classes for "all" view events defeats the purpose of the whole system, it has no added value(crud property already explains its a read event). Allowing a generic event to be triggered from a billion places not only results in the loss of granularity but also makes it very difficult for observers to monitor specific things happening in Moodle. If you have suggestions for more specific situations which would be consistent among plugins, please bring those forward, and we will consider creating abstracts for the same.

   Do not require redeclaration of all functions before you can use them (duplication is the worst sin in MVC patterns)

You don't have to redeclare a single method if you don't want to override anything. For example see mod/choice/classes/event/course_module_instance_list_viewed.php

   REMOVE the dependency on "other" for almost half of the events, like SERIOUSLY????

What is wrong with using information from other field to make the description of the event more rich?

    Provide usable examples of events to be created for real plugins, not some loosy-goosy partial examples that actually cover different plugins, but look like they are part of the chain

https://docs.moodle.org/dev/Migrating_logging_calls_in_plugins seems pretty clear to me. But feel free to suggest improvements.

   Create vocabulary for events, so not every plugin has to re-declare vocabulary (which also means that all of the events in Moodle are currently not multilanguage, but english only, just look at all the hardcoded strings in mod_assign)

There is already a vocabulary for verbs used in the events specs. As far as hard coded language strings in the get_description() are considered, these are designed only for admins. There is an issue to return translatable strings to be displayed to users in reports (see MDL-45106)

    If you want to make Events 2 actually usable, add hooks support as default
    Simplify the event handling and requirements, it's not a spaceship, they are log messages
    Provide proper error handling for events, breaking debugging output because a not yet fired event is invalid is retarded

See initial part of this post.

    Class autoloading would be useful if it actually auto-loads, not requiring increasing versions to trigger upgrade for every single change or addition

To make the autoload work faster we have implemented a cache, when you add a class you need to purge this. Changing version numbers is simple a way to do the same. If you are writing code, you should be doing this with developer mode on which doesn't need purging of the cache. (https://github.com/moodle/moodle/blame/master/lib/classes/component.php#L160)

I understand that the whole system is a bit more complex than what it used to be, but everything that is there, is there for a reason. We welcome all constructive feedback on how we can simplify this for plugin developers, but simply saying that everything is useless is not the right approach.

If you find docs that are incomplete or not clear, please feel free to provide suggestions and we will do our best to incorporate the same.

Hope this post clarifies a few things.

Cheers

Average of ratings: Useful (6)

Re: My Total developer rant with the Events API

by Tim Hunt - Tuesday, 2 September 2014, 12:36 AM

David's post raises issues with the logging API that I tried to raise when things were orginally proposed, and like David, my concerns were ingored too. I get the impression that no-one at Moodle HQ cares to properly understand what the alternative would be. Well, would have been. It is all a bit academic now, we are stuck with this API.

To comment on just one thing though:

David: requiring at least 20 LOC to achieve a simple "view" call

Ankit: As I mentioned earlier, it is not a simple log entry to say something has been viewed, it has tons of other things associated.

Tons is hardly a very precise number. There are, perhaps, half a dozen bits of useful information in an event. It is completely disproportionate to require 20+LOC to log that. (And 20 LOC is probably an underestimate.)

I can't be bothered to point out the other issues with Ankit's post.

Average of ratings: Useful (1)

Re: My Total developer rant with the Events API

by Conn Warwicker - Wednesday, 3 September 2014, 1:57 AM

If the new logging system is not for simply logging entries, perhaps it would be an idea to create a "sub-logging" system that is? A separate class you can extend which allows you to use it in a much more basic way?

Average of ratings: -

Re: My Total developer rant with the Events API

by D.A Alderson - Friday, 5 September 2014, 3:27 AM

Hey,

teachers-learners-teachers-learners....a poem in the making-let's not lose sight-what works eh...

Average of ratings: -

Re: My Total developer rant with the Events API

by dawn alderson - Friday, 5 September 2014, 6:01 AM

hi,

Having lied down in a very dark and quiet room for the last week-I now receive posts from this thread-as a consequence of posting earlier-no worries. I get it. But, have not the time to dwell here.

Can I be honest? This all sounds like tracking issues to me-yes important, of course, and given the buzz surrounding BB analytics-at a cost-yep I hear you all However, teacher-speak, this is a rib-nudge for ASSESSMENT IN THE MAIN-FROM WHERE I am standing...java, php-swahili...accessibilty (outcome vs process=concentration on fit for purpose) is the focus eh-maybe-to be honest, tail-wag-wrong- tree, so b it-just my thoughts really.

cheers,

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 5 September 2014, 6:30 AM

Dawn,

it is awesome how you can take a comprehensible story, then shuffle the words around and still create something that looks like it was once English, but I am trying to have a meaningful debate here. Feel free to unsubscribe if you no longer wish to receive emails.

David

Average of ratings: Useful (4)

Re: My Total developer rant with the Events API

by dawn alderson - Friday, 5 September 2014, 7:01 AM

David, hi

I hear your frustration, and I repsect that. Now, a few things:

1. Tim's credentials, nope more of a matter of experience-might be worth thinking on that.

2. Have you travelled around BB analytics, tried from the front-end; user interface-let me know what you think.

3. When Tim says:

I think that is called a bullseye-in darts-speak.

All objective and reasonable debate I would say. Nope, not going to unsubscribe-too interesting, to be honest. And, I actually know what I am talking about-really I do-wouldn't bother otherwise.

cheers,

Dawn

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 5 September 2014, 7:22 AM

Dawn, I meant it more in a way that your previous post did not make much sense to me, and seemed very disconnected from this discussion.

I have had the experience to work with BlackBoard Analytics, and where I think they are leaps and bounds ahead of Moodle that is not necessarily the way I think the Events API should work. That being said, it would be great if the output of the events API would allow for that kind of functionality, which it currently doesn't.

On the topic of analytics, I have developed a Piwik / Google Analytics plugin for Moodle, and can say that even the default analytics from that are much more powerful than anything in Moodle to date.

Average of ratings: -

Re: My Total developer rant with the Events API

by dawn alderson - Friday, 5 September 2014, 7:48 AM

No worrries David,

I can't do much about the way you see things-it is your way of seeing eh. And if I did not make much sense to you-then at least it now appears we are on the same page...so hey ho, connected-disconnected....the point remains that we all see things in our own way...and if we wish to see others' points of view...if we wish-or work hard to do so- then that is up to the individual.

I am tired of hearing- one must spell things out for others to understand-grown adults....I think that is neither here nor there.....I appreciate you get the BB point-and what a valid point that is-I can assure you.

When we step outside of what we know-and think about an alternative-then that is when we realise there is more than one way of seeing-I don't think we ought to be harsh about the historical decisions made with regard to moodle code-and judge its efficiency on those terms-it is about validity-does it do what it says on the tin? Yes-and some...and teachers and learners thinks so given its uptake...yes improvements are always needed with all things, that takes time and needs to be the right time for change.....events api....yep I think we all hear you- but fiddling with back-end-needs strong justification....in terms of the user.

If it would greatly benefit the user then play around with the spaghetti junction....if not then there are far more immediate things that need addressing like the advanced forum, in my view-as a teacher and experienced user of such systems and in terms of my knowledge of enhancing the student experience-horses for courses David.

cheers,

Dawn

Average of ratings: -

Re: My Total developer rant with the Events API

by Andrew Lyons - Friday, 5 September 2014, 3:34 PM

Hi David,

Google Analytics, and Piwik are both awesome and powerful pieces of software. A well integrated solution can yield fantastic results, but personally I think that they are insufficient for some of the purposes we look at here. Let me give you an example.

One of the things which we log is the creation of a forum post in Moodle. We can log appropriate information about the discussion, the post, the forum, the course, and the user writing the post. An appropriately configured Google Analytics/Piwik can almost certainly capture the same metadata - no love lost there.

However where I think it falls down is that it would be almost impossible to then get those analytics solutions to tell you who that post was visible to, and who it was in reply to - you may be able to get some of that information (and I'm happy to be proved wrong). It's also then difficult to associate that information with the users who subsequently read that post. Those analytics solutions may tell you how they got to read the post, but not necessarily and where they came from once they got there, but not which users didn't get there at all, so which were the users who did not view it?

Does the users who read the post have an effect on their academic performance. It won't then easily tell you whether that post was well received - did users vote for it for example. Perhaps it was graded - can those analytics tools give you all of the data to show you how many times that the user views the page and over what time period before they made their reply? Some of those things could be calculated given enough time, but is an external system really the right place to do so?

Equally, the current events API won't give you information about other useful things - how long was the user on the page, what did they look at when they were on the page? These things are more ideally suited to the analytics tools that you suggest.

Personally, I'd like to see an integration of the two approaches. Combining something like Piwik with Moodle's own logging could give some amazing insight into how people use the VLE, and interact with a course. How long did the user spend looking at the page before they replied? Did they then refer to it whilst doing so? How active has the user who hasn't read the post been, and how do their grades compare?

Just adding more thoughts to the mixing pot,

Andrew

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 5 September 2014, 5:30 PM

Andrew,

To illustrate my point let me ask you whether you could answer your own question based on functionality present in Moodle.
In the current situation I can't, and am quite confident you cannot either. Happy if you can prove me wrong.

David

Average of ratings: -

Re: My Total developer rant with the Events API

by Andrew Lyons - Friday, 5 September 2014, 5:51 PM

Hi David,

The data is now present so it is now feasible. Depends which part of the question you're asking me to answer though. It wouldn't be beyond possibility to create reports for this, but some of it is very specific so you'd really want to know what was of educational benefit.

Andrew

Average of ratings: -

Re: My Total developer rant with the Events API

by Elizabeth Dalton - Tuesday, 7 July 2015, 11:24 PM

I just came upon this discussion while trying to find out if the new logging system would tell me whether an instructor gave written feedback on an assignment, along with a grade. (I still don't know the answer to that question...)

I built extensive learning analytic reports on the old logging system using the Configurable Reports plugin, and now I am re-creating those to use the new logging system. What I have noticed so far:

1) I'm not seeing much better detail for the information I want to track. I was already able to tell who was reading forum posts, who was replying to whom, etc.

2) It's quite difficult to compare my old reports with reports that use the new logs-- I don't seem to be able to define criteria that will match the results I was getting before. I realize that more events are being logged (e.g. group creation and editing), but it is difficult to validate my new reports against old reports on the same courses when I can't figure out how to match the logic.

I don't develop modules (yet), so I can't comment on the rant that started this thread, but from a reporting point of view, I'll just say that whatever benefits the new event system is supposed to give us, I'm not seeing them yet. Maybe I'll see more advantages as I continue to work with the system, or as more features and infrastructure are added.

Average of ratings: -

Re: My Total developer rant with the Events API

by Matt Bury - Friday, 5 September 2014, 7:17 AM

Hi David,

I've been watching this discussion unfold with interest. On the coding front, I have to agree that "under the hood", Moodle looks pretty unwieldy and poorly put together.

In defence of your position, i attended a presentation and discussion of "PHP Object Calisthenics" by Guilhermo Blanco, a former developer at Yahoo! https://github.com/guilhermeblanco What came across among the developers at that event was that MediaWiki and Moodle were not the best constructed code, especially in terms of stability and complexity.

My preferred IDE for PHP is Netbeans and I've noticed that the more recent versions now include notifications if functions or classes get too long, according to "Object Calisthenics" rules. It's a nice reminder for other projects but disheartening when working with Moodle.

It appears to be the nature of the Moodle beast and it doesn't look like much will change any time soon. It'd be great to re-develop Moodle from the ground up following good/best practices which is precisely what coders like Guilhermo Blanco do. Perhaps we could find wealthy benefactors to fund such a project?

Average of ratings: -

Re: My Total developer rant with the Events API

by Tim Hunt - Friday, 5 September 2014, 4:56 PM

This 'lots of small objects and methods' approach to software is what is taught in certain schools of software development, and left to my own devices, it is probably how I would write code. However, it is not uniformly a good thing.

I have a specific example in mind:

Here is some code in Moodle 1.9: https://github.com/moodle/moodle/blob/MOODLE_19_STABLE/lib/questionlib.php
and her is how I rewrote it in Moodle 2.1+ https://github.com/moodle/moodle/blob/master/question/engine/questionattempt.php#L1166

There is one important respect in which the 1.9 code is better than the 2.x code. How easy is it to understand if you have not seen it before. This is a particularly important property for open source software.

To understand the 1.9 code, it is all there. You just just read it like you would a story. This is how humans normally communicate. In addition to the code there, you will only need to look at a couple of other classes

class question_multichoice_qtype extends default_questiontype

To understand the 2.0 code, you need to look at code from all these classes/interfaces:

qtype_multichoice_single_question extends qtype_multichoice_base extends question_graded_automatically extends question_with_responses extends question_definition implements question_manually_gradable, question_automatically_gradable
qbehaviour_deferredfeedback extends question_behaviour_with_save extends question_behaviour
question_state
question_attempt
question_attempt_pending_step extends question_attempt_step

I make that 14 classes. You probably need a debugger to step through the code. It comes as a big relief to me that there are actually question type and question behaviour plugins, and other activities beside quiz, in the plugins database. (It is the case that you don't need to understand all in the internal workings to create one of these plugins. Indeed, keeping this API https://docs.moodle.org/dev/Using_the_question_engine_from_module simple was a key design goal.)

Anyway, I spend over a year building that mess, there must have been a point:

The problem with the 1.9 code was that it was very hard to work with. It was easy to introduce a new bug when trying to fix an existing bug.
Also, there was not the concept of question_behaviour, just adaptive and non-adaptive mode, with manual grading stuck in there. We wanted to add more behaviours, therefore behaviour classes was pulled out as an invocation of the strategy pattern.
The new system has clear separation of concerns between classes. Once you understand which class is responsible for which things, you can find things and work on them without introducing bugs.
It is easier to verify a small method is doing the right thing, both just be reading and an thinking, and also because it is easier to unit test.

But I still worry about how hard it is for new developers. I say this based on having to try to explain bits of it in the quiz forum.

Average of ratings: -

Re: My Total developer rant with the Events API

by Visvanath Ratnaweera - Saturday, 6 September 2014, 4:38 PM

David

That is some gimmick of AI which withstood the Turing test!

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 5 September 2014, 6:28 AM

Ankit,

your post illustrates that there is a very big disconnect between what end users want in terms of functionality, what developers need in terms of documentation and clear defined designs, and what the ivory tower of Moodle HQ developments seems to be thinking.

For instance: saying the events api is much more extensive than the old logging system gets a thumbs up from me. However. What is the current benefit for the end user? The logs are as useless as they were before, no searching, no data processing to make anything of the data.

Then let's look at the developer perspective, does the new events API allow for better log entries? Ehm no, you still need to specify every input manually, but to make matters worse you now need to specify about 3 times as much compared to before.

In terms of default events, what do you think of events that actually make sense to an end user? It is completely useless to log whether a user viewed a resource as that can be done by using any off the shelve analytics, but what of:

Submitting assignment
Receiving feedback
Receiving grade
Passing grade for assignment
Activity completion
Course completion

All these events should be identical for all areas in Moodle, instead mod_assign seems to have gotten the treatment of all these events, where other modules have gotten less. Beats me why these events arent simply global.

Instead of having to create these massive event handlers for every log call you could have had correctly defined events that could be triggered anywhere allowing default hooks, allowing multilang support as you only fill in information.

If you cannot see the errs of your current ways you should try working with Moodle for a few weeks, instead of developing for it.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by Derek Chirnside - Friday, 5 September 2014, 7:55 AM

Re David's comment:

you should try working with Moodle for a few weeks, instead of developing for it. . . .

I have long thought the way to best (as in most effective and quickest) way to bridge the divide between the HQ and MoodleINC world view (ie the developers) and the others (ie the users) would be for developers to spend some time actually teaching with Moodle, administrating a course or even visiting some teachers and getting some feedback. We nearly got some of this with the researcher based at MoodleHQ a while back, but not quite.

-Derek

REF: One place where I have said this [at random] https://moodle.org/mod/forum/discuss.php?d=225087#p1094362

Average of ratings: Useful (1)

Re: My Total developer rant with the Events API

by Andrew Lyons - Friday, 5 September 2014, 9:13 AM

Hi all,

I come from a slightly different background to Ankit, so I thought I'd add my thoughts to the conversation.

I've been working for Moodle HQ for just under 12 months now and, prior to working for Moodle HQ, I worked for a University on the VLE implementation side. We were migrating to Moodle (from a custom-built VLE written in Lotus Notes - don't ask!) and, as a result, were lucky enough to be starting with a clean slate. As such, we were looking at a lot of requirements, and performing gap analysis compared with our previous VLE. So yes, I'm coming to the conversation as a Moodle developer, but also as someone working closely with end users, and University management vested in the success of the institution, and I've seen different parts of the story.

As part of our requirements gathering and gap analysis, one of the first things that University management wanted to know about, was logging:

How much is logged?
What is logged?
How is it logged?
Can we warehouse those logs, and export them to appropriate locations?

At the time (around three years ago), the new Events API did not exist and we had to score a big fat 'No' on the requirements and gap. Thankfully, now that can be changed to a yes and people can start to think about those integrations.

Yes, most of the time, people use those Moodle logs to effectively spy on their students (to see which users have taken part in activities, or which students haven't logged in recently), or as an audit trail when things go wrong. However, the logs can be much, much more than that.

One of the key reasons that our management wanted detailed logs was to look at how students were interacting with the VLE, and how that interaction affected those students. To use it to spot trends and relationships between that interaction with the VLE, and the resultant grades. To then use that information to spot outliers early such that they could be approached before grades suffered. To use it to spot where tutors were interacting more and what effect that had upon student happiness and results.

All of these things are now possible with the new API.

I remember being at the Hackfest where some of the early details for these logs was first fleshed out, and I remember being very excited about the possibilities. I did not work for Moodle HQ at that stage, and Moodle HQ developers were only about half the audience. There were developers from several other institutions, and Moodle partners who represented organisations with similar requirements. I was not the only excited person in the room.

I've also had the pleasure of attending several Moots over the past few years and again, this kind of detailed logging has popped up time-and-time again. This year at the Australian Moot I attended a presentation by Rose Elsom & Channa Herath of Westbourne Grammar School on their continuous reporting in Moodle - used for the same kinds of purposes as Lancaster University. I attended some sessions at the 2013 UK moot in Dublin from Alex Walker from Glasgow's City University. They were using metrics, analytics, and heat maps to monitor the effectiveness of the VLE to, in part, achieve similar things and to assess how effectively their users were interacting with the VLE.

Yes, some of these analytics could be gathered using external analytics tools... but that relies upon a tight integration with those analytics tools, and Moodle. Without a very tight integration, the data is far less meaningful and becomes pointless for some forms of analysis.

So I for one think that this is a positive step for many people.

One of the things that you point out is the current lack of event handlers for those logs, and the inability to read them properly. Well, to start with the same standard reports exist with some slight improvements, so we have not moved backwards. Since the previous data was so varied in quality and content it was often pretty useless, I encountered some bugs where some parts of Moodle tried to parse the data, only to be thwarted by plugins entering poor data - I'd say that the current status is an advancement over the previous state of play. Moving forward, we now have the option of writing more detailed reports - something that we couldn't previously do.

Thinking about my previous work at Lancaster, the requirements that they had were pretty specific. Tying pieces of information together from various University sources to then export that log data to a data warehousing system is not exactly something that should be built into Moodle. The only sensible way to go for their data warehousing and analytics is the route of custom plugins to integrate the many integrations we're written (primarily relating to enrolment in this case, but also potentially with library systems, lecture capture, and media server).

With all of this data, we now have potential to do much more and to analyse just how people do use Moodle.

I hope that this adds another viewpoint to the conversation. As I say, I can see a definite advantage to users - albeit not necessarily in the way in which you're thinking.

Best wishes,

Andrew

Average of ratings: Useful (4)

Re: My Total developer rant with the Events API

by David Mudrák - Friday, 5 September 2014, 3:12 PM

What is the current benefit for the end user? The logs are as useless as they were before, no searching, no data processing to make anything of the data.

I do not agree. I see it as sort of chicken-egg problem. We can't prove the real value of the new events without actual reports and analysis plugins. But we can't have reports until new events are being logged for some time to produce meaningful and useful data. We should also hear from people writing those reports on what they thing about the new API.

On another note, I am not sure that this discussion steers to a constructive conclusion. I fully appreciate the community feedback. But the tone like "Moodle should have been rewritten from scratch, or die" is as helpful as saying that HQ is full of incompetent amateur scripters who think that MVC is an abbreviation for a mobile toilet.

So, can we re-focus this on something we all can actually act-on in a near future? Let me start with something like

Is anybody willing to prepare some prototypes of eventual helper wrappers that would make the events API more contributor-friendly?
Are there some good examples of actual reports that make use of the new logging system?

Average of ratings: Useful (4)

Re: My Total developer rant with the Events API

by Martin Dougiamas - Friday, 5 September 2014, 3:47 PM

Hi there David B,

Ankit, Andrew and David M have all made some good replies but I'll reiterate:

The purpose of improving the logging/events to this level was mainly for two reasons:

1) We want to improve the quantity and quality of our "time-dependent" information (there are other APIs for accessing the databases). This will help drive Moodle research into online learning, which in turn will enable us to create the best analytics tools (better than anything else out there) which will in turn support teaching and learning processes. When I talk about analytics I mean AI, not pretty reports about "clicks" and paths. Moodle will act as your assistant - contacting you proactively and supporting you. All this work is coming up, and I hope to see the community developing a lot of plugins for it. We've started one simple one already for 2.8: https://docs.moodle.org/dev/Event_Monitor_specification

2) For larger sites, this torrent of logging (and real-time analytics) into internal tables would affect performance greatly, especially on clusters, so this new logging system has a way to stream that data into faster external databases, such as NoSQL ones. This is pluggable so you can make your own interfaces to anything you like, even TinCan.

I appreciate all this has made logging more work for developers but there are longer-term considerations here that I don't think you may have grasped (probably because it's not been communicated well enough).

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by dawn alderson - Friday, 5 September 2014, 4:37 PM

Morning all (afternoon for some, evening for others)

My take, hh.

@Derek

you should try working with Moodle for a few weeks, instead of developing for it. . . .

This is interesting Derek. I must add, I don't think it is an easy plan to expect an expert in one domain to shift into another domain overnight-and it is not fair to expect peeps to do that. Yep, I am aware of computer science stuff-but I am not as expert as the rest of em here-I rely on other's knowledge to inform me about that stuff-and I am happy to shell out what I know about T&L&A. In essence, the two domains should logically fudge/or sometimes explode-you know water-oil relationship...*grinning*

Now that brings me on to my response to Andrew's very well laid out post.

I would add, universities are not always top-down orientated in terms of decision making-and I know that was not the point Andrew. However, it is worth noting a little more detail about out infrastructure.

All universities worth their salt have an L&T&Ass committee, A TEL committee and so on. In turn, info from dept reps are fed into those committees, so stuff about logging at course level/module level/part-timers/international students, full on distance learning, and on...you get my drift eh.

Those uni level committees forward feed the minutiae from the depts/courses/modules student tracking systems to the decision makers committee-THE COMMITTEE....so that informed decisions can be made....it is not just a question of what uni management want-and what works in their eyes....it is about keeping an eye on grass roots....ergo the students' progress-to provide an excellent learning experience..

David's ref to chicken and egg.......I have a comparison in very simple terms to highlight my point for grass roots level: if the students do not engage with the forums...for whatever reasons-then what is the point of analytics for that purpose.....one might say then that a focus on student engagement is key-and the analytics-the meta stuff second in this context.....obviously a fine balance is needed and so I conclude-a T&L&Ass team at Moodle HQ- the bridge between user and dev- is a thought.

@David

MVC is an abbreviation for a mobile toilet. LOL!

@Martin

This will help drive Moodle research into online learning, which in turn will enable us to create the best analytics tools (better than anything else out there) which will in turn support teaching and learning processes.

-Brilliant!

cheers,

Dawn

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Friday, 5 September 2014, 5:56 PM

Hi Martin,

I think you must have misunderstood my primary point of this rant (it came with that warning!).
I do not question the need or purpose of the Event API at all, I think it was a long overdue improvement to the quality of events that can be logged. What I do question is the current implementation that forces developers like myself to build everything over and over again.

In the old situation you had the extremely limited add_to_log call which looked something like this:
add_to_log($course->id,'role','assign','admin/roles/assign.php?contextid='.$context->id.'&roleid='.$roleid,$rolename,'',$USER->id);

In the new situation you do not have any default calls, so even if you want to log a default event, you are still forces to create a custom handled, even if it does nothing.

So if instead it would be possible to have the following type default events without a need for a handled (but with the support for it)

\core\event\course_module_viewed::create($this)->trigger();

\core\event\module_submission_graded::create($this)->trigger();

That would make all events for these types behave the same, require the same default information, have default language strings, and reduce the amount of code for a default event to 1 line. In my opinion to make any analytics possible there must be common ground between similar or identical events, even if the resource is difference, otherwise you can never do a quantitative and qualitative analysis of the impact of certain learning activities as the underlying data cannot be compared.

Right now all these events only live in their own space (just look at the list of events), requiring every single module to reinvent the wheel, or be forced to copy paste existing code. Whatever way you look at it, that is a bad design choice.

In response to David M. The intent of my post was never to have a constructive discussion, but to vent frustration. I am very glad tho that there is a constructive discussion that sparked from that. I have also not yet seen any response from anyone outside of Moodle HQ that says the Events API is gods gift to Moodle, au contraire.

Average of ratings: Useful (2)

Re: My Total developer rant with the Events API

by dawn alderson - Friday, 5 September 2014, 7:08 PM

David B-OK,

now you have my attention, I am always inclined to play naughty school girl otherwise-I find rants noisy!

So, when you say:

So if instead it would be possible to have the following type default events without a need for a handled (but with the support for it)

\core\event\course_module_viewed::create($this)->trigger();

\core\event\module_submission_graded::create($this)->trigger();

I must be honest, this sounds logical. Any knock-on-effects in terms of efficiency? Would that be a headache workload for someone? Would that solution enable a wider participation of devs to contribute code in a more accessible way...or are these questions 'can-worms-type'?

cheers,

Dawn

Average of ratings: -

The content of this forum post has been removed and can no longer be accessed.

Re: My Total developer rant with the Events API

by lior gil - Sunday, 7 September 2014, 4:54 PM

In many places, when someone is only on the user/customer/front-end side, there's always some criticism on how things work and how it could be done a little bit better.

Then, sometime that person switches into the management/developer side and after a while his point of view naturally changes and becomes more broad. Things that before were "this isn't really good" turn into "it may not be optimal but it has some benefits in the long run, let's keep it".

We started migrating all our development to 2.5 and when 2.7 came out we skipped 2.6 completely. Turns out this was a good call. When I looked at the new and improved event2 what passed through my mind was something in the spirit of "Oh Moodle, what have you done?"

My team have the benefit of a pedagogy team that give us a different perspective on our work and helps us change our code design in order to fit the demands and ease the workflow of our end-users. Sometime It's a compromise on out part 'for the greater good'.

I seriously doubt Moodle HQ works that way (no, QA doesn't count here) and this discussion shows the two opposing points of views I described previously. When all you see is code you are at risk of neglecting some other aspects of it and I guess this is why a new event system was created in one version and became obsolete in the next version - because the newer version was better but the result was countless of wasted work hours of many developers around the world.

Also, please correct me if I'm wrong, as far as I understood, the main reason version 2.7 was declared to last several years was because of another rant about the need to constantly rewrite one's code every few months.

Personally, after the first initial shock, I converted my code to use the new events, even though I would've liked the events to accept some variations instead of creating one event for every purpose, even if they're only a little bit different from each other (got 36 events so far).

Someone else on my team found out a problem calling the events from a local plugin, which if true than is very serious thing, because the events don't just log data but are also used to trigger other actions using observers.

In short (yeah I know too late for that) maybe these discussions should have more weight on some of the decisions making in HQ because at least some of us voice the end-user experience and workflow of the work done up there.

Average of ratings: -

Re: My Total developer rant with the Events API

by dawn alderson - Saturday, 6 September 2014, 8:33 PM

An outstanding paper:

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6842585

(link is being fussy!)-ref here

Towards Scalable Systems for Big Data Analytics: A Technology Tutorial

H.Hu et al., 2014 IEEE (open source)

course you all may have seen it already, apols....but NoSQL appears to get a thumbs up...in the main.

cheers,

Dawn

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Sunday, 7 September 2014, 7:06 AM

Hi Dawn,

NOSQL in itself is quite meaningless as it's an acronym for Not Only SQL.
This regards the storage types of this kind of data, and where NOSQL excels is when processing unstructured data.

SQL based databases in general are much more aimed at reliable storing data (ACID compliancy), which is something NOSQL does not.

In the current implementation the NOSQL based storage engines from Moodle are quite meaningless, as they treat these NOSQL engines as if they are SQL (MongoDB etc.). Also, none of the reports in Moodle implement any of the NOSQL features as most Moodle installs will never use MongoDB as external storage. SO the technological "we built it because we could" prevailed over practical use here I think.

David

Average of ratings: -

Re: My Total developer rant with the Events API

by Darko Miletić - Sunday, 7 September 2014, 10:15 AM

Moodle does not support installation on nosql database. There are only some session and cache drivers, so your comment does not make much sense.

Average of ratings: -

Re: My Total developer rant with the Events API

by Visvanath Ratnaweera - Sunday, 7 September 2014, 3:32 PM

But Mongo DB Is Web Scale! See http://www.youtube.com/watch?foo=bar&v=b2F-DItXtZs.

(Found in https://moodle.org/mod/forum/discuss.php?d=205281#p895425. The original site http://xtranormal.com has apparantly changed hands and the cartoon videos are no more hosted there.)

Average of ratings: Useful (1)

Re: My Total developer rant with the Events API

by dawn alderson - Sunday, 7 September 2014, 4:31 PM

yeah yeah....right-I laughed loudly! Very funny.

But, let's keep to the thread, preferably this one....coz if we put this little tally together then a mix and match approach may not be that harmful:

@Martin

2) For larger sites, this torrent of logging (and real-time analytics) into internal tables would affect performance greatly, especially on clusters, so this new logging system has a way to stream that data into faster external databases, such as NoSQL ones. This is pluggable so you can make your own interfaces to anything you like, even TinCan.

Outstanding paper: Towards Scalable Systems for Big Data Analytics: A Technology Tutorial

H.Hu et al., 2014 IEEE (open source)

Possibilities

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Sunday, 7 September 2014, 7:12 PM

maybe read it in the correct context. Moodle support NOSQL (like MongoDB) for the new Events API.
In which reference my comment makes a lot of sense.

Average of ratings: -

Re: My Total developer rant with the Events API

by Visvanath Ratnaweera - Monday, 8 September 2014, 1:44 AM

David

Sorry for posting an inappropirate joke. I agree with you that it is not relevant to this discussion, Martin had mentioned NoSQL earlier.

I appreciate your effort, I must say.

Average of ratings: -

Re: My Total developer rant with the Events API

by Marcus Green - Monday, 8 September 2014, 3:50 PM

Like so much of the best humour that video speaks to a truth. So much technical commentary is ill informed. Doesn't mean NoSQL has no value, just that when something is fashionable is often over sold.

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Monday, 8 September 2014, 8:00 PM

Hi Visvanath,

my response was to Darko, I actually know and like your link, ranting away at ignorance ;)
So no worries

David

Average of ratings: -

Re: My Total developer rant with the Events API

by Darko Miletić - Tuesday, 9 September 2014, 6:53 AM

maybe read it in the correct context. Moodle support NOSQL (like MongoDB) for the new Events API. In which reference my comment makes a lot of sense.

I am sorry but your comment still does not make much sense. The idea of creating pluggable storage support for logging was one of the main reasons all this rehash was made in the first place. The general rant of all big users of Moodle is that it is hard to make good Business Analytics for Moodle due to poor logging implementation. Everything is mainly in just one simple table and not everything is being logged in the first place. The bigger the site - more problematic is the handling of the log table especially when client wants to keep log history forever. New event and logging API does not bring more logging but it offers a chance to:

Separate logging storage from central database thus decreasing it's workload
Save log information in potentialy much faster storage (any decent NOSQL) which would in turn offer chance for faster BI that can be scaled much more easily than with just one simple table

Of course this is just a bare start in the right direction, lots of things need to be either implemented or improved.

Average of ratings: -

Re: My Total developer rant with the Events API

by David Bezemer - Tuesday, 9 September 2014, 5:59 PM

So explain to me then what features of NOSQL the current Event API uses other than storage? (and when considering storage it talks to the NOSQL driver as if it were a regular DB). Yeah I thought so....

Or maybe the report then, are they using any of the benefits of NOSQL features like mapreduce, fulltext search or advanced indexing? I thought so again....

Currently using MongoDB is exactly as if you are using an external MySQL/PostgreSQL database, nothing more, nothing less. In which regards my comments still made and makes more sense then any of yours regarding this subject.

Average of ratings: -

Re: My Total developer rant with the Events API

by Darko Miletić - Tuesday, 9 September 2014, 8:14 PM

So explain to me then what features of NOSQL the current Event API uses other than storage?

You seem to think that I need to justify implementation details made by somebody else. I never talked about quality of implementation. The only thing I said is the reason (I think) Moodle HQ decided to make pluggable log storage which is to separate logging from central database and offer easier scalability of storage receptor for high-load sites. Nothing else. And that was in response to your statement that plugin for MongoDB (or other nosql database) is implemented because they could do it.

I also did say this:

Of course this is just a bare start in the right direction, lots of things need to be either implemented or improved.

Average of ratings: -