General developer forum

Discussion: Best way of making JS/YUI widgets work with AJAX

 
 
Picture of Per Ekström
Discussion: Best way of making JS/YUI widgets work with AJAX
 

Hi.

About a month ago I started a thread to get TinyMCE working with AJAX in Moodle, but I couldn't figure out a good way to do so. What I needed was a way to load in a tinyMCE form (and only the tMCE widget) in an AJAX call. Couldn't grok much so went into research mode and let things mature in my head.

So, now I'd like to start a discussion on what the best approach would be for a general mechanism that allows you to make AJAX calls within moodle. Like it or not, AJAX is becoming more and more widespread, and does have severe benefits with both bandwidth costs and page loading times. Things like dynamic reports, where you have a few filters, you want the filters to stay the same and only redraw the table/graph beneath it for example. Or if there are dynamic on-the-fly load-as-needed pages, like, say... The book module, or maybe the SCORM module? There are enough use cases for AJAX that I am convinced a general mechanism needs to exist in Moodle, so that one can extract parts of pages if one needs to. Note that I'm not saying Moodle should abandon it's policy of everything working without JavaScript (since I find that policy to be sound), just that it should be possible to extract portions of a page for AJAX calls.

Now, to the problem at hand. If you wish to make an AJAX call to update a portion of the site, ideally you want to only load certain parts and leave the rest alone. If you want to update a table you want to print only the table HTML - not headers and footers and everything else around it. Therefore you need to have some way of printing HTML snippets if the page is served with AJAX, and full blown HTML if the page is served without it. This requires a way of telling whether or not a call has been made through AJAX, and should be a part of $CFG (e.g. $CFG->isajax).

Second, you need some way to reload any eventual JS behavior, preferably without reloading every single bit of JS already loaded into the engine. If you update a graph in a report with new parameters, that graph needs to be re-initialised. If you reload a table, same thing. If you happen to have an AJAX-link in there, it too needs to work.

Now, let's discuss solutions. For the first problem I'd like to propose a new automatic theme layout called "ajax", and then just let the author deal with the loading of snippets. This requires a change to how the template engine works, but would otherwise be invisible to the developers (apart from the isajax thing). Therefore:

  • $CFG->isajax = ($_SERVER['HTTP_X_REQUESTED_WITH'] == 'xmlhttprequest' OR optional_param('isajax',0,PARAM_BOOL)) ? 1 : 0;
    • isajax is a GET parameter that can be sent to force an AJAX call
    • isajax is needed for two reasons - some toolkits still do not support HTTP_X_REQUESTED_WITH, and also, sometimes it can be nice to force an AJAX call for debugging purposes.
  • In $PAGE::set_layout(): if ($CFG->isajax) { $layout = 'ajax'; }
    • I'm convinced that a new layout is needed for these ajax calls, but there is no need for it to be visible IMO.
    • The only task of the ajax layout is to print the content as-is and provide hooks to load ajax widgets included in the page (like TinyMCE).
  • Now there's but one thing left, and that is to look at that module/block/report you're making:
    • all you have to do is treat everything as usual
    • If you need to reload a specific part (and only that part) through AJAX, use $CFG->isajax. If there is more than one section, use a get parameter like &section=foo (e.g. you wish to change the order of items in a course section maybe?)

As for the second problem, I do believe that will be taken care of by the ajax layout. So... Comments? Thoughts? Is it doable or desirable?

 
Average of ratings: -
Picture of Andrew Nicols
Re: Discussion: Best way of making JS/YUI widgets work with AJAX
Group DevelopersGroup Moodle HQGroup Particularly helpful MoodlersGroup Testers

Hi Per,

Actually, much of this is already in place and possible, but with a few alterations. We're already beginning to go in this direction and there are a couple of new features in 2.5 which do just this. For examples, see the new help tooltip system in 2.5, and the new course/category detail expansion (https://tracker.moodle.org/browse/MDL-38661). These both load sections of the page without loading the entire page.

It doesn't make sense to have scripts return their entire standard output for AJAX, but rather to have specific components of a page loaded on an as-needed basis. Doing that in a generic fashion is very difficult to achieve but we can (and have) put the tools in place to let you do this for individual situations.

Firstly, your suggestion of $CFG->isajax is already in place, but we've gone down a slightly different route. Rather than allowing you to specify ajax=true or something similar as a parameter, we ask you to use a different script. In this script, you define the constant AJAX_SCRIPT. See https://github.com/moodle/moodle/blob/master/help_ajax.php for an example of this. There are a couple of main reasons for this:

  • error handling - when you define AJAX_SCRIPT, we use a different renderer for error messages which return a JSON response compatible with M.core.exception and M.core.ajaxException so that any errors thrown by your code can be parsed sensibly and displayed to the user.
  • renderer selection - we have to define renderers early on in the page instantiation, and we select a different renderer if you have an AJAX script

Secondly your suggestion for $PAGE::set_layout is already achieved by use of renderers. Any time you use a script which defines AJAX_SCRIPT as true, this will mean that the output system will try to use a different renderer and fall back to the standard renderer if an ajax specific one is not found. You can see that logic at https://github.com/moodle/moodle/blob/master/lib/outputfactories.php#L132. An example of this in action can be seen at https://github.com/moodle/moodle/blob/master/lib/outputrenderers.php#L2972 - this is the error handler I mentioned above.

You can define any output you like for an AJAX_SCRIPT but we tend to use JSON. We're already beginning to do this and it's something I'll be pushing in the next few versions of Moodle.

With regards the instantiation of JS, this is something I still need to look into, but I think it should already be possible. Where possible, I've been trying to write new code, and to convert existing code to use event delegation so no changes are required to add new listeners; and we do have some examples of other instantiation JS being called when adding new components to the DOM (see the file drag/drop in course editing - though this needs an overhaul to use events).

All of this is also really well supported by YUI.

Andrew

 
Average of ratings:Useful (2)
Me!
Re: Discussion: Best way of making JS/YUI widgets work with AJAX
Group DevelopersGroup Moodle HQGroup Particularly helpful MoodlersGroup Testers

I did get this to work on this branch.

https://github.com/damyon/moodle/tree/WIP_MFORM_DIALOGUE

(That branch adds a tool to the admin menu where you can test loading an mform in a yui dialog via ajax - including a tinymce editor).

But after getting this to work I think this approach is too hacky and I don't like it. 

Cheers, Damyon

 
Average of ratings: -
Picture of Per Ekström
Re: Discussion: Best way of making JS/YUI widgets work with AJAX
 

Ok, I see. That's rather good news, actually. smile

I've been down the road of loading a separate script before, but it's a lot more hassle to maintain and doesn't account for things like, say, being logged out of a session, lazy devs not checking that proper permissions exists (you can write to the database while logged out? WTF?) and other fun things. Therefore I've started to make my own webapps with as few entry points as possible, since it reduces chances of security holes (but at the cost of a slight overhead of course). As you point out, though, it's not a trivial problem to solve.

The biggest advantage to using the $CFG->isajax approach as opposed to AJAX_SCRIPT is that you get all the logic in one place. From there, you can very easily validate all the AJAX output you make to be proper HTML, and it's also fantastic when you turn off JavaScript - everything just works. Because of that, I must say I really prefer doing AJAX pages in this way:

if (!$CFG->isajax) {
    $PAGE->header();
    echo '<p>Hello World!</p>';
    $PAGE->footer();
} else {
    echo '<p>Hello World!</p>';
}

Naturally, this is just a simple example - in reality the Hello World would be a require_once("template.html"), and all the instantiation of the template is done on beforehand. The only disadvantage I can see with this approach is that it requires a bit of juggling with $CFG->isajax, but otherwise I do not see any real drawbacks to the $CFG->isajax approach. But to each to his own, no? smile

From what version is AJAX_SCRIPT available BTW? 2.5 or?

 
Average of ratings: -
Davo
Re: Discussion: Best way of making JS/YUI widgets work with AJAX
Group DevelopersGroup Particularly helpful Moodlers

AJAX_SCRIPT has been there since Moodle 2.0 (unless I'm mistaken).

You might want to consider the following variation on your AJAX page:

ajax-page.php:

define('AJAX_SCRIPT', true);
require_once('../../config.php');
require_once('lib.php');
myclass::view();

view.php:
require_once('../../config.php');
require_once('lib.php');
...
echo $PAGE->header();
myclass::view();
echo $PAGE->footer();

lib.php:
class myclass {
  public static function view() {
     echo html_writer::tag('p', 'Hello world!');
  }
}

By routing most of your logic through a class in a library file, you eliminate many of the problems of multiple entry points, whilst retaining much of the code clarity that comes from them.

 

 
Average of ratings:Useful (2)
Picture of Per Ekström
Re: Discussion: Best way of making JS/YUI widgets work with AJAX
 

Yes, absolutely, you can do it that way (and I did for quite some time). My experience with doing it that way, however, is that you're far better off trying to minimize your entry points to your apps. Separating the logic from the presentation helps, a bit, but what happens when you for instance try to make an AJAX call to a script supposed to output JSON when you're logged out? You do get breakage, and it's a bigger risk that edge cases causes breakage. That's the primary reason I'm not really fond of multiple entry points.

In the end it's a matter of preference though and both styles have their advantages as well as their disadvantages. But, shouldn't it be possible to do something like:

if ($_SERVER['HTTP_X_REQUESTED_WITH'] == 'xmlhttprequest' OR optional_param('isajax',0,PARAM_BOOL)) {
    define('AJAX_SCRIPT',true);
}

[...]

if (isset(AJAX_SCRIPT)) {
    myclass::view();
} else {
    $PAGE->header();
    myclass::view();
    $PAGE->footer();
}

Couldn't that combine both into one "best of both worlds"?

 
Average of ratings: -
Picture of Andrew Nicols
Re: Discussion: Best way of making JS/YUI widgets work with AJAX
Group DevelopersGroup Moodle HQGroup Particularly helpful MoodlersGroup Testers

Hi Per,

I considered this too. Ignoring the fact that you wouldn't be able to use optional_param (because it's not available until config.php is defined, by which time it's too late to define AJAX_SCRIPT), this can work. I had a play with a proof-of-concept adding the code to lib/setup.php.

I felt at the time that it was still better in some respects to keep the entrypoints separate as this could lead to confusion later on. It's too easy to expect AJAX_SCRIPT to do all of the formatting too, and that may be the case, but chances are that it won't. Although some people will write their code as you suggest, there's a strong chance that people will do a lot of testing AJAX_SCRIPT all over the place in the same script, and will lead to nastier code to read.

Keeping the AJAX version in a separate script, appropriately namespaced, gives a clear indiciation as to purpose.

Perhaps it's worth re-opening the debate though... what are your main reasons for disliking multiple entrypoints?

Andrew

 
Average of ratings: -
Picture of Per Ekström
Re: Discussion: Best way of making JS/YUI widgets work with AJAX
 

Hi Andrew!

Sorry for being a timesink, I'll try to keep this brief.

The main advantages and disadvantages to single- vs multi entry points the way I see it are these:

Single entrypoint

Advantages

  • Less risk of security holes and edge-case bugs
  • Opens up for automatic testing and/or (X)HTML validation
  • Everything collected in one place, easy to follow and overlook
  • Almost impossible to break the "No JavaScript Requirements" rule
  • All links can go to "real" places with little to no extra work, thus not breaking middle-clicks
    • Compare <a href="view.php?id=14" onclick="load_ajax('foo',this.url+'&isajax=true')">this link</a> with a classic <a href="#" onclick="load_ajax('foo','ajaxscript.php')">ajax link</a>

Disadvantages

  • Not as efficient, intuitive or straightforward as multi-point
  • Source code files can get large, complex and hard to overlook, especially with multiple paths
  • Does require a bit of "PHP logic juggling", especially if you want automatic test benefits

Multiple entrypoints

Advantages

  • Clear purpose
  • Simple structure - one-script-does-everything
  • Fast and efficient - All unneccessary fat trimmed away

Disadvantages

  • Need to be very careful about logic in order to avoid security holes
  • Error handling might miss edge cases, which might give the user a bad impression
  • Hard to automate tests and automatic (X)HTML validation
  • Code can get rather complex - ajaxscript.php calls to ajaxscript2.php which calls to a template file that calls to ajaxscript3.php etc etc
  • Might possibly break the "No JavaScripts required" rule

Do note that many of the advantages/disadvantages I list are dependant on how you break up your code, and my bad experiences with multi-point may have clouded my judgement. In either case you need a solid structure and a steady hand, else you'll just end up with the old-fashioned PHP soup we've both seen once too many times... smile

Hope that made it a bit clearer on my position!

 
Average of ratings: -