What would be a good test that templates do not cause performance problems?

What would be a good test that templates do not cause performance problems?

by Tim Hunt -
Number of replies: 10
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

This was raised a the last developer meeting: https://docs.moodle.org/dev/Developer_meeting_May_2016

Moodle has a nice new template system (mustache). For what we read, it seem to be pretty fast. However, before we convert all the existing Moodle output to it, we want to be sure that won't cause a huge performance regression. How could we best do that.

My suggestion would be to take one or two significant bits of output: e.g. a forum thread, where you have a lot of posts, each of which contains some things like user pictures, buttons and icons. My other suggestion would be outputting a section of the course page, with half a dozen different activities.

Then really, just call both the old code, and the new code, in a loop a lot of times to compare the time. The only thing to do is to make sure, as much as possible, that it is a like-for-like comparison. That is, ensure both chunks of code that we are are doing just the work of outputting the HTML, and not doing work of getting data from the database.

Does anyone have a better idea? How much work would this be?

Average of ratings: -
In reply to Tim Hunt

Re: What would be a good test that templates do not cause performance problems?

by Damyon Wiese -
Some points about testing this.
1 - We have 2 mustache engines - do we need to compare Mustache.js vs mustache.php ?
2 - There was some limited performance testing of mustache.php on the original issue: https://tracker.moodle.org/browse/MDL-49152?focusedCommentId=341909&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-341909
Average of ratings: -
In reply to Damyon Wiese

Re: What would be a good test that templates do not cause performance problems?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I think Mustache in JS in not really replacing anything else, so less important to profile there. (Also, it only causes problems on the client's computer, not our server, so less of a problem tongueout)

So, I think focus on PHP for now.

The issue with a lot of the testing done in that bug is that it was done on a developer machine. (Caching off, developer debug, no PHP accelerator.) We only really care about performance a server tuned like a live server such as might be used in production. That is important.

Average of ratings: Useful (2)
In reply to Tim Hunt

Re: What would be a good test that templates do not cause performance problems?

by Dan Poltawski -
I think Mustache in JS in not really replacing anything else, so less important to profile there. (Also, it only causes problems on the client's computer, not our server, so less of a problem)

So, I think you are probably just inventing performance measurement which is probably not going to be the primary concern if everthing moves more client-sidey. In fact we found and fixed a big client-side issue (MDL-53667) with competencies which would give me far more concern than if templates are twice as slow.

We Moodle developers need to start taking client side performance a lot more seriously.

Average of ratings: -
In reply to Dan Poltawski

Re: What would be a good test that templates do not cause performance problems?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Any performance regression that rears its ugly head is serious, and should be taken seriously. MDL-53667 is interesting. I had not seen that before. (I note it came up in Moodle helper code, not the mustache core library.)

I am not 'inventing' performance measurement. We do performance measurement at the OU every 3 months, before each of our releases. Sometimes we we are the first to find performance regressions in core, and then we fix them.

If templates made page-load times twice as slow, that would be a serious problem, and if that was done throughout core, then it is not something that could be 'fixed' without basically reverting a whole release's worth of changes. Hence my suggestion that we get some measurements before heading too far in one direction.

Look, all the signs are the mustache takes performance seriously. However, we are engineers. We should measure and verify.

Average of ratings: Useful (2)
In reply to Tim Hunt

Re: What would be a good test that templates do not cause performance problems?

by Dan Poltawski -
Any performance regression that rears its ugly head is serious, and should be taken seriously.

I agree (and I'm sorry my comment was a bit flippant, bit like your comment about client side performance wink). In fact you'll find me banging the drum of us wanting to get us a lot closer to the webkit approach than we already are. While we do have some modest progress towards that, the important thing we need to get better is decent metrics.

My point was that I am not sure I entirely agree with you that this metric you are so focused on in isolation is as important as the emphasis you put on it. I expect that mustache templates will be slower, but I feel like we're talking about the microptimisation level and orders of magnitude mean that saving a database query will be far more important for most pages than template cpu degradation. 

We do performance measurement at the OU every 3 months, before each of our releases. Sometimes we we are the first to find performance regressions in core, and then we fix them.

And I/we thank you for that, but how can we as a community get this sort of measurement with decent real world-ish data (from you and other large sites) tested earlier, so you stop having to detect those performance regressions? That is a far more important thing to work on in my view.

Finally (and importantly), what are you going to do if you find the 'wrong' answer here - without actual evidence of real world problems I foresee a future of bugs open reporting the problems without a solution. I agree with doing this sort of measurement before something lands and its a shame we didn't get anything more extensive while it was in development - but you know how the world works, people want you to ship, compromises are made, we are all human and some things slip.

Average of ratings: Useful (1)
In reply to Dan Poltawski

Re: What would be a good test that templates do not cause performance problems?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Yes. smile

You are right that we need to focus on the big picture with performance, and that good metrics, regularly generated is probably a key next step.

Average of ratings: -
In reply to Tim Hunt

Re: What would be a good test that templates do not cause performance problems?

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

We need a good piece of function that now (or will soon be) using Mustache templates that was using renderers, like you mention.

You mentioned forum, but I think the forum is only using Mustache for its email output. Is there any core module completely using templates now? If not, maybe we can look to third-party plugins? I am currently looking at replacing the output functions of questionnaire with renderers. Maybe I should change that effort to templates, at least in a development branch?

mike

Average of ratings: -
In reply to Mike Churchward

Re: What would be a good test that templates do not cause performance problems?

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

So, to help with this, I'm trying to get up to speed on how to implement templates with renderers. The example work that was mentioned as a good place to follow is MDL-50464.

I can see the developer docs on templates here - https://docs.moodle.org/dev/Templates.

Is there more documentation somewhere? I ask, because I cannot figure out how all of the functions in "block_rss_client/classes/output/renderer.php" are used. Functions like "render_item" are not called from anywhere obvious, so I'm assuming there's some "auto-magic" happening in "block_rss_client/templates/item.mustache" that I'm not aware of.

Can anyone help?

mike

Average of ratings: -
In reply to Mike Churchward

Re: What would be a good test that templates do not cause performance problems?

by Mark Johnson -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
As far as I can tell, the only render_foo() function in that renderer that's being called is render_block(), which causes export_for_template() on each templatable object attached to the block object to be called.

Looking at the comments on MDL-50464 which was the issue to convert this block to renderers/templates, I think the extra render_foo() functions were just added to provide flexibility for someone overriding the renderer in a theme, since it allows each renderable to be rendered individually.  I expect Brendan Anderson who did the development can tell you for sure!
Average of ratings: -
In reply to Mark Johnson

Re: What would be a good test that templates do not cause performance problems?

by Brendan Anderson -

Mark, that's right. render_block is called, but render_item and render_feed are not. My thinking was that it might be possible to extend block_rss_client itself and not have to extend the renderer.

I was trying to provide maximum flexibility without a good understanding of how the functionality would be overridden/extended.

When exporting the block for templating, the feeds and items are too. Then they are rendered with an include inside the mustache file.
Average of ratings: -