Purge Cache takes Forever with Boost and Child themes

Purge Cache takes Forever with Boost and Child themes

by Chris Kenniburg -
Number of replies: 30
Picture of Particularly helpful Moodlers Picture of Plugin developers

In Moodle 3.3 we are getting issues with long times to refresh the page when clearing cache or saving admin settings.  It seems to affect all boos-based themes.  Specifically we are concerned with Fordson on Moodle 3.3 but we noticed quite a big difference in refresh times with Boost and several child themes as well.  

Commenting out the following lines seems to help dramatically but we loose the icons obviously.  

@import "../../boost/scss/fontawesome/moodle-path";
@import "../../boost/scss/fontawesome/font-awesome";

Is this just how things are going to be or are we doing something wrong with the themes?

Average of ratings: -
In reply to Chris Kenniburg

Re: Purge Cache takes Forever with Boost and Child themes

by Dan Bennett -

+1 with this. We are seeing the same issue since Moodle 3.2.

Any changes to Boost theme itself, or any child themes developed based off Boost, take a long time to just load the next page. And you can imagine how painful it is for a theme developer...

Average of ratings: -
In reply to Dan Bennett

Re: Purge Cache takes Forever with Boost and Child themes

by Kevin Wiliarty -

I assume that this is the price for not having to run grunt to process the sass files as you edit them. The sass files need to be compiled to regular CSS before the browser can use them. You can maintain the sass and css in parallel so that the css is represented in actual files and is always ready. It looks to me like Boost processes sass 'on the fly' and keeps it in the cache, which is fine ... except when you clear the cache. Note that there are no `.css` files in Boost.

Average of ratings: -
In reply to Kevin Wiliarty

Re: Purge Cache takes Forever with Boost and Child themes

by Chris Kenniburg -
Picture of Particularly helpful Moodlers Picture of Plugin developers

I am seeing up to 30 seconds to refresh the page on localhost and about that or a bit longer on production site.   I was worried I was doing something wrong with my theme but it happens on all the child themes to one extent or another.

I am guessing this is just the way it will have to be for now?

Average of ratings: -
In reply to Chris Kenniburg

Re: Purge Cache takes Forever with Boost and Child themes

by Marina Glancy -
Picture of Core developers Picture of Moodle HQ Picture of Moodle Workplace team Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

FYI tracker issue for this problem is MDL-58646

Average of ratings: -
In reply to Marina Glancy

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

@Maina, how did you at Moodle HQ develop the boost theme when each cache clearing and theme rebuild takes a long time? Did you use workarounds or did you just sit and wait? Or may there be other options?

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

If not from @Marina it would be interesting to learn from someone from Moodle HQ who is involved in the theme work how they deal with the long cache refresh times during theme development.

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Richard Oelmann -
Picture of Core developers Picture of Plugin developers Picture of Testers

Just a note to say that this isn't just about theme development per se - it impacts every time an admin changes a setting, or implements any feature that requires a purge of caches.

This isn't just a developer issue, its a usability issue for anyone actually maintaining and administering a site, whether developer or not, whether access to server side files or not.

Richard

Average of ratings: Useful (5)
In reply to Chris Kenniburg

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

After a lot of deep core code debugging I discovered that the treepostprocessor called in the function post_process($css) in outputlib.php is the reason for the extreamly slow CSS generation. When I disable the treepostprocessor in the theme config.php the CSS generation is fast again.

$THEME->csstreepostprocessor = null;

What does the treepostprocessor in Moodle do?

Average of ratings: Useful (3)
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Richard Oelmann -
Picture of Core developers Picture of Plugin developers Picture of Testers

Is that the bit that got added to deal with auto-processing the vendor prefixes?

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

The convenient SCSS compilers are written in C/C++ because the task is demanding. So it's no wonder that a PHP implementation takes very long. I will see if I can develop a »hack« to use a fast SCSS process instead for the work on the design. Iterative work on the theme will not be possible with the Moodle PHP SCSS compiling because each iterative step would take minutes - and there are many steps in the process.

Has anybody developed a way to use SCSS that is compiled fast? My basic idea is to compile the SCSS externally and include the resulting CSS file in the theme.

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Mary Evans -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Surely it would be simpler just to add the FontAwesome CSS file.

 

Average of ratings: -
In reply to Mary Evans

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

@Mary It's not only about the FontAwesome CSS, a major issue is the treepostprocessor.

And when these two issues may be solved the SCSS rendering in PHP is still much slower than with for example libsass. To get a convenient workflow it should be possible to use the proven standard SCSS renderers instead of the Moodle PHP ones.

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Richard Oelmann -
Picture of Core developers Picture of Plugin developers Picture of Testers

The problem with compiling the scss externally is that it then prevents the use of the rawscss and variables as settings

Average of ratings: -
In reply to Richard Oelmann

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

@Richard, I could imagine that Moodle writes the user added rawscss and variables form settings as a user preset SCSS file for example in moodledata. This file can be included into any external process for compiling.

Because of the speed advantage during theme development external SCSS processing would be a big win.

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Mary Evans -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

@Urs,

They say that PHP7 appears to make Boost theme a lot quicker, so I tested this out on my website last night when I upgraded from 3.2 to 3.3 and sure enough all worked well.

Mary

 

Average of ratings: -
In reply to Mary Evans

Re: Purge Cache takes Forever with Boost and Child themes

by Richard Oelmann -
Picture of Core developers Picture of Plugin developers Picture of Testers

@Mary, yes I'm using php7 and although there is a delay, I've never had the issues to the extent reported 

@Urs, I wasn't so much thinking of theme developers who may have the skills and the inclination to set up Grunt etc and process the scss externally. But it would need to work for 'ordinary' (if there is such a thing) site administrators,including the many who don't have server access.  After all, a theme developer isn't the primary target for theme settings. 

Could it work something along the lines of the php classes for Imagemagick? Relies on an external process, but controlled via the php, so still from within the main Web page code? (given that still requires additional php plugins installed, but we already have that with Moodle) 

R

Average of ratings: -
In reply to Richard Oelmann

Re: Purge Cache takes Forever with Boost and Child themes

by Chris Kenniburg -
Picture of Particularly helpful Moodlers Picture of Plugin developers

I am noticing about a 30 second to 45 second delay once you purge cache.  Never anything more than that.  So it is not unbearable but it does make testing a bit of a chore.  

Average of ratings: -
In reply to Chris Kenniburg

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

I am testing a »hacks« based workflow now with libsass in an external process and the compiling is done nearly instantaneous. With this workflow I can iterate. The »hacks« process seams to work, I don't know if Moodle will compile the same CSS when I remove the »hacks«. That I need to test.

Btw I work with the Mac program »CodeKit«, which is a web design build tool somehow similar to Grunt but with a GUI, which I prefer.

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by David Scotson -
Hi Urs, is there an issue to follow any work on this?

We've recently moved the Snap theme to sub-theme Boost, and this delay is somewhat annoying for developers.

Workarounds we've employed or things we've noticed:

* we have a grunt task that hits this url:  joule2.dev/theme/styles.php/snap/-1/all, this starts the CSS regeneration process as soon as we hit save in our code editors so hopefully it is nearly done by the time we switch to the browser and refresh or load a new page

* we also spotted the tree post process call was taking up a lot of time. We were wanting to skip this in our child theme, but it seems if you set it to null in the child theme, it just uses the value from the parent theme (Boost). Even if you replace the call with a new function that does nothing, it still parses the full tree which is where the time is mostly spent. Hacking the Boost theme

* The time taken seemed to be proportional to the length of the total CSS. Our dev VMs have a whole lot of plugins installed and took an extra 10 seconds compared with a default Moodle install. We also found a few bits of the SCSS code that were using @extend and generating silly amounts of CSS code. We fixed the ones in Snap and also filed a bug upstream about uses of @extend in Boost: MDL-58930

* There are existing PHP extension Sass libraries on github that call the C lib to go faster, though it sounded like they only supported Apache.

Average of ratings: Useful (1)
In reply to David Scotson

Re: Purge Cache takes Forever with Boost and Child themes

by Dan Poltawski -

is there an issue to follow any work on this?

There is a dramatic performance bug with fontawesome in 3.3 which should be fixed this week in MDL-58646

More wide ranging improvements are in epic MDL-59122 subtasks (NOTE: you need to be logged into jira to see epic subtasks for some annoying reason..)

Average of ratings: -
In reply to David Scotson

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

Hi David,

you seam to have found the same topics that slow down the theme development process. I plan to write about my workaround which seams to circumvent the issues quit ok as I can say by now. But I am extremely busy in the moment to get some tasks done so it may last some time until I can write.

And yes, I did hack boost to stop the tree post process there too.

Average of ratings: -
In reply to Richard Oelmann

Re: Purge Cache takes Forever with Boost and Child themes

by Urs Hunkler -
Picture of Core developers

Someone from Moodle HQ mentioned that the external process could be optional. Something like - if an external process exists use it else use PHP. That would be great.

Average of ratings: -
In reply to Urs Hunkler

Re: Purge Cache takes Forever with Boost and Child themes

by Chris Witham -

That sounds really useful. We maintain a frontend framework that we implement across a number of products, Moodle being just one of them. It would be far easier for us to use a grunt file to compile the SCSS for us outside of Moodle during theme development and then just ship our compiled, minified CSS to production within the theme instead of having the SCSS compile on the production server when settings change.

I understand there are some downsides to this like not using Moodle settings variables in the SCSS files but I think that's a trade off I'd willingly accept.

Average of ratings: -
In reply to Richard Oelmann

Re: Purge Cache takes Forever with Boost and Child themes

by Andrey Mindubaev -

Moodle could create all necessary files with rawscss and settings, @import all created files and sources files, and after that run external scss.

Average of ratings: -
In reply to Chris Kenniburg

Re: Purge Cache takes Forever with Boost and Child themes

by David Penner -

Seeing the same. Glad to see I'm not the only one!

In my case, until I disabled xdebug, I was having to wait upwards of 2 minutes (this is on a quad core virtual server with plenty of RAM sitting in my institution's data center) when purging caches or while using theme designer mode. Disabling xdebug brought it down to the 30 seconds others mention here. Which is still far too long for effective iterative work while theming.

Average of ratings: -
In reply to David Penner

Re: Purge Cache takes Forever with Boost and Child themes

by Bruno Castello -
Hi, how you managed to disable it? I am running latest Moodle version on my iMac, the only way I managed to bring it down to ~30 seconds was commenting the two lines about fontawesome in my SCSS files. Not really a good solution for a frontend designer, while we wait for a proper solution. Thanks in advance.
Average of ratings: -
In reply to Bruno Castello

Re: Purge Cache takes Forever with Boost and Child themes

by Richard Oelmann -
Picture of Core developers Picture of Plugin developers Picture of Testers

Try commenting out the cssposttree bit instead of the fontawesome, then ensure you are designing on a standards compliant browser that doesn't need all the vendor prefixes (or as much as possible anyway). Then re-enable the bit commented out when you need to test against vendor prefixed or older browsers, or when releasing to an environment where you have less control over the user browser choice (ie almost anywhere! smile )

Average of ratings: Useful (1)
In reply to Richard Oelmann

Re: Purge Cache takes Forever with Boost and Child themes

by Bruno Castello -

Thanks! But commenting out the csstreepostprocessor does not help at all - it takes about 2 minutes to reload after clearing the caches. Only when I comment out the lines about fontawesome it takes ~30 min.

Average of ratings: -
In reply to Chris Kenniburg

Re: Purge Cache takes Forever with Boost and Child themes

by Elliott Benzle -

Second this. There are numerous variations of this problem on the forums with the bottom line being that the CSS cache is painfully slow to rebuild and server settings need to be changed to allow enough time for the cache script to complete.

Additionally, any problems with any CSS files break the entire site. 

Average of ratings: -