Varnish
Has anyone used varnish in front of moodle? Did you find anything useful?
Varnish sounds pretty cool, but my brief reading around it makes me think it'd be almost useless in front of moodle (since auth around most scripts/shared files would cause most things to be uncacheable).
The only thing it might help with might be themes? Is that your findings?
Varnish sounds pretty cool, but my brief reading around it makes me think it'd be almost useless in front of moodle (since auth around most scripts/shared files would cause most things to be uncacheable).
The only thing it might help with might be themes? Is that your findings?
Re: Varnish
Others @ Catalyst use it for http://www.tab.co.nz/ there might have been some tests with Moodle. Nothing I know of.
Even if our stuff is not cacheable, having a reverse proxy as buffer frees up mem-heavy apache processes. Without it, mem-heavy processes sit there, spoonfeeding bytes to the tcp connection that might have a slow client at the other end, or a bottleneck anywhere along the pipeline.
For my current job however, I'm looking more at rproxy than varnish
And - we must make moodle more cacheable. For lots of things (file.php, mod/forum/view.php) we could just say must-revalidate, and then do something smart with the revalidation request. For file.php, a sha1 for the etag would do magic, Petr knows all about it. There a lot of work to do on that track.
Even if our stuff is not cacheable, having a reverse proxy as buffer frees up mem-heavy apache processes. Without it, mem-heavy processes sit there, spoonfeeding bytes to the tcp connection that might have a slow client at the other end, or a bottleneck anywhere along the pipeline.
For my current job however, I'm looking more at rproxy than varnish
And - we must make moodle more cacheable. For lots of things (file.php, mod/forum/view.php) we could just say must-revalidate, and then do something smart with the revalidation request. For file.php, a sha1 for the etag would do magic, Petr knows all about it. There a lot of work to do on that track.
Re: Varnish
"we must make moodle more cacheable"
Just seen this. A long, long time ago I added some 304 Not Modified support to file.php. We've been running the code in production for two years now. I know things are changing in 2.0, but it might be worth considering for a future 1.9.x release.Re: Varnish
Now that we've been running varnish in front of our Moodle cluster for a while, I've been monitoring what has been the top cache misses. These eight files make the top eight requests that caused a cache miss back-end request:
/lib/cookies.js
/lib/overlib/overlib_cssstyle.js
/lib/overlib/overlib.js
/filter/asciimath/ASCIIMathML.js
/lib/javascript-static.js
/lib/dropdown.js
/lib/ufo.js
/lib/speller/spellChecker.js
#9 was /login/index.php
The fact that every one of the top eight misses is a Javascript file that should be eminently cacheable jumps right out at me. I'm more of a sysadmin than a developer, though, so I'm not at all familiar with how the Moodle code handles cookies and delivers files. Surely there must be some way to have Moodle not require an auth cookie for JS files?
-Greg
/lib/cookies.js
/lib/overlib/overlib_cssstyle.js
/lib/overlib/overlib.js
/filter/asciimath/ASCIIMathML.js
/lib/javascript-static.js
/lib/dropdown.js
/lib/ufo.js
/lib/speller/spellChecker.js
#9 was /login/index.php
The fact that every one of the top eight misses is a Javascript file that should be eminently cacheable jumps right out at me. I'm more of a sysadmin than a developer, though, so I'm not at all familiar with how the Moodle code handles cookies and delivers files. Surely there must be some way to have Moodle not require an auth cookie for JS files?
-Greg
Re: Varnish
I'm running varnish in front of my Moodle sites. Not a huge amount of benefit, but we are seeing at least some content being cached.
We had some initial pain because our default config was stripping the auth cookie on requests for (png|jpg|gif), causing users to get stuck in an endless loop of login prompts. On many systems, that config makes excellent sense since most graphics are going to be static and served up without requiring authentication, but Moodle is a whole different critter. I think with some careful configs, we can probably get varnish to cache at least the theme files and graphics. I'm currently working on tweaking our varnish configs. I'll post when I know more.
We're also working a lot with Drupal and Pressflow with regards to caching. I suspect we might find some excellent lessons on how to maximize caching comparing the cache-optimized Pressflow code with the stock Drupal code.
Quoth Martin:
Wow. That would be stellar!
-Greg
We had some initial pain because our default config was stripping the auth cookie on requests for (png|jpg|gif), causing users to get stuck in an endless loop of login prompts. On many systems, that config makes excellent sense since most graphics are going to be static and served up without requiring authentication, but Moodle is a whole different critter. I think with some careful configs, we can probably get varnish to cache at least the theme files and graphics. I'm currently working on tweaking our varnish configs. I'll post when I know more.
We're also working a lot with Drupal and Pressflow with regards to caching. I suspect we might find some excellent lessons on how to maximize caching comparing the cache-optimized Pressflow code with the stock Drupal code.
Quoth Martin:
we must make moodle more cacheable. For lots of things (file.php, mod/forum/view.php) we could just say must-revalidate, and then do something smart with the revalidation request.
Wow. That would be stellar!
-Greg
Re: Varnish
Based on today's testing, I think even adding this little bit to the varnish config will help cache some of the static Moodle elements:
(in sub_vcl_recv)
if (req.url ~ "^/(theme|pix)/") {
unset req.http.Cookie;
}
I've seen a significant drop in cache misses for a bunch of theme files and there haven't been any side effects.
I'm still pondering how to cache a the js files in /lib and /filter. Those seem to be the next largest offenders hitting the back-end server.
By the way, if you're looking for a quick way to see what's topping the list of back-end (non-cached) hits, 'varnishtop -i TxURL' is your best friend. Run it whil users are hitting the site and it gives you a list of the top file requests to the back-end.
-Greg