General developer forum

Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users

 
This discussion has been locked because a year has elapsed since the last post. Please start a new discussion topic.
Picture of Rowan Chakoumakos
Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
 
Hello Moodlers!!

I am Rowan from GHOP!
I have searched through the documentation and could not find any articles on Gzipping Moodle.

demo.moodle.org could be reduced from 21k to 5k: a reduction of 76.19% by gzipping alone!
This does not only saves bandwidth, the page appears to load faster for the user.


Gzipping is usually achieved in two different ways:

Gzipping on the fly - When a user requests the page, the server gzips the page and sends this. This can increase the load on your server especially during high traffic.

Passive Gzipping - The server, usually during off-peak hours, creates a gzipped copy of each page. This obviously increases the amount of disk space utilized. When newer browsers such as Firefox request content, they first request a gzipped copy and thus the server sends this copy.

Implementing Gzipping in Moodle:

One simple way to implement gzipping on the fly, is to create a .htaccess file with a file handler. Essentially, any php file before being processed will get the instructions to gzip added to the top of the file.

Passive gzipping can be more difficult to implement because it requires setting up a cron job to run during non-peak hours. Also since Moodle is changing constantly, whenever a change occurs, the gzip verison must also be updated!

Any feedback or suggestions on this idea are more than welcomed smile

Thanks!




 
Average of ratings: -
Picture of Nicklas Lindgren
Svar: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
 
HTTP compression might be a good idea, or an unacceptable risk depending on your situation.

Most modern browswers support this, and those who don't will get uncompressed versions of the content.

However, there is a small group of browser versions who have bugs. Those who claim to understand HTTP compression but that in reality don't, or don't completely or correctly.

This group includes old Netscapes, MSIE6 before SP1 with Realplayer installed, old versions of Mozillas browser engine Gecko, and most likely some others too.

Nobody should use these browsers nowadays, but if they do they won't be able to access the Moodle, or will have serious problems doing so.

It all depends on how important you consider maximum accessibility, and what kind of contact you have with your users.



For HTTP compression in Apache i would recommend using mod_deflate (included by default) together with a conservative configuration enabling it, for example:

# Insert filter
SetOutputFilter DEFLATE

# Netscape 3.x is rumored to have unspecified problems
BrowserMatch ^Mozilla/3 gzip-only-text/html

# Netscape 4.x is rumored to have unspecified problems
BrowserMatch ^Mozilla/4 gzip-only-text/html

# Netscape 4.06-4.08 is rumored to have some severe unspecified
# problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip

# MSIE and browsers posing as it might handle it, there is
# however rumors about another bug than the one below
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

# MSIE 6 prior to SP1 (Jan 2002) might not handle it
# http://support.microsoft.com/default.aspx?scid=kb;en-us;Q312496
# Unfortunately there does not seem to be a way to differentiate
# between different MSIE 6.0 versions.
#
# BrowserMatch \bMSIE.6\.0 no-gzip

# Opera is rumored not to have had any serious problems handling
# compression
BrowserMatch \bOpera !no-gzip !gzip-only-text/html

# Konqueror has had bugs related to compression, but as far as i
# know none that prevented accessibility or readability of
# responses with Content-Encoding: gzip
BrowserMatch \bKHTML !no-gzip !gzip-only-text/html

# Old versions of Gecko have had problems with compression. I
# have not found a later confirmed bug than this one:
# https://bugzilla.mozilla.org/show_bug.cgi?id=deflate

# A crude way to filter Gecko versions based on the fix date of
# the bug:
BrowserMatch \bGecko/199 no-gzip
BrowserMatch \bGecko/200[01234] no-gzip

# Adobes PDF reader plugin has been known not to handle compression.
# Also PDF files usually contain compressed data.
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary

# Don't compress already compressed file formats
# Assuming that .exe means a windows self extracting file.
SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2|sit|rar|exe)$ no-gzip dont-vary

# This includes compressed image formats
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary

# Make sure proxies don't deliver the wrong content
# (Proxies might convert images between formats otherwise)
Header append Vary User-Agent env=!dont-vary

Passive gzipping is inappropriate for sites where nearly all content is generated by scripts, like in Moodle.
 
Average of ratings: -
Picture of David Berry
Re: Svar: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
 
We implemented GZIP compression about 3 weeks ago (because our techie likes playing). The crude figures are 5% speed increase and 75% bandwidth reduction. No checks have yet been performed for SSL and concurrency.

Thanks for the information on the browser (in)compatibility. I will have to look at that in more detail next term - but no-one has complained yet!

Regards

Dave Berry
 
Average of ratings: -
Me at the Moodle Moot NZ11
Re: Svar: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Particularly helpful Moodlers
Interesting... not something I have played with yet...
Can anyone expand what is meant by 'implemented GZIP compression' ?
(Just changing paths e.g. from /usr/bin/zip to /usr/bin/gzip ?)
I'd like to experiment on a test server...
This looks interesting...
http://www.whatsmyip.org/mod_gzip_test/phpgzip/
But I think I read somewhere that gzip doesn't work well with Moodle ???
Anyone?
Thanks so much, Stuart smile
 
Average of ratings: -
Picture of Richard Enison
Re: Svar: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
 
SM,

Well, I did a search of the Moodle docs (wiki) for gzip and here's what I found:

On the one hand,

http://docs.moodle.org/en/System_paths#Path_to_zip

but on the other hand,

http://docs.moodle.org/en/Backup_FAQ#How_do_I_backup_my_whole_Moodle_site.3F

So go figure.

RLE
 
Average of ratings: -
Picture of Iñaki Arenaza
Re: Svar: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Documentation writersGroup Particularly helpful MoodlersGroup Plugin developers

Richard,

that is about a different thing. Zip (the PKZip compatible one) is the format used to compress Moodle course backups, i.e., those made inside Moodle with its built-in backup tools.

That format is incompatible with gzip's one, which is fine to use outside Moodle (i.e., when you externally backup Moodle files, Moodle database, etc.).

Finally the issue being discussed here is HTTP compression when serving Moodle content the browser, which has little to do with the above wink

So three different scenarios, with three (well, just two wink) tools.

Saludos. Iñaki.

 
Average of ratings: -
Picture of koen roggemans
Re: Svar: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Documentation writersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Translators
A big thank you for pointing this out: we have our Moodle server at school behind a rather low upload bandwidth so compared with the speed we are used to at school, using Moodle was always slow at home.
Now, the largest coursepage I know of loads under 3 sec (on an unloaded server) cool
 
Average of ratings: -
Picture of Jenny Gray
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Plugin developers
Any-one interested in increasing client-side performance might find this discussion http://moodle.org/mod/forum/discuss.php?d=82578 helpful.

Gzip is one of the recommendations made by Yahoo - and something I keep meaning to look into!
 
Average of ratings: -
Martin Dougiamas
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Documentation writersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers
I used to have it enabled on moodle.org a few years ago, but I had reports of some browser incompatibilities (just as Nicklas describes here) and I had to reluctantly switch it off. moodle.org really has to be accessible.

Now however, perhaps the browser situation has improved. (Martin goes off to look...)

Actually, I see mod_deflate has improved over mod_gzip so that it's possible to target browsers with it. Great! Thanks, Rowan!

I've turned it on for moodle.org and demo.moodle.org!
 
Average of ratings: -
Picture of Ryan Smith
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
 
Martin,

Can you post all of your directives you used for mod_deflate in your conf?

Also, does this interfere with eAccelerator? Anybody know?

Thanks!
 
Average of ratings: -
Martin Dougiamas
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Documentation writersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers
I'm running eaccelerator v0.9.5.2 here, and it seems to be work fine. I've not tested files yet, see: http://moodle.org/mod/forum/discuss.php?d=66362

Here's what I've got in httpd.conf:
 
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript application/x-httpd-php application/x-javascript text/css
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
 
Average of ratings: -
Picture of Matt Gibson relaxing in the Alps
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Plugin developers
I tried getting this to work a while ago, but got nowhere. How can you tell if its working?

moodle.org (this page) still seems not to have it enabled according the Yslow output:


4. Gzip componentsnorgie_open_dna.gif

 
Average of ratings: -
Martin Dougiamas
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Documentation writersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers
I just added "application/x-javascript text/css" to the list of file types to process so that should be fixed now too.
 
Average of ratings: -
Picture of Jenny Gray
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Plugin developers
I'm still seeing the same output from YSlow that was reported above.

I'm running in Firefox 2.0.0.11 if that's any help.

The good news is that the list I see in YSlow for moodle.org is smaller than my site, so something must be working smile
 
Average of ratings: -
Picture of Matt Gibson relaxing in the Alps
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Plugin developers
Seems to be fixed for me.

moodle.org that is. Still not getting any success on my own setup. phpinfo shows that mod_deflate is enabled and I have pasted the above commands into httpd.conf, but no joy. apache error log is silent. Hmmm...
 
Average of ratings: -
Picture of Matt Gibson relaxing in the Alps
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Plugin developers
Seem Yslow has a problem reporting gzipping of js files when they have a far future expires header.

http://developer.yahoo.net/blog/archives/2007/07/high_performanc_3.html

(comments)
 
Average of ratings: -
Picture of Rowan Chakoumakos
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
 

Hello Martin Dougiamas,

Glad that I could help! smile
According to whatsmyip.org, moodle.org is saving 75.76% bandwidth big grin

Rowan

 
Average of ratings: -
Dan at desk in Moodle HQ, Perth
Re: Delivering Gzipped Content to Reduce Bandwidth and Increase Rendering Speed for Users
Group Core developersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers
I meant to reply to this post originally just to suggest that gzipping might not be worth the processing overhead in many moodle scenarios. (Where we are often more worried about server load than the reduced bandwidth gain).

Perhaps this is increasing the load on moodle.org atm ;)
 
Average of ratings: -