Images no longer appearing/corrupted - Moodle 2.4.6

Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -
Number of replies: 12

Hello. Long time Moodle admin, first time poster.

Moodle environment: 2.4.6+ (newest build as of 9/27/13)
PHP: 5.3.2 (Can not upgrade to 5.3.3 due to server limitations)
Server: Dedicated host on an Ubuntu box
File permissions: moodle_data set to 777 (temporarily)


We had a Moodle site that was at 1.9 for a long time. Due to some security issues (that have been resolved), we finally upgraded from 1.9 to 2.2 to 2.4 in July 2013. The upgrade process went smoothly, with all of our courses and course files, including images and PDF links, carrying over fine for each of the 60+ courses. The client made the decision to keep the site inactive for the public since June 2013 and had a target goal of updating course content for a relaunch in October 2013.

In mid-September, the client continued updating course but a funny thing happened - the previous Legacy Course File images and files stopped appearing in the browser. The images simply stopped showing up, appearing as red X as a broken image. When I would try to use the file browser (server files) to see the images, the thumbnail previews did not appear. Going to the file directly (based on the html path from Moodle) did not render the file either and instead showed it as what I can only assume is ASCII. This seems to be the case for all images on the site, not including ones that come with a theme such as icons and such (the thumbnails on the theme selector page are broken as well). I have included images for both of these scenarios so you can see the issue.

This issue seems similar to what was reported here - https://moodle.org/mod/forum/discuss.php?d=218764 - but that thread never had a solution to the issue posted due to some other issues not related for that user.

I have restored the site from a previous backup before the client started editing, both the files and the database, to no effect. I have had the host check permissions, make sure everything was in binary mode (files were never uploaded via ftp, all were added via Moodle's interface in the previous 1.9 version) and that was not the solution. If I download one of the legacy course files locally, specifically an image, I can not get a preview of it (In Preview on OS X) BUT I can open the file itself in Photoshop. 

Adding a new file, with a new file name presents the same issue of not rendering correctly, even if Moodle has never seen the file before. All files are going to the general Server files bucket. 

The path to the moodle_data folder is correct, the HTML output for the image itself is correct (not broken tags like referenced in the post I linked to). I think that the issue is related to the file.php and its associated libraries but I am not a PHP programmer so I am not sure.

I have cleared theme caches more times than I can count, updated to the newest possible version and still the images simply won't render. So essentially the issue seems to be something has happened to the moodle_data folder (which is stored outside of the site, as recommended by Moodle) and due to the way Legacy Course Files are saved there is no specific way for me to figure out if an errant file is causing it or anything.

I am officially stumped, the host is stumped and the client is really hoping we don't have to offload their images to a separate site and redo all of the uploads.

I will be happy to answer any questions or provide any more information if needed. Any and all suggestions are appreciated and thank you in advance!

Attachment ascii_image.jpg
Attachment themes_broken.png
Average of ratings: -
In reply to Adam Varn

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Jez H -

That looks like fun!

The most likely cause would be with the HTTP headers. The webserver tells the browser what kind of file it is, it is not simply the case that a browser sees "jpg" and handles it as an image.

This can be controlled in PHP and from memory file.php does a bit of this:

http://php.net/manual/en/function.header.php

It can *almost* certainly be handled by rules in the webserver / .htaccess file too.

Something which may be of use to you is a firefox plugin called "Live HTTP Headers":

https://addons.mozilla.org/en-US/firefox/addon/live-http-headers/

With that plugin you will be able to inspect the headers, check the content type, details of compression being used etc.

I would not start hosting externally, I am sure you can fix this in Moodle, it looks like the path and image files are correct which is the main thing.

Average of ratings: Useful (1)
In reply to Jez H

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -

Thank you Jez H for the quick response!

The site where Moodle is installed does not have a .htaccess file currently. What's the easiest way to get one in there (other than copying one from another, non moodle site)?

I will check this out in the meantime but any info on the .htaccess file would be appreciated.

Average of ratings: -
In reply to Jez H

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -

Checked with the host and they say PHP is fine so I am stuck with the htaccess issue it seems. I've attached a copy of the results of the Live HTTP headers in case that helps anyone (I edited out the site name for security purposes)

I guess I need to know if
A) do I need an htaccess file to solve the issue
B) what needs to be in there and how do I create one specific to Moodle?

As I said before, the site was working fine without any issues until suddenly...it wasn't. So nothing was changed on the server end.

Average of ratings: -
In reply to Adam Varn

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Jez H -

Hi Adam,

You do not need a .htaccess file, if there was one there it would have been worth checking it, I am not even sure that .htaccess could cause this, suffice to say if there is no file then that cannot be the problem.

Looking at the output from your text file I think the problem is this:

Content-Type: text/html

It should look like this:

Content-Type: image/png

Or show a different status with no content type at all.

As you have already seen, you are indeed being presented with "text" as opposed to the rendered image.

I am not sure where that would be coming from, I know Moodles file handler used to have code to set headers, but it could be set on the host too.

If images in the theme (your logo for example) are rendering ok that would suggest the problem is with Moodle.

Thngs dont "just break" though, something must have changed somewhere, are you sure your host has not applied any patches (in particular changes to PHP)?

 

Average of ratings: -
In reply to Jez H

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -

Hi Jez,

No, nothing was applied to the server (it's a managed server so unless I ask them to do something specific to it, they don't touch it). There are about 18 other PHP powered sites on this box and non have had any issues in the past few months, and are running fine so I am pretty confident that nothing was patched/changed.

I saw the text/html as well but how (or what) do you go about changing the headers? There has to be something in there that is causing these to be seen incorrectly, I just don't know what could have altered it. If it is coming from Moodle, where would I go about trying to figure out the cause of this, since you said it used to have code that set the headers (if the host isn't doing it)

Appreciate all the help on this, like I said it is pretty bizarre and considering we are getting into specific HTTP requests there has to be something underlying that is causing it.

Average of ratings: -
In reply to Adam Varn

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Jez H -

If you look at filelib.php there are many references to mime types, the function "get_mimetypes_array" retuns a long list:

https://github.com/moodle/moodle/blob/d45e65ccadbf4097a970655941ac1c5cae17f26d/lib/filelib.php

Moodle uses this code to set the header for given files, but I am not sure why it would be failing on your Moodle install.

You could try cloning your site with a more recent + release, personally I stick with the stable versioned releases and avoid the + builds.

If you backup your db and existing files, try and rebuild it with a more recent build and see if it still does it.

Before you do that you may want to re-visit the .htaccess thing and see if you can fix it that way:

http://www.htaccess-guide.com/adding-mime-types/

You could try creating a .htaccess file in your Moodle directory and add:

AddType image/jpeg .jpeg .jpg .jpe .JPG

Not sure it would work but for the two minutes it would take to test its worth a shot.

 

Average of ratings: -
In reply to Jez H

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -

Hi Jez,

I tried the htaccess route (thank you for the link) but had no luck, so that was a no go.

I looked at the filelib.php file but I am hesitant to start modifying that, for fear it would disrupt something else. Not really a coder either so I wouldn't really know too much of where to start.


I don't quite follow what you mean about cloning and using a newer release. Since I am on the latest + build already, how can I go back down to a different version? Won't that alter the DB? And by rebuild it, what do you mean exactly, like try to do a new install and then import the old courses in from scratch? 

Appreciate all of the help here.

Average of ratings: -
In reply to Adam Varn

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Jez H -

You cannot go back a version and I would not start hacking around with those files either, I just mentioned them because those are the kinds of areas where headers are set.

By cloning I just meant make a second copy of your site (on a different URL) and build that up for testing.

An easier approach would be to do as you suggested and use a clean install then import a couple of your courses and see how that behaves.

If it were me I would probably try a clean install of the 2.5.2 official release (9th September) and see how that behaves. Then import a course from your current system into it, and if that works try upgrading that 2.5.2 to the same point release you have on your live system.

At least doing that will give you an idea of whether there is a problem with your environment, specific Moodle version or something within your courses, and perhaps narrow things down a bit.

Average of ratings: -
In reply to Jez H

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -

I may go the route of making a new site from scratch but unfortunately, due to other sites on the server, I need to stay at PHP 5.3.2 so I can't go and do a full upgrade to 2.5.2 official. I am working with the host still to see if there is anything they can do on a master level with Apache and PHP to address the http headers issue but, due to the client limitations, may still offload the files temporarily while I continue to troubleshoot.

If I do spin up a new site I will be sure to update this thread with any results so that others with this same issue will be able to know what was attempted/not attempted.

 

Average of ratings: -
In reply to Adam Varn

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Jez H -

If your host supports it you can define different PHP versions in .htaccess for different applications with something like:

AddHandler application/x-httpd-php53 .php

AddHandler application/x-httpd-php52 .php

So you can (if your host supports it) run Moodle in a different PHP version to other apps on the same host.

Average of ratings: -
In reply to Jez H

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -

Jez (or anyone following this issue),

I received this update from my hosting company who was looking into the MIME type issue:

" We have confirmed the file is loading out of the DB with the correct mimetype and PHP is reading the file as image/jpeg; however, the data being returned to the browser is text/html. Everything is loading correct until it reaches the function ( send_stored_file ) located in the file ( ./lib/filelib.php ). At some point when file.php is executing this function, the mimetype is changing and we are not able to figure out why. "

So it appears that the issue lies with Moodle itself. I am not sure if I should make an issue tracker issue for this or what. I'm currently in the process of trying to upgrade to Moodle 2.5 using the method described by Jez but still would like to figure out the overall issue if possible.

I've attached the output from what the host found in the query and the images, hopefully this might help someone who maybe has some PHP knowledge and could take a look? As before, any help that can be offered would be appreciated.

Average of ratings: -
In reply to Adam Varn

Re: Images no longer appearing/corrupted - Moodle 2.4.6

by Adam Varn -

Just as a general update for anyone stumbling on this thread:

The issues appeared to come from a security vulnerability in 2.4.6 that was exploited through old plugins that had not been removed.

I was not able to actually pinpoint the actual effect of what this injection attack did in relation to the images not appearing, even after doing fresh code installs and looking at the general functions of files. I ended up offloading all of the files that were not appearing to Dropbox and linking them directly that way, as the local files appeared to be corrupted...

...however, once I switched the files out, I began the process of moving the site to a new server (this was going to be done anyway) so that I could upgrade to 2.5. Once I moved the site over, the problems persisted BUT I then removed the old plugins (hotpot, lastevent and journal) and then upgraded the site to 2.5. 

After doing the upgrade, the site was functioning perfectly. Images appeared, the icons appeared (they had disappeared for some reason) and everything is working fine now. I do not know what was causing the HTTP headers to get interpreted differently still but the upgrade to Moodle solved the problem.

So my advice is to just do the upgrade if your server allows for it. That will likely solve your problems.

Average of ratings: -