Images broken after site migration

Images broken after site migration

by Robert Meany -
Number of replies: 25

Hi,

This is my first time working with moodle, and I'm relatively noob with Linux/Apache/MySQL/PHP, so bear with me...

I was tasked w/ migrating our moodle site off of Site5 hosting onto a couple of local servers.  We are splitting our database and web roles due to a moderate volume of concurrent users.  So far I have managed to get everything working OK EXCEPT images.  All image content on the site (including theme-based images for the interface) load up with placeholders. 

Earlier, I had encountered an issue loading ANY content in the moodledata directory and was /hoping/ that would resolve everything.. But apparently not.  

The issue I had earlier was that I had uploaded the moodledata folder with Filezilla without specifying binary mode.  This caused it to auto-to ascii mode which corrupted the files in that folder.  After specifying binary and re-uploading, user uploaded documents will now download or display correctly in the browser.

 

I'll just lay out the process I went through to get moodle on the new servers.

I am running the latest LTS of Ubuntu Server on both machines.  MySQL is installed on one machine and Apache/PHP is installed on the other.  

The Moodle I am migrating is at version 2.5.1 (Build: 20130708)

I had originally installed a fresh moodle (latest version) to verify that, in its basic form, it was working (it was).  When I started the migration, I brought the fresh server down (by stopping apache) and removed the moodle and moodledata directories.  I did not touch the mysql database.

Our hosting service had placed the moodledata folder at ~/moodledata and the moodle web files folder at ~/www/moodle and pathed the config.php file appropriately

For the migration, I opted to have the new server respond to the same http header as our current publicly hosted Site5 server at moodle.eosmith.org  ... Eventually, the new server will take that DNS entry anyway.  I edited the hosts file on my PC to point to the IP of the internal server instead of the Site5 server so I could test it.  This way I can avoid having to run the replace.php script to update the url prefixes in the database.

- used Filezilla on my PC to FTP into the Site5 server and download the moodle web and moodledata folders.

- also FTPed down the dump of the MySQL database.

- uploaded the MySQL dump to the database server and the moodle web and moodledata folders.  

- imported the dump to the MySQL database with no errors.

- placed the moodle folder on the new server at /usr/share/moodle

- ran "find /usr/share/moodle /exec chown root {} \;" to set root as the owner of all files and directories under and including moodle.  did the same with chgrp.

- ran "find /usr/share/moodle -type d chmod 755 {} \;" to set permissions on all directories under and including /usr/share/moodle

- ran "find /usr/share/moodle -type f chmod 644 {} \;" to set permissions on all files under /usr/share/moodle

- did the above examples but with chown/chgrp www-data  and chmod 700/600 for directories/files in /var/lib/moodle

- modified apache httpd.conf and virtual directory configuration to respond to http://moodle.eosmith.org

- modified the paths appropriately in /usr/share/moodle/config.php

As I mentioned, content, links and styles all work correctly.  It's just images that won't display. I copied a PNG file directly into the root directory and opened it on the site to verify that it wasn't an apache configuration issue with regards to loading images... Directly accessing the .PNG file works fine.  I also verified that the filehash entry in the mdl_files table in the database was correct and pointing to a non-corrupt copy of the file in /var/lib/moodle/filedir.  The pathing on the URL appeared to be what the pluginfile.php file was looking for.

I'll attach my apache, php and moodle configuration files.

I would greatly appreciate any advice on this... Thanks in advance.

 

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Robert Meany -

I should also mention that I tried testing upload of a new image to see if that worked.  it didn't.. so it's not just limited to images that I attempted to migrate.

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Ken Task -
Picture of Particularly helpful Moodlers

Did you check apache error logs ... default location, if I re-call correctly, var/log/apache2/error.log

Might also check the access.log file as well.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Images broken after site migration

by Robert Meany -

The access.log file has an entry for the request but there is nothing in the error.log about it.  

Access log example

10.1.5.200 - - [12/Dec/2013:21:52:04 -0500] "GET /pluginfile.php?file=/1877/course/section/223/RamenMoode.png HTTP/1.1" 304 271 "http://moodle.eosmith.org/course/view.php?id=19" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Robert Meany -

Hmm... if I actually download the 'blank' image link, I get a PNG file.  It's not zero-sized.. In fact it's exactly the same size as the actual image file should be.  Which is leading me to believe that somehow the file is getting corrupted when it gets retrieved..

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Robert Meany -

... And I just verified that particular file in filelib itself is not corrupted by looking up its hash value and copying it out/renaming it with a .png extension.

 

.. So it's definitely something in the process of retrieving the file using the moodle libraries that is causing the corruption...

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Ken Task -
Picture of Particularly helpful Moodlers

Have you run MySQL queries for the images from the web server or from the DB server itself?

Might install FireBug extension for Firefox and see what it says.

Have anything like APC installed?

Have you tried purging caches in Moodle (either via Moodle UI or via command line)?

'spirit of shairng', Ken

Average of ratings: -
In reply to Ken Task

Re: Images broken after site migration

by Robert Meany -

I installed firefox and the firebug extension.  The only difference here is if I attempt to open one of the images directly

(specifically, I went to moodle.eosmith.org/pluginfile.php?file=/1877/course/section/223/RamenMoode.png)

I get the following message on the page: 

The Image "moodle.eosmith.org/pluginfile.php?file=/1877/course/section/223/RamenMoode.png" cannot be displayed because it contains errors."

 

I am not familiar with APC.  I have a pretty vanilla install of Ubuntu server though, with only the necessary packages to run moodle.  I'll double check and see what packages are installed on the server...

 

I'm not viewing the page from either the web server or the db- I'm viewing it from a client PC.  The web server and db are both CLI only, so I couldn't do that.   However, the web server is doing all the work and is only accessing the db for url -> file translation, which I did verify was working correctly.

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Have you done this with Debugging on?

Have you tried with 'slasharguments' set to 'No'?

Average of ratings: -
In reply to Howard Miller

Re: Images broken after site migration

by Robert Meany -

Slash arguments are set to 'No' and debugging is turned on.

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

This...

The Image "moodle.eosmith.org/pluginfile.php?file=/1877/course/section/223/RamenMoode.png" cannot be displayed because it contains errors."

says that slasharguments is 'on'. Unless you changed the setting after that of course wink

 

Plus - you should get much more than the 'contains errors' message either on screen or in your logs (you've looked in your web server error log?). Is debugging right up to Developer and the option to display the errors switched on?

Average of ratings: -
In reply to Howard Miller

Re: Images broken after site migration

by Robert Meany -

If I have slasharguments on, it shows the url like this: "moodle.eosmith.org/pluginfile.php/1877/course/section/223/RamenMoode.png"


With slasharguments off, it shows like: "moodle.eosmith.org/pluginfile.php?file=/1877/course/section/223/RamenMoode.png" 

Average of ratings: -
In reply to Howard Miller

Re: Images broken after site migration

by Robert Meany -

Yeah, debugging is turned all the way up to developer.  Only thing in the apache logs is that it successfully returned the file requested at the above mentioned url.

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Robert Meany -

Also, I have PHP set to log and display errors on the page when they occur (which I verified when I was messing with it and creating my own PHP files which did have errors in them)

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Wrong log... error log not access log

However, oddly, this works...

http://moodle.eosmith.org/theme/image.php/magazine/core/1382556608/help

What exactly are these files that are not displaying? If they are files uploaded into your course content, are they still there?

Average of ratings: -
In reply to Howard Miller

Re: Images broken after site migration

by Robert Meany -

You're looking at our production server which is not the one we're migrating to.

Average of ratings: -
In reply to Ken Task

Re: Images broken after site migration

by Robert Meany -

Oh, and yes, I purged caches.  Also made sure that cron.php was run.

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Ken Task -
Picture of Particularly helpful Moodlers

Can verify that png images are being served from the Moodle code folder ... accessing http://site/pix/help.png displays in browser.  That, however, is direct and does not involve php/moodle code.

What does:

php -i |grep 'PNG'

show?

'spirit of sharing', Ken

 

 

Average of ratings: -
In reply to Ken Task

Re: Images broken after site migration

by Robert Meany -
Yeah, I can open PNG files directly from the moodle web folder ( I actually manually took a hashed file from the library and renamed it with the .png extension, put it in the moodle web folder and opened it in the web browser.. it worked)
Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Mauno Korpelainen -

You should not have php end tag at the bottom of site config.php in moodle 2.X.

If you check that file the last comment says

// MAKE SURE WHEN YOU EDIT THIS FILE THAT THERE ARE NO SPACES, BLANK LINES,
// RETURNS, OR ANYTHING ELSE AFTER THE TWO CHARACTERS ON THE NEXT LINE.

and you have after that
?>
( and one empty line )

This was ok in moodle 1.X but moodle 2 config.php says

// There is no php closing tag in this file,
// it is intentional because it prevents trailing whitespace problems!


There is also one extra line before the first

<?php

Clean them away with some simple text editor and your site should run ok again.

Average of ratings: -
In reply to Mauno Korpelainen

Re: Images broken after site migration

by Robert Meany -

There weren't any trailing whitespaces, but I did get rid of the end ?> and it didn't seem to make any difference.

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Mauno Korpelainen -

Try to purge caches after config.php change - extra characters or wrong file encoding in config.php are a common reason for corrupted files in moodle. Are you sure that there is no extra line at the beginning - my text editor finds one in that zip package file...

Average of ratings: -
In reply to Mauno Korpelainen

Re: Images broken after site migration

by Robert Meany -

There was an extra line at the beginning and I did get rid of it.

Average of ratings: -
In reply to Robert Meany

Re: Images broken after site migration

by Robert Meany -

Well.. The issue seems to be fixed now... I want to say I have a definitive solution to the problem, but I can't say for sure what exactly did fix it.

 

But I strongly suspect the following:  

Even though I setup the hosts file on my PC to point to the internal IP address of our test server when browsing files at http://moodle.eosmith.org , I had not thought to change the hosts file on the web server to point to itself when referencing that address.  So, I believe that if any PHP files on the web server were attempting to reference data by web url, that it may have been attemping to download that data from our hosted live production server, which is where http://moodle.eosmith.org is currently registered to on DNS.

The tip about purging caches after making configuration changes was very helpful as well.  The change above did not immediately make any difference, but things seemed to correct themselves after doing that..  Which is also why I cannot say for sure that it was the hosts file that was the issue as I was messing around with settings in config.php at the same time.

The hosts file issue does stand to reason though because no one else seems to have reported this issue, and I'm doubtful most people use that method for testing a migration.

Average of ratings: -