General help

 
 
Picture of Doug Moody
Puzzling Graphics and data issue
Group Particularly helpful Moodlers

I recently had to move a moodle site from one hosting service to another (never mind the reasons, I will talk about that in another post)

Anyway, the old site is no longer accessible, but I did get a tar file of the entire site.

Since the tar file was so large, and I didn't want to include all the old files from my moved site, I chose to upgrade to the newest moodle using the zip file from moodle downloads. I pointed the new install to the old SQL database, and everything worked fine. I was able to see my site, and it appeared to come across fine.

However, some things are not quite right.

First, all the user pics are missing.

Secondly, all the course theme graphics, such as system icons, etc., are missing.

So, since I have all the old data, I need to know what directories to restore to my new host. I can put them manually into the correct directories, or if I need to create directories, I can do that too.

But can someone tell me where these graphic reside, and all the folders I need to restore manually should go? That will be a lot quicker than trying to restor an entire site.

Oh, and one more thing I noticed...My backup files show up in the restore dialogs, but I get php errors when clicking on them. Apparently moodle is lost when it comes to knowing where certain files ought to be, or it can't find them because they aren't yet on my new server. So its all related, but I just need to know what folders to restore and/or any utilities I should run to "sync" up the new locations.

Thanks in advance.

 
Average of ratings: -
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

I found the answer to my own question, and will post here for future questions...

You need to go to the moodledata folder and look for the folder in there called "filedir" It is likely a large folder, but replace the one that installs from a new install with the one from the old moodle site. Make sure that the one you put there corresponds to the database you have linked to this new m oodle site, or you run the chance of corrupting your database and site.

 
Average of ratings: -
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

I spoke too soon. Now my images are gone again.

There must be something askew in the whole thing. Can someone educate me on how images are shown in moodle? Where are they stored and where do they come from when being displayed?

 
Average of ratings: -
Picture of Ken Task
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Edit one of the missing images.  It should show the full URL of where Moodle thinks it should be located.  If you've moved a site, that URL to the image might show http://oldsite/moodle/whatever.  There is a search and replace utility (I know, some say don't use it, but it's better than editing 100's of images individually). One just has to be careful using it ... search for FULL URL http://oldsite/ replace with http://newsite/

Notice above uses 'http://' in front and ends with a '/'.  DO NOT USE just the hostname or an IP.  That will replace errantly.

Such an easy question ... where are the files stored?  You are in the right location ... filesdir ... but the filenames are no longer in humanly recognizable format ... ie, english.png but rather just a uniquely generated hash consisting of letters and numbers.

A png graphic linked to somewhere in a site might *BE* the following in filedir:

./fc/00/fc004a95109124037a443d360596bf40068edea7
PNG image data, 128 x 128, 8-bit/color RGBA, non-interlaced

In moodledata/filedir/fc/00/ folder there is a file by the hashname of

fc004a95109124037a443d360596bf40068edea7

That file is a png graphic.  Where did the name go?  It's now in the DB - table: mdl_files

select * from `mdl_files` where `filename` like "%.png%"

or better yet ... or more specific (if you have only the contenthash name and not the 'humanly recognizable name'):

select * from `mdl_files` where `contenthash` like "fc004a95109124037a443d360596bf40068edea7"

That will show:

course section 67 / cat-icon.png

 

 

It's an image in a course section called 'cat-icon.png'.

Neat huh? sad

So when Moodle generates that image in content, it has to search the DB for the file name and build an img src link in HTML so your browser gets it.  It will use the config.php file to find some of that URL.  If, however, an image that is already supposed to be in content, it might find the full img src URL in the DB ... errant that it might be, that's what Moodle builds and attempts to send to your browser.  You might see a 'broken image' box.

When one moves a moodle site one *MUST* move the DB AND the datadirectory.  Shouldn't install a fresh Moodle with new DB and datadirectory then replace the entire datadirectory with old datadirectory.  It is, however, ok to use a freshly downloaded code directory (moodle) that's close to the version you had.  Use the moved DB and moved data directory.  All that might do is upgrade a point version - ie, you had 2.2.5 and the freshly downloaded code directory is version 2.2.7+ (minor upgrade).

Think you need to review what you did and how to sort it out.

'spirit of sharing', Ken

 

 
Average of ratings:Useful (1)
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Ken,

I wish I could "LIKE" answers from moodlers here, because yours would definitely be the MOST liked!

Your writing style is clear, and you explain why things do what they do. I ahve searched for a long time for this kind of info, and you summed it up nicely in just a few short paragraphs.

You said that there is a search and replace utility. Can you point me to it please?

I'm glad you are here. Thanks so much!

 
Average of ratings: -
Picture of Ken Task
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Thanks for your kind comment ... reponse must have been a quirk as I am normally too long winded/'techie' (to some), etc..

You can "like' ... or at least rate the posting useful.  Please do so.

There is no link to replace in the Site Admin menu.  Login as admin level user and then using the URL line change to go to:

http://site/admin/tool/replace/

Simple form with warnings.  Again: Search for: http://oldsite/ Replace with http://newsite/

include the http:// in front and the trailing slash to prevent errant replacements.

'spirit of sharing', Ken

 
Average of ratings: -
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Ken,

Well, I tried all the tricks  you recommended. I got a little confused when y ou said to view the previous URL of my old site. The old URL is the same as my new one. The only thing that changed was my hosting provider.

I copied the URL of images that I saw (not sure where you got your image info, but I right clicked on theimage and copied the image URL. Here is a sample of what I saw:

 

http://mydomain.com/moodle/pluginfile.php?file=%2F13%2Fuser%2Ficon%2Fbinarius%2Ff1&rev=6393

Of course, this doesn't look like the sample you provided. Not sure what "pluginfile.php" means. Do you?

I am still not getting pictures. I am wondering if a restore of an old course (but only the pictures) that had pictures in it wuold help?

 

 
Average of ratings: -
Picture of Ken Task
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Didn't understand you kept the same FQDN and just moved to a new provider. You probably did say that and I didn't pick up on it.

Still though, moving the domain name to another provider does require a change to DNS I would think.  Since your server is to be accessed by students and it won't be behind a firewall with very restricted access, think it's safe enough to use your true domain name/site URL.  Others might disagree, but don't think we're talking that much of a security risk if the new domain name is shared here.  We could have a DNS issue or a routing issue (it does happen, even with remotely hosted sites) and providing at least the FQDN of the host one could do some 'digging' to see what's out there in DNS land for your system.

So - lets review a little ... you did a fresh install of a Moodle (within the same series number).  Did the site do a minor point upgrade? (ie, from x.x.2 to x.x.6?)  Then copied the filedir directory from the old site to the fresh install.  And used the DB from the old site in the new site.  Images don't show.  And user icons (that have been uploaded by users) don't show.

If that is correct, sounds 'logical' but a 'risky' way of doing things ... but it's done now.  So ... did *all* of the old filedir transfer?  Are ownerships and permissons of the copied filedir set correctly for new system?

You are 100% on target that the DB and the contents of the filedir must match.

Pluginfile.php I haven't pursued, but it looks related to user icons - not images uploaded to content.  In the example provided, it was an image used in content. Were users allowed to upload their own pics/icons?  Obviously, Moodle doesn't store images in an images folder of any sort.  But users do have ID numbers and would think their images/pics would be mapped to that ID ... somehow/someway.

Do you have anyway of checking the status of tables for corruption? (phpmyadmin?).  Moving this data around over wires can sometimes pickup errant characters ... not enough to crash something, but just enough to confuse.

You might have to take a step back.  Wipe out what you have but only IF you have the code directory, the original data directory, and an sql dump of the DB.  That tar ball you have could be un-tarred and then re-zipped ... one zip for the code, one zip for the data directory.  One could then upload the smaller zips.  Just make sure when you zip you do so recursively to get all of the moodle code directory and all of the data directory.  Probably wise to test un-zipping to see if you get what is desired when un-zipping on server.

Then, rather than attempting upgrading while moving at the same time, just get your old site up and running - where everything works.  Then look into upgrading without 'work-arounds' but following recommended process from Moodle.org:

http://docs.moodle.org/24/en/Moodle_migration

http://docs.moodle.org/22/en/Upgrading

Yes, version numbers are different above, but most of the time the process is the same.

'spirit of sharing', Ken

 
Average of ratings: -
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Ken,

I copied the link from some other graphics, and this is what I am seeing. Does this look more like it ought to look? If so, then what would you advise should be my next step?

http://mysite.com/moodle/user/view.php?id=859&course=11

 

http://mysite.com/moodle/user/view.php?id=712&course=11

 
Average of ratings: -
Picture of Ken Task
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Not having migrated a site in the same fashion as you have described and not having that much curosity to replicate such a site to see what got broke and how to fix it, I can only offer the following:

When looking at a user profile in a 2.4.1+ the image URL shows:
http://site/moodle24/user/profile.php?id=5

But, for a user that uploaded an image for their profile, getting image location in firefox shows:

http://site/moodle24/pluginfile.php/25/user/icon/boxxie/f1?rev=74

Know the image names are changed (like in 1.9) to f1.png, f2.png etc. but can only guess as to the ?rev=74 ... theme?  or revision of image?

The DB, mdl_files table does have 'component' with 'user' as the data and 'filearea' set to 'icon' for a user that uploaded an image.  There is also, in the same table, a column for 'user id'.

select * from `mdl_files` where (`component` like "user" and `filearea` like "icon")

There's all your contenthashes - thus where the images are located. 

However, I uploaded a .gif file for a user pic.  In the table is shows the icon file name to be f3.png and the mimetype, of course, image/png.  Moodle is converting them.

Did check out uploading user pictures in 2.4?
From the 'help/description':
Upload user pictures

User pictures can be uploaded as a zip file of image files. The image files should be named chosen-user-attribute.extension, for example user1234.jpg for a user with username user1234.

As far as links to other images through out the site, one will have to research the mdl_files table directly, I would think.  Remember that 'id=##' is found in many url's in Moodle.  I see course=11 which is easy enough to understand, but view.php?id=## represents what?  Inspection of /user/view.php file might give a clue.

So maybe, your best approach is to use 'Upload user pictures' IF that's the only issue you have with the migration and images.  However, think you did say that some other images/links to files were missing.   Suggesting one could be looking at the  tip of the iceburg here ... and if not using something provided in core (like upload user pictures) one will either get really good a manipulating the site via MySQL or have plenty of practice at mysqldumps and restores of DB's.  While very educational in pursuit, something I'd care not to do on a production site.

When something 'mostly works', it makes me un-easy.  Know you'd really not like hearing it again, but ... it might be in your best interest to start the migration all over again.  Get the site migrated and functioning first, then do the upgrades.

'spirit of sharing', Ken

 
Average of ratings: -
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Ken,

I will work on this on the weekend and report back how I get it resolved. But it probably will require a complete rebuild, as you suggest.

But an interesting phenomenon has happened. I use various browsers to test my site, and my favorite is Firefox. But, at the beginning of this whole thing, I get the pictures to show - once...

Anyway, Firefox was the browser at the time, and somehow Firefox has cached the .png thumbnails of the pictures (but no other browsers have persisted their caching, just Firefox). So I just richt clicked on one of the thumbnails and saved the png graphic to view it. Its size is 35 x 35 pixels. Of course not enough to save.

But I thought it was interesting that when I click on the studen'ts personal profile, there is no graphic that is linked, even in the cache. The only thing I have is the small png thumbnail.

That explains why I don't have full size images.

So, does that thumbnail come from the database, or from the moodledata folder? I suspect it came from the database. If that's the case, then I have a shot at recovering the images that live in the moodledata folder. But if those png images came from moodledata, then maybe that's all I have.

I also agree with you that when there are "abnormal" behaviors in m oodle, then it might be an indicator that things are not working properly elsewhere. That concerns me. When I have time, I might just rebuild the entire site. But that will have to wait till the end of the year when students are gone.

 
Average of ratings: -
Sketch...
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

And vote for MDL-35099 so that the Replace function gets fixed.

 
Average of ratings: -
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

Done! wink

 
Average of ratings: -
Picture of Doug Moody
Re: Puzzling Graphics and data issue
Group Particularly helpful Moodlers

I was able to retrieve my data. It took a lot of work, and the help I received from Ken Task was awesome. Thanks Ken. I am (mostly) back on track. As a matter of fact, I rebuilt the site completely from odds and ends. It was getting a little rough around the edges, so this presented me with an opportunity to tune it up.

I guess we all need a crisis from time to time to improve ;)

 
Average of ratings: -