Some resource URLs not correct after upgrading/migrating

Some resource URLs not correct after upgrading/migrating

by Matt Rice -
Number of replies: 8

A little bit of background info: It's been my job to migrate my school's Moodle from 1.9.9 to 2.5.3 (via 1.9.19 then 2.2.10 then to 2.5.3). Part of the issue is that I have had to set up a duplicate server (with the content from our current 1.9.9 site) so that the faculty can see how their content looks in the upgraded version (while not messing with the current server, on which classes are still running). In setting up this new server and migrating over the old content, some URLs are pointing to the wrong place (e.g. they are pointing to the old moodle URL "moodle.myschooldomain.edu/" instead of the new URL "upgradedmoodle.myschooldomain.edu/").

They way I see it, I have several interconnected problems:

  1. The URLs are pointing to the wrong place. This seems like it would be easy to fix (possibly with the Search & Replace admin tool), but is complicated by...
  2. I don't know if the resources to which the URLs point were migrated as well and
  3. If the resources were migrated to the new file scheme, I don't know to which resource to point the appropriate link

For #1: I believe I can get a list of incorrect URLs (e.g. a grep across the mysqldump of my upgraded db for "moodle.myschooldomain.edu")

For #3: The incorrectly referenced files still exist in my (current version) moodledata, so I can get at them.

It's entirely possible that I missed something during my rush to migrate/upgrade; if I've missed something, I would be glad to hear about it (and how to correct it!)

What I would like is a way to fix the links to point the correct resource. I would obviously prefer a script to run, but if I can get pointed in the right direction on how to migrate these things manually, I feel confident I could write a script myself.

As mentioned previously, the site in question is running 2.5.3.

Average of ratings: -
In reply to Matt Rice

Re: Some resource URLs not correct after upgrading/migrating

by Matt Rice -

After some more digging, it looks like I might be able to use (/backup/converter/moodle1/lib.php) moodle1_file_manager.migrate_file() to solve both 2&3, which would just leave #1 (which doesn't overly difficult).

 

Can anybody verify that I'm heading in the right direction with this?

Average of ratings: -
In reply to Matt Rice

Re: Some resource URLs not correct after upgrading/migrating

by Ken Task -
Picture of Particularly helpful Moodlers

Can't say I've taken the same approach to a migration, but from what you've described, think you might run into the same issues in the future IF both sites use the same DB.   The wwwroot variable in config of both sites gets inserted at the beginning of URL resources in the Moodle.  Teacher updates something in the sandbox moodle it will be mis-linked in production.   Suggest that one might need only the search and replace tool without the other more guru-ish 'stuff' - assumes your sandbox was a clone of the production and that teachers didn't help compound the problem by deleting a course/restoring a course, adding new content in sandbox. 

Suggest that one might need to consider keeping the 'sandbox' moodle completely separate from production moodle - different DB, different data directory, same version of code however.   Teachers do have the power to backup their production course and restore to sandbox where they can play without fear of loosing anything critical in producton server.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Some resource URLs not correct after upgrading/migrating

by Matt Rice -

Thank you for the advice, Ken.

I might not have been as clear as I could have been when I stated my situation: we currently have a site running 1.9.9 that is being used for classes. What I did was take a snapshot of that site (db, moodledata, etc), transferred that snapshot to a self-contained server, and upgraded the snapshot (to 1.9.19, then 2.2.10, then 2.5.3). 

The issue I'm seeing is manifesting itself on the upgraded snapshot; after upgrading, there are links back to the current (external, 1.9.9) site, which there shouldn't be. I have verified that the resources that are incorrectly linked from the upgraded snapshot do still exist in the moodledata of the un-upgraded source Moodle (the 1.9.9 version).

Average of ratings: -
In reply to Matt Rice

Re: Some resource URLs not correct after upgrading/migrating

by Ken Task -
Picture of Particularly helpful Moodlers

All moodles take the value of $CFG-wwwroot and inserts it before links to images/resources on the server (like files) and records those in the DB.

Original server: http://somemoodle.somenet.net

$CFG-wwwroot=http://somemoodle.somenet.net;

When one exports the DB links will have http://somemoodle.somenet.net in them.

Sandbox server: http://###.###.###.##/

$CFG-wwwroot=http://###.###.###.##/;

Importing the SQL dump from the DB of somemoodle.somenet.net intoi the sandbox server DB will include links that use the wwwroot variable http://somemoodle.somenet.net

Running search and replace will clean that up.  However, any HTML block will still have errant links as they convert URL's to some other format behind the scenes.   Search and Replace tool cannot convert those.  So HTML blocks you will have to manually edit/relink IF you desire them to work on the Sandbox server.

If the issue is HTML blocks ... may not be a reason to edit them all on the Sandbox server.  Could simply remove the HTML blocks on the Sandbox.   It's purpose is for a Teachers 'playground' anyway, right?   No one should be exporting from Sandbox to import to production, correct?

If I'm not understanding the problem, maybe some screen shots with info would help.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Some resource URLs not correct after upgrading/migrating

by Matt Rice -

I think we're on the same page, Ken: if I had done the Search & Replace on the upgrade server before I had upgraded, I probably would be ok, but because I didn't I have a mess to clean up. Likewise, if I had upgraded in place, I would have been ok because the $CFG-wwwroot variable would have been correct when the upgrade process was porting resources.

 

The crux of the issue I have now is that I need to recode the links on the upgraded Moodle to not point to the 1.9.9 instance (and instead point to resources on the upgraded server, and those resources might not exist because Moodle was treating them as external when the upgrade happened).

 

As to whether or not the upgraded server I have is a sandbox or not is more of an internal political issue, so I can't really ask for your assistance on that one.

Average of ratings: -
In reply to Matt Rice

Re: Some resource URLs not correct after upgrading/migrating

by Ken Task -
Picture of Particularly helpful Moodlers

I think I understand ... the upgrade to 2.2.x from 1.9.19+ step would have been the place to do search and replace and then check the entire site for functioning of such links.  How about legacy files?    That turned on?

Seems to me that regardless of how one did the 'Moodle March' from 1.9.x to 2.x + and then higher, the new file system set all files to 'legacy'.  Only way to 'avoid legacy' was to stop at 2.2.x and do the backup/restore to new/self there at that version for every course (not possible for some due to number of courses).

'spirit of sharing', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Some resource URLs not correct after upgrading/migrating

by Matt Rice -

Thanks for the advice, Ken. Just to be clear, are you suggesting that I try Search/Replace after upgrading to 1.9.19 (from 1.9.9) and before upgrading to 2.2.x (from 1.9.19)?

Average of ratings: -
In reply to Matt Rice

Re: Some resource URLs not correct after upgrading/migrating

by Matt Rice -

Just reporting back that using Search/Replace BEFORE the upgrade to 2.2 appears to solve this issue to my satisfaction. Thanks, Ken!

Average of ratings: -