Backup and restore

 
 
Mary Cooch
Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

I am trying to restore a 1.9.18 course into a brand new 2.4 Moodle . I have turned on debugging and am attaching the message I get (which  relates to "invalid file path" Could anyone advise as to what I need to tell our network manager so he can fix it? Thanks

 
Average of ratings: -
moi!!! it is what is is...
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Particularly helpful Moodlers

Hi Mary, I am no great IIS expert, but I have always thought the file path was supposed to not be so long. But that might not be it, rather, does the "advent" folder exist in the backup? While a file might exist in the file list, was it actually included in the backed up data? Can you look inside the zip file? That is usually the first thing I do when a backup does not restore, and it might yield a clue about the actual folder construct used. Cheers.   

 
Average of ratings: -
Mary Cooch
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Yes I must admit I thought it looked odd too. I made that course and I recall there being a lot of flvs in there but you're right;  I will open it up and see if it has anything dodgy inside it!

 
Average of ratings: -
Picture of Ken Task
Re: Pls help understanding Restore debugging message
Group Particularly helpful Moodlers

Might be a 'bug' in PHP itself on that platform.  Seems to me I saw something in a forum here that said same.

Error code: file_invalid_path
$a contents: C:\inetpub\moodledata/temp/backup/4e1d322cb5ad4016a3bed113ad8cd734/course_files/adventflv\day01.flv

See the last '\' in the path to the file ... right before the file 'day01.flv'?

All the others after moodledata lean the other way ... '/' which is the way Linux or Mac uses directory delimiters.

IF it's not a 'bug' in php on that platform for which there is a fix (or if there is no fix yet), the only thing I could suggest is to edit the xml files contained in the un-zipped backup and manually change the directory designators (the last one).  That's a pain, I know, but ....

'spirit of sharing', Ken

 
Average of ratings: -
Mary Cooch
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

OK I just tried to restore another course and this is the first line of what I got:

Debug info:
Error code: file_invalid_path
$a contents: C:\inetpub\moodledata/temp/backup/9e320bfec3ef29d3ebed790f1a872123/course_files\codebreakers.gif

and I see again that the last \ before one of the files inside the course is the "wrong" way round. So you might be onto something, Ken. The only problem is - I don't know whatsad

 
Average of ratings: -
Don
Re: Pls help understanding Restore debugging message
Group Testers
I would not worry about the slash direction. I've seen before in Moodle. I think the problem has to do with the file system , repository you have set up. Your 1.9.x is looking for /moodledata/(course number) . Your 2.4 needs legacy file system set up as a repository before restoring 1.9 course. I think wink
 
Average of ratings: -
Mary Cooch
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Hmm.. it would seem  not. I just tried this and I am still getting the error message. I'm sure I've restored 1.9 courses into 2 on other sites without needing legacy course files enabled but, despite being enabled I still get the same error on this site. Well ... perseverance wins in the endsmile

 
Average of ratings: -
Don
Re: Pls help understanding Restore debugging message
Group Testers
It was worth a shot.
 
Average of ratings: -
Don
Re: Pls help understanding Restore debugging message
Group Testers
There's a tracker issue open. You should add to the discussion. https://tracker.moodle.org/browse/MDL-37879
 
Average of ratings:Useful (1)
Mary Cooch
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Don - very useful thanks!

 
Average of ratings: -
moi!!! it is what is is...
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Particularly helpful Moodlers

Sorry to butt in here, but being fairly stodgy and preferring to do things the harder way, what happens if you restore the course to an earlier version? Is it possible there has been a small coding change that could be causing an issue with the fairly substantial leap from v1.9 that was not present in, say, a v2.2 or v2.3?

 
Average of ratings: -
Mary Cooch
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Colin -et al - thanks to Helen I now see there is a current problem with restoring 1.9 courses with files pointing to course files - /MDL-37774 In the meantime , while waiting for the fix. I saw in the tracker Don linked to  here MDL-37879 someone has added a workaround - it says (quote) 

in the file of ...../moodle/backup/converter/moodle1/lib.php
at the line of 1216
// BACKUP RESTORE PROBLEM OCCURED BY THIS SECTION!!!
if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH))

{ throw new moodle1_convert_exception('file_invalid_path', $sourcefull$ }

/// END OF ERROR SECTION!!!
If you comment these 3 lines, backup restoration from 1.9 to 2.4 is completed successfully.

On my local host Moodle (not our school Moodle!) I thought I would try this but I don't know what "comment" means! I know where the file is but what am I meant to do? I simply replaced those lines with the lines above, but that just gave me a new error message I had never seen  before, so I must be doing something wrong.

 
Average of ratings: -
Walking on the snow towards Lago Nero...
Re: Pls help understanding Restore debugging message
Group DevelopersGroup Particularly helpful Moodlers

Hi Mary,
you need to comment out the sentence as per PHP rules. You can use /* */ or // at the very start of each line, e.g.:

        // PARAM_PATH must not be used on full OS path!
/*
        if ($sourcepath !== clean_param($sourcepath, PARAM_PATH)) {
            throw new moodle1_convert_exception('file_invalid_path', $sourcepath);
        }
*/

HTH,
Matteo

 
Average of ratings:Useful (1)
Mary Cooch
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Ah thanks- I will try this later

 
Average of ratings: -
moi!!! it is what is is...
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Particularly helpful Moodlers

Hi Mary, I have been tied down with huge gobs of administrivia, so sorry i have not been here, how did you get on with this?

 
Average of ratings: -
Picture of Ken Task
Re: Pls help understanding Restore debugging message
Group Particularly helpful Moodlers

When restoring a 1.9.x backup, the files will automagically come in as legacy (at least, that has been the case with versions 2.3.x and below). 

When Mdl2 sees a zip backup, it states it will attempt to convert the .zip backup to an .mbz (a Mdl2 compat backup) before restoring.  It does this in moodledata/temp/backup/

On Windows a '/' is a legal character in a file name, normally. A '\' is not a legal character in a file name as that is Windows delimiter for folders/directories.  However, with UTF8 character set the following applies:

Naming Conventions

The following fundamental rules enable applications to create and process valid names for files and directories, regardless of the file system:

  • Use a period to separate the base file name from the extension in the name of a directory or file.
  • Use a backslash (\) to separate the components of a path. The backslash divides the file name from the path to it, and one directory name from another directory name in a path. You cannot use a backslash in the name for the actual file or directory because it is a reserved character that separates the names into components.
  • Use a backslash as required as part of volume names, for example, the "C:\" in "C:\path\file" or the "\\server\share" in "\\server\share\path\file" for Universal Naming Convention (UNC) names. For more information about UNC names, see the Maximum Path Length Limitation section.
  • Do not assume case sensitivity. For example, consider the names OSCAR, Oscar, and oscar to be the same, even though some file systems (such as a POSIX-compliant file system) may consider them as different. Note that NTFS supports POSIX semantics for case sensitivity but this is not the default behavior. For more information, see CreateFile.
  • Volume designators (drive letters) are similarly case-insensitive. For example, "D:\" and "d:\" refer to the same volume.
  • Use any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:

    • The following reserved characters:

      • < (less than)
      • > (greater than)
      • : (colon)
      • " (double quote)
      • / (forward slash)
      • \ (backslash)
      • | (vertical bar or pipe)
      • ? (question mark)
      * (asterisk)

http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx

What actually controls the restore process are the xml files created when converting the course into 2 format.

If you can, take a gander at: moodledata/temp/backup/ and see if there is a directory with the hash '9e320bfec3ef29d3ebed790f1a872123'.  There is probably a files.xml file that checks the 'course_files' folder also found in the '9e32....' folder.

Here's what one entry in a files.xml looks like:

  <file id="13">
    <contenthash>dee7a75c740150ee236ec6e0df7661b8d00ad862</contenthash>
    <contextid>180</contextid>
    <component>course</component>
    <filearea>section</filearea>
    <itemid>10</itemid>
    <filepath>/</filepath>
    <filename>audio-headset.png</filename>
    <userid>3</userid>
    <filesize>7624</filesize>
    <mimetype>image/png</mimetype>
    <status>0</status>
    <timecreated>1303376359</timecreated>
    <timemodified>1303376359</timemodified>
    <source>$@NULL@$</source>
    <author>Name of author here</author>
    <license>cc-sa</license>
    <sortorder>0</sortorder>
  </file>

These two lines:

    <filepath>/</filepath>
    <filename>audio-headset.png</filename>

determine the 'path' and the filename.

If the backup is coming from a Linux/MacOSX server and one is restoring to a Windows platform, the restore php scripts/routine must convert those directory designators or know what to do with them or not do with them.

You might have discovered a 'feature'! :|

'spirit of sharing', Ken

 
Average of ratings:Useful (3)
Mary Cooch
Re: Pls help understanding Restore debugging message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Wow thanks Ken - I will read that in great detail when I am awake (early morning here) But one thing I do know is that is IS a 1.9 course from a Linux server and is being restored to a Windows platform..

 
Average of ratings: -
Mary Cooch
How safe is this workaround?
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

The one earlier in this thread and which I will paste below (thanks Matteo). Our network manager is not keen to use unofficial patches but we are still unable to restore 1.9 courses to 2.4 and are struggling a bit. Could anyone reassure him this won't break our Moodle? (worked fine on my local host but my local host isn't going to house 1000 userswink)

Hi Mary,
you need to comment out the sentence as per PHP rules. You can use /* */ or // at the very start of each line, e.g.:

        // PARAM_PATH must not be used on full OS path!
/*
        if ($sourcepath !== clean_param($sourcepath, PARAM_PATH)) {
            throw new moodle1_convert_exception('file_invalid_path', $sourcepath);
        }
*/

HTH,
Matteo

 
Average of ratings: -
Walking on the snow towards Lago Nero...
Re: How safe is this workaround?
Group DevelopersGroup Particularly helpful Moodlers

Hi Mary,
AFAIK it breaks nothing if you restore backups into a Windows server from a known source i.e. a known Moodle instance: it would be nice not to have a mixture of \ and / when restoring a backup coming from a Linux hosted Moodle, time to re-ping the Tracker?

IMHO while waiting for the official fix - which will probably found the root for those \/ -  the safest option is to clone the production site and test the import there, the wiser is to create a new backup from your local Moodle instance where the course backup has been successfully imported: maybe using a Windows->Windows backup will not trigger that issue.

HTH,
Matteo

Edited @ 2013-02-26 14.48 CEST to cleanup the last statement (not mother tongue wink)

 
Average of ratings: -
Mary Cooch
Re: How safe is this workaround?
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Thanks Matteo! Unfortunately we can';t clone the production site because we don't have access to the database (without paying a lot of money -long story) so we are restoring the courses .onto a clean 2.4 - hence the problem. But I will pass on the message.

 
Average of ratings: -
Picture of Emma Richardson
Re: How safe is this workaround?
Group Particularly helpful Moodlers

Can you update the original site and then export and import to the new site?  I always found that the easiest way to get courses over the 1.9 to 2.? hurdles.  If not, I would be tempted to import to a 1.9 version, update and move forward that way.

 
Average of ratings: -
Picture of Benjamin St.Germain
Re: How safe is this workaround?
 

After applying the work-around, we have imported 14 courses with no problem, and plan to import dozens more this way.  There really is no other solution at this point besides to wait.  All that function does is "clean" the path to fix the alternating slashes and some other cleanup, but the "unclean" path works fine for the import process to access course files.  If there really is a problem with the file, an error will probably the thrown somewhere else in the restore process when it tries to actually put that file into the new class.

 
Average of ratings: -
Mary Cooch
Please help with this restore error message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Our network manager applied the workaround over the weekend but I am getting this error when I try to restore a course - does anyone have an idea? Is it a case of not copying/pasting correctly or something else?

Parse error:syntax error, unexpected T_IF, expecting T_FUNCTION in C:\inetpub\wwwroot\moodle24\backup\converter\moodle1\lib.php on line 1219

 
Average of ratings: -
Picture of Benjamin St.Germain
Re: Please help with this restore error message
 

Sounds like typo in the attempt to comment out the function.  You can do a multi-line comment as shown above, or individual line comments with two forward slashes:

        //if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH)) {

        //    throw new moodle1_convert_exception('file_invalid_path', $sourcefullpath);

        //}

 
Average of ratings: -
Mary Cooch
Re: Please help with this restore error message
Group Documentation writersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup TestersGroup Translators

Just to add to the thread  -this did fix it and he can now restore courses, so thanks all.

 
Average of ratings: -