Moodle in English: Restore Course from cli using admin/cli/restore.php

Hi,

I think if there is a basic course backup.php there should be a similar restore.php so you can be able to at least backup and restore files directly form the server without having to install other third party tools such as moosh.

I wrote one of my own trying to keep it simple and similar to the options provided by the admin/cli/backup.php

You are welcome to use it or make any thoughts/suggestions.
I 've made a git branch for it here.

I have tested it in version 3.8 but it should work with all others since it uses only moodle api's.

restore.php

Lazaros

restore.php

Prosjek ocjena:Useful (2)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - utorak, 10. prosinca 2019., 12:09

Hi Lazaros,

Is there any option to have the restore get the course fullname and shortname from the backup this would make automating mass restore much easier?
so ideally could specify
restore.php --categoryid=1 --source=backup1

thanks

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 10. prosinca 2019., 14:53

Hi,

No, I tried to keep it in the same philosofy as the backup.php which does not make mass actions but it is a nice idea although risky to restore entire folders.
I mean, what if courses exist? or multiple backups of the same course exist in the folder?

So if the "source" is a folder and a category is given instead of a courseid or shortname we could restore all backup files found in the directory to the given category using original names. If courses exist it will overwrite them

I will implement it as well and we can test it to see how it goes.

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - utorak, 10. prosinca 2019., 17:42

Hi,

I think if the course exists already then it should add a number to the end this would allow the batch to continue. Or as you mentioned it could just overwrite, for what we are looking we have folders with all courses backed up and restoring to a new site so it should not have duplicates

Thanks

Regards

Simon

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - utorak, 10. prosinca 2019., 19:11

Hi Lazaros,

If think it's a (very) good idea.

You should also create a tracker item, to have it included in core Moodle (once well tested).

Séverin

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 10. prosinca 2019., 19:37

Hi,

Thank you for the comments,

I have implemented what Simon asked and pushed it to my git branch CLI_RESTORE_MOODLE_38_STABLE.

As Simon suggested, if you pass a directory and a categoryid as an option then it mass restores all .mbz files found on the directory given to the category given, overwriting existing and creating new courses if they don't exist.

On the other hand if you pass a file as a source then you can either specify a courseid or a shortname to find and overwrite, or you can specify a categoryid to overwrite or create in that category with specific shortname (if also provided)

I haven't finished the tests but it seems working as expected. Ignore my previus attachment and get the one from my branch if you like to test it (use this link).

Yes I have to open a new issue as well.

Regards,

Lazaros

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 10. prosinca 2019., 20:09

Opened new feature issue MDL-67488

I have renamed my branch to MDL-67488-MOODLE_38_STABLE

Please find my latest version there

Prosjek ocjena:Useful (2)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 10. prosinca 2019., 23:21

I guess if we are restoring entire categories it would also be nice if backup.php could backup entire categoryid to directory.

Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - srijeda, 11. prosinca 2019., 05:19

Hi,

Just tested restoring with 3 sample courses, one is being created however the course is not fully restored, the other 2 were not created i have attached the errors

Also tested restoring the same 3 coursed individually with the new code and they look like they restored correctly

a lot of our courses have a course format called grid https://moodle.org/plugins/format_grid if the same courses are restored within the web interface the grid format restores correctly however from restore.php the format changes

thanks

regards,

Simon

error.log

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - srijeda, 11. prosinca 2019., 17:08

Hi,
Simon some of the errors are related to the backup file and we should investigate.
What troubles me is that the restore works in single file mode and not in directory mode.

It seems I forgot to add an important include file (which I now did).

Could you please try again with latest code and let me know?
Could you please tell me what version of moodle and php cli version you are using?
(php -v shows)

regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - srijeda, 11. prosinca 2019., 05:50

backup.php by categoryid to directory is a great idea

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - srijeda, 11. prosinca 2019., 17:17

Thanks Lazaros for the (quick) work on this

Yes, of course that could be very helpful to be able to bulk backup courses from a category. I think you should create a separate tracker item for that (and link it with MDL-67488).

About your restore feature, i also think it could be useful if one could ask, with an option, to only create non existing course (without trashing existing ones), or at the opposite, only reset and recreate existing courses.

Séverin

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - srijeda, 11. prosinca 2019., 19:15

Hi,

Looks like the only way to create a new course while restoring with specific shortname is using backup::TARGET_EXISTING_DELETING, while the TARGET_NEW_COURSE option always creates new courses adding numbers at the end in both full and short names (at least in latest master branch in which i tested). Correcting the target mode seems to solve the problem and create the courses correctly.

Fullname is always drawn from the backup file and no number is added at the end.

I think we don't want to make the script too complicated to specify everything. Besides we can't specify all restore options from cli.

The restoration uses default admin restore course options.
Severin,
If source is a file we have 2 modes:
1) We provide categoryid: A new course is created with specific shortname (if provided/otherwise the one from the backup file is used) or a course is overwriten if provided shortname (from backup or cli) exists.
2) We don't provide categoryid: An existing course is searched by shortname or id and is overwriten and an error is thrown if no course is found.

If source is a directory: (we have to provide categoryid)
1) Everything from that directory gets restored and gets created/overwriten using backup short and fullnames

To do what you suggest we have to add 2 more options (one or the other) lets say for ex --c (only create) or --o (only overwrite existing) with default option being to do both of them. I guess we are talking for source directory mode only.

I ll try to add that as well and see if we can test it.

Regards,
Lazaros

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - srijeda, 11. prosinca 2019., 20:13

Hi,
I have added options --reset and --create for only overwriting existing or only creating new courses when restoring. My tests show ok.
If none specified then it does both.

Please test if you can.
Regards,
Lazaros

PS: Simon I will have to look into grid_format to test and find out more. Please check the latest version though to see if something is diferrent.

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - četvrtak, 12. prosinca 2019., 06:20

Hi Lazaros,

I have tested the updated version with an individual course restores ok except grid format, restoring from folder did not restore any courses, I have attached log

thanks

error2.log

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - četvrtak, 12. prosinca 2019., 13:16

Hi,
Simon I need to know more to help.
1) The first error finds multiple categories with id 17 (???) so I don't know in which the restore attempt was made. Is this category empty and new? Could you try to create an new empty one and test again?
2) Did courses 7FR and 8FR already exist?
3) What moodle and php cli version was used?
4) Did you use the backup.php to create 7fr and 8fr backups?

we will deal with fixing grid after being sure category restore works

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - petak, 13. prosinca 2019., 10:23

Hi Lazaros,

1 - the category id is 17 and is now the only category on the site
2 - quite possible I have now removed all courses to retest
3 - Moodle 3.8+ (Build: 20191205) - PHP Version 7.2.11
5 - Yes they were backed up using backup.php from moodle site running Moodle 3.7.1+ (Build: 20190816)

after re-running the restore it looks like the 7FR restored fully
8FR was created and partially restored
the 3rd course did not restore

thanks

regards,

Simon

error3.log

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - subota, 14. prosinca 2019., 01:17

Hi,
I have updated my code to include checks for same names in same category and not just same short or full names.

I found a bug in the restore dbops class when TARGET_NEW_COURSE mode is used. If a new course restore is requested from backup the code checks if shortname or fullname already exists and adds the word 'copy #' and '_#' in the names. It does not check though if category id is the same and thus restores new names again and again when those exists in another category. I have to report an issue on this..

Simon the new code will probably create all (*.mbz) courses, but I don't think if it will manage to restore them completely. That has to do with the restore options used. It troubles me that as you say, single restore works (except grid format) because I use the same exact options in either directory or file mode restore.

Please try the new version and let me know on your findings. When restoring from UI do you choose new course mode or do you overwrite existing Course? Do both of these work for grid format?

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - ponedjeljak, 16. prosinca 2019., 05:37

Hi,

the new version has the same result, one restored ok the others not. When restoring from the UI i would pick new course and this will restore fine with the grid format.

this guy is having a similar issue
https://moodle.org/mod/forum/discuss.php?d=347777

thanks

regards
Simon

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - ponedjeljak, 16. prosinca 2019., 17:06

Hi,

Simon,
Thank you for pointing similar cases,
I am unable to reproduce the problem. I did some changes and there are more to come but the script works for my backups so I need help figuring out what happens in your case.

I made a debug version (download latest) that prints out more staff. It also restores in new course mode so it might work.
I need you to test and send the error log for further study.
I am sure we will figure this out

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 17. prosinca 2019., 01:08

Hi,

Simon,
I have installed grid_format in my site and have tested backing up and restoring in bulk a category with grid formated courses from folder.
I can't reproduce the problem. So either it's fixed or something else is wrong at your site.
Please let me know when you test this and send the error log.

I have also opened bug MDL-67531 which is related in a way. When restoring, if same fullname or shortname exists in any category the suffix " Copy #" and "_#" is added to the restored Courses even if the Restoration Category is empty which is wrong. It should check for dups only in the same category not all.

Regards,
Lazaros.

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - utorak, 17. prosinca 2019., 09:25

Hi Lazaros,

I have re-run the newest restore the first course has restored successfully with the grid format

the second is created and not fully restored

thanks

regards,
Simon

error4.log

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - utorak, 17. prosinca 2019., 10:07

Hi Lazaros,

I have just re-made the backup files with the include users option turned off and tested restoring - this restored all 3 courses with the grid format successfully - so it must have something to do with how is restores the user data from site to site both of the 2 servers are using ldap and have been fully synced. For the majority this will be fine as they will be reset at the end of the year however we will still have around 50 courses that need to include user data
thanks

regards,
Simon

error5.log

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 17. prosinca 2019., 15:31

Hi Simon,

Good to know the code worked. As I can verify too there are more problems with restoring course backups, not just from cli that have to be solved. Usually the errors occur when restoring quizzes or multiple answer questions, not just users and they appear in UI restorations as well.

In some of my backups (usually large courses) I am unable to restore backups and I get similar errors with temp tables hanging in the end. I think this is a general moodle problem that has to be addressed. Any moodle backup has to be able to restore otherwise we can't really rely
on the backup system. Right now I have courses that I am unable to restore from backups.

If as you say restore from UI works then we just have to see what restore options your UI uses. The only options that I specify in my restore script that could affect the process are TARGET_NEW_COURSE or TARGET_DELETE_EXISTING and MODE_GENERAL.

The first 2 are chosen if the course has to be created as new or overwrites an existing one.
The second one has to do with the way the controller restores data and can be any of MODE_[ GENERAL | HUB | IMPORT | SAMESITE].

We have to investigate differences and if one of those restores the courses completely without error then we might want to make an extra option for it on the restore script, so we can try different approaches.

I will also have to ask if it's possible for you to test single restore of one of the courses (with grid format), to see if overwrite works.

I will update the script with "--mode" option for you to test.

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 17. prosinca 2019., 16:50

Hi Simon,

You can now test different restore modes (--mode=general is default). Try samesite, import or hub and see if that works.

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - utorak, 17. prosinca 2019., 21:24

Hi,
Using the latest version I managed to restore Course backups that were not restoring even from UI using mode "import". I think though that it did not restore the users.

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - srijeda, 18. prosinca 2019., 05:33

Hi Lazaros,

I tested with and without the --mode options and all are failing
14 => 0,
)]
Error code: dmlwriteexception !!
!! Stack trace: * line 489 of /lib/dml/moodle_database.php: dml_write_exception thrown
* line 1331 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
* line 1377 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->insert_record_raw()
* line 87 of /backup/util/dbops/restore_controller_dbops.class.php: call to mysqli_native_moodle_database->insert_record()
* line 462 of /backup/controller/restore_controller.class.php: call to restore_controller_dbops::save_controller()
* line 213 of /backup/controller/restore_controller.class.php: call to restore_controller->save_controller()
* line 164 of /backup/controller/restore_controller.class.php: call to restore_controller->set_status()
* line 206 of /admin/cli/restore.php: call to restore_controller->__construct()
!!

regards,

Simon

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - srijeda, 18. prosinca 2019., 15:40

Hi Simon,

That's what I call progress lol.
This seems like a database error though and not script error probably.

Is your database working correctly?? Can you check space on your server?

Regards
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - četvrtak, 19. prosinca 2019., 04:51

Hi,

I have improved admin/cli/backup.php to backup entire category using --categoryid=number, or to backup course in specific filename with --destination=path/file.mbz

You can find the tracker here MDL-67553

backup.php is available on my repository here MDL-67553-master/admin/cli/backup.php

Regards,

Lazaros

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Simon Brock - četvrtak, 19. prosinca 2019., 10:54

Hi Lazaros,

the backup by category works great thanks, not too sure what happened with the restore errors yesterday but downloaded the latest one and looks to be restoring well, some of the ones with questionnaire are failing I will send you a message with a example course
thanks

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 20. prosinca 2019., 20:03

Dear Simon,

I have tested your backup in a fresh 3.8 moodle installation (after installing grid_format plugin as well) using my latest restore.php version and it checks out as is, either in NEW_COURSE or in EXISTING_DELETING target and mode general.

It is normally restored with signle error "Invalid sequence found in restored multianswer question 39" with grid format without any users (they were not included in the backup as I can see).

In my attempts one time when EXISTING_DELETING target was detected (because the course existed) it failed, but redoing the same exact operation succeeded. (" php admin/cli/restore.php --categoryid=3 --source=/tmp/foo/"). It seems to be a bug in the way the restore controller creates the questions. Runing same restore command over and over I can see that first fails, second succeeds, third fails, fourth succeeds etc. It only happens when overwriting existing course.

This happens because course contents are not entirerly emptied before restoring and create conflicts and that is why I changed restore code so that if it is overwriting a course it first removes all course content.

That worked. I uploaded the new version and is ready to test.

Regards,

Lazaros

"Default exception handler: error/error_qtype_match_subquestion_missing_in_db Debug:
Error code: error_qtype_match_subquestion_missing_in_db
$a contents: stdClass Object
(
    [id] => 165543
    [questiontext] => <p>Spotted and claimed the East Coast of Australia for the Crown and named it New South Wales</p>
    [questiontextformat] => 1
    [answertext] => Captain James Cook
)

* line 174 of /question/type/match/backup/moodle2/restore_qtype_match_plugin.class.php: restore_step_exception thrown
...

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - ponedjeljak, 23. prosinca 2019., 18:22

Hi, no corrections, I think it's done.
For anyone finally reading this, the issue tracker is at
https://tracker.moodle.org/browse/MDL-67488
and my repository for restore.php is at
https://github.com/lazarosfs/moodle/tree/MDL-67488-MOODLE_38_STABLE

Recursive backup.php is at

https://github.com/lazarosfs/moodle/blob/MDL-67553-master/admin/cli/backup.php

Regards,
Lazaros

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - četvrtak, 19. prosinca 2019., 17:34

Hi Lazaros,

Sorry for not coming back here more, i really lack time for that and for testing your developpments, but i really appreciate what you're doing.
I hope all this could be integrated for Moodle 3.9.

Just a question (or another suggestion) : when you backup a directory, do you include all subdirectories? Perhaps it could be an option (with or without subdirectories), to allow more use cases.

Keep up the good work,
Séverin

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - četvrtak, 19. prosinca 2019., 19:07

Dear Severin,

Thank you for your kind words. It would be nice for them to be integrated although I see no iintrest what so ever in the tracker.
On my opinion having a verified backup and restore procedure through cli is something that should have been implemented a lot of years ago because it automates tasks and provides easy sleeping at nights :p

PS: Simon I have received your backup and will test it and get back on you.

Your idea of backing up recursiveley is definetly great and I will implement it with an extra option.

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - četvrtak, 19. prosinca 2019., 19:24

Like you, i think this kind of things should have been in Moodle since years. But priorities are sometimes hard to follow/understand ; it's the "magic" (or problem) with open source communities and software.

But sometimes it just needs one man to take time to develop the thing to have it integrated some times after that.

I've wrote about this discussion and tracker items you created in some related tracker items ; hope it will bring some people, and help to have it included.

Thanks in advance for recursive backup

Regards,
Séverin

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 20. prosinca 2019., 03:19

Hi,

I have finished with the backup.php cli script. You can now use --r option to recursively backup all courses under the categoryid and subcategories.

Please test and verify.
Latest version is in MDL-67553-master/admin/cli/backup.php

Regards,

Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 20. prosinca 2019., 05:39

Hmm, on second thought I think the subcategory finding query in backup.php needs a bit of tweaking, I will have it fixed.

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 20. prosinca 2019., 06:13

It works on my tests.
Fedback appreciated

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - petak, 20. prosinca 2019., 16:52

Very nice to see your quick updates on this

Sorry won't have time to test...

But if you want to maximize chances to have it integrated in Moodle 3.9, you should follow instructions about new feature development process, and when everything tested, submit for peer review (and perhaps add a patch label).

HTH,
Séverin

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 20. prosinca 2019., 18:12

Hi Severin,

I did follow the guide and the branches are labeled and tested in the tracker but I don't have permission for peer review. I will add a patch label and see if anyone is interested in peer reviewing the code.

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Bruno Malaval - četvrtak, 9. siječnja 2020., 23:01

Hi,

I'm interesting by this feature and just test backup and restore with the following versions :

backup : https://github.com/lazarosfs/moodle/blob/MDL-67553-master/admin/cli/backup.php
restore : https://github.com/lazarosfs/moodle/blob/MDL-67488-MOODLE_38_STABLE/admin/cli/restore.php

I'm on the latest moodle 3.8.1, on a test server

Backup seems to work fine. I get a warning :
---

loading controller from db
PHP Warning:  max(): Array must contain at least one element in /var/www/moodle-3.8/course/format/lib.php on line 287

Warning: max(): Array must contain at least one element in /var/www/moodle-3.8/course/format/lib.php on line 287
setting controller status to 1000
-----

But nothing anormal, it's an empty course (no sections)

I have problem with the restore process
The category contains 12 courses. I backup the category and try to restore the courses in another category

Here's the commands I use :

sudo -u apache /opt/rh/rh-php72/root/bin/php backup.php --categoryid=449 --r --destination=/var/lib/moodle/backup_prod/

sudo -u apache /opt/rh/rh-php72/root/bin/php restore.php --categoryid=488 --create --source=/var/lib/moodle/backup_prod/

You can see the error in the attach file, it seems to be relative to questions of a quizz

Then, I try to restore the same course alone

sudo -u apache /opt/rh/rh-php72/root/bin/php restore.php --categoryid=488 --courseshortname=newmbox1 --source=/var/lib/moodle/backup_prod/sauvegarde-moodle2-course-1647-cours1-20200109-1523.mbz

And it works !!

In the meantime, Great job, thanks a lot

Bruno

error_restore_category.txt

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 10. siječnja 2020., 02:01

Hi Bruno!

Thanks for your kind words,
We seem to have a problem restoring some course quizzes for the first time.
This also happens in overwrite mode and I had to clear course contents first and then restore to make it work.

I guess this is what happens in your case and it seems logical that the first restore task probably tries to create a new course and restore the backup and that fails. The single restoration command finds an already existing half restored course in 488, and restores overwriting correctly.

To verify that this is the case I will need you to repeat the process exactly as you mention but also provide the last succesful log of the single restoration. Then we will figure out a way to overcome this in "new_course" mode as well.

It would also be nice if possibly if you could send me an example course .mbz that fails to make my own tests. (message me with a private link to download)

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 10. siječnja 2020., 14:02

I would also check to see if restoring to a new course using the ui succeeds.

The error indicates that some quiz restoration options were not given. That might also be possible due to not having defaults in the cli restore mode or not being able to fetch them to restore the quizz properly.

Either way we have to investigate further.
Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Bruno Malaval - petak, 10. siječnja 2020., 16:53

Hi,

I've made new tests this morning. By the way, I'm on Moodle 3.8+, not 3.8.1

Each time I try to restore courses, it's in an empty category. I remove all courses for each test.

I try :
Restore the course which causes problems alone
by your script => it's OK
by the UI => it's OK

Restore all courses from the category
by your script as mentioned in the post above => ERROR
by your script as mentioned in the post above, but with the option " --mode=import " => it's OK

Here's trace from the restore process with the import mode, for the problematic course

== Performing restore of "Cours MoodleBox" (cours1) to (cours1 with id 2440 in category 489) from file /var/lib/moodle/backup_prod/sauvegarde-moodle2-course-1647-cours1-20200110-0816.mbz (backupid is 77f1a0b0dfb45d82fc710d4e70483764 target is NEW_COURSE and mode is import ==
instantiating restore controller 50b8e42fd6c91fec85848ffe96437128
setting controller status to 100
loading backup info
loading controller plan
setting controller status to 300
applying restore defaults
Unknown setting: questionbank
setting controller status to 400
checking plan security
setting controller status to 600
saving controller to db
calculating controller checksum 85a2ca9ef7d8e313e1f543b05bcc7ea4
loading controller from db
setting controller status to 700
saving controller to db
calculating controller checksum 7223fd149ef4d1be9e714adf32a901df
loading controller from db
setting controller status to 800
saving controller to db
calculating controller checksum 6e89e899317aa003d6f496717b8263bb
loading controller from db
processing file aliases queue
setting controller status to 1000
saving controller to db
== finished. ==

I send you link to download the mbz file from this course by pm

thanks

Bruno

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 10. siječnja 2020., 18:27

Dear Bruno,

I downloaded your backup .mbz and tried everything I could but was unable to reproduce the problem on my test 3.8+ server.
Every test completed successfully in mode general and target either new_course or existing_deleting which is what the restore.php uses in either category or single restore mode.

Mode import restores without users so maybe that is where the problem is located.
Are the rest 11 courses restored?

What happens if you repeat the restore without deleting the courses?

Regards,
Lazaros

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 10. siječnja 2020., 20:03

PS: Hmm I will have to correct that, I managed to reproduce, debugging, will let you know.

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - petak, 10. siječnja 2020., 20:40

Hi,
Bruno I found the problem. The error is created when destroying the restore controller between sequential file restorations.
Removing "$rc->destroy();" fixes the issue.
I have pushed the new file to my repository. Please test and confirm.

Regards,
Lazaros!

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Bruno Malaval - petak, 10. siječnja 2020., 21:44

Hi,

I've just test with same files and confirm : it works !!

Thanks a lot.
Bruno

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - četvrtak, 6. veljače 2020., 19:04

Hi,

Just discovered an existing plugin which seems to make the same thing :
https://moodle.org/plugins/tool_brcli

But i really think it would be far better (and logical) to have it included in core

Séverin

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - četvrtak, 6. veljače 2020., 19:54

Hi,

I think that is great news, I hope I knew that before I put all this work in it.
It definately needs to integrate to moodle core but I still see no interest for either the plugin or my work to do so.

Thankfully my scripts work fine with all versions I have tested and I use them all the time so I hope people can find either mine or the plugin to do the job.

I have also indicated that there is a bug with the restore course naming. Moodle checks for same short/long names and adds _(number) to them before restoring. The thing is that it does not check for same category when counting.
So if you have a course named "test" in every category and try to restore a backup it will be created as test_14 for example even if it is the only one in it's category.

I have created a patch for that too at the tracker I hope sometime those all find their way to the core

Regards,
LS

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - četvrtak, 6. veljače 2020., 22:03

Hi Lazaros,

Yes, you've done great work for all these improvment, new feature and bug correction.

I know things can sometimes be (very) long to be officially corrected/included, and it can be discouraging.

I think you have to update status of your tracker items MDL-67553, MDL-67488 and MDL-67531 (and rebase code) so that they can be peer reviewed, before testing and integrating.

You should follow instructions to have them peer reviewed for bug correction and also for new feature. Keep up the good work and patience.

HTH,
Séverin

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - srijeda, 26. veljače 2020., 18:55

Hi Lazaros,

I've wrote in your different tracker items that they're waiting for peer review, hoping someone will have a look on them soon.

You should certainly also act on them, at least rebasing with latest code, and perhaps doing other things, following links provided in my previous messages.

HTH,
Séverin

Prosjek ocjena:Useful (1)

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Lazaros Sotiriadis - srijeda, 26. veljače 2020., 21:46

Hi,
Thank you for the ping Severin, I will try to rebase, I am out of time unfort..
Regards,
LS

Prosjek ocjena: -

Re: Restore Course from cli using admin/cli/restore.php

napisao/la Séverin TERRIER - petak, 24. travnja 2020., 22:55

Hi Lazaros,

I hope you're fine, with the actual Covid-19 crisis.

Just a reminder that your 3 tracker items MDL-67553, MDL-67488 and MDL-67531 are in the list of issues waiting for peer review. I suppose that, if you could rebase on latest code, that would ease the remaining process, and let better chances to have it included in Moodle 3.9. No new changes should be accepted for Moodle 3.9 after 4th of may, as stated here.

Hope you'll find a little time for that.

Regards,
Séverin

Prosjek ocjena: -