Environment
Moodle version 3.11.15+ (downloaded today) Windows 2022 (I know I know I know). IIS 10. PHP 7.4.29. MS SQL.
When I tried to restore a course from a server file, I get as far as choosing a category. I would be stuck at choosing categry and noticed that the list of course restore in progress grow in the 'Restore into existing courses' section, as I retry the continue button on the 'Restore to a new course' section. Looks like everytime I click the button, two Restore In Progress were spawned. PHP error logs say Context ID not found, no matter which category I chose.
Followed this link https://docs.moodle.org/402/en/How_to_rebuild_context_paths
DBCC on MS SQL found no error.
The context_cleanup_task rebuild was run but no effect. The array of every query seems to be empty... whatever it means.
The database was restored from a copy backed up right after moodle 3.11.15+ update a weekago. The codebase was downloaded today and database updated accordingly (to get all latest built). All plugins status resumed to the status as of a week ago. No plugin errors on PHP.
Any suggestion would be really appreciated.
I don't understand what we are trying to do ...
"The database
was restored from a copy backed up right after moodle 3.11.15+ update a
weekago. The codebase was downloaded today and database updated
accordingly (to get all latest built)."
'a weekago' could mean already behind! Key is build date when doing things your way.
Problems (unknown) with above process, would probably lead to the issues you are seeing now with courses.
Migrating to a dev instance, other server, or similar, requires code, DB dump restored, and minimal moodledata/filedir/ The code versions match what's in the restored DB dump to a new DB (change to config.php).
The 3.11.15+
says:
"Built Sporadically
1 day 7 hours ago"
Hmmmm ... would have to look at the change log *differences* to see if there wasn't something related to the errors you are seeing now.
'SoS', Ken
Hi,
Sorry it took so long to circle back. I went through a process of eliminations. Here is the current situation with course restore problem.
Environment: Windows server 2022, IIS 10, PHP 7.4.29 Moodle version updated from to 3.11.15+ (as per original post) and to 3.11.16+. I wasn't sure if I did anything wrong with my git process so here's how I update this instance.
- I delete all content from moodle directory after backing it up
- git clone from moodle
- set branch tracking to MOODLE_311_STABLE
- copy back web.config (for IIS), config.php and all missing plugins from backup
- run upgrade.php
The site successfully upgraded to 3.11.16+. All the plugins are there. Login fine.
Then attempt to restore a course into a specific category
php restore_backup.php --file=m:/mdata/moodledata03/repository/StandardCourse/coursename.mbz --categoryid=181 --showdebugging
It failed with the following error.
== Extracting backup file to: M:\mdata\moodledata03/temp\backup\restore_64f0e6fd3f18a ==
== Preprocessing backup file ==
== Cleaning temp data ==
!!! Exception - error/restore_check_course_not_exists !!!
!!
Error code: generalexceptionmessage !!
!! Stack trace: * line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
Developer error log is turned on in Moodle. PHP error log the following.
[31-Aug-2023 12:15:17 America/Los_Angeles] Default exception handler: Exception - error/restore_check_course_not_exists Debug:
Error code: generalexceptionmessage
* line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
[31-Aug-2023 12:16:14 America/Los_Angeles] Default exception handler: Exception - error/restore_check_course_not_exists Debug:
Error code: generalexceptionmessage
* line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
I went to the specific category 2 Course restoration in progress copy 1 and copy 2 appeared. In the moodledata\temp\backup there is a folder with content in it. Only one.
Then I tried to restore via UI. I managed as far as choosing Destination. Everytime I chose a destination and click Continue, the page simply refresh with 2 additional Course Restoration in progress copies in the Restore into an existing course section on the same page. And more php error in log file.
[31-Aug-2023 12:21:31 America/Los_Angeles] PHP Notice: Context id 390881 does not have valid path, please use context_helper::build_all_paths()
Sorry it took so long to circle back. I went through a process of eliminations. Here is the current situation with course restore problem.
Environment: Windows server 2022, IIS 10, PHP 7.4.29 Moodle version updated from to 3.11.15+ (as per original post) and to 3.11.16+. I wasn't sure if I did anything wrong with my git process so here's how I update this instance.
- I delete all content from moodle directory after backing it up
- git clone from moodle
- set branch tracking to MOODLE_311_STABLE
- copy back web.config (for IIS), config.php and all missing plugins from backup
- run upgrade.php
The site successfully upgraded to 3.11.16+. All the plugins are there. Login fine.
Then attempt to restore a course into a specific category
php restore_backup.php --file=m:/mdata/moodledata03/repository/StandardCourse/coursename.mbz --categoryid=181 --showdebugging
It failed with the following error.
== Extracting backup file to: M:\mdata\moodledata03/temp\backup\restore_64f0e6fd3f18a ==
== Preprocessing backup file ==
== Cleaning temp data ==
!!! Exception - error/restore_check_course_not_exists !!!
!!
Error code: generalexceptionmessage !!
!! Stack trace: * line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
Developer error log is turned on in Moodle. PHP error log the following.
[31-Aug-2023 12:15:17 America/Los_Angeles] Default exception handler: Exception - error/restore_check_course_not_exists Debug:
Error code: generalexceptionmessage
* line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
[31-Aug-2023 12:16:14 America/Los_Angeles] Default exception handler: Exception - error/restore_check_course_not_exists Debug:
Error code: generalexceptionmessage
* line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
I went to the specific category 2 Course restoration in progress copy 1 and copy 2 appeared. In the moodledata\temp\backup there is a folder with content in it. Only one.
Then I tried to restore via UI. I managed as far as choosing Destination. Everytime I chose a destination and click Continue, the page simply refresh with 2 additional Course Restoration in progress copies in the Restore into an existing course section on the same page. And more php error in log file.
[31-Aug-2023 12:21:31 America/Los_Angeles] PHP Notice: Context id 390881 does not have valid path, please use context_helper::build_all_paths()
- line 516 of \lib\accesslib.php: call to debugging()
- line 697 of \course\classes\category.php: call to has_capability()
- line 1190 of \course\classes\category.php: call to core_course_category::can_view_course_info()
- line 1840 of \course\classes\category.php: call to core_course_category::get_course_records()
- line 1890 of \course\classes\category.php: call to core_course_category->get_courses()
- line 293 of \course\classes\management_renderer.php: call to core_course_category->get_courses_count()
- line 310 of \course\classes\management_renderer.php: call to core_course_management_renderer->category_listitem()
- line 161 of \course\classes\management_renderer.php: call to core_course_management_renderer->category_listitem()
- line 501 of \course\management.php: call to core_course_management_renderer->category_listi in C:\inetpub\wwwroot\s-moodle\lib\weblib.php on line 3284
[31-Aug-2023 12:21:31 America/Los_Angeles] PHP Notice: Context id 390880 does not have valid path, please use context_helper::build_all_paths()- line 516 of \lib\accesslib.php: call to debugging()
- line 697 of \course\classes\category.php: call to has_capability()
- line 1190 of \course\classes\category.php: call to core_course_category::can_view_course_info()
- line 1840 of \course\classes\category.php: call to core_course_category::get_course_records()
- line 1890 of \course\classes\category.php: call to core_course_category->get_courses()
- line 293 of \course\classes\management_renderer.php: call to core_course_category->get_courses_count()
- line 310 of \course\classes\management_renderer.php: call to core_course_management_renderer->category_listitem()
- line 161 of \course\classes\management_renderer.php: call to core_course_management_renderer->category_listitem()
- line 501 of \course\management.php: call to core_course_management_renderer->category_listi in C:\inetpub\wwwroot\s-moodle\lib\weblib.php on line 3284
During the time, I updated a different instance to Moodle 4.0 from the the same set of code and I was able to restore the same course. Their server environment and PHP version are the same. I can't update this one to 4.0 yet so I need to fix it. As of now I am trying to update all additional plugins, which I am not sure is helpful but I can't think of anything else to try.
Any suggestion?
It's been many many years since I used Windows and command line ...
I wonder about this:
php restore_backup.php --file=m:/mdata/moodledata03/repository/StandardCourse/coursename.mbz --categoryid=181 --showdebugging
If one issued just php restore_backup.php it is supposed to show the help for that command since no options were provided. Did that help show linux 'slashes'?
How would php on Windows interpret the path to the backup.mbz file if the slashes are forward leaning (like /) rather then what is native to Windows backward leaning (like \)?
First line of debug shows confusion to me:
== Extracting backup file to: M:\mdata\moodledata03/temp\backup\restore_64f0e6fd3f18a
I see a / in front of temp which is to be a directory. All the slashes there are correct for Windows except for that one.
Does/did a directory restore_64f0e6fd3f18a get created in moodledata\temp\backup\?
'SoS', Ken
in config.php, I have this
$CFG->dataroot = 'M:\\mdata\\moodledata03';
no trailing / or \.
The help of restore_backup.php shows /. But I had tried \ as well. I tried windows \ for the filepath with no effect.
PHP error logged the following
[01-Sep-2023 07:54:53 America/Los_Angeles] PHP Notice: Context id 390892 does not have valid path, please use context_helper::build_all_paths()
$CFG->dataroot = 'M:\\mdata\\moodledata03';
no trailing / or \.
The help of restore_backup.php shows /. But I had tried \ as well. I tried windows \ for the filepath with no effect.
PHP error logged the following
[01-Sep-2023 07:54:53 America/Los_Angeles] PHP Notice: Context id 390892 does not have valid path, please use context_helper::build_all_paths()
- line 516 of \lib\accesslib.php: call to debugging()
- line 697 of \course\classes\category.php: call to has_capability()
- line 1190 of \course\classes\category.php: call to core_course_category::can_view_course_info()
- line 1840 of \course\classes\category.php: call to core_course_category::get_course_records()
- line 1890 of \course\classes\category.php: call to core_course_category->get_courses()
- line 1649 of \course\renderer.php: call to core_course_category->get_courses_count()
- line 1522 of \course\renderer.php: call to core_course_renderer->coursecat_category()
- line 1562 of \course\renderer.php: call to core_course_renderer->coursecat_subcategories()
- line 1623 of \course\renderer.php: call to core_course_renderer->coursecat_category_content()
- line 1522 of \course\renderer.php: call to core_course_ren in C:\inetpub\wwwroot\s-moodle\lib\weblib.php on line 3284
[01-Sep-2023 08:05:57 America/Los_Angeles] Default exception handler: Exception - error/restore_check_course_not_exists Debug:
Error code: generalexceptionmessage
* line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
On screen it displays this
== Extracting backup file to: M:\mdata\moodledata03/temp\backup\restore_64f1fdd40db5b ==
== Preprocessing backup file ==
== Cleaning temp data ==
!!! Exception - error/restore_check_course_not_exists !!!
!!
Error code: generalexceptionmessage !!
!! Stack trace: * line 497 of \lib\setuplib.php: moodle_exception thrown
* line 105 of \admin\cli\restore_backup.php: call to print_error()
!!
There were other directories created but none with that name.
This may be a stupid question... I saw a lot of warnings about LF to be replaced by CRLF when I did GIT ADD (or GIT COMMIT). Could it be where the problem is?
"This may be a stupid question..."
No such thing when it comes to Moodle! Any platform!
Now I don't do Windows ... but here is something I'll share that aftected the running of task from command line. Help showed the raw ... due to the slashes in the task to run, bash shell environment messed that up. I finally put the specific task enclosed in quotes (not smart quotes) to be able to run them from CLI.
Wonder if you did the same with the path to the .mbz ... enclose that path in "'s. Treats it as a literal ... ie, do not parse!
"I saw a lot of warnings about LF to be replaced by CRLF"
Well, that is a diff between Windows and Linux shells so in your case, they were just warnings and really haven't messed up anything when it comes to files acquired by git.
If you use git to acquire to a test directory moodle code see what it says with that. You'll need an editor that can show machine language so you can see what's hidden at the end of lines in files. Dunno of an editor for Windows that does that.
'SoS', Ken
Notepad++ is a Windows editor that will let you see what is at the end of each line. View > Show Symbol > Show End of Line
If it shows a bunch of CRLF, then do Ctrl+f, click on replace. in the Find what: type \r\n then in Replace with: type \n
You can then click, Replace, and do them one at a time, or, like I do, Replace All.
Remember, if you mess it up you can always do a Ctrl+z and put everything back the way it was.
If it shows a bunch of CRLF, then do Ctrl+f, click on replace. in the Find what: type \r\n then in Replace with: type \n
You can then click, Replace, and do them one at a time, or, like I do, Replace All.
Remember, if you mess it up you can always do a Ctrl+z and put everything back the way it was.
I use Sublime Text but I don't customize the code. I simply used it for config file and ini file editing. After my post, I researched more about LF and CRLF on Moodle on Windows to find no straight forward answer... I tried "" the path but no effect. And you won't believe what happen next.
I traced my note on all my changes. I compared the php.ini of the 2 moodle instances in question and noticed a difference in session.save_path setting. I didn't document that change somehow. I changed the one that doesn't work from "c:\windows\temp" to "/tmp" and tested again. VOILA ! It went through! Both GUI and CLI version of restore_backup.php worked now!
I traced my note on all my changes. I compared the php.ini of the 2 moodle instances in question and noticed a difference in session.save_path setting. I didn't document that change somehow. I changed the one that doesn't work from "c:\windows\temp" to "/tmp" and tested again. VOILA ! It went through! Both GUI and CLI version of restore_backup.php worked now!
Weird! ... but Congrats!
A tip ... maybe ...
in code/admin/cli/ there is a cfg.php script - which shows all settings for the moodle.
You might be able to redirect to. text file for your server docs:
cd code/admin/cli/
php cfg.php > allsettings.txt
Then access via browser:
https://site/admin/cli/allsettings.txt
to acquire.
Once acquired delete the .txt file.
'SoS', Ken
thanks Ken. I used cfg.php to find and correct some incorrect settings during this Moodle update. It was useful.
Now that I am calmed down from the excitment of resolving the problem, there were a few other things I did today including enabling fastcgi logging, increasing Rapid Fail Maximum from 5 to 25, ensuring PHP Manager on IIS and Handler Mapping are consistence. All pertain to website, not CLI so I think they are not so relevant.
The thing that may be more relevant is folder permission. There was an error in permission 02777 on config.php. It was missing a digit so I corrected it. But it doesn't resolve the issue even after i rebooted the Windows server. Out of gut-feeling, I went to remove all IUSR permission from all data folder leaving just IIS_IUSRS with Full Control in there and rebooted the server. As I expected wwwroot became 'unwritable'. I ensure IUSR was in IIS_IUSRS group but effective permission check on the folder shown that IUSR has no permission. I then add IUSR back to the data folder with Full Control permission and rebooted Windows again. Restore was still failing. These were all done about 2 hours prior to the php.ini config changes.
I am not sure if it takes time for permission to propagate on the server? It doesn't seem to make sense. I have checked again just now to see that effective permission for IUSR for the data folder is now Full Control.
Now that I am calmed down from the excitment of resolving the problem, there were a few other things I did today including enabling fastcgi logging, increasing Rapid Fail Maximum from 5 to 25, ensuring PHP Manager on IIS and Handler Mapping are consistence. All pertain to website, not CLI so I think they are not so relevant.
The thing that may be more relevant is folder permission. There was an error in permission 02777 on config.php. It was missing a digit so I corrected it. But it doesn't resolve the issue even after i rebooted the Windows server. Out of gut-feeling, I went to remove all IUSR permission from all data folder leaving just IIS_IUSRS with Full Control in there and rebooted the server. As I expected wwwroot became 'unwritable'. I ensure IUSR was in IIS_IUSRS group but effective permission check on the folder shown that IUSR has no permission. I then add IUSR back to the data folder with Full Control permission and rebooted Windows again. Restore was still failing. These were all done about 2 hours prior to the php.ini config changes.
I am not sure if it takes time for permission to propagate on the server? It doesn't seem to make sense. I have checked again just now to see that effective permission for IUSR for the data folder is now Full Control.
Have a sandbox server with multiple versions of Moodle - 3.9.highet to 4.1.highest.
All linux severs have same ...
Am ujsing Apache for all - document root /var/www/html/
/var/www/ however is defined as apacher user's home directory - for all practical
purposes ... and that's where moodledata directories go.
./config.php:$CFG->directorypermissions = 02777;
./config.php:$CFG->directorypermissions = 02777;
./config.php:$CFG->directorypermissions = 02777;
./config.php:$CFG->directorypermissions = 02777;
In config-dist.php
// 4. DATA FILES PERMISSIONS
//=========================================================================
// The following parameter sets the permissions of new directories
// created by Moodle within the data directory. The format is in
// octal format (as used by the Unix utility chmod, for example).
// The default is usually OK, but you may want to change it to 0750
// if you are concerned about world-access to the files (you will need
// to make sure the web server process (eg Apache) can access the files.
// NOTE: the prefixed 0 is important, and don't use quotes.
$CFG->directorypermissions = 02777;
But the note above doesn't explain why the prefixed 0 is important.
Maybe:
https://superuser.com/questions/382827/what-does-directory-permission-02777-do
This octal digit is made up of setuid, setgid and sticky bits, all of which have comparatively complex effects.
The 2 sets the setgid flag. When present on a directory, all files created within it will inherit its group, instead of the creating user's primary group.
Suid, guid and sticky bit
But, all that is Linux ... have no idea how that translates to Windows OS and !USER etc.!
Don't think I've ever see a chart that compares Linux to Windows on this topic!
Have an issue of my own .... have a google bucket mounted on all servers.
In *past versions of moodle*, had been able to add a symlink in moodledata/repostory/ for the Google bucket mount point and Moodle would accept that setup for a file system repo.
No longer!!! GRRRRR! A Google Bucket is cheaper than any local data disk.
Of course, in this case, don't really know if it's moodle or the Google end ... Google isn't standing still and recently have noticed OS updates for gsfuse.
I can still use a directory in the Google Bucket for CLI site backups as well as course backups via Moodle php/moosh scripts!!!!!?????? But not in the GUI! ... MORE GRRRRRR!
Pardon the 'piggy back' and not trying to 'hijack' your thread for your problem!!!!!
Just needing to vent!!!! :|
'SoS', Ken
Didn't read the whole thread. But do you really have a /tmp directory in a Windows machine?