Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Dominique Palumbo -
Number of replies: 7
Picture of Plugin developers
Hi,

We have a lock problem with Moodle 4.1.5+ on Debian\apache\phpfpm 7.4x on a cluster. Mostly with memcache, we don't go to redis for now, we disable the MUC and we've less lock in the UI. But we still have some lock in the UI and also everyday in the CRON (mostly on send notification).
The lock factory is now by default on the filesystem. Is it better to put it on the DB ? We use postgressql.

The error are : A lock was created but not released
but also
Adhoc task failed: core_course\task\course_delete_modules,Coding error detected, it must be fixed by a programmer: The course module 156651 could not be deleted. Unable to acquire a lock for caching.: /var/www/moodlecode/lib/modinfolib.php(644) #0 /var/www/moodlecode/lib/modinfolib.php(2817): course_modinfo::build_course_cache()
#1 /var/www/moodlecode/course/lib.php(973): rebuild_course_cache()
#2 /var/www/moodlecode/course/classes/task/course_delete_modules.php(66): course_delete_module()
#3 /var/www/moodlecode/lib/cronlib.php(367): core_course\task\course_delete_modules->execute()
#4 /var/www/moodlecode/lib/cronlib.php(198): cron_run_inner_adhoc_task()
#5 /var/www/moodlecode/lib/cronlib.php(76): cron_run_adhoc_tasks()
#6 /var/www/moodlecode/admin/cli/cron.php(178): cron_run()
#7 {main}
Backtrace:
* line 367 of /lib/cronlib.php: call to core_course\task\course_delete_modules->execute()
* line 198 of /lib/cronlib.php: call to cron_run_inner_adhoc_task()
* line 76 of /lib/cronlib.php: call to cron_run_adhoc_tasks()
* line 178 of /admin/cli/cron.php: call to cron_run()
In french sorry
Impossible d'enclencher un verrouillage pour le cache => Unable to engage a lock for the cache
Adhoc task failed: mod_forum\task\send_user_notifications,Impossible d'enclencher un verrouillage pour le cache.

Any idea and help will be greatly appreciated.

Thanks in advance.

Have a nice day.

Dominique.
Average of ratings: -
In reply to Dominique Palumbo

Re: Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
The advantage of putting the locks in the db is that it's much easier to see what the locks are doing and/or manually clear them if something goes wrong and you need to.
Average of ratings: Useful (1)
In reply to Howard Miller

Re: Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Dominique Palumbo -
Picture of Plugin developers
Hi Howard,

Thank you for the advice. I'll discuss of it with my colleagues.
The locks where usually clear by the system because no tasks, adhoc tasks or users get stuck forever, no clear all caches was neccessary for unlocking, for now.

I had the feeling that it can be the speed of the filesystem shared on the cluster that is too slow. But it's just a feeling.

Do you know if the lock->release() is asynchronous ? If it's synchrone my feeling is wrong...

Have a nice day and thank you for your answer.

Dominique.

Average of ratings: -
In reply to Dominique Palumbo

Re: Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
I'm not sure I understand your question. A process "asks for" a lock. If it's already locked it exits. Otherwise, it runs and then clears the lock.

The problem comes when the process fails and doesn't clear the lock. Locks have a lifetime but the default is 24 hours...
Average of ratings: -
In reply to Dominique Palumbo

Re: Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Mark Johnson -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

In this case, you're dealing with the Cache Lock API rather than the general Lock API. Rebuilding the coursemodinfo cache requires a lock to prevent concurrent rebuilds. Since MDL-67020 cache locks are stored in the same location as the cache, to ensure that caches like coursemodinfo can be stored locally. However, cachestore_memcached doesn't support locks itself. If I remember correctly it therefore falls back to using the cachelock_file plugin. You are probably best off setting coursemodinfo cache to use cachestore_file or cachestore_redis since they support locking natively.

Average of ratings: Useful (4)
In reply to Mark Johnson

Re: Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Dominique Palumbo -
Picture of Plugin developers
Hi,

Howard, I think Mark have explain better my problem than me : )

Mark, I'll discuss to install redis instead of memcache. And I hope it will solve the problem (but I don't think I'll get it quickly).
I'll read your post (and others) in the tracker to better understand the mechanism and the problem.

If I've right understand what I've read for now, is it possible than the filesystem we use doesn't have a native lock support or the lock is desactivated ? And it can be the source of the problem ?

Thank you Mark and Howard for your time and answers it really help.


Have a nice day.

Dominique.
Average of ratings: -
In reply to Dominique Palumbo

Re: Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Dominique Palumbo -
Picture of Plugin developers
Hi,

We also think that can be the speed of the filesystem (NFS) that cause sometime the trouble. If we find something, I'll let you know.

Dominique.
Average of ratings: -
In reply to Mark Johnson

Re: Error in Moodle 4.1.5+ in the UI and cron : Unable to acquire a lock for caching

by Dominique Palumbo -
Picture of Plugin developers
Hi,

After going to cachestore_file , as i said, I still get locks errors. But, I think one or more of the clusters still use memcache (I don't know why exactly, I configure it from the UI and reset all caches on all clusters). But I continue to get errors. I'm sure I had create a worst situation when some clusters use memcache and other filesystem for caching... After cleaning all caches on all clusters, temp directories, local etc, revalidate than the memcache is not used anymore. I stop to get lock error on cron.
I know my description was not clear, but I really do not understood what append and thing about strange origins of the problem.

I just want to add, when you stop to use memcache for the MUC. Validate TWICE on the memcache that no cluster still use it.

Thanks all for your help.

Have a nice day.

Dominique.
Average of ratings: -