פרסומים שנוצרו על-ידי Visvanath Ratnaweera

תמונה של Particularly helpful Moodlers תמונה של Translators
Probably repeating myself. But this nuisance happens regularly. I was trying to post the reply in the screenshot, tried at least 5 times, always the same error shown appears, nothing posted.

 
Then changed the browser - from Firefox to Brave, if that is important. Got the security code by mail. Could post after that. Ref. https://moodle.org/mod/forum/discuss.php?d=473873&parent=1898650
תמונה של Particularly helpful Moodlers תמונה של Translators
Since nobody seems to know what the script https://j.6sc.co/j/c8971021-fc65-4bbd-9a05-b6e57875e886.js does, I asked Gemini. The reply is not flattering.
 
[Quote]
This script is a 6sense WebTag, a tracking and deanonymization tool used by B2B marketing and sales teams.

Specifically, the URL https://j.6sc.co/j/[ID].js loads a configuration for a specific company (identified by the unique ID in the filename) to track visitor activity on their website.

What the script does:
De-anonymizes Visitors: It attempts to match your IP address and other digital "fingerprints" against 6sense’s massive "Company Graph" database. This allows the website owner to see which company you work for (e.g., "Someone from Microsoft is visiting") even if you never fill out a form or identify yourself.

Intent Tracking: It monitors your behavior on the site—which pages you visit, how long you stay, and what products you are looking at.

Predictive Analytics: 6sense uses this data to calculate a "buying stage" (Awareness, Consideration, Decision, or Purchase). It tells the sales team if a specific company is "heating up" and likely to buy soon.

Personalization: The script can trigger real-time changes to the website. For example, if it identifies you as being from a specific industry, the site might automatically show you relevant case studies or banners tailored to your business.
תמונה של Particularly helpful Moodlers תמונה של Translators
A disclaimer first:
 
> Currently, we do not really know what else to look for. We use Postgres, by the way.
 
I assume that you are talking to the large community. AFAIC, consider me as another patient in the same clinic who has a history of the same symptoms, knows them "from inside". But that doesn't make him a doctor! He talks from the prescriptions he received, mostly just recollections, always centered on his particular case. That is not what we want.
קריצה
 
> Quizzes are used extensively in only one course. While they do exist in other courses, only this particular course contains a large number of questions and quizzes. It is possible that no one has ever attempted to copy a quiz in any other course.
 
It could be just the size, or broken data due unfinished transactions. Probably both. Then you should ask, "Why the size"? The easiest explanation is that the now infamous feature/bug has started duplicating question categories and triggered a vicious circle. If that is true, patching the but is not enough, one has to clean the (data) mess! That is only possible with a deep understanding of the database model and being able to identify the bloat - likely what only Moodle makers know. A commercial contract to a Moodle Partner could be the sensible solution, if your organization is big. It sounds like Microsoft selling virus cleaners for Windows, but that was a thing.

>> Is it the only course that makes use of a large collection of question categories?
>
> Yes
 
See above.

>> And consequently contains large number of duplicated categories?
>
> I wouldn't say so. I see about 40 categories in the question bank for this course. Most of them seem to contain questions. However, all of them have names that were apparently chosen by the teacher. There's nothing like "My category (copy)", for example.
 
Interesting. Most of our duplicated categories have "(Kopie)" in their names. Can you vow to the fact all the questions in that course are unique (the versions not counting)?

>> Is it a "forever course"? People come and take exams and go, the course remains - with all the user data, most importantly the exam results?
>
> Yes, it is an ongoing course that is updated once a year.
 
Do you reset the course yearly? Then it is not a "forever course" in my jargon. The forever courses happen because the institutions are supposed to archive exam results five years, for example. Therefore you'll find which have past exam submission spanning five year or more. (_More_ because nobody thinks of cleaning them - they trust the Moodle admin to be an archiver and historian.)

>> Do you duplicate a few quizzes each time?
>
> I can at least say the teacher duplicated quizzes in 2025 (Moodle 4.2; no problems) and again in 2026 (Moodle 4.5; huge problems).
 
Well, they might be having exam seasons tied to the semesters. If my theory above is correct, i.e. the course continuously accumulate questions, then one day it overflows - without an apparent major incident.

>> Do they get their questions randomly from question categories?
>
> No. The teacher would like to use this feature but has had problems in the past. In some cases, no suitable question could be found. She did not use this feature for the 2025 or 2026 quizzes.
 
If you say, the many questions are unique and quizzes were not fetching questions randomly, I am possibly following a red herring. 

>> How many versions your questions have on average?
>
> About four, but very few questions have many versions (up to 43).
>
>> Are your quizzes set always to take the latest version?
>
> I think so. In mdl_question_references, version is always NULL.
 
OK. I don't know how relevant the number is. But at least AFAIC, they add another layer of (unwelcome) complications. 

>> If you can make it run on the CLI, that might release the blockade.
>
> I tried to duplicate a quiz using the CLI with sudo php import.php --srccmid= --dstcourseid=, but it didn't work. The process hasn't finished even two hours later. It caused a huge amount of DB queries like the ones I mentioned in my original post.
 
That is not good. Something is brocken: either the DB has become too big for the machine, or the data has anomalies or just a bug.
 
Talking of the DB how big is a SQL dump? What is the size of the DB reported by the DBMS? How much RAM do you give the DB?

>> The huge database queries sound like Moodle is wanting to back up huge amount of data in to the database and getting timed out. I would dive in to that. Why? What data?
>
> We removed all questions that had no entry in mdl_question_references, as well as entries in mdl_files that had component 'question', but were not linked to a question.
 
As said earlier, there were many receipe to make the DB leaner. The above is just one. See those long discussions I've linked in my previous reply for more. Again, I can't definitely say which one would help, which ones will cause more trouble.
 
I'm at my wits' end. Remember, no doctor here. Hope, qualified people will come to help.
 
Edit: You checked the obvious, monitoring the server closely when it runs out of steam and the timeout values are reasonable, right?
תמונה של Particularly helpful Moodlers תמונה של Translators
Grüsse nach Norden!
חיוך

You're right. Many of us are in this unfortunate situation characterized by massively duplicated question categories and experiencing all sorts of difficulties - some operations taking incredibly long or not ending at all or loading the server so heavily that it crashes and even giving errors due to inconstant data caused by those crashes and more - we immediately think that we have the same "problem". Maybe, but not the same cause, simply because there were/are many causes and each has his own combination/complication. In this thread we need to look at your case. But it is not that easy, we automatically think along our own experience. Still giving it a try.

You say, the problems occur mostly in one course. What is special about that course?
- Is it the only course that makes use of a large collection of question categories?
- And consequently contains large number of duplicated categories?
- Is it a "forever course"? People come and take exams and go, the course remains - with all the user data, most importantly the exam results?
- Do you duplicate a few quizzes each time?
- Do they get their questions randomly from question categories?

You said, that the all the question categories that are being used in this course are at the course level, none at course category level, not even at the top (site) level? That would eliminate certain suspicions, later versions of Moodle 4, especially 4.5 LTS, has shown that site level question categories cause disproportionately high loads.

Are you certain that database tables are consistent? I don't mean "unused" records but also integrity damaged. I saw that you've started doing that. You'll find many user contributed "health checks" in the discussions. There was even an official /tool/health/ sometime. I was developing my own, but then this versioning of questions came, which was too much for me.

Talking of question versioning, how many versions your questions have on average? Are your quizzes set always to take the latest version?

Also a good thing that the clone behaves identically. So you can also try some potentially destructive medicines.
 
Are the prod. server and the clone in the same machine, the same horse power?

The huge database queries sound like Moodle is wanting to back up huge amount of data in to the database and getting timed out. I would dive in to that. Why? What data? It is possible that this is a one-time blockade. If you can make it run on the CLI, that might release the blockade.
 
The large number of files in the quizzes are suspicious too. If there are images in the questions, Moodle should not duplicate them, they are stored once in the repository. That reminds me a bug Atto(?) had, it converted images dragged-and-dropped to Base64, a) undermining this process b) the (bloated) Base64 need to be loaded directly in to RAM!
עצוב
 
All that said, you have professional DBA in the team I assume. In the case of MySQL/MariaDB they have to be "vacuum cleaned" after maltreated.
תמונה של Particularly helpful Moodlers תמונה של Translators
Generally, early Moodle 4.x had a bug which made the number of questions to grow exponentially. There are many, many resulting discussions in this forum. This is a kind of a summary: What is the state of the duplicating questions in Moodle 4.5 LTS?
 
In the meantime there are updates like  Duplicating a quiz duplicates questions in a question bank and a reciepe Huge Question Bank Upgraded Successfully to Moodle 5.0. I say "generally", since I don't know whether these things are related.