After running Moodle for more than 3 years now, our database (and moodledata) is getting really big. This not only created trouble for our future upgrades, but also slowed down the server performance. Although we could trim down by purging records of the mdl_log table (based on the "time" field), but there are also mdl_course, mdl_quiz, mdl_quiz_states (or mdl_question_states), and other tables (as well as associated user files) that are getting really big (or taking a lot of disk space). Just curious, is there any people out there who are running a big moodle installation that has successfully purged (permanently, instead of just marked as deletion) data from some of these big tables based on a certain criteria (say, only keep one year of data, or three semesters of data)? Thanks!
I think it would be a good idea to put it as a wish list item that in the admin block there will be an option for the admin to purge data based on some criteria.. just my 2c.
"I deal with this by creating completely new installations each school year. I then load all the classes, but only the course data, via backup and restore. We have a script that automatically creates the user accounts and assigns them to the proper classes. I keep the previous year's installation at a separate URL (ie. http://2005-06.school.com) That way we have a usable archive for teachers to reference and our current installation is free of cruft. "
Dear Bryce Thornton,
I need more details about the end of year procedures and any automated scripts that will help doing that?
Can you help me please?
There already is the course_reset functionality, which discards most of the student-generated content (e.g. forum posts) and as of recently wipes out all the data about quiz attempts.
And you can of course, delete entire courses. And deleting a course should delete all associated data.
However, it is probably more a process question. How long do you need to keep information about finished courses available (there may be legal reqirements)? You probably need a procedure that says after X months, take a backup of the course and archive it, then wipe out all student data. Then after a further Y months, Completely delete the course from the system.
I would do this as a manual process in the first instance, perhaps helped by some custom reports (it is easy to stick new ones in the admin/report folder) that list courses that have been finished more than X months ago.
Later, when you are confident, you can consider completely automating it.
What version of Moodle does this. We are running 1.6.3+ (looking into upgrading) and I really REALLY need to do this. Creating a new course and restoring does not restore all the images nicely. I have reset but it does not reset the quizzes and it is too time consuming to do every single one using the "show all attempts"/"delete attempts" route. Is there another way to clear out all quiz attempts.
I have a ton of work, deleting quiz attempts ahead of me if I can not find an automated solution. Thanks to ANYONE who can help!
* Maintain a well-performing site for current activities
(This means finding a way to trim data)
* Minimize downtime and disruption for users.
(This means consistent URL for active site. Limited migrations of courses.)
* Maintain course data for 7 years.
* At end of academic year (in August), make a clone of the current site and give the clone a name representative of its 'archive' purpose, e.g., moodleAY0607.school.edu. AY0607 = Academic Year 2006-2007, August to August.
* Trim active site of all courses no longer active or being developed.
* Trim archive site of all current/ongoing and future courses. (Keep it available and maintained for 7 years.)
We've developed some additional 'trimming' reports to the existing reports to facilitate this. For example, we've developed a 'trim courses' report that tries to identify courses that may no longer be active and gives us a single 'click' delete with a button. Same for users.
The one thing we'd like to improve here is the deletion of users. When we decide they're ready to go (e.g., they haven't shown up on the new site for a few semesters) then not only set their "deleted" value to 1, but to really delete them from the user table. If what Tim Hunt says is true, these users should have little to no data in our trimmed system...but we'd still like to trim our user table in our 'active' instance.
The real problem is not to purge data based on some criteria
or to delete logs older than some time.
The real problem is to archive data based on some criteria,
i.e. to delete data from current Data Base and at the same
time to insert the same data into the second Data Base "HISTORY"
related to educational history records about students, teachers, courses and etc.
The both Data Bases have to be usable through Moodle user interface.
Yes, Admin -> Server -> Archive
The Time Machine is my 2c.