Archival of Student Information

Archival of Student Information

by James Snell -
Number of replies: 7

One of my Moodle sites provides small certifications for users that register with email self-registration. The students generally only need access to this system for a few months at most and then they're gone. Right now I have them auto unenroll after a time. However, their accounts remain in the system at all. Right now I've got nearly 8000 such users. I'd like to work out a schedule for account deletion, but I don't want to lose the record that they were a student.

Is there some moodle mechanism for archiving accounts or something similar? Even if I could define a Moodle database activity that deleted account information is copied to before the actual deletion, that'd be fine. I certainly can devise my own system for this, but it would seem to me others have this issue and probably have solutions for it.

So, any tips/pointers?

 

Minor Background Details

I casually administer a few moodle installations. I'm not an expert, but I do have some decent skills too. My users are fairly basic in their needs. My Moodle systems are kept current (so 2.6.x right now).

I operate on Ubuntu 12.04. Two of my servers live in Amazon EC2 and the rest are internally hosted off of VMware ESXi.

Average of ratings: -
In reply to James Snell

Re: Archival of Student Information

by jason everling -

I am not sure if this maybe is what you are looking for but this may be the easiest route..

Setup a 2nd Server for Archiving, a clone of the 1st server, so Apache, MySQL.

After you get the second server running you can use a couple of commands to stream the data "moodledata" "database" from the main server for archiving.

For purposes of this write-up let's say you send your MySQL Backups to /opt/mysql/backups on the 1st server, you would need to setup SSH properly for this to work, and you have setup the same directory on the 2nd server.

Server 1 Crontab :

/usr/bin/mysqldump --user=root --password=yourpassword databasename | gzip -9 > /opt/mysql/backups/backup_`date +\%Y\%m\%d\%H\%M\%S`.sql.gz

/usr/bin/rsync -avz /opt/mysql/backups root@ipaddressofServer2:/opt/mysql/backups

The above commands are going to take a MySQL backup and then at regular intervals copy them to your second server.

Now since the "moodle" application only changes when you upgrade it does not need to be archived so on the 1st server we can add another command to crontab to copy "moodledata" at regular intervals

/usr/bin/rsync -avz --exclude 'cache' --exclude 'codecoverage' --exclude 'sessions' --exclude 'temp' --exclude 'trashdir' /var/www/moodledata root@ipaddressofServer2:/var/www

The above command if you notice does not have the --del option which would mirror server 1 and since we want to "archive" we want to keep all the data on the 2nd server. You can have this in crontab run every 5min if you want, up to you. The 1st tie it runs it might take a while so you can do the 1st copy manually then add the entry to crontab since the rest will be incremental changes.

That is basically it, now if you want to go a step further you can create "virtual hosts" in Apache on the second server for each "term" the users are enrolled and name them according like term1.domain.com points to its own moodle install location with its own database restored on the second server. You can set this up for each term. Now each virtual host must have its own moodle folder but all of them can share the same "moodledata" folder. You would though however need to manually restore a new database for each term on the 2nd server and point each "config.php" to its individual database but all off them would point to the same moodledata folder.

You would now be able to keep the live server up to date and delete whatever you need to, if you need to go in and see past data you would just logon to one of the 2nd server virtual hosts and pull that data. This would also make it a snap to restore data to the live system.

Thanks,

JASON

 

 

 

Average of ratings: Useful (1)
In reply to jason everling

Re: Archival of Student Information

by Hittesh Ahuja -

Hi Jason,

very informative. thanks a lot.

We have been upgrading Moodle once every year, so we started from 1.9 -2.2 , 2.2-2.5 and this month we would be upgrading from 2.5- 2.6 .To facilitate the upgrade, we have a read -only version available for users whilst the upgrade is happening. So I take a copy of the live instance, clone it and try to make that read only. Now ,we've tried setting the db and the file store to read only but that stops us from getting into the moodle site. So, last year, to make the site 'soft read only' we incorporated a few core hacks so the moodle site looks as if its read only , but its just bypassing users from updating the log table and doing any updates /writes. 

I havent come across any simpler way for the site to behave purely readonly when we are upgrading. 

Once we have successfully upgraded, we archive the read only version so thats its archived, but also users cant make any changes to that version, hence read only.

Is this how you do your archiving process ? or is it just a copy of the old instance but with write permissions ?


Hittesh

Average of ratings: -
In reply to Hittesh Ahuja

Re: Archival of Student Information

by jason everling -

The above post that I made is how we archive Moodle, we reset all our courses at the end of every semester so that our Production Moodle instance stays clean. The archive is setup almost completely read only set in the database, you have to set all the tables to read-only, NOT in my.cnf, then update the tables that need write permissions in order for faculty to login/view the courses. The main tables I know of right now that we have read/write on are mdl_backup* , mdl_cache*, mdl_config*, mdl_log*, mdl_stats*, mdl_user_lastaccess. The ones with * is all the tables that start with the table name. We do not have cron running on the archive server, if you have cron running you would need allot more tables read/write.

We do the backup and restores to the production instance, we do not let faculty run them. Students are not allowed to login, faculty are in a seperate OU than students so that keeps students from being able to login.

Also, I use Git, I can upgrade our production instance with having only 10 minutes of downtime. I stage the upgrade and have prepared the Moodle application with the updated plugins and such so that when the day comes to upgrade I move the old moodle application out of the webroot and then just copy over the ready to go local repository then run the upgrade. I also of course test all this days before on a local dev server.

JASON

Average of ratings: -
In reply to jason everling

Re: Archival of Student Information

by Hittesh Ahuja -

"The main tables I know of right now that we have read/write on are mdl_backup* , mdl_cache*, mdl_config*, mdl_log*, mdl_stats*, mdl_user_lastaccess"

So basically,a db user is given only SELECT,UPDATE,INSERT,DELETE on the above tables ? which other tables have only SELECT permissions. With this effect, what happens when an academic tries to update something, say, an assignment title and hits save. Is the academic presented with an mysql error ?


Hittesh

Average of ratings: -
In reply to Hittesh Ahuja

Re: Archival of Student Information

by jason everling -

Basically yes, those above tables should have the standard Moodle permissions, all the other tables should be set at Select.

The errors shown is more less what you have set in Moodle for debugging or display messages setting, ours is set to none. Its just a standard Moodle error page, must be fixed by a programmer error.

If you are to worried about the database and error messages you can also just leave it as is and just edit the teacher role and change the architecture type to non editing under Site Administration > Users > Permissions > Define Roles and edit the Teacher Role. They would not be able to delete or modify anything since non editors cannot. I set this up just to try it on our spring 2014 archive, so far it behaves like it should I did haven't change anything else. I didn't use this before because I never thought to do it until a month or so ago I had to create a new role for auditors to view all courses but not modify anything.

JASON

Average of ratings: -
In reply to jason everling

Re: Archival of Student Information

by Hittesh Ahuja -

I am trying to do exactly the same thing now.. create a 'teacher read only' role so that all they can do is view things like blocks, activites , resources etc and prevent them from adding,deleting or updating anything. Once happy, we would come up with a similar role for students smile

Will post my progress here if anyone is interested. 

Average of ratings: -