How Do You Delete Old Assignments?

How Do You Delete Old Assignments?

by Jez H -
Number of replies: 25

Can anyone tell me what they do to clear down old assignments?

We get tens of thousands of assignments per year all of which go into Moodles file system alongside course content.

The only way I am aware of to clear down old assignments (to free up storage space) is to reset the course, but we don't want to do that.

What we want is to keep the grades, course content, forum posts etc for users to refer back to but remove old assignment files.

I would be grateful for any ideas / suggestions on the above.

Thanks

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Fred Riley -

This isn't an answer, but please see a related post I've made to this forum as answers to that will be relevant to this query.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by ben reynolds -

The one thing I know about this is that Moodle flushes files that do not have a link to them. That means Moodle knows how to test for files with no links. So, if you could see what that test is, it would tell a programmer what needs to be done to delete links in an appropriate way.

I'm not a programmer. I may have said in English what Fred said in databasese in his related post.

Average of ratings: -
In reply to ben reynolds

Re: How Do You Delete Old Assignments?

by Jez H -

Thanks Ben,

I will test that but doubt it would work well from a students perspective, they will probably see a broken link or no link at all.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by ben reynolds -

Since this is database work a programmer or super super user would need to know how to do, you would need to announce in a news forum or some such that the files are no longer available. I'm guessing that the links would no longer exist.

As we have requirements here in the US to preserve these records, we don't actually anticipate removing them until 2 or so years after the course has ended. At which point we archive the course and only go back to it as needed, which, so far has been, essentially never.

Storage being so cheap now, I'm wondering if you really need to delete them.

Average of ratings: -
In reply to ben reynolds

Re: How Do You Delete Old Assignments?

by Jez H -

Hi Ben,

We are looking at coding something to do this.

We will be dealing with perhaps 100,000 assignments per year, some are quite large, video assignments for example.

We take full system backups which gobbles up more space.

It also means these backups take longer to run, consume more network traffic, storage etc.

I guess we will keep our assignments for at least 18 months but will want to retain the Moodle course itself for perhaps three. The problem is you cannot separate assignment submission from course content.

Another little statistic, in 12 months running 50% online submisison (rest offline) we more than doubled the size of a Moodle instance with several years worth of teaching materials within it, it was built in 2008 and we imported existing courses into it then.

I guess over time if left unchecked our Moodle file system will become perhaps 80% unneeded assignments and 20% useful content.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by ben reynolds -

Those are some compelling numbers. Or, more accurately, daunting.

I'm not sure what you mean about assignment submissions can't be separated from content.

This will be intriguing to follow.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Derek Chirnside -

What happens if you delete the assignment module?

Do all the assignments get deleted in some sort of cron process?

Do grades get deleted?

-Derek

Average of ratings: -
In reply to Derek Chirnside

Re: How Do You Delete Old Assignments?

by Perry Way -

Q: What happens if you delete the assignment module?

A: The folder containing the source code gets deleted ultimately, plus the tables to support the module if they are tables only for that module. (i.e. most modules use the `mdl_user` table but it's not part of their module)

Q: Do all the assignments get deleted in some sort of cron process?

A: Not exactly. What happens is there is a definition of how a module is supposed to implement code. Some of those items are callback functions which are called when you delete the module. One of those callbacks generally contains all the database code to drop the tables that that module relies on and created when you installed the module.

Q: Do grades get deleted?

A: I do not know the answer to this one, sorry! Would like to know, though, because I have a curious mind!

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Jez, I have struggled with the very same question, so thanks for asking.

I wonder if one approach might be to have a completely separate Moodle installation for archiving courses.  When appropriate, a Moodle course that is over can be backed up with all user data and then restored in this "archive Moodle installation".  Once archived, the course can be removed from the production Moodle installation, which I think will remove all of the submitted assignments and other course related data (right?).  Using this approach, daily backups of the "production Moodle" installation would probably be smaller.  The archive Moodle installation would not need to be backed up daily, just maybe once a semester.

I am offering this idea in order to get some feedback from both yourself and other people here on Moodle.org.  I am wondering if anybody is currently using this scheme?  Yes, I realize that if your school has many courses that this backup/restore procedure could take a bit of time.

Average of ratings: Useful (1)
In reply to Rick Jerz

Re: How Do You Delete Old Assignments?

by Jez H -

Hi Rick,

The alternative we were looking into was a separate repository type and modified assign plugin. In short assignments would still run though a single Moodle instance and its file API but the physical files would go somewhere else where they could be more easily managed.

The repository type would only exist on the back-end, that is to say you would  not see it listed in the file picker!

A specific risk we wanted to avoid with this approach is our drives filling unexpectedly. If your Moodle instance runs out of disk space you have a real mess on your hands. When you start working with video assignments the problem becomes more acute.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

I know what you mean about video files, I use many myself for instruction.  However, as the instructor and content creator, I never put my videos into Moodle.  They are always on a different server outside of Moodle.  But I can understand someone wanting to keep all content in the Moodle course, which then puts it into the moodledata directory.

Average of ratings: -
In reply to Rick Jerz

Re: How Do You Delete Old Assignments?

by Jez H -

We use Kaltura for video, its pretty good but we are not using it for assignments though we may do at some point. However if we did that Kaltura uses more space as it keeps the original and creates transcoded copies. We would then need a process to clear those doen too!

There are issues when you upload many videos to Kaltura in a short timeframe as happens with assignments. Also some assignments for media students require submission of source files rather than watchable video. Using Kaltura would be to improve the grading process for tutors (play video in grading page) rather than improving storage efficiency.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Dave Balch -

I think the way to do this reliably would be to create a local plugin containing a cron job that scanned assignment submissions past a certain age, and deleted any files attached to them.

You'd probably want the plugin to:

  • Limit how many assignments it clears each cron run (to avoid timeouts when there is a lot to do).
  • Record the timestamp of the last submission you cleared, so the next clearance can start there (for efficiency).
  • Indicate on the submission that they have been cleared for maintenance (to avoid student confusion).

It would also be a good idea to communicate to your students that their submitted files will eventually be deleted (so they should ensure they've a copy of everything they want to keep).

My group takes a different approach (which I need to update for M2.7 when we upgrade this year), by downloading all the submission files, grades, etc. in a course, and archiving the files on a separate filesystem, then deleting the courses altogether (as student access expires). The archive copies are kept to comply with exam regulations.

HTH

Average of ratings: -
In reply to Dave Balch

Re: How Do You Delete Old Assignments?

by Jez H -

Hi Dave,

That is the kind of thing we were thinking, the problem is how to remove the files without breaking the relationship with the files table which was also discussed here:

https://moodle.org/mod/forum/discuss.php?d=259586

If you want to keep copies on a different file system then you would need (I think) some kind of repository plugin to send files off there and keep the data you need to retrieve them. That is to say its one thing to copy the files, another to retain controlled access to them afterwards!

I am not sure there is much benefit in doing that. In terms of course content Moodle will only keep one copy of a file so if you roll over a course the files within it will not be duplicated. You bloat your database somewhat, but making courses read only may be a better approach, then clear down the courses once you are finished with the assignments... unless you have a particularly long retention period beyond the life of the course.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Dave Balch -

I wouldn't mess with the files table, but use the API - probably $fs->get_area_files().

 

Something like this pseudocode:

submissions = local_get_100_oldest_submissions()
foreach (submissions as submission) {
    assignment = submission->assignment
    submissionfiles = fs->get_area_files(assignment->context->id, 'mod_assignment', 'submission', submission->id, "timemodified", false);

    foreach (submissionfiles as file) {
        file->delete();
    }
}
Average of ratings: -
In reply to Dave Balch

Re: How Do You Delete Old Assignments?

by Jez H -

I would need to look into the API more, I would imaging file->delete removes both the file and db record?

What I was hoping to do is update the file records to point at a single file, imagine 10k assignments all with the same contenthash (and other required settings) pointing at a single pdf telling you your assignment has been cleared down.

That way there are no broken references in the db, no broken links for the users.

 

 

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Dave Balch -

> file->delete removes both the file and db record?

That's the plan; hence no concerns about anything breaking.

> That way there are no broken references in the db, no broken links for the users.

Well, technically that's correct, but practically and philosophically I'd say they were broken.

If a user clicks a link of "my-assignment.txt" to get their assignment but instead gets a clear-down PDF, it would violate the principle of least surprise - which would be particularly unfortunate if they hadn't know it would happen and don't have a separate copy of their work.

I don't see any reason that tampering with the files table wouldn't work, but it sounds like a more risky approach that might not be great for the students.

Average of ratings: -
In reply to Dave Balch

Re: How Do You Delete Old Assignments?

by Jez H -

Hi Dave,

If you delete the files and a student clicks through to their old assignment they will get a 404 error or a coding error... or something undesirable.

I very much doubt deleting the files will remove the link in their graded assignment, and if it did remove it they would wonder where it had gone.

I think if we do this we would send a notice to students in advance warning them of a pending clear down, after that the linked document would give details of the retention policy / process, but ultimately its a way to try and maintain the integrity of the db / file system.

If a subsequent version of Moodle updates / moves files that will cause problems too (if db records are removed). An example of this was many moons ago my predecessor cleared down the users table, we then hit a problem upgrading to 1.7 which introduced roles / contexts, the upgrade was trying to apply permissions to items for which there was no longer a record in the db.

There really doesn't seem to be an easy or a right way to do this sad

 

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by dawn alderson -

Hey,

wasn't sure where to put this response....something of a simultaneous discussion going on in the developer forum too...(Jez's thread-link)

OK. Just a thought.  In an ideal world, it seems to me it would be absolutely awesome if, the focus for development was on 'individual student assignment profile-that can be easily accessed/stored' as opposed to a focus on improving the facility to delete a 'cohort assignment load', in the first instance.

I suppose what I am suggesting is a kind of PLE zone....whereby say for example I had 50 students enrolled on a three year undergrad course....and say 47 pass through to year 2, but 3 of those need to re-sit/repeat-then those exact same students struggle in year 3...in the same way...need a re-sit/re-submit or whatever.....

I would like to track back through their assessment profile/work submitted, in particular when awarding their overall degree.....cross-check mitigating circs with assessment attempt and so on.....wheras the other 47-aggregate scores/grades  would be on record, but I would not necessarily need to go back and look at their subs for year 1.....does this make sense?       

Deleting cohort assignments obviously needs a healthy timeframe between when they were submitted and when the students exit from the courses.....however, being able to track individual assignments for deletion is also worthy of consideration as some student subs might need to be held for longer than others.

Bit of a wordy response sorry, hope someone gets it- wide eyes  understand if not mixed

cheers,

Dawn   

 

 

Average of ratings: -
In reply to dawn alderson

Re: How Do You Delete Old Assignments?

by Jez H -

Funny you say that, I added a line to my previous post along the lines of:

"assignments (and grades) should belong to the student and not to the course"

and then deleted thinking it was a bit of a digression.

You should (in my unqualified opinion) be able to handle student assignments, feedback and grades independently of course content.

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Dave Balch -

> If you delete the files and a student clicks through to their old assignment they will get a 404 error or a coding error... or something undesirable.

Really? I don't know for certain, but would expect that the student would be able to view the assignment page (seeing their grade), but the files simply wouldn't be present.

Average of ratings: -
In reply to Dave Balch

Re: How Do You Delete Old Assignments?

by dawn alderson -

...course, there is Turnit in compatibility/functionality with some potential hybrid-iz-able.....features too....additonally- a focus on possible development of gradebook/assignment/subs/tracking/student PLE assessment zone/deletion of out- of- date assessments/ docs/files and-on.....sounds super!

wink

 

D 

Average of ratings: -
In reply to dawn alderson

Re: How Do You Delete Old Assignments?

by Jez H -

Hi Dawn,

We use TurnItIn and that looked like it could be a big help.

Unfortunately TurnItIn is a nightmare, a lot of Universities have assignment deadlines around the same time (over same few weeks) and there are regularly issues at these times.

To my knowledge its not suitable for other forms of assignment, video / media files for example.

So yeah it can help, but it brings its own set of problems.

Interestingly lectures seem to use it more for formative work (working on drafts / checking originality) than for the final assignment. I think part of the reason for that is if you submit a draft under a formative assignment (setup in Moodle for this purpose) and then submit a final version against a different assignment later on TurnItIn flags the duplication between the two.... at least it did last I heard... I don't teach I push buttons smile

Average of ratings: -
In reply to Dave Balch

Re: How Do You Delete Old Assignments?

by Jez H -

Hi Dave,

I dont know for certain either, guess there is only one way to find out and that is to test it!

I had assumed (always a bad plan) the link / reference to the file was held in one of the assignment tables and would remain visible if the file were simply removed.

I will have to do some more digging...

Average of ratings: -
In reply to Jez H

Re: How Do You Delete Old Assignments?

by Dave Balch -

I figure that, so long as you use the API, you shouldn't end up with any dangling references - that's the point of it after all! smile

Average of ratings: -