Updating a file everywhere it is used in Moodle 2.0

Updating a file everywhere it is used in Moodle 2.0

by Tim Hunt -
Number of replies: 36
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

There is much wingeing about the lack of this feature, so I decided to make a proof-of-concept: https://github.com/timhunt/moodle-local_fileupdater

At the moment this just does two things:

  1. Lists all the places in Moodle where a particular file is used (and you have to manually upload the file in question to a folder in moodledocs first)
  2. There is an implementation for a update_file_contents function that can be used to replace a file with updated contents.

and it is limited to admins only.

Obviously, more work is needed, for example:

  1. Let the user select the file to search for using the filepicker.
  2. Link up 1. and 2. so that you can actually update the contents of the file with a new file (again using the file-picker to upload the new file).
  3. Add more sophisticated permissions checking, so we can let teachers use the tool to update just the files they have permission to change.

3. is actually really hard to do in general, but a reasonably workable approximation should not be too hard.

I am not proposing to do any more with this now. I should not even have spent the two hours this prototype took. It was just a bit of displacement activity. So, please, anyone, feel free to clone that git repository and do more work. Thanks.

Average of ratings: Useful (8)
In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Mark Drechsler -

Thanks for your efforts Tim - even a little POC work to get people thinking about how this could potentially work is good in my books, even if we are a pack of whingers ;)

In reply to Mark Drechsler

Re: Updating a file everywhere it is used in Moodle 2.0

by Mary Cooch -
Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

If nobody whinges then useful changes never get madebig grin

In reply to Mark Drechsler

Re: Updating a file everywhere it is used in Moodle 2.0

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

@Mark: POC work ? what is it?

In reply to Joseph Rézeau

Re: Updating a file everywhere it is used in Moodle 2.0

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

"proof-of-concept" - as in the first line of my post.

In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Thanks, Tim.

May I add that I find it happens too frequently in these forums that people use acronyms without taking the trouble to expand them to their fuller form, probably thinking that everyone knows them. And "those people" probably include myself.thoughtful

Joseph

In reply to Joseph Rézeau

Re: Updating a file everywhere it is used in Moodle 2.0

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That is why there is a Glossary in this course: http://moodle.org/mod/glossary/view.php?id=851. It seems that POC has already been added, but the moderators have not yet approved it.

It would be really nice if we could head this thread back on-topic now.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Anthony Borrow -
Picture of Core developers Picture of Plugin developers Picture of Testers

I am happy to see updating a file throughout Moodle being discussed. Is there a tracker issue created for it so that folks can vote and comment? Peace - Anthony

In reply to Anthony Borrow

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Deleted user

Re: Updating a file everywhere it is used in Moodle 2.0

by Stefan Nychka -

Hi.

I'll briefly address #2 (more accurately the subject of this thread ... when I say #2 below, that's what I mean) and #3.  I don't know about #1.

First, I'm guessing there's a pretty good chance 2.1 will have #3 especially, maybe #2.  That's slated for the end of this month, hence there's a really good chance things'll all be working the way you want before next March, or at least you will have enough info. well prior to make a decision.

Second, note there is a Legacy Course Files Area (LFC Area) that basically works the same as the old Files area, except with better privacy.  All links to a file there would be "updated" when that target file was changed.

I am, though, concerned because the LCF Area is not supposed to be used (I believe because of privacy/security issues), and I'm assuming this also means it won't exist for ever, but I haven't yet seen work explicitly being done to support the link-once-update-multiple-times feature present in the old Files Area and the LCF Area.  The closest I've found is a mention of file synchronization in documentation, some related issues describing workarounds in 2.x, and a tracker issue (23306) on related improvements.

P.S. I'm not trying to say you should move to 2.x now, either.  Sticking with 1.9.x for now is reasonable.

In reply to Stefan Nychka

Re: Updating a file everywhere it is used in Moodle 2.0

by Stefan Nychka -

There may be a workaround for the issue mentioned in the subject.  See my first, currently only, post in the "Link into server files/private files" thread.

Please test, criticize, or point to an older, better explained post if one exists.  I have looked in the past, but not recently.

In reply to Stefan Nychka

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Deleted user

Re: Updating a file everywhere it is used in Moodle 2.0

by Stefan Nychka -

Hi.  I see.  Note what you mention (your #1 in a previous post) is not what I'm directly addressing.

I'm addressing the more general concept of updating a file that is the target of many links w/o having to modify those links.

So, I don't think the link to the other thread will help much with #1.  It was just meant to update my 1st post here; so, it was addressing #2 in your previous post.

Cheers

P.S. #1 by the way, is a good idea.  I'm not trying to say it's not.  It's just that it isn't currently as important to me, although I could see it being very useful to many others (including me at some point in the future ... our department's method of sharing Teacher info. across terms isn't all that sophisticated.)

In reply to Stefan Nychka

Re: Updating a file everywhere it is used in Moodle 2.0

by Stefan Nychka -

Sigh.  It doesn't work.  It breaks across a restore.  See the linked-to thread in my previous post (this post's parent).

In reply to Deleted user

Re: Updating a file everywhere it is used in Moodle 2.0

by Mark Hilliard -

Matt,

Your experience with teaching staff Moodle EXACTLY paralles mine!

Getting teachers to put basic files and work on-line is a simple, practical starting point that opens the door to all the other wonderful "interactive" things Moodle can do later when they ready to use them!!!!  Where you start to get initial success can be CRITICAL to long term adoption.

Losing the long established 1.9 file upload tool's simplicity has gutted our successful model for early adoption strategies.  Can we get back some key functional parts????  work tim did here say's "yes"  IF it can be continued!

Average of ratings: Useful (1)
In reply to Deleted user

Re: Updating a file everywhere it is used in Moodle 2.0

by Rosario Carcò -

I fully agree with you. See my post here, from an admin's point of view:

http://moodle.org/mod/forum/discuss.php?d=174702#p788253

And as I write in some posts there:

Multiple reference IS A PROBLEM, even a button to UPDATE ALL REFERENCES will probably cause more harm than benefit, especially when teachers work in teams on different courses sharing one and the same file(name) and file-areas. Who will help them to keep track of the different versions they upload and the courses/resources where they want to keep one version rather than the new one. So I would have to teach them a fix again: NEVER UPLOAD a file with the same name! Change the name like that file01.pdf, file02.pdf so as to be sure you can reference the correct version in any course/resource, otherwise you will certainly loose track. And I would have to tell them also: NEVER USE THE UPDATE ALL REFERENCES button, unless you are the only teacher of your courses and you know well what you are doing.

And attaching files to activities/resources is ok, but as activities/resources belong to a course anyway, I do not see why the files should  not be saved into an OS directory like it was done in 1.9, so as to keep backup&restore very simple for system administrators.

Rosario

In reply to Anthony Borrow

Re: Updating a file everywhere it is used in Moodle 2.0

by ben reynolds -

Tim,

I was just grousing Wednesday in a Moodlerooms webinar on 2.0's file system that I didn't believe the Open University could possibly be going manually through individual activities to change links. I guess this means the OU is?

For example, we create a number of copies of a course, each of which links to one file. If the file needs to be altered because of a typo, in 2.0, I'm going to have to change each link in each course.

Grouse: complain, grumble. Not quite as peevish as whinge.

In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Petr Skoda -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers
Hello,

the use of update_file_contents () breaks the fundamental rules of our new file api - you must not change files that belong to some other plugin, no exceptions allowed. Only the owner of the file is allowed to change it.

The solution is to finish the implementation of file_browser class and use it for all search, read and write operations. This class uses callbacks to plugins to find out what is and what is not allowed to do with files.

There are some file areas that must not be changes (assignment submissions, versioning data, etc.), sometimes other information such as revision flag must be updated after any file change. The problem is that you as an admin or teachers never know what operations are allowed to be done with each file - do not ever change files from outside of any plugin.

Petr
In reply to Petr Skoda

Re: Updating a file everywhere it is used in Moodle 2.0

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

So, you are saying that if we want to implement this functionality (and a lot of people seem to want it) then the only safe way to to it is:

  • implemente a new callback {frankenstyle}_update_file_contents_if_appropriate($oldhash, $newhash); that any plugin could choose to implement - and something similar for core file areas.
  • make local_fileupdater work by looking over all the plugins, and calling this callback, if it is defined.
In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Petr Skoda -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers
No, I meant to add this functionality to the file_info and file_storage classes. The reason is that this general functionality could be used by webdav, web services and anything else that wants to modify plugin & core files - it is supposed to include all access control logic.

The code in lib/filebrowser/* needs a lot more work but I believe it could be very useful in the future.
In reply to Petr Skoda

Re: Updating a file everywhere it is used in Moodle 2.0

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Ah, OK. My understanding is clearly still lacking.

I hope that either you get time to work on that at some point (are you listening Martin?) or that you can describe your ideas in enough detail that someone else can do it.

I assume that filebrowser uses (or will use) call-backs in the different plugins to know what files they have, and what it is safe to do with them.

In reply to Petr Skoda

Re: Updating a file everywhere it is used in Moodle 2.0

by Mark Hilliard -

Petr,

I am happy to see developers are still looking at this issue seriously and from reading the posts I see how the "rules" of ownership are imposing limits to mass updates...

Maybe we need to "brand" links to files with the owners "stamp" so teachers have some "clue" what will/won't updateable... wink  The more I work with Moodle 2 the more I like so many of it's features...if we can just find ways as a community how to improve/address some of the key lost functionalities of 1.9 files mgnt tools, I think the community would make a huge adoption.

Besides the tracker...where else can we follow any ideas for improving/changing the 2.0 file mgnt tools?

In reply to Mark Hilliard

Re: Updating a file everywhere it is used in Moodle 2.0

by Stefan Nychka -

Hi.

Which ticket contains this improvement, where 2.x can update a file everywhere?  I'd like to vote for it and follow its progress, but I don't see where in the tracker you can vote for/follow this particular issue.

(The ticket I listed previously doesn't contain it at a glance; none of the sub-tasks imply they include this improvement.  I did a quick search of the tracker just now, and had done a more thorough search earlier this year, but to no avail.  If it is in one of the subtasks for that ticket, fine, just let me know ... I don't mind digging if I know I'll eventually find what I'm looking for.)

In reply to Petr Skoda

Re: Updating a file everywhere it is used in Moodle 2.0

by Derek Chirnside -

Just curious Petr:  Where are the rules?

I have found this: http://docs.moodle.org/dev/File_API

I think this will turn out to be too inflexible.  I realise most solutions are a can of worms.

Today I read this: http://rlcommunity.remote-learner.net/mod/book/view.php?id=17

I quote: "Why use the Alfresco ECM as the File System for Moodle?

With ELIS 1.8, Remote-Learner has vastly improved the ability of Clients to use Alfresco ECMS, a world class content management system, as the main file system for their Moodle sites. So why would an institution want to use Alfresco as the file system for Moodle? There are number of great reasons; here are just a few.

  • Alfresco contains versioning capabilities that the Moodle file system does not. This means if a teacher or administrator accidentally or purposefully overwrites a file in Moodle (e.g. updates a homework assignment with new content), Alfresco retains both the original version of the file and the updated version in case the teacher or someone else would like to access the original file. Without Alfresco as the file system for Moodle, the original file in Moodle would be lost.
  • Alfresco will not actually delete files even if the teacher accidentally deleted it from Moodle. Instead, Alfresco will retain a copy of the deleted file where the site administrator can retrieve it and undelete it in Moodle.
  • Alfresco contains rich search capabilities. This enables users to search for file content across all of the various Moodle classes an institution has - they will see only the content they have permission to see. The Alfresco file permissions are automatically set by Moodle's roles system, although the site administrator can override or modify these. This means teachers working in Moodle have a straightforward and efficient way of finding and accessing content saved in Moodle by others via Alfresco. The Moodle file system does not include such file searching capabilities.
  • When used with ELIS 1.9 - our Alfresco implementation supports organizational and sub-organizational folders, where everyone in an organization has certain permissions to view and/or edit the files in the organizational folder.
  • Finally, files put into Alfresco can be linked to from any number of Moodle courses - when the file is updated in one place, the changes are instantly visible in all the courses the file has been used in"

I wonder can Moodle vanilla be set up to link with Alfresco easily?  Or does it require a hacks and coding.  I'm curious about the ELIS community version: does it have core changes in Moodle to enable the repo link to work as described here?  Or is it just a simple change to Alfresco?  If there are hacks, do they meet the Moodle coding guidelines, and could changes occur in Moodle?

In reply to Derek Chirnside

Re: Updating a file everywhere it is used in Moodle 2.0

by Hubert Chathi -

ELIS is currently only available for Moodle 1.9 (though we are currently working on porting it to Moodle 2), so the comparison in that document is only in relation to Moodle 1.9's file system.

Moodle 2 already has an Alfresco repository plugin (among others).

In reply to Petr Skoda

Re: Updating a file everywhere it is used in Moodle 2.0

by Jean-Michel Védrine -

“Rules are mostly made to be broken and are too often for the lazy to hide behind”

In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Mark Hilliard -

Tim,

Excellent POC work. This kind of "file picker" improvements could bring the larger community of experienced, power users to get behind Moodle 2.0 bandwagon!

But, am I correct in that Petr has stipulated "reasons" that it shouldn't be implemented??

So, help me out here...did Petr give you a solution or option to deploy this type of "single replacement" file fucntionality or NOT?  If not, I want to know what the $%^&#@ dev's were thinking when they doomed teachers to be updating links to common files one by one by one...

Mark H.

In reply to Mark Hilliard

Re: Updating a file everywhere it is used in Moodle 2.0

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Petr did not say this should not be implemented. He did say that my naive implementation was dangerous, and outlined what the correct implementation would look like, within the Moodle architecture. I agree.

I really don't get this "doomed ... to be updating links to common files one by one by one" refrain. I know it is a popular moan at the moment, but when do you actually need to upload multiple copies of the same file in your course? I think it is much more likely you might have to upload the same document to several different courses, but the updating that is just as much of a pain in 2.0 as in 1.9.

So, if you really think this is an issue, give me some user stories about building a course that requires repeat uploads.

In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by sam marshall -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Tim: I think the issue with files, like for download on the home page, is not a real problem. Also if it's 'files' that are reused in other places the solution is easy, upload it to the homepage as a file resource and then link to it from several places.

More likely the problem would be with images - say you make an image of, I dunno, the periodic table. You might use that image in 2 forum posts, a wiki page, 18 different quiz questions, etc. But now somebody discovers another element! Oh sh-!

(I agree that the problem with this ought to affect relatively few situations though.)

--sam

In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by ben reynolds -

Tim said:

> I think it is much more likely you might have to upload the same document to several different courses, but the updating that is just as much of a pain in 2.0 as in 1.9.

In 1.9, I upload one file SCHEDULE.PDF to Site Files, copy the link, put it in a template course, CDW3Template, before I replicate identical courses with different teachers -- CDW3A, CDW3B, CDW3C, CDW3D, and so forth. I replicate the courses. Later, I discover that my idiot student worker used the 2009 calendar not the 2011to make SCHEDULE.PDF. In 1.9, I overwrite SCHEDULE.PDF with the new, corrected file. The links in all identical courses still point to the correct file.

In 2.0, the procedure is to upload the new SCHEDULE.PDF, copy the link, and go through each course, one by one, replacing the link.

As I understand it, Moodle 2.0's internal file naming scheme does not give the same name to the newly uploaded file, and so, you must relink in each course.

In reply to ben reynolds

Re: Updating a file everywhere it is used in Moodle 2.0

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

There was a bug in early 2.0.x versions that mean you could not upload a file of the same name over the top of an existing file. That has now been fixed.

In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Rosario Carcò -

There are at least TWO scenarios in which teachers upload new versions of their files:

  1. Single Teacher, multiple courses of the same sort, with different classes: every file he/she uploads may be a first draft, a second improved draft, and a final one. Every draft being refined and tuned and improved with the students collaboration, something you develop during the course
  2. Multiple Teachers, multiple courses of the same sort, with different classes: dito

A real course area, with files lying on the OS file system, still has its reasons to continue to exist like in 1.9, even if it were only to make life easier for system admins who have to recover sigle files from tapes, when teachers lose them or acidentally overwrite them with the wrong version of the same file, which in principle is the very same danger/mess of the new file system. The only difference is that in 1.9 a lost or overwritten file belonged to the course's file area and could easily be retrieved from tape. Now in 2 you never know which version of the file uploaded with the same name and hence replaced is still referenced by which resource. And copying the whole file into a resource's file area only makes things worse, because teacher know this copy of the file only by its original file name and not by the metadata produced by Moodle to correctly handle it internally. So teachers will definitively get lost. A slightly better approach would really be to keep only ONE copy of an uploaded file, so as to store it really only once on the OS' file system and save a lot of disk space, but then a certain file name could only be used once. And if the file's name would be allowed to be used in each course, we return to the point where we startet, the course-file-area of 1.9

As I already said, resources/activities/etc. belong to a course anyway and hence keeping the legacy file system of 1.9 is the best solution for teachers and system admins.

Why do you not combine the CONS of both file systems?

  • I would even agree on creating not a course-file-area any more, BUT on a personal basis, i.e. per given teacher or group of teachers (cohort?)
  • this area would be created transparently on the OS' file-system to facilitate backup&recovery to/from tapes, no hash no scramble no lookup-tables necessary
  • so a file would exist only once and could be referenced in different courses/activities/resources/etc.
  • uploading a file with the same name replaces any older version and all references to that file would still continue to work, no link-update necessary
  • course backups would include all referenced files to be consistent when you restore them
  • restore of backup-zip-files would include an intelligent mechanism to RElink and save the referenced files again into the new teacher's/cohort's file area. So this process has to be interactive so as to specifiy the teacher/cohort before starting the restore
  • access permissions should be inherited from the activity/resource/etc. where the file is referenced and not from the file-area where the file resides
  • so as not to offer the students the opportunity of guessing a file name like it was in 1.9 links should not show the original file name but a randomly generated ID when the links in activities/resources/etc. are created
  • file-areas of single teachers or teacher teams (cohorts?) should be treated as private and hence should not be accessible for others
  • still there should be a global Moodle file area like site files in 1.9 accessible to all teachers and admins only. Files from there have to be downloaded first and then uploaded into the teachers'/cohorts' file area, or copied internally to avoid criss-cross references and orphaned links if someone deletes a file there
  • multiple references to one file are still dangerous in such shared file areas but we have to leave this to the responsibility of the teacher teams which have to agree on a common policy if they reference a file in different courses/activities/resources/etc.

As per http://moodle.drupalgardens.com/article/tue-06152010-1918/ftp-files-moodle-20-repository-api

>>
Then you can go ahead and create folders for each teacher, or course or however you want to structure the files, and give the content creators (probably the teachers) upload permissions to the folders using FTP or whatever method you allow.

The Moodle 2.0 admin can then add those directories as sources for files in the file picker.
>>

is far too complicated for a medium to large scale site like ours. This should be handled internally by Moodle as I proposed, one directory per teacher oder teacher team. Whether the sysadmin wants to expose those directories per FTP or WebDav should be possible and stay in his/her responsibility.

Rosario

Average of ratings: Useful (1)
In reply to Rosario Carcò

Re: Updating a file everywhere it is used in Moodle 2.0

by Rosario Carcò -

>>Why do you not combine the CONS of both file systems?

Please excuse my english, I thought of CONVENIENCES, but you use PROS for it. So combine the pros of both systems into one.

Rosario

In reply to Tim Hunt

Re: Updating a file everywhere it is used in Moodle 2.0

by Paul Ganderton -

Thanks for this Tim. I've only just seen the thread (and I'm no-where near a developer ofr coder) and it does seem to touch on areas that are key to me in getting Moodle to critical mass in my school. We're currently with 1.9.x until I and a few othrs can really sort out 2.x.x (probalby 2.1.x given the issues I've seen raised in the forums).

Do I understand it correctly that the old file upload system is not kept in 2.x.x? That work is being carried out to ensure that an updated file is updated in all linked instances?

If a minor irritation might be aired: when I move a file in 1.9.x it breaks the links, whereas if I do the same in Dreamweaver, it doesn't. Has this been corected in 2.x.x? If not, might it be considered?

Thanks.

In reply to Paul Ganderton

Re: Updating a file everywhere it is used in Moodle 2.0

by Derek Chirnside -

Paul, part response only.

File handling is completely different now.

Your questions:

1. Do I understand it correctly that the old file upload system is not kept in 2.x.x?

Yes Have you seen http://docs.moodle.org/20/en/File_handling_2.0

2. That work is being carried out to ensure that an updated file is updated in all linked instances?

Hmm.  Not sure how to answer this questions.  Tim has done a POC and Petr has said it breaks some rules to do this.  So the answer is probably No, but there is at least one developer trying to work on this.

3. If a minor irritation might be aired: when I move a file in 1.9.x it breaks the links, whereas if I do the same in Dreamweaver, it doesn't. Has this been corected in 2.x.x? If not, might it be considered?

I've not seen this referred to for a long time.