Moodle 2.2 Confusion on Server Files and Private Files

Moodle 2.2 Confusion on Server Files and Private Files

Nancy K Hoke གིས-
Number of replies: 38

Greetings - We have just created our Moodle 2.2 testing sight.  I am the administrator and I have been exploring and creating accuonts and courses.  I have begun to explore and understand the File Picker and the repositories, but I am confused on a couple of things I have encountered when in the File Picker.

1. I uploaded documents through Private Files. These files do not appear in the Server Files area.  Are these two separate locations?

2. Can a file be moved or shared from Private Files into Server Files?

3. In training faculty should I train them first in Private Files and then move them into other areas of the picker at a later date?

4. Server Files can be shared across courses and faculty and Private Files cannot. Is this correct?

As always thank you so much for your help - Nancy K. Hoke Khalifa University Abu Dhabi

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Nancy K Hoke

Re: Moodle 2.2 Confusion on Server Files and Private Files

Mary Cooch གིས-
Documentation writers གི་པར Moodle HQ གི་པར Particularly helpful Moodlers གི་པར Testers གི་པར Translators གི་པར

Hello Nancy

1. Yes - private files are like your personal online USB drive - just yours. Server files are like the university network - people with relevant permssions to folders can access them anywhere in Moodle. So if you were a teacher in French course and German course you could access both course's files via Server files -but you couldn't access the Chemistry course for instance

2. Yes - you can move private files to server files by adding them to the course page where they will be displayed. You can't move them without adding to the page

3. That is up to you - I don't do that personally because it is an extra and unneccessary step. They can simply upload directly into the course and to server files. They only need private files if they feel reassured having their files privately stored on Moodle and for later use. There is a video here -

4. I think that was covered in point number 1

Don't forget the documentation on files  -here's something for those moving from 1.9 to 2 http://docs.moodle.org/22/en/Course_files

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

Mary, thanks a lot for this. It is just what I did not understand correctly after a rough first approach through the new file system docs. And here are other thoughts/solutions that might be of interest:

http://moodle.org/mod/forum/discuss.php?d=196713

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

Mary I am making my first tests and in my Moodle 2.07 I just discovered that STUDENTs have access to the SERVER FILES like the Teachers! And they can even choose and upload such a file for example in an assignment.

THIS MUST BE A BUG, no student should have ever access to server or any other upload area used by teachers!

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Mary Cooch གིས-
Documentation writers གི་པར Moodle HQ གི་པར Particularly helpful Moodlers གི་པར Testers གི་པར Translators གི་པར

They can choose and upload someone else's file you mean? I haven't experienced that with my Moodle 2 sites but I am no longer on 2.0.something - can you confirm you are logged in with a regular student account, no permissions changed and that they can access other people's files via server files, as opposed to their own?

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

Because of my PHP Version being 5.2.14 I could not install Moodle 2 higher than the 2.0x line.

After having installed PHP Version 5.3.8 I will be able to upgrade this Test-Server to its latest Version 2.2 or 2.3

But for the moment being I will check again that the account I used is really a students' account and report back.

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

YES, in this version they actually can choose the teachers' files:

 Everything should be default, no special permissions for students set. I activated the legacy files but hope this has nothing to do with it as students are not allowed to access this one, which is not displayed in the above file picker.

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།:Useful (1)
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Martin Dougiamas གིས-
Core developers གི་པར Documentation writers གི་པར Moodle HQ གི་པར Particularly helpful Moodlers གི་པར Plugin developers གི་པར Testers གི་པར

The problem here is a misunderstanding.

You've used a "folder" resource to store your files in.  By default, yes, this is accessible by students - it's actually designed that way so that you can share a lot of files with students.

It is possible to prevent students accessing a folder activity - you just need to go into the permissions settings for that activity and remove students from some of the capabilities there.  This would prevent what you are seeing above.

Another way is to use your Private Files area for your files.

 

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།:Useful (1)
In reply to Martin Dougiamas

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

I am completely stunned, no breath, no words... and I thought the new file system were designed for the better security!

I mean, if I run into such a problem being the Moodle Admin since 2004 what are the problems my teachers are going to face as soon as we switch to Moodle 2.x ??

I only followed the video above which is intended for teachers. And only by chance and mistake discovered that students can access my files!

In the thread where we discuss the new file system I advocate for only two things:

- make file handling for Admins as easy as it was in Moodle 1.9 with plain-text or even hashed directories so that we can restore missed files directly on a OS' file system basis and not by going through the Moodle tables to discover what file got which hashed name in which hashed directory. There might be cases in which you do not have access to your Moodle Tables but to your OS' file system

- enable folder or file areas for teachers that work in teams, so that they can see what the other colleagues have uploaded. And if they upload a newer version of one and the same file, to be able to follow what is going on. In the Moodle 1.9 course-file-area this was implicitely possible. Now it seams to become completely impossible to work in teams without tracking separately which files have been uploaded, which ones have been linked/referenced and which versions of the same files have been linked/referenced in which courses/resources/activities/etc.

And now you say that even students can share those files with you as teacher....

In this case teams of teachers can share their files too, and security to dissallow the students to see them has to be achieved by misterious security/capability settings. I say "misterious" for our teachers, not for us admins. But as Admins we should be able to set this on site-level-basis, so as to exclude really every student from seeing the teachers' files.

And if server-files is a huge server-repo where all teachers can see all uploaded files, how could we manage to hierarchically split it up into smaller folders, so that every teacher and every team knows exactly where to look for own files? Of course I could set up sort of

- Team 1

   - Sub-Team 1

   - Sub-Team2

- Team 2

  - Sub-Team 1

  - Sub-Team 2

based on my 10 schools/universities and their subjects, teams etc. which is a really big overhead for something which was implicitely present in the course-file-areas of Moodle 1.9

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།:Useful (2)
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

So we are going to face another sort of problem:

- in Moodle 1.9 the students could guess what files were stored in the course-Files folder and access them even without them being linked in the course

- now you are saying that students can access the whole server file area which is even worse, because it is located at a site level, not only at course level

- using this server-files area would mean that teachers could simply share their files with the students without even linking anything in their courses, making Moodle behave like a drop-box. I know we can not prevent teachers from simply uploading their pdf-files into a course, but this way this is even inviting them to do so without even setting up their courses.

- as the old legacy course files area is also a site-level upload area, at least this one can not be accessed by the students, as I stated in my tests above. So this one would be much safer to use for teachers and teams of teachers even if I were to create the mentioned hierarchical folder and sub-folder structure for every school/university and every teacher-team.

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།:Useful (1)
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Mary Cooch གིས-
Documentation writers གི་པར Moodle HQ གི་པར Particularly helpful Moodlers གི་པར Testers གི་པར Translators གི་པར

I am sorry I don't quite understand what you mean  here Rosario? You can't upload files to a repository; you can only add them via "add a resource>file or >folder" at which point of course the students are meant to see them. Can you explain a bit more where the problem lies?

Also - I am confused by this thread as it seems we are having two conversations at the same timeསླ་བསྲེས་ Or maybe I am just not paying attention!

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

Maybe it is not the new file system per se, BUT ME, who is messing things up, because I didn't understand it correctly. So please help me.

We are talking about the SERVER FILES, as showed in my example above.

I uploaded several files, following your video. Thanks to the video I discovered how teachers can upload files into their PRIVATE FILES area.

But if I upload a file into the SERVER FILES area, Students can:

- see them

- use them to upload them e.g. into an assignment (which means this could be the solution the teachers already uploaded not knowing that this could be visible to students too!)

If this is all correct and no misunderstanding of mine, we can go further and think about practical solutions for my 10 Universities Moodle site https://moodle.fhnw.ch

The solutions to find are:

a) teams of teachers should be able to see each others files

a2) in a big site like ours, it should be possible to have folders/areas only accessible to those teams/schools/universities SEPARATELY and not for teachers/students of the whole site (SERVER FILES is a good idea, but of little use on a 17'000 user and more than 5'000 courses site)

a3) this was implicitly possible in Moodle 1.9 where only the teachers registered in one course could access the courses's file areas and noone else. And only the enrolled students could access the files published by the teachers via links to files or links to folders.

b) if one and the same file is uploaded/referenced/linked into resources/activities maybe in different versions like myDocument01.pdf and and at a later stage myDocument02.pdf, every teacher of the team should be able to see whether all the links/references have been updated or if there are reasons to leave one reference/link to the original myDocument01.pdf whilst in other courses/resources/activities the new reference to myDocument02.pdf is needed.

b2) if this is not VISIBLE, a team of teachers has to set up a complicated tracking notebook to know where/in which course/resource/activity they referenced which version of the file. This was implicitly given in Moodle 1.9 where you had to choose different file names if you wanted to link SEPARATELY and at the same time to different versions, and using the same file name meant simply to replace the file contents with the new one without having to renew the link.

c) there are other aspects at the OS' file system level:

c1) handling backup-zip-files has to be quick and IMMEDIATE (which means without going through the database and without going through the GUI, unless bulk-operation-scripts are integrated to move/delete backup-zip-files), e.g. when Moodle fills the data disk over night and you have to free it up in a hurry next morning (I experienced this at least 3 times since 2004, when the backup did not correctly rotate the zip-files)

c2) handling missed files has to be quick, like in Moodle 1.9 where I could restore missed files individually from my tapes. Now I would have to go through the GUI or the database to find out the hashed-dir and the hashed-filename and only then I would be able to restore it from tape, assuming no other file got the same hashed-dir and hashed-filename in the meantime. Because teachers often contact me one year after they gave their course, if they repeat courses on a one year term. So they tell me they miss something and I simply restore the old backup-zip-file or any other file from tape, restore the course into a new course and finally we can partially merge back into the original course what is missing, or we decide to delete the original and keep the restored course. All this was very very handy and practical on Moodle 1.9

So, if the new file system doesn't offer such handy solutions any more, this means I have to find solutions first and I have to instruct my teachers to use new tricks.

As Derek says, Moodle is no CMS, so why didn't we keep the new file system lean an clean as it was in Moodle 1.9 ? Some security-enhancements would have done it.

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

To be clear, SERVER FILES is a way of going through all the files that the user has access to already.

If you upload to a folder resource and then do not either hide the folder or override the permissions on it, then you have published the files on the course for the students to see. It does not matter whether they access them via the SERVER FILES file repository or via the main course page, they still have access to them either way (and allowing students to submit from server files is, for all practical purposes, identical to them downloading the file they have access to anyway, then uploading it again as their assignment submission).

a) teacher collaboration on files, 2 options:

1. create a folder then HIDE it, so students can't see it or access the files in it (if students can access the files when the folder is hidden, then that is a bug to be fixed, but I don't think they can)

2. create a custom plugin (could be a local plugin, block or something else) - I don't think this would be a huge job to do.

b) 2.3 adds aliases which allow files to be updated wherever they are used, when the 'real' instance of that file is updated

c1) automatic backups can be set to save in an external folder, not inside the Moodledata, so you can manage them that way if you want

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Davo Smith

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

Thanks Davo, and sorry for my misunderstanding:

>>

If you upload to a folder resource and then do not either hide the folder or override the permissions on it, then you have published the files on the course for the students to see. It does not matter whether they access them via the SERVER FILES file repository or via the main course page, they still have access to them either way (and allowing students to submit from server files is, for all practical purposes, identical to them downloading the file they have access to anyway, then uploading it again as their assignment submission).

>>

I took RECENT FILES and PRIVATE FILES and SERVER FILES like sort of global (site) file areas where Students and Teachers would see only their own files. If I follow correctly, SERVER FILES shows everything that was PUBLISHED inside the course where the teacher and the student is enrolled... Ahaaa

>>automatic backups can be set to save in an external folder, not inside the Moodledata, so you can manage them that way if you want

I was not sure on that, I thought Moodle would keep track of them in its file-table even if they are collected on a OS' directory.

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

You have 3 options with auto backup:

  • Save in moodle files
  • Save in external folder
  • Save in BOTH moodle files and external folder

If you choose option 2, then nothing is saved inside Moodle files.

Be aware, that switching between methods does NOT transfer previous backups to the new area (or ever remove them from the old area - this includes if you have set to only keep the last x backups).

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Davo Smith

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

So you need option 3 to let Moodle remove/keep a given number of course-backups??

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

No.

If you specify to keep 3 backups, then:

  • store backups in Moodle files will keep the last 3 backups in Moodle files
  • store backups in an external folder will keep the last 3 backups in an external folder
  • store backups in both will keep 3 backups internally and 3 backups externally.

However, if you were storing the backups internally (and assuming you have 3 backups stored there) and then you switch to an external folder, then you will start off with 0 backups in the external folder and 3 backups still stored internally. After the backup has run 2 times, you will have 2 backups in the external folder, but still the 3 original backups stored internally. Run the backup another couple of times and there will be 3 backups in the external folder (1 will have been deleted), but still the original 3 backups stored internally.

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

After thinking over night how I could have run into such a misunderstanding of the SERVER FILES, I think we should differenciate things better:

- PRIVATE FILES: file area with all uploaded user files

- RECENT FILES: VIEW of recently uploaded files

- SERVER FILES: VIEW of uploaded AND published files. The files can be seen by all teachers and students of the same course or courSES, giving quick access to the files published/linked in course folders and course links. Is this VIEW spanning over ALL/SEVERAL courses where teachers and students are enrolled ? or does it show only the files published INSIDE the course where you use the file picker at a given moment ??

Is that correct? Then maybe we should call it rather COURSE FILES to better reflect, that this is not to be confused with SITE FILES we had in Moodle 1.9 where all teachers could access files in one pool, but no students (unless the student could guess the URL). Then SERVER and SITE FILES would be synonyms.

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Rosario Carcò

Re: Moodle 2.2 Confusion on Server Files and Private Files

Derek Chirnside གིས-

I've come in a little late here Rosario.

You say: "now you are saying that students can access the whole server file area which is even worse, because it is located at a site level, not only at course level"

No, this is not true.  I think I can vaguely see how you have come to this confusion (see martin's Post clarifying this) but rest assured, it is not true.

Somebody please check this reasoning (Mary??):

  1. Server files are those actually uploaded as in add resource > file.  NOT available unless you have teacher permission.
  2. Add a resource > Folder IS available to students.  Unless it is Hidden to students.  This is the purpose of folder, to make files available to students.
  3. There is no equivalent of Moodle 1.9 Course files in Moodle 2+ (Ignoring for the moment legacy files)
  4. As a kind of work around we do two things:
    1. Use private files to upload a lot of files we may want to make available to choose from ourselves.  Not sharable with other teachers.  (They are private, after all)
    2. Use a HIDDEN folder to  make files available to other teachers on the course.  Like a pseudo-version of the old Course files or the Legacy course files.  Available to other teachers, not students.
    3. Use the File system reposisitory (which has it's own issues)
    4. The external repository solution: Use dropbox.  Use Sharepoint.  (There are a few other options here . . .)
    5. ??  I can't think of other work arounds.
  5. Short of using a file repository (Alfresco etc) there is currently no nice simple way to use Moodle as a repository of files like the old Course Files area.  As Nancy Hoke said: a new paradigm is needed.  I will give references to this if you like, but that is where Moodle has moved on to.

In other words, there is no nice way to have a bunch of files up on a Moodle 2.1+ site accessible by the teachers to link, unlink, move, change, share, do good work, create chaos etc, with files COMPLETELY siloed by course with all the security holes as in Moodle 1.9 course files area.
This is what you get with Files 2.0: sharable content between courses, less duplication of files.  ie the Server files.  (With all the clicks, which may be fixed in 2.3)

There is Legacy files.  I wondered at some stage if this may move on to being a "Moodle Files Repository Lite" but no way I think now.  As has been said often. Moodle is not designed to be a CMS.

I have yet to find a robust workflow for course deployment/creation for Moodle 2 as it comes outof the box.  We all have our cludges and work arounds.  I hope Files in 2.3 will assist this: multiple file upload, drag and drop maybe.  But even then, we are back to dealing more from our computer HDD <> Moodle than from anything Course Files like in Moodle 2.

Good luck Rosario

-Derek

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།:Useful (2)
In reply to Derek Chirnside

Re: Moodle 2.2 Confusion on Server Files and Private Files

Rosario Carcò གིས-

Derek,

>>Server files are those actually uploaded as in add resource > file.  NOT available unless you have teacher permission.

My walk through above proves the opposite! I uploaded a file (hmmm... does it matter whether I uploaded as TEACHER or as Moodle-ADMIN? I guess I uploaded as ADMIN) and it went AUTOMATICALLY into SERVER FILES.

Then I logged in as STUDENT and I was able to see that file and I could even upload it into an assignment. So could there be a difference if uploading as teacher or as admin? Uploads of admins being available to students, uploads of teachers only to teachers?

>>Use a HIDDEN folder to  make files available to other teachers on the course.  Like a pseudo-version of the old Course files or the Legacy course files.  Available to other teachers, not students.

Good idea, but very very hazardous, as there could be files containing the solutions. So one erroneous click and the students could gain access to all the secrets the teachers might have in there...  མིག་ཁྱབ་

>> there is currently no nice simple way to use Moodle as a repository of files like the old Course Files area

I have proposed an idea to Matteo Scaramuccia who programmed the DAVroot Plugin:

Why not use Moodle as I proposed above to have separate file-areas directly on the OS' file system for different schools/teams/etc. And teachers could access those file areas directly with their respective WEB-DAV-clients to upload/delete files.

In the Moodle GUI (file-picker) they would finally link the files to the courses/resources/activities they want to.

Permissions on those file areas could be granted by enrolment of the teachers in the given courses.

A practical example:

- I would create a file area for a team of teachers, sort of /teamA, /teamB in moodleData/filedir/teamA, moodleData/filedir/teamB

- I would grant access to this areas in Course 1, Course 2, etc. and implicitly to all teachers enrolled in Course 1, Course 2 (or directly to the individual teachers, if we wanted to keep independent from the courses)

- the teachers of teamA and teamB respectively could connect to https://moodle.fhnw.ch/teamA and https://moodle.fhnw.ch/teamB respectively using their WEB-DAV-clients

Thanks for the "Good Luck", but if needed, I will try to work on this with Matteo.

Rosario

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Nancy K Hoke

Re: Moodle 2.2 Confusion on Server Files and Private Files

Nancy K Hoke གིས-

May I add to this discussion? My nextquestion is in regard to managing the files. From what I am reathere and from I see there is no delete for a file! Especially in the early days when faculty  is exploring, as I am now, I have uploadedmany documents as I learned hodots navigate the system. I wanted to go in and delete some of the files and I am not sure how to do this? If I delete the link to the file does that delete the file from server files? I came upon the statement that the system cleans itself up. What does this mean? Will a report be generated of files that are not being used? 

Question 2 - Is the only way to upload a file is through resources and selecting File or Folder? I know I can to Private files directly, but can faculty just tofiles to upload their documents and then create the link later? 

Thank you for your help. Nancy K. Hoke 

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Nancy K Hoke

Re: Moodle 2.2 Confusion on Server Files and Private Files

Mary Cooch གིས-
Documentation writers གི་པར Moodle HQ གི་པར Particularly helpful Moodlers གི་པར Testers གི་པར Translators གི་པར

Hi Nancy -yes if you delete the link then it will go from server files in time so it's not a problem.  If faculty is really keen on uploading files and creating the link later then I'd suggest either private files as mentioned before or making  a folder on the course page and uploading to that folder (which colleagues can do together) and hiding it from students' view with the eye.

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 Confusion on Server Files and Private Files

Nancy K Hoke གིས-

As always thank you again - in making the Folder for the course page - that would be using Folder in Resources?  Which is different from creating a folder when you are in Adding a New File?   - Nancy K.

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 Confusion on Server Files and Private Files

Philippe Decloitre གིས-

Hello everyone.

I've just read this post where Mary Cooch has again shown her prowess in deambulating through Moodle! Impressing indeed!

Trève de compiments དགའ་འཛུམ་

Here is a question related to file managing:

I have several activity files (say HotPot, but it could be others) all linking to ONE & same video file.
When I create theses files I use a relative path /video/myvideo.flv.
This way, wherever my files are working (this or that course), they can always reach the intended video.
So far so good.

With Moodle 1.9 I would upload the whole set, create an activity for each HP file, which in turn would call the same video file, and voilà!

Not so with Moodle 2.2

So far here is how I proceed:

I create an activity, on that occasion upload the whole set, select one of the HP file to be summoned by the activity, then duplicate the activity ("X2" sign) and select the next HP file to be called up and so on.

This works very well and is quite straight forward until you realize that Moodle has not only duplicated the links but the files themselves. This means that the video file is n times on the server (if you have 10 activities revolving around the same piece of video, that's a hefty pack!), especially when it's time to re-upload it to an other Moodle site (way too large a bulk)

Surely there must be a way, that would prevent this mushrooming of copycats, while still allowing for relative path.. but I really wonder what it is...

Mary, are you still there? དགའ་འཛུམ་

Philippe

 

 

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Philippe Decloitre

Re: Moodle 2.2 Confusion on Server Files and Private Files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

You don't worry about the space on the server.

The way Moodle files work is that two files with the same content map to a single physical file stored on the server. So if you upload the same video file 5 times to the server (or if you duplicate it 5 times on the server), then it is only actually stored once (but the Moodle FilesAPI stores 5 references to the same physical content). If you replace one of those files with a different video, then the original file is still there for the other 4 references, but the changed reference now points to the new file (although, on that last point, Moodle 2.3 is introducing a new option to replace the updated file everywhere it is currently used, but, as far as I understand, only for files that have been duplicated on the server, not for those that have been uploaded separately).

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Davo Smith

Re: Moodle 2.2 Confusion on Server Files and Private Files

Philippe Decloitre གིས-

Hello Davo,

Thanks for taking the time to answer.

As I understand, Moodle does a very good (though unseen) work at managing files. If so, why is it,  that when I want to backup my set of activities, the mbz file is so huge (making it impossible to transfer it to another Moodle)?

Thanks again, I understand you are not responsible for the Moodle's behaviors, I'm just trying to sort things out དགའ་འཛུམ་

Philippe

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Philippe Decloitre

Re: Moodle 2.2 Confusion on Server Files and Private Files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

The best I can offer with working out why a backup file is so big, is to download the file, rename it from filename.mbz to filename.zip and then extract the files.

If you look in the 'files' folder, then (within each of the subfolders) you will find all the files included in the backup. Find the biggest files, note their name (which will be something like '2742eb81e7c3c72848c94d53789ceaeb39cd48f4') then open up 'files.xml' and search for those names (some names will occur more than once if you've reused the file). Immediately below the file name, you should find details of the activity that included that file (along with the original file name).

However I would add, with reasonable confidence, that the size of the backup is directly related to the number of large files attached to activities included in the backup. It is possible for a very active course backed up with user data to have a large amount of database data included, but I would usually expect videos, audio recordings or a large collection of images to make up the bulk of the storage space.

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།:Useful (1)
In reply to Davo Smith

Re: Moodle 2.2 large backup files

Mary Cooch གིས-
Documentation writers གི་པར Moodle HQ གི་པར Particularly helpful Moodlers གི་པར Testers གི་པར Translators གི་པར

(I've changed the name of this thread as I think we are going into the area of backups) I have a point to make/query to raise here too. I made a course with videos in that I backed up to use in another Moodle. But it came out too large so I decided to compress the videos and reupload them as different, smaller filetypes. In 1.9 I could have deleted the others but even though the previous videos no longer appeared on the course, Moodle still wanted to include them in the backup and so my back up file was no smaller at all!! I thought Moodle deleted non-displayed files after a certain time so I went back a few days later, backed it up again but it was still the same size. Hmm..སླ་བསྲེས་

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Mary Cooch

Re: Moodle 2.2 large backup files

Philippe Decloitre གིས-
Hello Davo and Mary, i had indeed done some scrutinizing of the backup folder. I must admit I did not go much further than peering at the exadec. File names wondering if Moodle was going the way Microsoft did when it let go the old .ini and DOS file for the directory System, Not a trove for the common teacher I am! Anyway, I used the search tool to find any. Flv file... there are none to be found. Does Moodle transform them too? PS: I only backed up one series of activties, no other data whatsoever (what happened to the sellect/deselect all button at the start of the backup process?) Philippe
དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Philippe Decloitre

Re: Moodle 2.2 large backup files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

As mentioned above, it does not 'transform' the files, in the sense that the contents of the file are left untouched. However, it does rename all files to a long string of (apparently) random letters and numbers (it is, in fact, a unique identifier, based on the content of the file, which means that two files with the same content will have the same name and will only be stored once); in the process the file extension is lost. That is why I suggested you should search based on file size (not name) to find the largest files and then look up their names in files.xml.

Alternatively, you could search within the file 'files.xml' for '.flv' and look to see how many flv files are included, but, you would have to bear in mind that any files with the same 'contenthash' value would only be stored once. I have wondered, at times, whether there would be any value in a simple tool that could open up a Moodle backup and list all the files contained within it (including sizes, types and the activity they are attached to), but that might imply that I had time to write something like that ...

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Davo Smith

Re: Moodle 2.2 large backup files

Philippe Decloitre གིས-

Thanks Davo for putting all the "grey" (or hidden) processing to light. That does explain a lot to me.

I added the extention .flv to the files expected to be remains of what I had uploaded earlier and there they were!

That means Moodle keeps all the files even those I have deleted.  In itself it's just a site manager's problem, but the hurdle is that I have then to make out what IS part of the backup and what is NOT if I want to have a workable (uploadable) backup. I thought the machine was supposed to do that after I had gone through the backup forms.

As to cleaning itself up (during a "Cron" as Lawrence mentionned _Thanks Ray for jumping in_), my Moodle is pobably not well bred because the older video files that should not be in there, were removed weeks ago (rather untidy དགའ་འཛུམ་.

Well, so, I suppose, what I am to do is pick up my own scopae and mopae..

Thanks for the lesson.
Philippe

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Philippe Decloitre

Re: Moodle 2.2 large backup files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

I'll do a quick test tonight to see if I can reproduce the problem you have reported and let you know the outcome.

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Davo Smith

Re: Moodle 2.2 large backup files

Philippe Decloitre གིས-

...I stay tuned དགའ་འཛུམ་

 

 

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Philippe Decloitre

Re: Moodle 2.2 large backup files

Davo Smith གིས-
Core developers གི་པར Particularly helpful Moodlers གི་པར Peer reviewers གི་པར Plugin developers གི་པར

I after some experimentation I've not managed to get any deleted files to appear in a backup. This is what I did:

I created two file resources. I attached a single file to each resource. I then deleted one of the resources. I also edited the other resource, deleted the file and uploaded a replacement. I also uploaded a different file with the same name and overwrote the original and uploaded one other file, but clicked on 'cancel' on the form, to discard it.

This left one file resource with one attached file (but with 4 other files that had been attached and deleted in some kind of way, but not cleared out of the database by cron; not to mention several draft file records).

I ran a backup, then downloaded and unzipped the backup file. As expected, there was only one file contained (the final attached file).

While this is not completely exhaustive proof that backup cannot include deleted files, it would strongly imply that the size of your backup is directly realted to the number of current active files in your course.

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།:Useful (1)
In reply to Davo Smith

Re: Moodle 2.2 large backup files

Philippe Decloitre གིས-

On my side after several attempts, even trying to backup just a few tags (not related to media in any way) I still have all those large media files.

I am not going to go much further on this as I am not sure a cron is apropriately running on my hosting server.

That might be the problem. (might it?)

I will return to you on this thread when I am sure everything is run properly (I've had a few glitches while running backups _missing ID parameter messages and the like..)

Thanks for the updates...

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -
In reply to Philippe Decloitre

Re: Moodle 2.2 Confusion on Server Files and Private Files

Eric Hagley གིས-

Good afternoon Phillipe (or maybe it is good morning in your part of the world!)


I'm hoping you (or someone else) may be able to help me. I, like you, have numerous instances where I have quiz questions based around an audio or video file. In 1.9 I could copy and paste the link to the file, after the initial upload, to other instances/questions.

Doing that in 2.2 results in "brokenfile" instances. I seem to have to upload the file every time. From what the rest of the discussion states, each time I upload the file I'm not actually doing so - rather Moodle sees the file being uploaded and points to the equivalent one that I've previously uploaded (that's how I read the ongoing discussion). However, this is rather time consuming and I wonder if there is an easier way than uploading the same file numerous times. (BTW the audio files that I upload don't seem to show in the "recent files" section)

Just to be clear these are the steps I take:

1. I have created a new quiz question. I want to add audio to it and other questions (same audio file).

2. I highlight some text and click the link button.

3. I click the browse button and upload the audio file.

4. I click "update". I save the question.

5. I want to use the same audio file in the next question. I again select a piece of text, click the link button, click "recent files" (can't see it there so have to go back to upload) go back to upload and upload the same file again. Previously if you did that Moodle would ask you do you want to overwrite the old file. This doesn't happen.

I would love to be able to just copy and paste the link but if I can't do this, what should I do?

I'm running 2.2.2 with 5.5.21 MySQL

Any ideas you or any other kind soul could offer, would be very much appreciated.

Kind regards,
Eric

དཔྱ་སྙོམས་ཀྱི་སྐུགས་ཚུ།: -