accidently deleted a repository file

accidently deleted a repository file

by Craig Cleator -
Number of replies: 18

Version 2.5.1

Hi there,

I was over my account size limit so I went into my moodle data folder and then into my filedir folder and started deleting older large files. In the process I deleted a repository file that is responsible for enabling me to Manage my private files. Now all I get when trying to manage and upload into my private files is an error message explaining the file does not exist. My problem is I do not know which file and is there a replacement file I can get my hands on, or will an upgrade solve my problem?

Cheers

 

 

Average of ratings: -
In reply to Craig Cleator

Re: accidently deleted a repository file

by Ken Task -
Picture of Particularly helpful Moodlers

No backups?   Shame, shame!

Suggest trying to trick Moodle into thinking it has what it needs, but that depends upon the *exact* error concerning that file (location)

So turn on debugging.   Attempt to access what ever you clicked upon, etc. to see what Moodle says ... specifically.   You can then return here and respond (by pasting the text of the error) into a response of this forum.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: accidently deleted a repository file

by Craig Cleator -

Hi Ken,

Thanks for the help and yes I know I should be running a backup on my site. I have had the site for nine years and an ongoing battle with my server provider and space. Back-ups tend to create massive files on my site.

Here is the debugging message I get:

Debug info: [dataroot]/filedir/0d/74/0d74d5947507bca548de06af166f8513fa3a0048
Error code: storedfilecannotread
Stack trace:
  • line 175 of /lib/filestorage/stored_file.php: file_exception thrown
  • line 846 of /lib/filestorage/stored_file.php: call to stored_file->update()
  • line 399 of /lib/filelib.php: call to stored_file->set_source()
  • line 260 of /lib/filelib.php: call to file_prepare_draft_area()
  • line 65 of /user/files.php: call to file_prepare_standard_filemanager()
 Thanks
Craig
Average of ratings: -
In reply to Craig Cleator

Re: accidently deleted a repository file

by Ken Task -
Picture of Particularly helpful Moodlers

Let's see if it's only one and get info from the DB.

Run this query:

select * from `mdl_files` where `contenthash` like "0d74d5947507bca548de06af166f8513fa3a0048"

What does that show/list?

Work around?   We'll give Moodle what it's looking for ... but not really.

Manually, with whatever tool you have to work with files/browse files ...

First browse to moodledata and see if there is a '0d' directory.   If there is NOT create one.   Set permissions and ownerships like the other folders you see in that area.  If there is a '0d' directory, see if there is a '74' directory inside.   If not, create one (permissions/ownerships same as folders/files above).   Inside the '74' directory is there a '0d74d5947507bca548de06af166f8513fa3a0048' file?   If not, create a text file with that as the filename.   Same ownership/permissions as files/folders above it.

From the trace, looks like trying to update a file that was still in draft.

When you run cron jobs there are routines that deal with file system and does move files from draft to a component area (doesn't really 'move' physically, rather the reference in the DB is 'moved'.   Are there any errors in running cron?

Since you've been running a long time, same database?  Have any tools to check the status of tables - especially the mdl_files table status?

'spirit of sharing', Ken

Average of ratings: Useful (2)
In reply to Ken Task

Re: accidently deleted a repository file

by Craig Cleator -

select * from `mdl_files` where `contenthash` like "0d74d5947507bca548de06af166f8513fa3a0048"

What does that show/list?

The search pulls up 29 rows, all with the same filename, fliearea where one row = private, one row = content and the rest = draft. All have the same contextid = 102 except for the filearea = content > contextid = 4231 with component = mod_page.

Work around?   We'll give Moodle what it's looking for ... but not really.

Manually, with whatever tool you have to work with files/browse files ...

First browse to moodledata and see if there is a '0d' directory.   If there is NOT create one.   Set permissions and ownerships like the other folders you see in that area.  If there is a '0d' directory, see if there is a '74' directory inside.   If not, create one (permissions/ownerships same as folders/files above).   Inside the '74' directory is there a '0d74d5947507bca548de06af166f8513fa3a0048' file?   If not, create a text file with that as the filename.   Same ownership/permissions as files/folders above it.

There was a folder called 0d. However, there was not a '74' within so I created it changed the permissions to reflect the top folder and then inside 74 created a text file with the filename as specified.

From the trace, looks like trying to update a file that was still in draft.

When you run cron jobs there are routines that deal with file system and does move files from draft to a component area (doesn't really 'move' physically, rather the reference in the DB is 'moved'.   Are there any errors in running cron?

Tried running a cron - job we shall see.

Since you've been running a long time, same database?  Have any tools to check the status of tables - especially the mdl_files table status?

I have been having troubles recently with my mdl_files table as draft files have been repeated for several of the same files. After a few weeks I get my server provider telling me to reduce the database under 500 mb. I manually search for all draft files and delete from database. It is a pain but seems to work. I still need to set up a cron job that will take care of this but not sure how.

Cheers

 

Average of ratings: -
In reply to Ken Task

Re: accidently deleted a repository file

by Craig Cleator -

Actually Ken, in my moodledata directory there are several folders named '0d'. They all show up in filedir but deeper in numbered folders. So within filedir/0d/74 I created a file called  '0d74d5947507bca548de06af166f8513fa3a0048' with permissions 750 or the same as folders above.

Back in moodle here is the error debug info I get.

Debug info: [dataroot]/filedir/a9/6d/a96d80ff2a77aa3fe204e8ce2c4c9399033b4685
Error code: storedfilecannotread
Stack trace:
  • line 175 of /lib/filestorage/stored_file.php: file_exception thrown
  • line 846 of /lib/filestorage/stored_file.php: call to stored_file->update()
  • line 399 of /lib/filelib.php: call to stored_file->set_source()
  • line 260 of /lib/filelib.php: call to file_prepare_draft_area()
  • line 65 of /user/files.php: call to file_prepare_standard_filemanager()
Average of ratings: -
In reply to Craig Cleator

Re: accidently deleted a repository file

by Ken Task -
Picture of Particularly helpful Moodlers

'moodledata directory there are several folders named '0d' ... well, yes, that possible with hashing, but they do NOT, cannot exist at the same level ... ie, just below moodledata.

The first error you got pointed to a file we created and placed manually.

This next error is for a file that should be in moodledata/a9/6d/

and it's hashed name is a96d80ff2a77aa3fe204e8ce2c4c9399033b4685

If you can't purchase a plan that allows DB larger than what you are using right now,  then this problem will continue to happen IF you manually remove records in the DB without removing also the actual files.

Only way I can help is to tell you that - and to suggest (IF your time is worth anything and you desire to NOT have these problems) you purchase a plan that allows for larger DB's.   If the provider cannot do that ... search for another provider.

Continuing to manually remove records and files leaves a lot of room for human error.   One of these days you could end up destroying your Moodle site.   Only hope and re-course is for the provider to restore your site.   Uhhhh, your provider is taking snapshots of your space, aren't they?   That's probably not a free service and will cost.

IMHO, this does illustrate that maybe Admins of a site do need tools inside the Moodle UI to 'deal with' the new file system.   But, wouldn't expect any such tool from Moodle HQ coming anytime soon.   Hasn't reached critical mass yet.

http://docs.moodle.org/26/en/Cron

Since you are remotely hosted, check with provider about how to setup cron jobs.

'spirit of sharing', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: accidently deleted a repository file

by Craig Cleator -

This next error is for a file that should be in moodledata/a9/6d/

and it's hashed name is a96d80ff2a77aa3fe204e8ce2c4c9399033b4685

 

Inside moodledata/filedir/a9/6d is a file named: a96d7e5f2e15f7d3a701311417fae4ea7ff578c3

Should I add the delete this and add a file with the name: a96d80ff2a77aa3fe204e8ce2c4c9399033b4685

or should I leave the file and add a second with that name?

I will look into setting up a proper Cron job.

Thanks

Average of ratings: -
In reply to Craig Cleator

Re: accidently deleted a repository file

by Craig Cleator -

Hi Ken,

 

It appears that the problem goes beyond just the private files repository. Trying to edit several pages produced the following errors.

Debug info: [dataroot]/filedir/06/2f/062fb70cdaabee8481b6e1166645290d1f1bb16e
Error code: storedfilecannotread

Debug info: [dataroot]/filedir/4b/b3/4bb35d22580d5e7585644a63b5e856ce3ea6d473
Error code: storedfilecannotread
 
Debug info: [dataroot]/filedir/7b/d6/7bd6914263db69e9a33595a69e552767cc95f153
Error code: storedfilecannotread
 
Debug info: [dataroot]/filedir/a9/6d/a96d80ff2a77aa3fe204e8ce2c4c9399033b4685
Error code: storedfilecannotread
 
Debug info: [dataroot]/filedir/c0/bd/c0bd0f02b726c9a3aca98d0d34e54339a2c539f8
Error code: storedfilecannotread
 
As far as I know this is it.
Average of ratings: -
In reply to Craig Cleator

Re: accidently deleted a repository file

by Ken Task -
Picture of Particularly helpful Moodlers

Thought I suggested that this might involve more than one manual re-creation to satisfy Moodle (maybe not).

That's not too many to manually re-create, is it?  Considering you get the site usable again.   You might run into more of them later.   If so, you'll just have to do the process again ... and don't forget to re-edit whatever that pointed to the man-made text file.

'spirit of sharing', Ken

 

 

Average of ratings: -
In reply to Ken Task

Re: accidently deleted a repository file

by Vani Bheemreddy -

Hi, 

I have a similar issue. 

Can I delete files from Server Files repository? Will this impact the courses where these files are added as I understand there could be some courses where files are added as 'alias/shortcut' and some courses as just a  'copy'?

Average of ratings: -
In reply to Ken Task

Re: accidently deleted a repository file

by Nidhi Tiwari -

Hi Ken,

I have create file as per your guidance but still having same error.

Debug info: [dataroot]/filedir/b7/4b/b74b7a349d7f87d8e5eb5f77db03c9401f7cb82f
Error code: storedfilecannotread

My file : /home/dev-galaxy/Allprojects/moodledata/filedir/b7/4b/b74b7a349d7f87d8e5eb5f77db03c9401f7cb82f.txt

What I have missed Please help me. I have run admin/cron also.

Thanks,
Nidhi

Attachment error.png
Average of ratings: Useful (1)
In reply to Nidhi Tiwari

Re: accidently deleted a repository file

by Nidhi Tiwari -

Hi Ken,

I have used the way to create file with appropriate permission but it doesn't work for me. Finally I have deleted that record from DB. This I have to do for at least 10records as per error notification I have changed the value of contenthashin below query :

select * from `mdl_files` where `contenthash` like "0d74d5947507bca548de06af166f8513fa3a0048"

It works for me smile

Thanks,

Nidhi

Average of ratings: Useful (2)
In reply to Nidhi Tiwari

Re: accidently deleted a repository file

by Nishant Pandya -

Hey nidhi,

Thanks a lot smile You save my day. This is the correct answer for this issue.

Thanks,

Nishant


Average of ratings: -
In reply to Nidhi Tiwari

Re: accidently deleted a repository file

by Ken Task -
Picture of Particularly helpful Moodlers

Files in filedir do not have file name extensions.   So if you had renamed the false file you created: b74b7a349d7f87d8e5eb5f77db03c9401f7cb82f.txt

to b74b7a349d7f87d8e5eb5f77db03c9401f7cb82f

note no filename extension, that would have matched what's in the DB.

See in another post, you've found the records in the DB and dropped those records.   That's half of what you need to do to keep filedir accurate and in sync with the DB.  

Make sure you check if there is a file in filedir/xx/xx/[contenthash] and remove it as well if it's not linked any where else in your Moodle.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: accidently deleted a repository file

by Nidhi Tiwari -

Hi Ken,

Thanks a lot for your quick reply smile . I have deleted records from DB but didn't maintain its list. Now I am not aware from filedir which file should be deleted. Also I am facing a issue of missing images due to this. Please suggest what should be the best way to repair this.


Thanks,

Nidhi

Average of ratings: -
In reply to Nidhi Tiwari

Re: accidently deleted a repository file

by Ken Task -
Picture of Particularly helpful Moodlers

@Nidhi ...

In a previous post you shared a query:

select * from `mdl_files` where `contenthash` like "0d74d5947507bca548de06af166f8513fa3a0048"

And that query found 10 references in the DB?  Are those the records you deleted?

In a similar situation, I selected contenthash,filename,filesize from ... blah, blah.   That way I knew if the file was a mimetype different from what was expected.   There could be more than one file in filedir/0d/74/

Assume you deleted those references in the DB (your query above).   I should have warned that since DB is out of sync with what's really in filedir, removing the reference from the DB should be done only when/if it's verified that there is no other file in that filedir/xx/yy/ directory.

So I'd do the query and then from ssh via command line:

cd /pathtomoodledata/filedir/

cd od/74/

and then I'd look to see what file (if any) was there:

ls -l

IF I saw a 0d74d5947507bca548de06af166f8513fa3a0048 file there, I'd check out what it was:

file -b 0d74d5947507bca548de06af166f8513fa3a0048

that command returns the file type/mimetype.

Because I had done the DB query with the filename, I could copy the 0d74d5947507bca548de06af166f8513fa3a0048 file out to document root (on my server /var/www/hmtl/) and rename it at the same time:

cp 0d74d5947507bca548de06af166f8513fa3a0048 /var/www/html/imagefilename.jpg (or whatever the filename was in the DB).

I could then look at the file via browser (later would remove the file from /var/www/html/

So now you have images missing ... like I said ... and like the warning.txt file says in filedir ... be careful and really know what you are doing when messing with any of it.

So ... if you can see the same errors as before ... missing [dataroot]/filedir/xx/yy/xxyyanothercontenthash

then to get around this now, manually create a file that has nothing in it at that location:

cd [dataroot]/filedir/xx/yy/

touch xxyyanothercontenthash

That last command will create a contenthash file name that is 0 bytes in size.

Now you can go into Moodle UI and edit the link to the file.  You are really editing the DB (again) while uploading the true image again to get the physical file into the sea of files contained in filedir matching what's in mdl_files.

You have to do that process for each errant reference ... one at a time.   I would not do a global sort of delete of the records in the DB without manually checking each one of those contenthash named files.

Hope this helps and doesn't make matters worse.

'spirit of sharing', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: accidently deleted a repository file

by Hartmut Scherer -

Hi Ken,

Thank you for your post to give Moodle DB what it's looking for. As a site admin I could no longer edit my profile. 

Debug info: [dataroot]/filedir/e5/34/e5348160aa956f0a9eb4ba8a6146ab9365c17303

Error code: storedfilecannotread

I added an empty file in e5/34 and then I got another error message

Debug info: [dataroot]/filedir/a3/2f/a32fa4fde0a3ceb81233c25ac1975236b4a044e2

Error code: storedfilecannotread

I got a third error message. All messages had to do with a missing picture for the profile. After I added an empty file into the last directory, I could edit my profile again.

I am very glad for your detailed instructions, Ken. Your advice is extremely helpful.

With kind regards,

Hartmut

Average of ratings: -