I need to migrate a site with a very, very large moodledata folder. Is it safe to delete any of the following folders in moodledata prior to migration?
/cache
/localcache
/sessions
/temp
/trashdir
Thanks,
Floyd
I am pretty sure all of them - the only ones I am not too sure on are sessions and temp but by their names alone, I would think that would be fine. To be safe, I would just move them elsewhere, then migrate the rest.
Thanks, Emma. I'm quite certain /sessions and /temp can be deleted. It's good to have your opinion on the others.
I look forward to seeing you at the Mountain Moot this year.
-Floyd
Look forward to seeing you too - should be a fun Moot as always!
Emma, is there any link/ explanation of what actually these individual folders do?
also, if this is possible, to give individual path to each of these folders in config file? like for example I am bit aware of cache and localcache, but can it be possible that if I want to keep cache separate on some other drive? if yes, how would I be linking it in config file?
Not that I know of Usman. I was just suggesting to move them in case for some reason, they were needed on the new site then they could be copied over at that time. I would guess it would need some core hacking to direct to a separate cache folder ... unless there is something in the cache settings...could you maybe set up as a new cache and direct in there?
That I have to look into, for performance reasons as so far I have been directing DB's to faster drives on Windows/IIS installations, and one recent install of moodle on Windows server that take my suggestions by putting DB on separate SSD drive, their performance is ridiculously fast, so just had been thinking as cache/localcaches are used by moodle for storing temp/mostly used files, so if they can be directed to faster drives independent of moodledata directory, then performance can be even faster (just a thought).
Now seems it's time to rub the lamp for Ken, Howard, Visvanath or even Matteo had been putting his efforts around Caches.
Hey! Did a feel a 'rub'? What? You were expecting a genie? ... REALLY! ;)
Ummmm ... Errr ... highly technical.
Googling finds this:
https://mariadb.org/significant-performance-boost-with-new-mariadb-page-compression-on-fusionio/
The Persona folks have reputation with MySQL.
Check this one out:
https://www.percona.com/blog/2013/10/03/inexpensive-ssds-database-workloads/
Clips:
Large Buffer Pool Size – The larger your bufferpool hopefully more dirty pages you can keep and as such more writes will come to already dirty pages effectively eliminating IO to SSD. So even if SSDs means you often can do with smaller memory amounts to achieve performance you need you might still want memory to increase your SSD life time.
Large Innodb Log Files – The larger are your Innodb Log files the more you can delay writes and as such the more logical writes can be merged into the single physical write. SSDs are fast so you often can get crash recovery speed you need with a lot larger log files than you had on HDDs. The combined log files size of 10GB may well work for your application.
Innodb Compression – Innodb Compression reduces amount of data written to tablespace hence can increase life time. Note in some MySQL versions compression increases amount of space written to Innodb Log Files, in this case it is good combined with Innodb Log Files being on Hard drive.
And the summary:
Depending on your application, inexpensive SSDs might be good fit for your workload, though enterprise-grade flash storage might be a good choice for very write-intensive applications even if it costs a lot more due to a much higher endurance capacity. Make sure you understand how many writes your application does and keep thinking about how to optimize your system to do less writes with SSDs – as reducing writes is a key to a longer SSD lifetime and as such lowers costs.
Back to reality (i.e., some ISD requirements or even Corps with outsourced IT who want nothing to do with Moodle).
Advantages might be negated IF there were such hard requirements as ALL DB's to be on a class B range of private IP address not of the same range/blocks/etc. of web server hosting moodle code. Same applies for location of sub-directories of moodledata. Case in point ... and you've already experimented with one ... sessions directory in moodledata ... to me that needs to be on web server ... filedir (if requirement was all servers storing files [of any sort] on yet another private IP block not of same range as web server.
Do know for a fact ... contents of moodledata/temp/backup can be manually erased as long as there is not a process using that space.
Sometimes one goes lickety-split ... to the next bottleneck. :\
'spirit of sharing', Ken
I can imagine a PCIe based drive (FusionIO) performing much faster than a standard SATA based SSD, just looking into the near future when Intel's Optane 3D Xpoint technology will come to data centers, with 7x performance of standard SSD and lifespan of traditional hard drive, only bottleneck would be PCIe lanes.
And if DB is on a remote DB server? If moodledata/filedir is on a NFS (pick your poison FS) remote server?
What comes before application in the ISO model?
Just askin'? ;)
'spirit of sharing', Ken
of course fiber then, but it barely will be needing a remote server with that much of through put.
I believe the way overall technology is moving forward the need of distributed computing only remains as redundancy requirement, else 1 CPU having 64 logical cores (Talking about AMD's Zen platform) would be enough to cater everything in one box.
Usman -
Just checking on your thoughts about deleting the directories I mentioned. Is there ever anything in the /cache or /localcache directories that will cause a problem if it is deleted?
Thanks,
Floyd
Floyd, not that I am aware of cache and localcache crucial to functioning of moodle, in fact , I get them deleted in cases where cache's get corrupted (blank screens on new installations), and as soon as browser is refreshed, they pop up right away.
So shouldn't be issue, but just in case, if you compress them, they could be of little size (this is solid assumption)
Just took a risk by testing on an old moodle site by deleting both cache and localcache to see if something really breaks, for inside both folders there were sub-folders related to theme and few javasript/css files.
on page refresh it did took few seconds to re-build the cache folders, but nothing was broken on the site itself.
As for sessions directory, it was loaded with session keys files, and deleting and refreshing page whilst sessions folder was recreated automatically, had no effect on installation at all - but if the site is making use of stats plug-in, they may lose stats temporarily until new data is filled up- though compression ratio on files inside the folder was 60%, so you can expect smaller folder size when compressed (extraction will take time).
Hello Usman,
Am working Moodle 3.0 and my Bluehost account says server is too full. Have generated a report that shows the biggest server hogs. Can i delete safely from moodledata/cache/cachestore_file/default_application/core_string/en_-cache?
Jeanne,
by BlueHost account getting full, I assume you're referring to the number of files? as they dont actually limit the file size, but there is limitation on number of files that one can keep on shared servers, which last time I know was 50,000 (but they safely allow up to 100,000)
Of course you can delete CACHE, LOCALCACHE and as well SESSION folder, as that will have tiny txt files sitting there and doing absolutely nothing, but do take care to delete all this when no one is using moodle (or put in maintenance mode), as deleting session folder at least, you'll be logged off from moodle as well.
Deleting CACHE/LOCALCAHCE folder will take a few seconds for your site to load as it will re-build cache, but deleting SESSION folder will have no effect apart you getting logged out of your session, so do try deleting SESSION first, and see if the number of files have came down, if not then proceed with CACHE folders, but do remember, for BlueHost, it takes some 4 hours to refresh the file count.
You can always have a look inside (may be mentioned as file count in BH's panel), and see which folder/folders are holding most number of files as well.