I have some more information on this. The organisation uses a iscsi SANs which will not allow or support NFS. All I need is a share on a on the SAN for our Moodle data folder, however I think that myself and the Windows Network team are talking different languages.
In the past I have requested a storage area on the network and mounted this area to the web server using an NFS mount
thanks in advance for any help
I use a share on a SAN and mount it with samba from a linux box. Seems to work perfectly.
Thanks for that, yes this is my thoughts. A SAN doesnt have to support NFS or SAMBA or CIFS. All I need is a share on the SAN and then I can use a variety of methods to mount that share on our Ubuntu machines
I am now being told that the SAN really presents itself as a server connected to the SAN rather than a share. Well I dont care if it presents itself as a dustbin as long as I get a shared folder on it.
Thanks for the heads up on this
Albert
Since you say it is an iSCSI SAN just have them create a LUN for you and mount using the iSCSI in Ubuntu..You will need to install the iSCSI packages then provide your IT dept with your iSCSI id then the volume will show up in Ubuntu then you can mount it and your web servers. iSCSI in Ubuntu, go down to the Initiator section since you do not need to setup the target; http://www.howtoforge.com/using-iscsi-on-ubuntu-10.04-initiator-and-target .
You can then use OCFS2 as the clustered file system for all the servers, For Ubuntu great guide here http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html
Jason
Wonderful many many thanks
Great
Albert
Ummmm
I have now been told if its a LUN then its not Iscsi rather a fibre channel?
Some further digging me thinks
Cheers
If it's a LUN served of fibre channel then it appears as a local scsi disk on your Linux server.
You might need to do something to rescan for the new disk then run a partitioning tool such as gparted to format the disk & mount it.
I don't know how you scan for disks on ubuntu. The command varies between distributions.
I think I have two choices here. Firstly I could get this iscsi Lun working and stick the Moodle data folder in it. And then have two sets of application code bases on my two web servers (these can be synchronized using a cron based rsync command periodically.
Or I could use OCFS2 as a clustered file system for both the Moodle-data and the Moodle application code base.
Although the later sounds rather tricky.
Too many choices
Me again. Is it OK to connect two web servers moodle-data folders to a single ISCSI Lun?
Cheers
I don't know where that last response came from, Even if it is Fibre Channel you can still mount on both web servers at the same time. Just use OCFS2 as the shared cluster file system. It will work perfectly fine and what it is meant for, concurrent read/write to a single lun shared by many servers. Up to 32 I believe. Here is full http://www.oracle.com/us/technologies/linux/025995.htm
You DO NOT put the /moodle core on the shared OCFS2 partition, only /moodledata. We use a main repo for the Moodle core files that is updated via git then rsync'd to the rest of the web servers. It rarely changes.
Also since it is a shared LUN on OCFS2 you DO NOT need to worry about putting MUC ( Moodle Universal Cache ) into memcache. Just use the standard disk cache in /moodledata. It will be as fast as you disks can spin.
Jason
OK this is still ongoing. Apparently a fibre channel LUN isnt available so my options are getting thin. Its an ISCSI San but isnt on a linux network but a windows network, does this make any difference?
I have details on how to create an iscsi lvm target on a sans but LVM is a linux technology and i am sure that the network team use some form of windows software to create iscsi targets. Does this make a difference/
Cheers
No it doesn't matter, can be any type of ISCSI Target, Your Linux OS will not care which it runs on.
All you need to do is give them your iSCSI iqn name like iqn.some.host.or.numbers to them to allow your host to access that LUN then add the IP Address of the iSCSI Target to your config like so;
sudo iscsiadm -m discovery -t st -p 10.10.10.1
You would then see the available LUNS, 10.10.10.1 would be your targets IP Address.
You then follow the rest of the initiator setup here;
http://www.howtoforge.com/using-iscsi-on-ubuntu-10.04-initiator-and-target
Jason
Hi Jason
Many Many thanks for your help on this matter, its the first time that I havent just been able to request an NFS share to do the job. I have found out this morning that the reason I cant discover the SAN using iSCSI initator is the fact that that the SAN is on a protected, isolated network.
So the only way of accessing a volume is to create a VM in the ESXi and mount the volume from there!
Which means another Ubuntu VM in the mix, which is just another VM that could crash. So in terms of reliability is not as good as a share or accessing an ISCSI LUN, and it also used valuable VW Sphere resources, processor and RAM. Too me this daft but may be the only way forward
I am currently looking at an Openfiler which may be the appliance that I need. However I think it is something that the network team would need to install and manage so im not sure that they would go for this
Many thanks indeed
Are your servers that you are going to host Moodle also on vSphere? First time that you had mentioned it. If so than you do not need to worry about a SAN.
Create 2 New VMs for Web Servers with basic 20GB Disks for the OS then add a new hard disk to one of the VMs in vSphere BUT must be configured correctly for sharing across multiple VMs.
So, Edit Settings > Add > Hard Disk > Create a new virtual disk > Choose size and choose "Thick Provision Eager Zeroed" > under "Virtual Device Node" choose a "SCSI (1:0)" then finish.
After the disk has been zero'd out go back into "Edit Settings" > click the new SCSI controller 1 and choose "Physical" Virtual disks can be shared between virtual machines on any server.
Edit the settings for the 2nd VM and choose add hard disk, choose existing, browse to where you created the disk for VM 1 , make sure you assign it the same SCSI (1:0).
You now have a virtual hard disk shared between your machines that you can then setup OCFS2 on.
If not, let me know what is available to you, like your setup so I can get a clear picture.
Jason
The more I think about this, the more I think that maybe the easiest option here would be to create 2 Vm web servers, have them as you suggested with 20GB OS disks and have one with a second disk for the moodledata folder and just share this disk using NFS. Would this work? I mean in terms of data consistency or is it simply that using any shared drives on VSphere would require a clustered file system.
Cheers
The easiest route since you are on Vsphere is a shared disk. You don't really want to go NFS route, you would degrade your I/O performance especially for the new versions of Moodle. Just as I mentioned before setup a 2nd disk to be shared between the two and setup OCFS2. It really is a quick and painless setup. We use it throughout for both Windows and Linux application servers. If you go NFS you would then have to setup a shared cache using memcache or similar requiring allot more work and configuration.
You can use this guide which is targeted for Windows Failover on Vsphere but is the same process for setting up the shared disk for any OS. http://pubs.vmware.com/vsphere-51/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-51-setup-mscs.pdf
Jason
Thanks for all your help but I think there is a little confusion here as the VM team have told me that this can only be achieved using a fibre channel, we only have ISCSI SAN.
And the doc in your link specifically mentions that "
Disks that are shared between nodes. A shared disk is required as a quorum disk. In a cluster of virtual
machines across physical hosts, the shared disk must be on a Fibre Channel (FC) SAN."
I think NFS is the only way forward for this
cheers for assistance, most helpful
Load balanced setup with a shared moodledata root. Always either had an NFS or GFS share provided for the moodledata root.
I think I might just ask for a 4th VM with a large disk, install Ubuntu on it and configure it as an NFS server and put the moodle data root on that. I was just trying to avoid having the 4th machine as this could go down so would be less reliable.
2 web servers
1 MySQL DB
1 NFS server
Cheers
More thought!
I dont think that will work either as I am still trying to share an ISCSI drive with two web servers! Doh
Albert
Even through the DOCS say that, you can still use it! We DO NOT have Fibre Chanel and still use a virtual shared disk in Vsphere. Our Vsphere environment is mainly a DAS Switched Environment with some 10GB ISCSI and it works just fine.
Here is a link to setting up OCFS2 using ISCSI to connect multiple hosts for concurrent read/write.
http://knowledgelayer.softlayer.com/procedure/connect-multiple-servers-single-iscsi-lun
Thanks,
Jason
Hi
Yes i have been doing some research and although both microsoft and VMware say no your solution does work and has done for other people as well. I already have OCFS2 set up on my servers, so just need a shared ISSCI disk configured and a target given to me
Many Many thanks for your help
Its much appreciated
Albert
Well we have tried to do this but we get an error:
An error was received from the ESX host while powering on VM MWeb2.
Cannot open the disk '/vmfs/volumes/MWeb1/MWeb1.vmdk' or one of the snapshot disks it depends on.
Failed to lock the file
Does the second VM need to have the SCSCI controller 1 set to "Physical, virtual disks can be shared between virtual machines on any server"
Cheers for everyones help
To answer my own question Yes IT Does
Thanks for everything Jason and Co
Albert
Ok Managed to get the shared disk setup on VSphere without any errors. With an additional step added to Jasons instructions i.e. making sure both VMs ISCSI controllers have set to share disks.
Both now Debian machines can see the drive if i fdisk -l
The drive is formatted to OCFS2 and OCFS2 and O2CB both start and work with no errors.
I can mount the first or the second machine to this drive but cannot mount them both as I get the error:
mount.ocfs2: I/O error on channel while trying to join the group
What have a done wrong here? The only issue i can think off is that following jasons instructions further up this post didnt work. The second VM throw a error on startup stating that there was a lock error.
So I altered the second VM's ISCSI contoller settings to also allow the sharing of disks and this worked with no errors in VSphere. Although I dont understand why as its both VM's sharing the same disk on the first VM. I do not share disks on the second VM.
I have folloed these instructions :http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html
However there is no mention of ISCSI targets? Do I need this step as well?
Cheers
It works !!!!
The cluster.conf has be the same for each server, this sounds obvious but the only difference that i had was each server had its own cluster details at the top of the cluster.conf and the second server under it. But no, just copy the first servers conf and paste it into the second
Thanks
Going to write a blog on this one
Yay! Im glad you have a cluster up and running now. I have done the shared disk so many times in Vsphere setting the second disk to shared must have slipped past me.
We actually use the shared disk on non-clustered Vsphere hosts also, just as long as each ESXi host has access to the same LUN.
Jason
