Demo site setup

Demo site setup

by Sarah Jane -
Number of replies: 24

Hi there, can anyone help me please I want to set up a demo site similar to the moodle demo, where people can go in have a play and the system restores itself after a period of time - I understand this has something to do with the cron instructions.

If anyone can send me these I would be very grateful.

I am working with moodle 2.0.8+

Sarah x

Average of ratings: Useful (2)
In reply to Sarah Jane

Re: Demo site setup

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Sarah

Such instructions are operating system specific. On the Unix command line they are simple and ideally suited to be made shell scripts which you can execute manually or timed.

Here are the steps necessary with some examples, partly from other discussions.

1. Install the demo site, say http://yoursite/demo, with whatever the Moodle version. Usually you need to do it only once.
Details: http://docs.moodle.org/en/Installing_Moodle

2. Develop the site to the state which you want to duplicate.

3. Take a site backup
Stop working on the demo site
$ cp -a /path/to/demo /path/to/backup ; save the Moodle code
$ cp -a /path/to/demodata /path/to/backup ; save the Moodle data
$ mysqldump --opt -u root -pSECRET demodb > /path/to/backup/demodb.dump ; save the database

4. Let people "play"

5. Restore the site to the "original" state
$ rm -rf /path/to/demo
$ cp -a /path/to/backup/demo /path/to
$ rm -rf /path/to/demodata
$ cp -a /path/to/backup/demodata /path/to
$ mysql -u root -pSECRET -e "drop database demodb"
$ mysql -u root -pSECRET -e "create database demodb default character set utf8 collate utf8_unicode_ci"
$ mysql -u root -pSECRET demodb < /path/to/backup/demodb.dump

Now if you put those commands in step 4 and 5 to a shell scripts, say backup.sh and restore.sh, you can run them as single commands whenever you like. If you are sure backup.sh should run periodically, 10 past every hour for example, you add that to the crontab of somebody (likely the root).
# crontab -e
10 * * * * /path/to/restore.sh

For details http://docs.moodle.org/en/Cron_with_Unix_or_Linux.

HTH
Average of ratings: Useful (5)
In reply to Visvanath Ratnaweera

Re: Demo site setup

by Sarah Jane -

Wow, this is going to take some understanding smile I really appreciate your reply and fingers crossed I will get this to work.

Many thanks x

In reply to Sarah Jane

Re: Demo site setup

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
No problem!

Once you reach the top, think of sending me some good Karma by rating the post.
wink

Some additional info:

- Related posts:
-- http://moodle.org/mod/forum/discuss.php?d=174304#p777496
-- http://moodle.org/mod/forum/discuss.php?d=200610#p877225
-- http://moodle.org/mod/forum/discuss.php?d=193117

- Typo: "put those commands in step 4 and 5 to a shell scripts" should read "put those commands in step 3 and 5 to shell scripts".
In reply to Visvanath Ratnaweera

Re: Demo site setup

by Colin Fraser -
Picture of Documentation writers Picture of Testers

For Windows users, there are a couple of options, but as I have never tried them, this is more of an intellectual exercise rather than a recommendation. You can download and install a program called CRONw, This does lots of things that the cron does in a linux environment. The other thing you can do is to use Task Manager and set up a schedule to link to a "task.php" file on your server which will do as you instruct it to. You should be able to get sufficient information to write a 'task.php" file even without a lot of php knowledge.

 

Average of ratings: Useful (1)
In reply to Colin Fraser

Re: Demo site setup

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Colin

If one could write a "GUI interface using visual basic to track the killers IP address CSI" in a hurry, this demo site setup must be breeze .
ROTFL
In reply to Visvanath Ratnaweera

Re: Demo site setup

by Sarah Jane -

Hello Visvanath

I have finally found the time to go through your instructions on creating a demo site as above.

My demo I want is be is http://myhub.pd-hub.com

I have created 2 files

One backup.sh - one restore.sh

The backup.sh have the following in the file.

$ cp -a /home/roberygp/public_html/myhub /public_html/myhub/backup ; save the Moodle code
$ cp -a /home/roberygp/public_html/myhub/myhubdata /public_html/myhub/backup ; save the Moodle data
$ mysqldump --opt -u root -pSECRET myhubdata > /public_html/myhub/backup/myhubdata.dump ; save the database

The restore.sh has$ rm -rf /home/roberygp//public_html/myhub
$ cp -a /home/roberygp//public_html/myhub/backup/myhub /path/to
$ rm -rf /home/roberygp//public_html/myhub/myhubdata
$ cp -a //home/roberygp//public_html/myhub/backup/myhubdata /public_html/myhub
$ mysql -u root -pSECRET -e "drop database myhubdata"
$ mysql -u root -pSECRET -e "create database myhubdata default character set utf8 collate utf8_unicode_ci"
$ mysql -u root -pSECRET myhubdata < /home/roberygp//public_html/myhub/backup/myhubdata.dump

The cron is set as

/public_html/myhub/restore.sh

But I am getting a message from the cron:

/bin/sh: -c: line 0: unexpected EOF while looking for matching `''
/bin/sh: -c: line 1: syntax error: unexpected end of file

Would you be able to guide me to where I have gone wrong please. x

In reply to Sarah Jane

Re: Demo site setup

by Guillermo Madero -

Hi Sarah,

I notice that you are using double slashes "//" in your restore.sh script.

Also, shouldn't the path "/public_html/myhub/backup" actually be "/home/roberygp/public_html/backup"? Assuming the latter path is the correct one, I think that your scripts should be set up as follows:

** backup.sh

$ cp -a /home/roberygp/public_html/myhub /home/roberygp/public_html/backup ; save the Moodle code
$ cp -a /home/roberygp/public_html/myhubdata /home/roberygp/public_html/backup ; save the Moodle data
$ mysqldump -u root -pSECRET myhubdata > /home/roberygp/public_html/backup/myhubdata.dump ; save the database

Notes:

1. I took off the "--opt" parameter, as it is on by default.

2. If your database is set up as a transactional db (InnoDB), as it should, the mysqldump command would be best defined like so:

$ mysqldump -u root -pSECRET --skip-lock-tables --single-transaction myhubdata > /home/roberygp/public_html/backup/myhubdata.dump ; save the database

** restore.sh

$ rm -rf /home/roberygp/public_html/myhub
$ cp -a /home/roberygp/public_html/backup/myhub /home/roberygp/public_html ; restore the Moodle code
$ rm -rf /home/roberygp/public_html/myhubdata
$ cp -a /home/roberygp/public_html/backup/myhubdata /home/roberygp/public_html ; restore the Moodle data
$ mysql -u root -pSECRET -e "drop database myhubdata"
$ mysql -u root -pSECRET -e "create database myhubdata default character set utf8 collate utf8_unicode_ci"
$ mysql -u root -pSECRET myhubdata < /home/roberygp/public_html/backup/myhubdata.dump ; restore the database

Cheers!

In reply to Guillermo Madero

Re: Demo site setup

by Sarah Jane -

Thank you so very much for your support and help.

I have read, tried to understand and changed the sh as advised.

Regarding our comment shouldn't the path "/public_html/myhub/backup" actually be "/home/roberygp/public_html/backup"?

the Myhub is a subdomain with the main domain being pd.hub.com would that make any difference to the script?

Thank you again xxx

 

p.s. I have just had a cron job come back with the same message

Cron <roberygp@md-16> cd '/home/roberygp//public_html/myhub/restore.sh

/bin/sh: -c: line 0: unexpected EOF while looking for matching `''
/bin/sh: -c: line 1: syntax error: unexpected end of file


Any advise reagdrign this? Thanksx

 

 

In reply to Sarah Jane

Re: Demo site setup

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Sarah

Yes, that is the approach I suggested: Have two scripts for backup and restore ready. By executing them in different order you can achieve many effects. Guillermo helped to iron out the bugs in your scripts, therefore I'm not going into details. Just two hints on how to debug:

1. During initial phase, you can copy the commands in those scripts line-by-line into
the console and execute them. Checking the output and its effect.

2. Later you can change the first of the script to "#!/bin/sh -x" (without the quotes) and
run them. You'll see a longer report as the output. You can also collect it to a file by "./scriptname > scriptname.out" for example.

About the cron job:
> The cron is set as
> /public_html/myhub/restore.sh
>
> But I am getting a message from the cron:
> /bin/sh: -c: line 0: unexpected EOF while looking for matching `''
> /bin/sh: -c: line 1: syntax error: unexpected end of file

Sound suspicious to me. A crontab looks like this:

# crontab -l
2,22,42 * * * * /usr/bin/php /path/to/moodle1/admin/cron.php > /dev/null
3,23,43 * * * * /usr/bin/php /path/to/moodle2/admin/cli/cron.php

moodle1 is a Moodle 1.x, the output goes straight to the wastebasket.
moodle2 is a 2.x, the output is sent to the owner of the crontab, in this case the 'root'.

Once you have it running, repeat the exercise in Windows!
smile

BTW, I had a similar idea, see "A testbed for Moodle" https://moodle.org/mod/forum/discuss.php?d=214084 in the "Testing and QA" forum.
In reply to Visvanath Ratnaweera

Re: Demo site setup

by Sarah Jane -

Hi Visvanath,

The one thing I love about working with moodle is that it really stretches someone like me and having you guys to guide us in the right direction is comforting.

I will do as you suggest and see what happens.

Thanks a million. x

In reply to Sarah Jane

Re: Demo site setup

by Ken Task -
Picture of Particularly helpful Moodlers

Qualifications for this response: not a programmer, but have scripted in linux OS.

Change this:

cd '/home/roberygp//public_html/myhub/restore.sh

to this:

cd /home/roberygp/public_html/myhub/; restore.sh

The "tick" after cd indicates what follows is a 'chunk' to be considered one piece.  There is no 'closing tick'.  In shell scripting, depending upon the shell being used, the ';' character is a way of putting more than one command on a single line and is a separator between commands.

In the original example of the script, the text following the ';' characters on a single line were intended to be escapes from execution and comments.  Again, depending upon the shell, // begins a comment and the text for the comment has to be on one line.  If, in your editor, comment text wraps to a new line, the script will fail execution at that point and one gets some interesting errors reported.

'spirit of sharing', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Demo site setup

by Sarah Jane -

Hi Ken,

wow, thanks for this. I did in fact edit my cron job has previously instructed and this is the message from cron job now.

 

$ rm -rf /home/roberygp/public_html/myhub
$ cp -a /home/roberygp/public_html/backup/myhub /home/roberygp/public_html ; restore the Moodle code
$ rm -rf /home/roberygp/public_html/myhubdata
$ cp -a /home/roberygp/public_html/backup/myhubdata /home/roberygp/public_html ; restore the Moodle data
$ mysql -u root -pSECRET -e "drop database myhubdata"
$ mysql -u root -pSECRET -e "create database myhubdata default character set utf8 collate utf8_unicode_ci"
$ mysql -u root -pSECRET myhubdata < /home/roberygp/public_html/backup/myhubdata.dump ; restore the database


Does this indicate that it is now working? Or should I change this to the information you have stated above. Please forgive my ingorance in this matter - I am learning quickly though!

Sarah x

In reply to Sarah Jane

Re: Demo site setup

by Ken Task -
Picture of Particularly helpful Moodlers

Remove the following 'comments':

; restore the Moodle code

; restore the database

from the script.  I'd also place a ';' at the end of each line.

Think the restore.sh script needs to be moved out of the myhub directory.  This command:

rm -rf /home/roberygp/public_html/myhub

is removing the directory and it's contents so that means it's also erasing the restore.sh script.  Probably not wise to leave such a script in 'public_html' as that is accessible via the web.  Think you might be able to execute such a script from /home/roberygp/.  Move the restore.sh script there.

And change the command given in cron job to:

/home/roberygp/restore.sh

This assumes that restore.sh has execute attributes - ie, is executable.  Creating a file with .sh extension doesn't automatically make it executable.  Example of attributes where a 'upnew' script is executable:

-rwx------ 1 user group    220 May  1  2009 upnew

The 'rwx' means readable by user, writable by user, executable by user.

If your shell is set to 'bash' then one could add a shebang atop the script:

#!/bin/bash
--- script here ---

When calling the script one has to use the full path to the script.
Script restore.sh located in /home/roberygp/ would have to be called with:

/home/roberygp/restore.sh

Some references for study of the 'cp' command as well as the 'rm':

http://linux.die.net/man/1/cp

http://linux.die.net/man/1/rm

WARNING: be very carefull with the 'rm -fr' command.  Stongly advise one has a complete backup and know how to restore before scripting.

'spirit of sharing', Ken

In reply to Ken Task

Re: Demo site setup

by Ken Task -
Picture of Particularly helpful Moodlers

Follow up to own posting ... hmmmmm ...

Have to ask ... are you simply copying and pasting without editing?

If so, that won't work for everything shared by others.  Example: the word 'SECRET' is your password.  Your passwords, if secure passwords, will probably have a mix of characters/symbols.  So that the shell doesn't get confused and thinks one of those characters is something it's supposed to act upon, enclose the password in between 'ticks'.

Example: 'M00dlE!'

Also, the user 'root' was used in the example.  More than likely, on a shared system, you will not be able to use the user 'root'.  You have to use the 'user' setup with your database that has ability to interact with the DB's for your account.

Finally, think it best to check with the system administrator/service provider about running shell scripts.  Could be a policy issue with provider.

'spirit of sharing', Ken

In reply to Ken Task

Re: Demo site setup

by Sarah Jane -

Ken, I found that now the site takes for ever to load - So I replace the cron job with your ainstructions above thinking this might help but now I get this message

/bin/sh: restore.sh: command not found


Sarah x

In reply to Sarah Jane

Re: Demo site setup

by Sarah Jane -

Ok, so with a little "playing" and a lot of reading I have etered the following cron job

/usr/local/bin/php /home/roberygp/public_html/myhub/restore.sh

with this I have received

$ rm -rf /home/roberygp/public_html/myhub
$ cp -a /home/roberygp/public_html/backup/myhub /home/roberygp/public_html ; restore the Moodle code
$ rm -rf /home/roberygp/public_html/myhubdata
$ cp -a /home/roberygp/public_html/backup/myhubdata /home/roberygp/public_html ; restore the Moodle data
$ mysql -u root -pSECRET -e "drop database myhubdata"
$ mysql -u root -pSECRET -e "create database myhubdata default character set utf8 collate utf8_unicode_ci"
$ mysql -u root -pSECRET myhubdata < /home/roberygp/public_html/backup/myhubdata.dump ; restore the database


So my question is is this working? should I not receive something saying done?

Have I got this completely wrong?

Sarah x

In reply to Sarah Jane

Re: Demo site setup

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Sarah

Now I understand the connection between your scripts and cron: You are sending them directly to cron without testing! Am I right?

Then you are conveniently ignoring a basic principle in scripting: "You can't automate any work which you can't do manually." Do them manually first, i.e. without cron.

There are two levels of doing this. See my hints (numbered 1 and 2) in the post https://moodle.org/mod/forum/discuss.php?d=201622#p936129 above.
In reply to Sarah Jane

Re: Demo site setup

by Guillermo Madero -

Oh, oh! Stopping to think (congrats!!) about what you said, "with this I have received", has made me realize... did you enter the command lines as shown here, that is, with the preceding dollar symbol? If so, apologies!

Taking into account Ken's recommendations and using the original path you had for the backups: /public_html/myhub/backup (instead of the one I thought was the correct one: /home/roberygp/public_html/backup), the content of each script should actually be as shown:

backup.sh

cp -a /home/roberygp/public_html/myhub /public_html/myhub/backup;
cp -a /home/roberygp/public_html/myhubdata /public_html/myhub/backup;
mysqldump -u root -pSECRET myhubdata > /public_html/myhub/backup/myhubdata.dump;

restore.sh

rm -rf /home/roberygp/public_html/myhub;
cp -a /public_html/myhub/backup/myhub /home/roberygp/public_html;
rm -rf /home/roberygp/public_html/myhubdata;
cp -a /public_html/myhub/backup/myhubdata /home/roberygp/public_html;
mysql -u root -pSECRET -e "drop database myhubdata";
mysql -u root -pSECRET -e "create database myhubdata default character set utf8 collate utf8_unicode_ci";
mysql -u root -pSECRET myhubdata < /public_html/myhub/backup/myhubdata.dump;

Before testing each script, please do follow Visvanath's advice ("how to debug") about testing each line manually and about the cron set up.

Average of ratings: Useful (1)
In reply to Guillermo Madero

Re: Demo site setup

by Sarah Jane -

Guys, you have all gone beyond just helping and I am truly indebted to you. I really though this would be a project I could do, but I am beginning to realize my limitations. I wonder if anyone could recommend someone to assist me in this - willing to pay. I have tried to request this from moodle partners and still waiting for their reply. Again thank you all - but I think I need to accept defeat on this one. Sarah x

p.s. Best bit of advice was to check with our hosters - it seems they do not currently support shell scripts! Oh well back to the drawing board! smile

In reply to Sarah Jane

Re: Demo site setup

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Sarah

That is bad news. But understandable in a way. Hosting providers are reluctuant to give full shell access, let alone shell scripts, simply because these things are two powerful - they can make and break! You need a "private" server for that: either a https://en.wikipedia.org/wiki/Virtual_private_server (VPS) or a https://en.wikipedia.org/wiki/Dedicated_hosting_service (dedicated server).

Can you explain what you plan to do from a user's point of view? Are you going to create and maintain the original site? Is it one site or many? What Moodle version(s)? Who are the test users: a closed group or will this be open for all visitors? Are the server specifications important? How many users do you expect working at the same time? Are you going to do load tests? Does the power have to be similar to your production server? For how long do you need the site?

I ask all these questions because I have these two ideas pending:
1. "A testbed for Moodle" https://moodle.org/mod/forum/discuss.php?d=214084
2. "The set of Moodle test courses (or sites)" https://moodle.org/mod/forum/discuss.php?d=213950

The server is a VPS just for testing. So no problem if it breaks. Depending on your plan I can reserve a site for you.
In reply to Visvanath Ratnaweera

Re: Demo site setup

by Sarah Jane -

Visvanath,

Thank you very much for your reply. I will go and checkout your ideas and get back to you. I feel a little bruised as I was determined to educate myself in this and now I have hit a brick wall. smile

Anyway I really am so very greatful for all your help and support.

Sarah x

In reply to Sarah Jane

Re: Demo site setup

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
I hope, that brick wall wasn't mine!
smile

Seriously, this is a common situation. The "theory" is boring and seemingly pointless - like those commands cd, cp, rm & Co. But in a pratically useful situation they come in combination with other things, in your case the commands mysql and mysqldump, and suddenly becomes demeaning.

Still, in this particular case, you are lucky. All these commands belong to a narrow subject area. One has a good chance of writing simple shell scripts like these if the individual commands (lines) are clear. This is a classic subject and has plenty of literature.

In fact, I offered an online course some time back then cancelled due to lack of interest: https://moodle.org/mod/forum/discuss.php?d=193117#p840816 (the whole thread is informative).