I am looking for instructions on setting up a staging environment where we can test and then deploy changes seemlessly to our production environment.
We are not a school, so we do not have the luxury to take our site down for several months during summer while we upgrade moodle, customize code, etc. We must do it in real time, and we must do it with confidence that our changes will not break the system, lose users or data, etc. So we need to test in a staging environment that mirrors the production environment in close to real time. When we are satisfied that testing is complete, we would like to be able to "press a button" and promote the staging version (code, database, moodledata) to production.
What is the current best practice?
Well, to clone a site, you need a copy of the code, the moodledata folder, and the database.
For copying moodledata, rsync is a good tool. For copying the database, you can either take a backup (perhaps you have one anyway as part of your disaster recovery) and restore it on another machine. Or you could look into any replication mechanisms provided by your dbms. For the code, well the whole point is that you want to test some new code.
I don't think you need a real-time mirror, at the Open University, the procedure is that we develope a bit, either custom development or merging in the latest Moodle release with our existing developments. Then 5 weeks before we are due to release we clone our live site onto our acceptance test server, and upgrade it to the new code. Then we may spend some time setting up some new courses to demonstrate any new features that need to be tested. Then we test and fix any bugs. Then we do the upgrade during our standard systems maintenance period, which means taking the live system down for about an hour one Tuesday morning.
For copying moodledata, rsync is a good tool. For copying the database, you can either take a backup (perhaps you have one anyway as part of your disaster recovery) and restore it on another machine. Or you could look into any replication mechanisms provided by your dbms. For the code, well the whole point is that you want to test some new code.
I don't think you need a real-time mirror, at the Open University, the procedure is that we develope a bit, either custom development or merging in the latest Moodle release with our existing developments. Then 5 weeks before we are due to release we clone our live site onto our acceptance test server, and upgrade it to the new code. Then we may spend some time setting up some new courses to demonstrate any new features that need to be tested. Then we test and fix any bugs. Then we do the upgrade during our standard systems maintenance period, which means taking the live system down for about an hour one Tuesday morning.
Tim,
I will attempt to summarize your suggestions. Since I am trying to automate the process with scripts, I have numbered steps.
Let's say the production site is:
and the test site is:
(test site should only be accessible to testers)
If I use CVS for version control, how does this change the steps?
I will attempt to summarize your suggestions. Since I am trying to automate the process with scripts, I have numbered steps.
Let's say the production site is:
and the test site is:
(test site should only be accessible to testers)
- put production site into maintentance mode
- backup the production site: database, code, moodledata
- take the production site out of maintenance mode
- restore the database, code, and moodledata to the test site
- edit /var/www/test/config.php, changing site-dependent CFG variables
- run http://mysite.com/test/admin/replace.php to fix URLs
- add new features, fix bugs, update Moodle to newer version, etc.
- test, test, test
- when ready to deploy...
- put production site into maintenance mode
- backup production site (database, code, moodledata) in case of disaster
- rename /var/www/prod as /var/www/prod.old
- copy code from /var/www/test to /var/www/prod
- edit /var/www/prod/config.php, changing site-dependent CFG variables
- run http://mysite.com/prod/admin/replace.php to fix URLs
- perform sanity checks
- take site out of maintenance mode
If I use CVS for version control, how does this change the steps?
6. run http://mysite.com/test/admin/replace.php to fix URLs
Well remembered, I always forget about this step, but it is hard to test without it.
I would add:
6½ Do a quick check on the restored site to make sure it seems to be working before you update it. (Just visit a few pages in your browser and make sure nothing is obviously broken.)
And I would split your point 7. to make it clearer.
7.1 Update the code in /var/www/test to the version you plan to deploy. (This might be a place where you use a CVS command.)
7.2 Go to http://mysite.com/test/admin/index.php to run the upgrade, watching carefully for any errors.
7.3 Configure your test courses to include examples of any new functionality you want the testers to cover.
12, 13, 14 is another place you might use a CVS command.
You need 14.5, go to http://mysite.com/prod/admin/index.php to run the upgrade, watching carefully for any errors.
15 is wrong! You have just upgraded your production database which already contained all the right URLs. There is no need to change them.
If you do the updates using CVS, then there are two cases to consider. One where you are directly using the moodle.org CVS server to update the files, and have any local customisations just as edits to those files. The more sophisitcated option is when you have your own source code repository (in which case look at SVN or GIT in preference to CVS) to manage your local customisations and tracking the official Moodle releases. Some of the details above will vary depending on which of those options you take, and I chose not to get into the details when typing the above. If you have not already done so, you might want to read CVS_for_Administrators.
Well remembered, I always forget about this step, but it is hard to test without it.
I would add:
6½ Do a quick check on the restored site to make sure it seems to be working before you update it. (Just visit a few pages in your browser and make sure nothing is obviously broken.)
And I would split your point 7. to make it clearer.
7.1 Update the code in /var/www/test to the version you plan to deploy. (This might be a place where you use a CVS command.)
7.2 Go to http://mysite.com/test/admin/index.php to run the upgrade, watching carefully for any errors.
7.3 Configure your test courses to include examples of any new functionality you want the testers to cover.
12, 13, 14 is another place you might use a CVS command.
You need 14.5, go to http://mysite.com/prod/admin/index.php to run the upgrade, watching carefully for any errors.
15 is wrong! You have just upgraded your production database which already contained all the right URLs. There is no need to change them.
If you do the updates using CVS, then there are two cases to consider. One where you are directly using the moodle.org CVS server to update the files, and have any local customisations just as edits to those files. The more sophisitcated option is when you have your own source code repository (in which case look at SVN or GIT in preference to CVS) to manage your local customisations and tracking the official Moodle releases. Some of the details above will vary depending on which of those options you take, and I chose not to get into the details when typing the above. If you have not already done so, you might want to read CVS_for_Administrators.
I don't think QA or release management is same thing as "upgrading." QA is at much higher level. It depends on knowing what means to install, upgrade, and migrate moodle (three separate documentation articles that sometimes bleed into each other) as well as version control (more articles). The documents tend to focus on low-level technical details, not high level issues.
For example, very first line of the Moodle migration document says
There may be times when you need to move your Moodle site from one server to another
This is completely idiotic statement. Of course you must move your Moodle site, and you must do it often (unless you are a thrill seeker who likes experimenting on the live site). You must have plan for it, and you must have scripts to automate such.
No wonder people getting confused. That is just my comment as outsider, looking at moodle documentation. Best way to "improve" such would be to toss away and rewrite, working from top-level use caeses down.
For example, very first line of the Moodle migration document says
There may be times when you need to move your Moodle site from one server to another
This is completely idiotic statement. Of course you must move your Moodle site, and you must do it often (unless you are a thrill seeker who likes experimenting on the live site). You must have plan for it, and you must have scripts to automate such.
No wonder people getting confused. That is just my comment as outsider, looking at moodle documentation. Best way to "improve" such would be to toss away and rewrite, working from top-level use caeses down.
You need 14.5, go to http://mysite.com/prod/admin/index.php to run the upgrade, watching carefully for any errors.
If you copy the code from /var/www/test in /var/www/prod, are you sure that going to http://mysite.com/prod/admin/index.php would run the upgrade ? How could Moodle detect that an upgrade is necessary ?
AFAIK, this is important, because you need the upgrade of the database, don't you ?
If you copy the code from /var/www/test in /var/www/prod, are you sure that going to http://mysite.com/prod/admin/index.php would run the upgrade ? How could Moodle detect that an upgrade is necessary ?
AFAIK, this is important, because you need the upgrade of the database, don't you ?