Upgrade from 2.6 to 2.9 and from mysql to mssql

Upgrade from 2.6 to 2.9 and from mysql to mssql

by Ingeborg (Inky) Chandler-Mertz -
Number of replies: 25

We needed to upgrade moodle but could not upgrade PHP. Installed a new windows server with new PHP 5.6. Installed clean Moodle which worked perfectly.

Exported Mysql db (tables) to Access and then imported them into the Moodle (mssql) db.

I get the following error:


Coding error detected, it must be fixed by a programmer: moodle_database::update_record_raw() id field must be specified.

Anybody been able to do an upgrade like this before and fixed this error. We tried many ways of converting MySql to MSSql and failed at every attempt except the Access import export method.


any help would be appreciated. Even if we have to redo everything using a different method.

Average of ratings: -
In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Have you made sure the php_mssql.dll is active in the php.ini file? I would suspect that this is a possible cause of an error. More likely the conversion itself was not properly done by Access. 

I am pretty sure you can do a dump via PHPMyAdmin into the mssql format...but you may have to look for a plugin to do that. From memory, I seem to recall a dump command that read something like mysqldump --compatability=mssql or something like that. There  is a number of additional elements that are useful in MySQL that are not used in mssql that might lead to errors. Aside from that, there was a tool, Server Migration Assistant from the Dark Side, that was supposed to do it cleanly, that might still be available. 

  

In reply to Colin Fraser

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Ingeborg (Inky) Chandler-Mertz -

Thanks. I will take a look into that a little deeper. The php_mssql.dll is active because we definitely had it all working before importing the data from the old database. So it has something to do with the table / table structure from the old db. Thank you for the reply.

In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Leon Stringer -
Picture of Core developers Picture of Particularly helpful Moodlers

Have you tried the Microsoft SQL Server Migration Assistant for MySQL (http://www.microsoft.com/en-us/download/details.aspx?id=42657)? The documentation was a bit of a long read but I eventually managed to work out the steps required.

If you're migrating servers just because of the PHP version, is it an option to just use MySQL on Microsoft Windows Server?

In reply to Leon Stringer

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Ingeborg (Inky) Chandler-Mertz -

We want to use MSSQL because we already use it and would like to incorporate the moodle backup into our existing plans. MySql was too cumbersome as a separate database system. Hopefully I don't have to import all the users and courses to redo everything. But if I have to that is what I may do. I will certainly look into the solution you found and see if that will work.  Thank you for finding that. We can use our old site until we find a solution but cannot seem to upgrade on the old server for some strange reason, hence the starting over on a new server.

In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi

You wrote:
>>> Exported Mysql db (tables) to Access and then imported them into the Moodle (mssql) db.

That not how it works. Moodle has its own DB abstraction layer which is decided during installation. See $CFG->dbtype in $moodle/config.php. You can't change it on-the-fly!

> MySql was too cumbersome as a separate database system.

Could you elaborate on that?

> We want to use MSSQL because we already use it

Doesn't seem to be a strong argument (for me).

> and would like to incorporate the moodle backup into our existing plans.

A Moodle https://docs.moodle.org/en/Site_backup contains more than the database. So you need new plans anyway.

> Hopefully I don't have to import all the users and courses to redo everything. But if I have to that is what I may do.

You are ready to pay a high price for a non-convincing reason. May be yours is a "Microsoft house" and people feel insecure if something is non-MS. Why you call this an _upgrade_ is a puzzle to me!
;-P
In reply to Visvanath Ratnaweera

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Ingeborg (Inky) Chandler-Mertz -

Thanks but your answers are not helpful at all. We already have at least 30 mssql db's running all our business applications. So moving Moodle to mssql is a very logical step for our organization. OS is not always the right solution. I also understand it is not the same db structure as mysql. I was hoping for answers that might help me convert the db's to the correct structure. That is all.. If it's not possible then i will just export and import all users and courses. Was hoping for a less time intensive procedure.

In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Ok, je non comprend pas..but try this... 

Moodle can be installed using the mssql database instead of MySQL, simple operation, point the installation to the created database, not the MySQL.

Create the database in mssql and then in the config.php file. point the $CFG-->dbhost to the new database. There may be a number of other things you will have to do to get Moodle and mssql talking together, as pointed out here, in this Moodle Doc

Using something like phpMyAdmin, do a dump of the database, and then reload the dump file as a flat file to the mssql, that should work. I really do not know enough about mssql to know how or if it will accept a flat file, but I am thinking this is pretty much a standard operation for all the databases I have used.

And, you are right, sometimes we forget that other people's priorities are not in our particular perceptions of the world, we should have been more helpful and less critical. blush 

In reply to Colin Fraser

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Ingeborg (Inky) Chandler-Mertz -

Thank you Colin. This is our best option right now. I will see if I can export differently out of mysql. That has been our main struggle . i got the new moodle install, on the new server, working perfectly with mssql. It's getting my old data that is the big issue. I will go look into it again. I appreciate you trying to find additional solutions for me that may work.  I don't give up easily. Thanks again.

In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Just a thought.... if you extract a csv file for Excel out of the MySQL, then is it possible to then upload to the mssql? May save a lot of time... BUT, you would have to use phpMyAdmin to do it. 


In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Hi Inky, 

\beginrant  mmm "Don't underestimate the power of the Dark Side", it is incredibly difficult to escape once in its clutches. MySQL "too cumbersome"? I doubt it. Would be more honest to say, "We are used to mssql and don't really want to change."? More and more studies are coming out in favour of Open Source tools, propreital software licenses are just too restrictive and expensive. Who wants to keep paying a Microsoft Tax? As MS have been issuing certification for MS products, it becomes harder and harder for MS trained personnel to make the leap of imagination to step outside an MS based solution. But reality suggests that MS is not the only way. \endrant

I would suggest a separate server, with a flavour of Linux and using entirely open source tools as a beneficial environment to run your Moodle in. 

 

Average of ratings: Useful (1)
In reply to Colin Fraser

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Colin

The big picture, Free and Open Source Software (FOSS) vs. Proprietory Software is a very big one. Coming down to the OPs refusal of MySQL, I don't think that is because MySQL is FOSS. For one after the aquision by Sun/Oracle whether MySQL is Free is queationable and secondly, if no FOSS then the OP can not take Moodle!

Rather I too think it is a NO to change. What I wanted to point out was that the price will be uncomfortably high.
wink
In reply to Visvanath Ratnaweera

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Well, I recently saw a study (I can't find it now, of course..duh!) that compared costs, and the report suggested that OS was a more viable option financially than PS, but this article says it better than I ever could...

Average of ratings: Useful (1)
In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Luis de Vasconcelos -

How many courses do you have? If it's not too many then one option you could consider is to create a new clean installation of Moodle on the MSSQL database. Then you backup all your courses on the old MySQL site (incl. user data)  and then use those course backups to restore the courses and user data onto the new MSSQL based instance of Moodle.

In reply to Luis de Vasconcelos

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Ingeborg (Inky) Chandler-Mertz -

It's quite a few.  That is my last resort. But I will do it that way if I have to.

In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

But you really should not have to. 

One of the major discussion points of MoodleMootAu15 was this issue of bulk handling of courses. This same thing being one of the major development tasks requested.

There was a plugin developed that did the same job as phpMyAdmin that was easy to install and run called Moodle Adminer. This should allow you to export the database as a csv file in Excel format. If this works, that would make the task a lot easier...  AFAIK, there is no size limitation.

In reply to Leon Stringer

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Ingeborg (Inky) Chandler-Mertz -

I tried this tool and it also gave us errors and was not able to convert the databases. We have since given up and to save time have once again installed mysql and moved the DB over from the old server. Everything is working. We will be trying to redo this whole process in the future when we have more time. 

Thank you everyone for your patience, suggestions and time.

In reply to Ingeborg (Inky) Chandler-Mertz

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Sorry that you have not been successful yet, but please, do not feel disheartened by that. As I said above, the power of the Dark Side is quite often far too strong to counter easily. It should be a simple matter of transferring data on agreed sql standards, but it seems that they are not interested in any standard they do not set for everyone else. 

Please, Ingeborg, keep us posted as to your future success. You might be able to help others at some point. 

In reply to Colin Fraser

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Colin

Do you know that a Moodle site on MySQL can be transferred to MSSQL, or you believe it could be done or you need to do it one day and hope somebody will come up with a solution?
;-P
In reply to Visvanath Ratnaweera

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I've been here with some clients.  As database common standards are a myth, moving databases is a hellish job.  IMO the OP should stop moaning about MySQL and get on with it or start with a fresh installation and back/restore or recreate the courses.  The big lesson here is proper planning in the first place.

Also consider that Moodle is developed and tested on MySQL and PostgreSQL. While it should be fine on other supported DBs it wouldn't be my first choice.

Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

No Visvanath, I don't know if it can be done right now. What I am suggesting is that the internal mechanisms of each database do not need to be the same. (Overlooking Howard's comments, which is, admittedly, not a sensible thing to do..smile ..) IF they use a common, standardized SQL file output, which it should, then it doesn't matter how that file is read inside the environment and it won't matter which SQL compliant database is used. I suspect it would also make the Moodle data extraction layer a lot simpler, but, I really don't know. 

Using phpMyAdmin, I created a *.SQL file backup of my database. Opening that file and going over it fairly carefully, and if I had access to an old Borland Interbase server I used to use, I think I could restore that SQL file to that server, recreate that database despite the age of that server. I couldn't use it on my Moodle I think, but the database would be restored and accessible.  I don't have access to an MSSQL server anymore, so I can't test that out, but I don't see why it wouldn't work, unless there is something in MSSQL that does not read the SQL file.

There should be an easy mechanism to convert MySQL to MSSQL, I know it is easy to export to a CSV file from MySQL that can be opened in Excel, but I am not sure how to get from there to an MSSQL database. You might, if you have the time, do it one table at a time, I understand that currently  this is the only likely way it can be done, but I have no MSSQL server to practice this on. Obviously the tools developed so far have either fallen in disuse and not upgraded, or been ignored and not maintained. This is another instance where the Dark Side could really help, release the original source code as OS and let someone else work on them.

So I am hoping there is developed a straightforward way of doing it.... but I am not expecting it anytime soon.   

ASIDE: I loved using Interbase. Its small footprint and incredibly undemanding admin requirements makes it ideal for smaller users of Moodles. It is fully scalable, and offers separate encryption/db handling routines that allows devs to maintain strong security with minimal inputs well that is what I found anyway. Pity that it isn't used as one of the preferred db options by Moodle. 

EDIT: Just looking up what has happened to Interbase, it might still be usable as Firebird, an OS implementation originally forked in 2002. mmm long bow here and now way off topic...smile   


In reply to Colin Fraser

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

With respect (which means I am going to insult you, of course tongueout ), have you *tried* moving from one database back-end to another? I have, and it's a fiendish process. Something always seems to go wrong because there's no such thing as 'standardised SQL'. I can't in all conscience recommend it - especially if the OP has a system that works. 

Still, it's not my hours of getting stressed out wink

In reply to Howard Miller

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Well... yes I have but that was a long time ago, in a galaxy far, far away...no that was Star Wars...I had an Access db to Interbase to develop, but then when I started I realized I had made my own nightmare. No-one else wanted the job and no-one suggested a way of doing it. What I ended up doing was creating the database in Interbase, the tables, adding some legitimate data, making a backup of it in an SQL file. I then backed up the Access db, but as a text file I think it was, or a flat file of some description that could be read in my text editor, not Notepad,  then hand copied and pasted data  in the same structure as the SQL file, ran it and after about the 100th go, and a week later, managed to get it all in. (Or was it copy and pasting from the Access table viewer, the data one table at a time into the SQL file, not sure now.) I really got sick of dropping the database, then recreating it and after editing the SQL file, eliminating error after error, running it again. Repeat after repeat, but it still worked out quicker than rebuilding the entire database from scratch. It was when I started this process that I learned that what the Dark Side referred to as "SQL" was their version of it, not the well known and well abused SQL92 "standard".   

Don't get me wrong, DBase4 had its own rules as did something else I was using at the time, so none of them were really "compliant" with SQL92. It has all moved on since then, and with newer standards, I would have thought that more and more db servers would be closer and closer to that standard... mmm SQL2003 isn't it? (Not having followed the development of SQL since 2001.) Your comments, Howard, suggest that ego still rules decision making processes.

In reply to Colin Fraser

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I'm not sure if you are insulting me or not (your ego comment has rather gone over my head) tongueout

Here's the thing (and just my opinion), Moodle is developed and fully tested on MySQL and PostgreSQL. While it *should* work on other supported database systems you are increasing the risk. If you are planning on using non-core plugins then it's even more likely that they have never seen MS SQL, Oracle or whatever. So, with that in mind, I'm not a big fan of running on more marginal databases. 

This is the same argument as running Moodle on (say) Ubuntu (good idea) as opposed to Solaris (bad idea), for the simple reason that it's what everybody else does. Which in IT land conveys a big advantage - you know it's been well tested even if informally. 

To make matters worse, all databases have their oddnesses and quirks. Transferring from one database to another is non-trivial and Moodle does not support it. If you do that then you are on your own. It hasn't been tested. 

Having said all that, if you have a compelling reason to do something in the "outfield" then fair enough and good luck. But, in these forums, I have to tell people that they might be stocking up a bad day for themselves when I know that they are wink

In reply to Howard Miller

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Colin Fraser -
Picture of Documentation writers Picture of Testers

 surprise  I wouldn't even dare to try and wind you up Howard!!!! 

Nah.. the egos of those people who make the design decisions. They cannot accept an independent "standard" for things like "output" or "transactions" or "triggers". No-one asks them to stop developing, stop innovating, but it seems that it is just not possible for them to produce an SQL file that all SQL based databases can read and execute. How difficult would that be? 

Oh.. I liked using Interbase for a number of reasons, mainly because of its exceptionally low admin overhead and as HDD space was not cheap at the time, its inordinately small footprint. I do understand that it hid a backdoor issue, which was resolved,  but even so, it was still an easy tool to use in a variety of languages, C/C++, Delphi/Kylix, PHP, et al, maybe even Javascript. I don't know what its limits are anymore. Perhaps, if I had the "knowledge", I would every try to create a data extraction layer for Firebird, but I don't and do not have the time to learn it. 

A DBMS is like booze, everyone has their own favourite poison, (except for the self-righteous, like me who do not imbibe), and are very reluctant to try something new, unless they have no choice.     

Average of ratings: Useful (1)
In reply to Colin Fraser

Re: Upgrade from 2.6 to 2.9 and from mysql to mssql

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I have plenty of sympathy. A big database is a non-trivial thing. It takes quite a bit of experience to get your head around all the peculiarities of a new system. Well, you have to start somewhere and learning new stuff is good for the CV wink