Integration with Student Enrolement Systems

Integration with Student Enrolement Systems

by Paul Duff -
Number of replies: 14

Hi All

Pitching for ideas here.  We started using Moodle in the summer last year and are convinced that it was the right choice.  Martin's enthusiasm and drive have produced a superb product, which is maturing at an unprecidented rate - a fact that hasn't gone unrecognised by the 600 or so other sites that have adopted Moodle since we first started to evaluate.

As a medium sized FE college in the UK, we felt it unrealistic to undertake administration manually and wanted to dynamically pre-load Moodle with Teachers, Students, Courses and Enrolements.  We have relatively sound database skills but lacked any PHP coding experience, so we plumped for 'data stuffing' direct into the mySQL database.  Basically, we set up triggers on the Oracle based student enrolement system, and programmed a management machine to replicate enrolements, withdrawals, transfers, new courses etc directly into the Moodle database - this has worked well and we now have something like 20000 students on 2500 courses (only a fraction of which are active I hasten to add)

Here's the rub - 'data stuffing' can't really be a long term solution.  It's been fine for our first year, and with the advent of 1.2, we'll need to revisit the code to make sure that we don't break anything in the revised database structures.  We will amend to take account of groups, but that's it really.

What Moodle could really use to support scalability is some way of interfacing directly with external systems to replicate data real-time - or maybe it's already there and I've missed it.  The batch import scripts I've seen look good, but manual batching isn't generally timely enough.  I recall that this has come up in the forums from time to time, but haven't seen a solution.

In my opinion, 'data stuffing' is too external to the system (not to mention downright dangerous), and to be really successful, the solution needs to be part of Moodle itself.  I can think of two possibilities, although there are undoubtedly others:-

  1. Moodle gets taught how to interface with external databases directly - probably a coding nightmare!
  2. Possibly an extension to the batch process where a CRON job looks for a data-file to process every few minutes.  Some 'MiddleWare', either on the Moodle server or on an external machine could fill the data-file with transactions.

In either case, the process would ideally need to handle new users (teachers and students - probably from separate sources), new courses, add students to courses, assign teachers to courses, assign to groups, and probably be able unenrol/delete as well - basically, an external script calling the internal routines for pretty well eveything that can be done manually within Moodle. 

The questions that come to mind are:-

  • Does anyone know of a current solution?
  • Is a solution planned? and if so, anytime soon?
  • Is someone already working on this?
  • Martin might take this as a commission, but his time seems pretty well mapped out for the next few months.  If he didn't have time, would someone with the PHP coding skills like to take it on?  We'd certainly be willing to contribute in any way that's useful/helpful.

Any thoughts or comments anyone has would be extremely useful.

Ta
Paul

Average of ratings: -
In reply to Paul Duff

Re: Integration with Student Enrolement Systems

by Mark Burnet -

Somewhere in the answer, I would hope that SIF would bring the promise that it has so far not delivered.  I am hoping (dreaming) for an Open Source Zone Integration Server to help me knit my Student Information to my Library to my Moodle, etc. etc.

Sorry, no answers, just more questions.thoughtful

In reply to Mark Burnet

Re: Integration with Student Enrolement Systems

by Paul Duff -

I confess that hadn't occurred to me.  The Microsoft solution might be appealing, but for my money, their solutions are usually designed to lock you in, although in fairness, they have moved closer to standards based solutions recently.

PS I forgot to set Arial/Verdana font in this post and was amazed that it displayed that way regardless - bet everyone but me already knew that - cool!!

Paul

In reply to Paul Duff

Re: Integration with Student Enrolement Systems

by Mark Burnet -

It was my understanding the Microsft had backed away from the SIF project in the last few years, based just on that complaint (and from the other SIF members!).  I know it is still driven by commercial interests, but there is hope for a decent framework that would be open for independent development.  There was a commercial shootout recently, but the field (and the market) must be limited.  Isn't this type of narrow market that Open Source lives for?  Still dreamin'......

In reply to Mark Burnet

Re: Integration with Student Enrolement Systems

by Charlie Reisinger -
You may find this of interest:

http://openzis.tigris.org/

These folks are working on an open source zone integration server. I haven't used/deployed it but perhaps it holds future promise.

Charlie

In reply to Paul Duff

Re: Integration with Student Enrolement Systems

by David Delgado -
Answers below...


Here's the rub - 'data stuffing' can't really be a long term solution. It's been fine for our first year, and with the advent of 1.2, we'll need to revisit the code to make sure that we don't break anything in the revised database structures. We will amend to take account of groups, but that's it really.

Right now, you can use Moodle API to do those thihgs (try, ~/moodle/lib/datalib.php ). It is a much more stable thing to do that dealing directly with the database. In a near future, say "Early 2004", in Moodle 2.0 there will be a "New architecture for Enrolments with plugins (similar to the existing Authentication architecture) so that external systems (eg Student Records, Paypal, LDAP etc) can control student and teacher access within courses". That will help you a lot.


What Moodle could really use to support scalability is some way of interfacing directly with external systems to replicate data real-time - or maybe it's already there and I've missed it. The batch import scripts I've seen look good, but manual batching isn't generally timely enough.

There is a "text file" user import in Moodle already in Moodle 1.2. You can also use external authentication to authenticate users with an external database. You get that at admin/config/users/authentification menu.


Does anyone know of a current solution?

A much better solution, right now, is what I told you above: use Moodle API and External authentication facilities currently avaliable in Moodle.


Is a solution planned? and if so, anytime soon?

Teh "New architecture for Enrolments with plugins" I told you about.


Is someone already working on this?

Martin is working on this. smile If you need anything that is not included/planned in Moodle, you can just make it yourself (if so, please share it with us) or purchase its development in Moodle.com. As Martin says, "Not only will you get your new feature sooner, but it can be added to the main Moodle distribution, so it will always be maintained within Moodle in future versions." There are many features that have already been purchased this way, such as importing of quiz questions from external text files, backup/restore of courses and even groups.


By the way, there is also a standard for those kind of things: IMS Enterprise. Perhaps we should take a look at it.


Hope this all helps. smile
In reply to Paul Duff

Re: Integration with Student Enrolement Systems

by Scott Elliott -

You've probably already thought of this, but why not use external authorization with moodle and your oracle database?  If you already have the users setup in an oracle database, no need to duplicate and worry about synchronization.

I personally haven't used it, but it may be the solution you are looking for!

In reply to Scott Elliott

Re: Integration with Student Enrolement Systems

by Paul Duff -

Hi All

Thanks for the advice guys, but we already authenticate to an external database - Novell NDS via LDAP - that bit's fine, but works only for users.

The area I was trying to clarify was pre-making the users, the courses and then pre-enroling users onto the courses in Moodle.  For us, that information is aleady in NDS, so I guess another option I hadn't considers could be the population of the Moodle database from NDS via LDAP, which is capable of providing the link between the external source and the Moodle database - basically, NDS would just be an alternative source to Oracle per my original post.

In some respects, NDS could be a better source since it already contains the students, the courses and additionally, the staff, which wouldn't necessarily be recorded in a student enrolement system.  Unfortunately, the same problem exists - how to make Moodle replicate changes as they occur - not in a manual batch mode.

XML might be another option worth considering too.........

Paul

In reply to Paul Duff

Re: Integration with Student Enrolement Systems

by John Dell -
It's been quite a while since I've played with Novell NDS, but I believe there is a product from Novell called DirXML that gives you an XML view of your NDS tree.  There are all kinds of nifty features in there including updateable views so when something in the NDS tree changes, it fires off a message or whatever to update in the defined targets.

John
In reply to John Dell

Re: Integration with Student Enrolement Systems

by Paul Duff -

Agreed - same as the triggers in Oracle from the original post really.  It's the nifty process at the Moodle end which actions those triggers that I'm chasing...........currently we action those triggers externally, where something closer to Moodle would seem to be the logical goal.

Paul

In reply to Paul Duff

Re: Integration with Student Enrolement Systems

by John Dell -
Ok. I see your point. Sounds like you would like moodle to offer an XMLRPC service or something like for adding users to courses, etc. I believe there are a couple of different efforts to provide a web service library for PHP such as XML-RPC for PHP but I've only done light reading on that. Would be an interesting project.

John
In reply to John Dell

Re: Integration with Student Enrolement Systems

by Mark Burnet -

Here is Novell's idea of  how SIF and NDS will look from their perspective and product offerings:

http://www.novell.com/coolsolutions/education/features/a_sif_overview_ed.html

Looks complicated to me.thoughtful

In reply to Mark Burnet

Re: Integration with Student Enrolement Systems

by Paul Duff -

The Novell paper makes sense really - but we would need a SIF compliant agent for Moodle - and that's also a fairly logical requirement given the intended audience.  This does rather make my point that the 'middleware' needs to be much closer to Moodle.

SIF shows promise, but there's more than one way to skin a cat.........

Paul

In reply to Paul Duff

Re: Integration with Student Enrolement Systems

by Mark Berthelemy -
Hi all,

The IMS Enterprise specification provides exactly the transaction mechanism you're looking for. The bit is doesn't do it define the transport mechanism - ie. whether to use export/import or SOAP etc.

The specification also needs tying down to an application profile, as currently there are too many optional items to make it easy to plug & play. There is work going on in UK at Higher Education and schools level to try to tie down an application profile. I can give more details if anyone's interested.

All the best,

Mark
In reply to Mark Berthelemy

Re: Integration with Student Enrolement Systems

by Mark Roggenkamp -

What about OKI? I believe it picks up where the IMS Enterprise spec leaves off (i.e. defines the interfaces to the system whereas the IMS Enterprise spec defines the data format)

http://web.mit.edu/oki

Their focus in their examples is Java but the spec itself is somewhat language neutral.

Mark