Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Number of replies: 98
I'm really pleased to finally be able to release a much requested extension to Moodle 1.9 which we developed for Cumbria and Lancashire Education Online to provide a basic way to export data from School MIS systems and use this in Moodle.

This extension is now in CVS along with export reports for Capita SIMS & Serco CMIS.

This tool was designed specifically to help ease schools administration of
users and user creation from their MIS system and comes in two parts:

1) User creation from MIS System report - this will generate manual accounts
from user details exported from an MIS system. It is specifically aimed at
those schools which don't have the skills to manipulate the data from their MIS
system themselves and NOT to be a flexible replacement for the current user
upload functions in moodle. For example it disables email addresses and doesn't
take real addresses. It takes the pain away for schools which were previously
manually creating every single user account.


2) Bulk Class Enrollment from MIS System report - this will allow roles to be
assigned 'a class at a time' based on class groups generated from an MIS
system. This means that teachers setup their moodle course, and then choose
'class 3a, 3b and 3c' rather than picking each individual pupil.

There are some known limitations:

1) No integration with moodle groups
2) Role assignments are not dynamic (e.g. when class 2A changes in MIS system,
it isn't reflected in the class which has already been assigned moodle).
3) Importing data from MIS system is a manual process

The next step to address this is, is to work on some Moodle core
features for 'class based enrolments'. This implementation was designed to be
intentionally simple, knowing that good moodle core support for 'class based
enrolment' is essential to take this further.

I'd love to here any comments you have on this extension and on moodle core support for 'enrollment groups' so please leave your comments. I'm going to be at the San Francisco Moot next week, so it'd be great to talk to you there too!

Dan Poltawski
Lancaster University Network Services Ltd
Average of ratings: Useful (1)
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Attaching screenshot which demonstrates how the bulk enrolment looks
Attachment mis_screenshot.png
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Berry -
Thanks for your efforts, Dan. Now all I need to do is get CVS to work through LancsNGFL/CLEO wink

Dave Berry
In reply to David Berry

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Hi Dave,

I think the contrib infrastructure should generate a zip file to download automatically at the next generation run at:

http://download.moodle.org/patches/cleo_mis_tool.zip

Though hopefully anthony can clarify this (I was on tight time constraints to upload this so didn't get a chance to consult him sad )
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Oh and if you can't wait, the UK CVS mirror is kindly hosted in CLEO, so hopefully you should be able to access this via uk.cvs.moodle.org
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Anthony Borrow -
Picture of Core developers Picture of Plugin developers Picture of Testers
Dan - Actually, it is better to use http://download.moodle.org/download.php/patches/cleo_mis_tool.zip as it helps us to gather statistical information on the downloads. I just tried to see if the file had been created and the download works fine. Peace - Anthony
In reply to Anthony Borrow

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Marc Grober -
I didn't find anything in M&P on this. Shouldn't there be an M&P listing if its in the CVS??
In reply to Anthony Borrow

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Berry -
Got the zip. Thanks to all of you and now I can play!

Dave Berry
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Paul Taylor -
Hi Dan:

I'm not sure if this is just me, but this has happened now on a few sites. It seems that the assign.html messes up the interface for the Assign System Roles. If you try to work with it once these files have been patched, the page is blank. Debug gave me this:

Parse error: syntax error, unexpected T_IF in /var/www/moodle-19/admin/roles/assign.html on line 109

Once I put the old assign.html file back, it worked again.

Paul
In reply to Paul Taylor

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Did the patch apply cleanly? It sounds like it didn't?
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dale Jones -
Just noticed this today, Dan, and I can't wait to try it. Well done for the great work.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Paul Garratt -

I am using this tool and one of the schools wants the usernames to be forename.surname.  How do I stop it from removing the fullstop when they are created?

Thank you

In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Adrian Stock -
Hi Dan,

That sounds wonderful. Just what I have been begging for for the last two days on another forum, and you released the tool just in time.

Where can I get it? Or does it sit on a server? Which URL?

Do you have simple instructions to go with it? The sort of thing you give your school moodle administrators to enable them to carry out the job.

Could you send me a copy please; perhaps a link in this forum or by email?

Thanks, and enjoy your trip to San Francisco. You have deserved it.

Best wishes,

Adrian
In reply to Adrian Stock

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Hi Adrian,

Similar to as I said to Dave, I think it should appear automagically as a zip file here: http://download.moodle.org/patches/cleo_mis_tool.zip within the next day or so.

In the zip file, there is a readme file with instructions on how to install this. And a link to current instructions which are available on the CLEO website for usage.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Matt Gibson -
Dan, you're a legend!

I'd love to get going with this, but I'm going to need to customise it a bit as our current setup uses SIMS to populate our active directory and we then read moodle accounts from there. Useful as we can then use NTLM SSO.

The really useful part would be to get the class lists to refer to the existing ldap accounts - how big a hack is needed would you say?

Also, if the class s updated in SIMS, does the change get reflected in the course automatically, or is it just a one off upload that needs to be manually managed?
In reply to Matt Gibson

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
&gt The really useful part would be to get the class lists to refer to the existing ldap accounts - how big a hack is needed would you say?


Not a hack at all, you can happily use that feature without the user upload feature and don't need to do anything to allow this (other than turning off the 'have you uploaded your users yet' warning).

The tricky part is getting the SIMS enrolment number into the moodle user account idnumber field (which is required for linking the two). I don't know whether the NTLM plugin already uses this field for something, or if it allows propagation of the idnumber field from AD - but in an ideal world you'd put the sims enrolment number in an AD field and that would sync across.

&gt Also, if the class s updated in SIMS, does the change get reflected in the course automatically, or is it just a one off upload that needs to be manually managed?


Sadly it doesn't get reflected automatically, its simply a one time operation (which just grabs all the current users in a course and assigns them so you don't have to).

This is obviously the way forward, it really needs core support to allow this to be done robustly.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Matt Gibson -
Great news! I know that NTLM/LDAP uses the username as the main unique reference so getting the SIMS number will mean making an extra field in the users table. I'm going to have to find out more about how our school deals with duplicate names, but the hack I was thinking of was making this field and referencing it instead of using the id number.

Have you got a 'put this in core' tracker issue yet?
In reply to Matt Gibson

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
> Have you got a 'put this in core' tracker issue yet?


I'm afraid its not quite that simple. (if it was i'd already have done it). In core we need to establish a central enrolment mechanism which factiitates this and isnt limited by it
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
Is this made more difficult by the fact that different schools create computer user accounts in different ways? Some base it on admission numbers, other on names, etc. This could cause issues for linking in with AD/LDAP authentication and I would be interested in seeing how this is being tackled. A standardised username?
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Nope its not an issue for moodle, we have an idnumber field in moodle for just this linking purpose and the current authentication plugins and enrollment plugins work fine with this.

The weakness is that in a lot of schools we can't automatically populate enrollments based on incoming MIS data, and instead need to put the power in teachers hands to be the controller of this. So like in this extension, a 'site wide class groups' concept.

My vision is to that the current enrollment plugins could be extended to do two things, either automatically update course records like now, or update 'class groups'. Changes to class groups would then automatically propergate to moodle courses which have enrolled this 'group'. (Teacher says 'I want to enroll class MATHS2A MATHS2B and MATHS2D to this course' rather than 'johnny, babra, dorris etc').

Lots of thought needs to be put into this infrastructure though, e.g. how do we interact with the current course based groups, how do we build these bulk enrolment changes in a scallable way, how do we interact with the current enrollment plugins (I don't think we need two sets of plugins for this).

Good core support for this is the way forward (hopefully i'll be able to work on this at some stage in the future). KISS was the principle behind this plugin, it tries to keep itself to itself and is a 'stepping block' to save time while we come up with a full solution.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Berry -
Sorry Dan, but I am still having some problems. The turning off of the "have you uploaded your users?" doesn't appear to be straightforward - or at least not to me sad

If I ignore this there is then an error message about "resetting courses". After bypassing this it appears to populate the courses but, at the end, there is nothing.

I will get there in the end but any pointers gratefully received.

Thanks

Dave Berry
In reply to David Berry

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Hmm, not sure what the problem is there..

Here is a patch to remove the checklist warning (I guess this should probably be the default, the annoying warning was to guide people through it).
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Berry -
That patch works but the uploading of courses still fails. It looks like some new database tables are created when the users are uploaded with the MIS tool. These are then checked before the courses are created. Obviously, if I don't use the user upload, these tables won't exist and an error is produced. At least I think that is what is happening.

The webserver reports:-

Table 'moodle.mdl_cleo_mis_class_members' doesn't exist

DELETE FROM mdl_cleo_mis_class_members
  • line 1377 of lib/dmllib.php: call to debugging()
  • line 321 of local/cleo_mis/cleo_mis_lib.php: call to delete_records()
  • line 56 of local/cleo_mis/cleo_mis_uploadclass.php: call to cleo_mis_reset_classes()
in /var/www/lib/weblib.php on line 6879, referer:

I hope this makes sense to you.

Dave Berry
In reply to David Berry

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Hi Dave,

It does indeed sound like the database tables aren't being created - have you hit the yourmoodle/admin/ page? (If so did you get an 'Local database customisations version is now xxxx' message?

Dan
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Berry -
blushblushblush

My excuse is that we kept on clearing the users, resetting AD......

Sorry

Dave
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
After looking for the best way to apply this patch on a new WAMP box (XAMPP 1.6.6a) I pulled down patch.exe from gnuwin32.sourceforge.net but open running the patch command receive the error 'could not find file to patch at input line 3'.

Is there anything in particular in the patch I need to edit to point it at my moodle dir?
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
Ignore my stupidity ... in my rush to get this one I neglected to change to the right directory in my script ...

Instead I now get patch.exe terminating 'in an unusual way'. Perhaps I should move this of the test box running XP and onto a server 2003 box instead. Does anyone have a better way of patching rather than patch?
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
After some jiggery pokery it appears that the patches did actually work and I can import the users and classes.

I am now looking at fixing the account names (the import fails to consider that sixth form student may have joined the school after being elsewhere, not to mention students changing schools).

Likewise we need to sort out email accounts too to give them the right email address.

Any ideas appreciated.
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Berry -
I have just been looking at this. It looks like the easiest way would be to alter the cleo_mis_uploaduser.php file. At line 100 the void email address is created and then disabled on the next line. Fortunately, for my school, the first part of the student email address is also the same as the username. Changing CLEO_MIS_VOID_EMAILDOMAIN to the school domain "should" give the correct address. I have yet to try this so, not being a PHP programmer, I might be wrong.

In reply to David Berry

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
This looks good. I'll try and look at it tomorrow. I am not a PHP coder either and I might look around to see if someone can help recode a few lines to fit things in the way we need.
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
If I change the code to the following can anyone spot any major issues?

// create email address and enable
$tmpuser-> email = $tmpuser->firstname . surname . '@domain.com';
$tmpuser->emailstop = 0;


In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Matt Gibson -
$tmpuser-> email = $tmpuser->firstname . surname . '@domain.com';

would probably not work.

$tmpuser-> email = $tmpuser->firstname . $tmpuser->surname . '@domain.com';

probably will.
Average of ratings: Useful (1)
In reply to Matt Gibson

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
Instead of surname it is lastname, but yes ... it did work.

Many thanks.
In reply to David Berry

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
I've not looked at this code recently but it sounds sensible. Note that I designed this intentionally to:

a) Not fit all needs
b) Not create real email addresses

So I wouldn't expect this to work necessarily anyway by design.

Its quite interesting that you say the username style aligns (and we offered a few styles to aid this), but does it still align with the collision detection we do (e.g. there are two Joe Bloggs, do you use middlename or increment or what)? I offered the different username styles to help fit in rather than be the definitive solution.

Having said that, patches are welcome and i'll try to incorporate them if you have something which make sense!

Dan
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Berry -
We populate our AD with a CSV file which is output from SIMS - similar to the technique you use in your MIS upload tool. There is then some pre-processing using Excel macros which pick up the clashes (and other things). These are then altered manually. It may be an ugly solution but it works. Some similar pre-processing on the TSV file that your report generates would avoid climbing the PHP learning curve but still give a flexible solution.
In reply to David Berry

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Dave,

Similar to I said to Matt, I think if you can modify your Sims > AD import to include the sims enrolment in the AD schema somewhere, and then configure the moodle ldap plugin to map that field to idnumber then you can continue to use AD for auth!
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Geoff King -
We've tested this here. In the end I mapped the AD 'description' field to the idnumber in Moodle. I also chose to map Office to Registration Group as you import that information too.

However no synchronisation occurs. The MIS Upload Tool creates manual accounts rather than ldap accounts. This is fine for a standalone Moodle install but it means that Moodle doesn't query ldap at all for these accounts.
In reply to Geoff King

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Steve Ludlow -

In the cleo_mis_uploaduser.php file change the following line

$tmpuser->auth = 'manual';

to

$tmpuser->auth = 'ldap';

must be lower case

In reply to Steve Ludlow

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Geoff King -
Yes, I've figured that out. I also had to alter some other settings too. Thanks for the pointer though. smile
In reply to Geoff King

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Oh, I was envisaging using the LDAP auth plugin for user creation rather than the upload tool (like i've said a few times - its intentionally inflexible).

So you'd then configure the ldap plugin to map AD description to the idnumber field in moodle and then solely use the upload classes function of the MIS tool and not the upload users function.

Does that not work?
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Geoff King -
No because ldap user records are only created on user login. Thus unless everyone's logged in, you get incomplete class lists when you upload those.
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
Well, it looks like the patches didn't fully work as the bulk enrolment tool is not happy. (doesn't exist)

I will wait to try this again in a few days once I have built a server 2003 machine to host it on rather than the test XP box.
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
No. Moved it onto Server 2003 R2 running XAMPP 1.6 and I still cannot get patches to apply.

The patch tool has been downloaded from GNUWin32 and it is not a happy chappy.

Are there other who have happily used this patch tool to apply the CLEO patches?

I am reluctant to start putting in a *nix box to put Moodle on as none of the other staff have *nix skills and I won't be around.

Any advice appreciated.
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Geoff King -
Your patch files are in *nix text format (LF only). Convert them to DOS text format (CR/LF). You can either do this in *nix using the 'unix2dos' command or from windows by loading and resaving them in Wordpad.
Average of ratings: Useful (1)
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Steve Ludlow -

I've managed to apply the Cleo Patches on a Windows 2003 Server using GNUWin32.

Taking me all morning and an obscene amount of nicorette chewing gum to finally work out how to do it.

if you need a hand email me sludlow@chaucer.ac.uk

Average of ratings: Useful (1)
In reply to Steve Ludlow

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
Thanks for the help from both of you on this.

After applying the patches I have now found a .rej file for admin_roles_assign_php.patch.

***************
*** 199,204 ****

if ($frm = data_submitted()) {

if ($add and !empty($frm->addselect) and confirm_sesskey()) {

foreach ($frm->addselect as $adduser) {
--- 208,227 ----

if ($frm = data_submitted()) {

+ // CLEO MIS START
+ // For each group, add the list of users to the role assignent list
+ if(confirm_sesskey() and !empty($frm->cleo_mis_groups) ){
+ foreach ($frm->cleo_mis_groups as $cleo_class) {
+ if($cleo_class = clean_param($cleo_class, PARAM_INT)){
+ $cleo_group_users = get_records('cleo_mis_class_members', 'classid', $cleo_class);
+ foreach($cleo_group_users as $user_record){
+ $frm->addselect[] = $user_record->userid;
+ }
+ }
+ }
+ }
+ // CLEO MIS END
+
if ($add and !empty($frm->addselect) and confirm_sesskey()) {

foreach ($frm->addselect as $adduser) {


---------------

As a result of this I think that this is the cause of not showing the class lists when trying to enrol students on a course.

Anyone come across this before?
In reply to Tony Sheppard

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
Well, after a restart of the server it seems to be happy. Not done anything else ... and I am just glad it is working.

I can finally get on with other things ... such as training staff how to use it!

Thanks to all who helped, especially Steve.
In reply to Steve Ludlow

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Sorry for the pain - as I said in the other thread, i was reluctant to provide pre-patched files due to the security implications of me being slow to update the prepatched file against core.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Fountain -

Not sure if it is related, I'm still trying to track what has happened but has anyone else who has installed this lost the ability to add blocks to the front page?

Dave

In reply to David Fountain

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by David Fountain -
Scratch that last message it appears to be a coincidence, I've just tested again on a new installation and have got the blocks editing where it should be.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dark Nova -
Thanks for the great work on this tool!

From our point of view the only things missing are:

Usernames can be Admission Number
The created Username's are link to LDAP (as we are using a single sign-on)

In reply to Dark Nova

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Thanks for the feedback!

As i've said in other posts, the upload user feature was designed to be simple and not an exhaustive.

> Usernames can be Admission Number

It'd be really simple to achieve this with the new advanced upload user features in Moodle 1.9, have a play with that tool as it allows many possibilites for username generation.

> The created Username's are link to LDAP (as we are using a single sign-on)

If you can put the sims adminission number in LDAP then you can continue to use the tool by putting the adminission number in the idnumber field using the ldap auth plugin.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dark Nova -
Hi Dan,

I appreciate that the design is to be simply not exhaustive, i was only provide some feedback smile

The issue with native moodle bulk users uploader is: it doesn't remove users that aren't in the CSV, i would need to specify users in the bulk upload user CSV for deletion.... It would be far easier just to say "If the user isn't in the CSV, Remove User"

I am presuming that your import does this?

Currently we are using the SIMS importer by David Willington: http://moodle.org/mod/forum/discuss.php?d=15012

Which automatically import’s the users as an LDAP user rather than your Sims Import which imports it as a Manual user! Also it any users who aren’t in the CSV are removed.

Is this something you’ll be looking to implement in the Future?

All the same, thanks for a great import I know people have been waiting for this one for a while. All the best.

DK

In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Mark Drabble -
My user accounts are made up from lastname-initial and sims admission number (eg Student-A1234) I can changed the coding to generate StudentA1234 but I am struggling to get the "-" symbol in the user name.

I have never done any coding before so I guessing as to some of the changes.

I have change the coding to add $dash = "-" and then added it to the section that creates the user names b

$dash = "-";
$idnumber = $user['idnumber'];

MIS_USERNAME_LAST_FIRSTINITIAL_IDNUMBER:
// student-a1234
$username = $lastname . $dash .$initial . $idnumber;
break;

When I import users, the $dash is being ignored. Can someone point me in the right direction please.

Many thanks

In reply to Mark Drabble

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Only alphanumeric characters are supported by default in moodle usernames, you need to enable the setting:

'Allow extended characters in username'

This is in: Admin > Security > Site Policies
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Mark Drabble -
Thanks Dan, I now have it working

In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Geoff King -
Got tripped up by a display bug in the user upload form. When you enable the advanced options and select lastname + first inital + yoe as your user generation scheme it doesn't work as advertised. I checked the code and there is nothing wrong with it, rather it seems that the descriptions for the option is transposed.

Heres a patch to fix it.


In reply to Geoff King

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Thanks Geoff!

I've applied this to CVS, so it should be fixed in the next zip build. You can also track changes at our public git repository:

http://git.luns.net.uk/?p=cleo_mis_tool.git;a=summary

(I've also created a branch there for to remove the 'have you uploaded users prompt).

Any more patches welcome!
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Geoff King -
Here you go then. These two patches are the sum total of what I needed to alter to make CLEO MIS Upload + LDAP + Moodle all work together. The first patch alters the user upload routine so that LDAP accounts are created. It also removes the password generatation and the force password change sections as these are not required for LDAP.

The second patch enables email for users, you must change CLEO_MIS_VOID_EMAILDOMAIN to reflect your email domain of course.

Note these were quick and dirty hacks so we could get our test system up and running, so there's no pretty interface in the user upload form to go with these changes.
In reply to Geoff King

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
I've not found time to integrate these patches properly yet. But i've created a branch and applied these to the repository.

Thanks again for sharing your efforts. Hopefully i'll get time to integrate this properly.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Geoff King -
Gah, that patch wasn't entirely right either. He's a patch. smile
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Michael Baron -
Why would i use this? seems like a long winded way of do running a report from sims, logging into moodle and going to user upload and locating the .csv
In reply to Michael Baron

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
The user upload function is designed for users who are not comfortable manipulating users themselves. It is not inteded to replace the user upload csv and designed for a niche of users. (Although i'd argue its at least the same amount of effort for both).

The class upload allows teachers to be given the power to chose groups of users at once easily at role assignment time which is the key advantage.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Michael Baron -
I downloaded the Zip file and looked at the instructions, Seemed quite confusing like half the instructions don't seem to work/i don't know what I'm doing. For example the patching stage of the installation completely eludes me. Any help/Advice?
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dark Nova -
Hi,

I've copied the patch.exe from patch-2.5.9-7-bin.zip and placed it in C:/windows/system32/

When i try to run the patch tool

D:\apache\www\moodle>patch -p1 < ./patches/lib_moodlelib_php.patch

I get the following error message:

patching file lib/moodlelib.php
Assertion failed: hunk, file ../patch-2.5.9-src/patch.c, line 354

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Can anyone advise me where I'm going wrong?

Cheers

DK
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Allan Morris -
Hi Dan.

Very nice piece of code. Has come in very useful in our school importing around 1200 students from SIMS into class groups. Only problems found so far are mine as I changed the account name generation code to the one we use and got it wrong for about 5% of students sad - will try harder next time.

We use groups for some courses so if you ever consider additional features I'd vote for this one being integrated somehow. Currently I have it working by assigning one class at a time to the course then adding those new ones to a group and then the next one ...

Thanks for releasing it.

Allan.
Cheadle Hulme HIgh School
Stockport.

In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Michael Baron -
Anyone else having problems installing this? anyone who isnt any advice please?
In reply to Michael Baron

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Allan Morris -
Hi Michael. What problems are you having precisely? How far did you get?
In reply to Allan Morris

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Michael Baron -
Not very far =[ mainly due to my lack of use with moodle. Ive moved over the folders and thats about it. The patch commands in the installation guide dont work and are completely forign to me. Im guessing theres more to it than what the readme.txt tells me (reinforced by the fact the dude in the post above) Any slightly more detailed instruction notes for people like me?
In reply to Michael Baron

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Hi Michael,

I assume the problem which you are having is applying the patches on windows.

I'm not familiar with windows patch tools (I have never tried to apply a patch on windows) so I can't offer much advice on how to do it.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Matthew Slatter -

I had a little fun installing this on a Windows 2003 box myself, but as soon as I found a Patch program for windows it went fine. 

There's a great little program on Source Forge that works a treat http://gnuwin32.sourceforge.net/packages/patch.htm.

Matthew

Network Manager

Marple Hall School

Stockport

In reply to Matthew Slatter

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Matthew,

Do you think you could offer some brief notes on how to do it on windows which I could add to the readme file to help others?

Dan
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Actually, you should probably refer people to Development:How_to_apply_a_patch, and Matthew, it would be really great if you could update that page, if necessary, in light of what you just learned.
In reply to Tim Hunt

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by andrew singleton -
Hi, 

I've copied the patch.exe from patch-2.5.9-7-bin.zip and placed it in C:/windows/system32/

When i try to run the patch tool

g:\inetpub\wwwroot\moodle>patch -p1 < ./patches/lib_moodlelib_php.patch

I get the following error message:

patching file lib/moodlelib.php
Assertion failed: hunk, file ../patch-2.5.9-src/patch.c, line 354

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Can anyone advise me where I'm going wrong?

Cheers

Andy
In reply to andrew singleton

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
What this means is that the version of moodlelib.php that you have, and the one that was used to create the patch, are sufficiently different that the patch program cannot work out what to do to apply the changes.

If the documentation page is to be believed, in this case, patch should create a file called something like lib/moodlelib.php.rej, which contains those changes that patch was not able to apply automatically, and you can try looking at that and seeing if you can edit moodlelib.php by hand to make equivalent changes.

Alternatively, you can try asking the person of created the original patch if they can create a new versoin of the patch for the latest Moodle 1.9.3+, or whichever version of Moodle it is you are trying to patch.

Yes another possible (sledgehammer to crack a nut) solution, would be to install Eclipse, which is the IDE I like. It has a nice graphfical tool for applying patches, that lets you fix merge problems manually.
In reply to Tim Hunt

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -

> Alternatively, you can try asking the person of created the original patch if they can create a new versoin of the patch for the latest Moodle 1.9.3+, or whichever version of Moodle it is you are trying to patch.


I've now done this, there was 113 line difference between the versions. My version of patch didn't abort at this.

Really, this should be updated to use events, but when this was originally written there wasn't an events framework - also i'm not sure the local/ hook supports events yet. If not Penny is working on it anyway smile
In reply to andrew singleton

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Mark Drabble -
Hi Andrew

I found that because its a windows box you may need to add --binary to the end of your command line, so you command line should look like g:\inetpub\wwwroot\moodle>patch -p1 < ./patches/lib_moodlelib_php.patch --binary

HTH
Mark

In reply to Mark Drabble

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Not sure why that should work as the problem was the number of lines difference since the patch was generated.

One way of getting it to work with the existing patch would be to tell patch to allow more 'fuzz' to account for the (113 lines) difference.

To do this you'd probably do something like:

g:\inetpub\wwwroot\moodle>patch -p1 -F 120 < ./patches/lib_moodlelib_php.patch
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by andrew singleton -

Hi Dan,

So for all the patches do i add the f 120 bit to the command i am trying to run?

thanks for looking at this

In reply to andrew singleton

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
If you download the new version of the package the new patch should fix the problems and you shouldn't need to. Are you having problems with the other patches?
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Phil Jones -
This looks great but I seem to have one problem...

I only want to use the upload classes function. I have installed everything and uploaded my classes from the sims report.

When I go to "list classes" it shows the relvent subject, e.g. Art, but when I click on this, no classes appear below.

I checked the databases and the users and classes are all in there.

Any ideas?

Phil
In reply to Phil Jones

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Phil Jones -
Ah, OK, found the problem- my 1.9.2+ version was missing some of the ajax/json libraries.

Thanks for your help- great tool
In reply to Phil Jones

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Not sure I helped! Apologies as I was away. I'm afraid it doesn't really degrade without JS. It did previously by showing all classes intiially - that behaviour should probably be reinstated..
In reply to Tim Hunt

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Thanks Tim, i'd not come across that page before, i've now linked the readme file to it.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Ciaran Gilligan -
Thanks for developing this tool. I used it successfully last year but am now puzzling over what I need to do in preparation for next year. Will it be a case of deleting all my users and creating fresh user accounts for September?

If there's anyone else in the same position I'd be glad to hear what your plan is...

Many thanks
In reply to Ciaran Gilligan

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Andi York -
I'd also like to know this as deleting all the user accounts could be quite problematic and time consuming every year.
In reply to Andi York

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Tony Sheppard -
Hi Andi

The IDnumber field should be the admission number if I remember rightly so that if you run the import again with the updated user details (including year group, classes, etc) should just update.

I would suggest Stephen takes a snapshot of the server and test it as a virtual machine. You will have to manually edit the report from SIMS to change the year groups / classes for next year though until year end is done.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Paul Garratt -

Can anyone point me in the right direct on how to add more users to a registration group?

Example:  A pupil joins half way though the year, would I need to run the reports again to get the info?

Thanks

In reply to Paul Garratt

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Yes, if you re-run the report the groups will reflect the new registration group status.

(Note however that any groups 'assigned' will not be dynamically updated - this is by design)
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Paul Garratt -

Hi Dan

Can you advise me on creation of users.  I am trying to create student users with a . in their name but it removes this when we run your report.

So example joe.bloggs

Can you advise me on what I need to change to stop it removing the full stop

Cheers

In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Jon Williams -
Hi,

I am just trying to get my head around this, i have run my two sims reports, but am wondering can i specify the students passwords, we want to keep them the same as another account ie date of birth and initials

i.e. 130875JW

is this possible?

Also for the username can we set this to year of entry, 4 letters of surname and 3 letters of firstname

i.e. 05willjon (2005,williams,jonathan)

thanks

J


In reply to Jon Williams

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
The report generated does not export date of birth, so couldn't with it as it stands, but it would be possible to add.

Same with the username style (perfectly possible, but not implemented with this tool) - you could almost certainly achieve this with moodle's standard user import tool.
In reply to Dan Poltawski

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Paul Clarke -
Hi all

I am new to moodle, I am working as the techincian at St Thomas More in Crewe. We have looked at options such as FROG, nice but expensive. We currently have uniservity which our staff find difficult to use. Our Senior staff feel that we need to look in a different way.

I have set up a moodle and have set up all the user accounts, teachers and pupils. Then I notice this thread as we are obviously going to need to integrate with SIMS to provide the access for parents/carers. So if I use this utility will I need to dump all the users I have imported? I am not using LDAP at the moment but will switch if it is the only way to get all of this working.

The problems I have is that we are not on a flat network. SIMS is on one network and the Curric is another. There is a trust between the two domains so staff can log onto SIMS using their curric usernames and passwords. What I do not want to do is create everybodys Curriculum usernames in AD again. I have staff that would go into melt down!!

I am pretty confident in trying any scripts etc but I thought I would voice a few concerns first.

How close to getting the BECTA access for parents are with with SIMS/Moodle? Is it a viable solution to providing "upto date" information such as behaviour and attendance?

I don't want to go down a path and spend a long time trying to get this to work just to hit a brick wall and then explain that we may need to go with FROG. Does anybody know of any funding grants for this? BECTA want us to do it, but who will pay for it?

Paul

In reply to Paul Clarke

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Ballance -
If you are thinking about meeting the parental access that the government have specified with Moodle then I wouldn't waste your time. Capita/Sims have their system pretty tightly locked down and the Moodle developers are all working hard on getting Moodle 2.0 to see the light of day.

Moodle 1.9.x doesn't even support global groups out of the box. I have written some code that handles the Sims integration for the schools I work for, but I have had to hack the core to do this. I am working on some code to give parental access to Moodle based on Sims reports - but we are not going to even try and do the live behaviour and attendance stuff ourselves. Capita have it all stitched up and we're buying into their Sims Learning Gateway product. I've had a play and I don't really like it, but it does give you live access via the web to Sims data. It meets the legal criteria. It also looks like a horrible Sharepoint botch-job, but Capita are Microsoft all the way down the line.

Hope this might help,

dan
In reply to Dan Ballance

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Guy Thomas -
Picture of Core developers Picture of Plugin developers
I have worked on an open source solution for Facility CMIS and a commercial solution for SIMS.
You do NOT have to hack the core of Moodle to achieve a solid level of integration.
The commercial solution I have helped to develop provides the following without ANY hacks:

1) Account Management Based On SIMS data -
Automatic account generation / deletion for learners, employees and parents.

2) SIMS Teaching Group Linkage to Moodle Courses with synchronisation so that users are automatically enrolled / unenrolled and move between groups as the data changes in SIMS.
* This does not work on a Moodle course per SIMS teaching group basis - a very restrictive course provisioning model used by other integration tools. The integration tool I have helped develop enables you to link as many SIMS teaching groups to a single Moodle course, or as few SIMS teaching groups to as many Moodle courses as you wish.

3) MIS Portal for parents, learners and teachers / employees.
* Parents can access their child's attendance, timetable, Moolde calendar, course grades and soon assessment, behaviour and achievement too.
* Teachers can view their own timetable and other teachers timetables. Teachers can also view any learners details in the same portal provided to parents
* Learners can view their own details in the same way as their parents providing the system has been set up to let them do this.

4) On the road map, lots of new features including write backs to SIMS - fancy taking your registers in Moodle?

Oh, BTW - the solution I am involved with does NOT invalidate your end user licence agreement with Capita as it uses Capita approved APIs.
Anyone who is serious about write-back integration with SIMs should look at a commercial solution which uses approved APIs.


If you are interested in this product, please visit:

http://www.schoolanywhere.co.uk/sims-moodle-integration.php


In reply to Guy Thomas

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Liam Winterbottom -
Hi Guys,

My name is Liam, I represent the Moodle Partner Guy has mentioned regarding the SIMs integration into Moodle.

If you would like a free demonstration of how the software works please don't hesitate to call 01535 604026 or drop me a message & I will contact you.


Thanks in advance,

Liam Winterbottom
SchoolAnywhere Official Moodle Partner
WebAnywhere Limited

In reply to Liam Winterbottom

Re: Release of CLEO MIS Upload Tool, for integration with Capita SIMS/Serco CMIS

by Dan Poltawski -
Hi Liam,

I notice you have posted on a few threads about this. Do you have plans to release the code to this integration?

Dan