Here at Glasgow Uni we are about to start spending quite a lot of time in looking at a web-services api in Moodle. Our initial efforts will be towards providing basic administration functions (eg, list,create,dealete => users, categories, courses etc.). We urgently need this capability ourselves.
The big implementation decision seems to be which underlying protocol to use. I guess this boils down to SOAP, XML-RPC or REST, or even perhaps more than one of these (like Amazon provides both SOAP and REST).
My thoughts are that in line with Moodle being reasonably simple to use and to develop that XML-RPC and REST would be the front runners. I think SOAP suffers from a rather considerable learning curve. Using REST in its purest form is also problematic - for example, it would be hard to get info about a course with (say) a GET to http://moodle.org/ws/course/3, it would probably look like http://moodle.org/ws/course/index.php?id=3 which is not quite in the spirit of REST.
Anyway, thoughts, decisions or direction very much appreciated. I hope to have some server code and example clients to spark debate and further development as soon as possible.
+1 for making it as RESTful as possible.
You might be able to get around the problem you mention with judicious use of mod_rewrite, although that would pose problems for those using IIS and those who can't easily use .htaccess for whatever reason.
I'm looking forward to seeing this!
You might be able to get around the problem you mention with judicious use of mod_rewrite, although that would pose problems for those using IIS and those who can't easily use .htaccess for whatever reason.
I'm looking forward to seeing this!
I don't think the use of mod_rewrite is an option. It really needs to work out of the tin. I remember only too well the fun and games we had with the 'slasharguments' feature in Apache 2 and IIS, and (mostly) that only needed one directive added. It potentially needs to work with PHP 4.1 too, which is another possible issue.
I just checked, and mod_rewrite is loaded by default on my OS X Moodle servers (at least, I don't remember enabling it). It's also turned on in a third-party web hosting service I use for some stuff. Buuuuut.... it's not turned on by default on my Debian Moodle server.
So, yeah, it looks like coverage is a little spotty (even side-stepping the IIS isue).
Shame, that. It'd be great to have nice pretty URLs for everything.
So, yeah, it looks like coverage is a little spotty (even side-stepping the IIS isue).
Shame, that. It'd be great to have nice pretty URLs for everything.
I've been doing a lot of thinking/reading and I'm coming around to the conclusion that I didn't think I would - SOAP.
Why?
* REST cannot really be implemented properly in PHP without getting involved in mod-rewrite which is not an option.
* XML-RPC is simple but perhaps too simple.
* SOAP understands character encoding properly - I'm not sure if this is a big issue, but it might be.
* SOAP understands WSDL
* The php libraries (eg, NuSOAP) are not really any harder to use for SOAP or XML-RPC for a client writer. I think *either* are probably easier to use/document than REST.
My other investigation is that of effective security for web services. The choice seems to be to use HTTP authentication or to pass the username/password with the request. Neither are amazingly elegant, the latter is probably easier to code and support and both would allow the use of SSL if required. However, any thoughts or better ways to do this most appreciated. We probably need to look at 'trusted' connections with key pairs too as an addition or alternative to username/password authentication.
Why?
* REST cannot really be implemented properly in PHP without getting involved in mod-rewrite which is not an option.
* XML-RPC is simple but perhaps too simple.
* SOAP understands character encoding properly - I'm not sure if this is a big issue, but it might be.
* SOAP understands WSDL
* The php libraries (eg, NuSOAP) are not really any harder to use for SOAP or XML-RPC for a client writer. I think *either* are probably easier to use/document than REST.
My other investigation is that of effective security for web services. The choice seems to be to use HTTP authentication or to pass the username/password with the request. Neither are amazingly elegant, the latter is probably easier to code and support and both would allow the use of SSL if required. However, any thoughts or better ways to do this most appreciated. We probably need to look at 'trusted' connections with key pairs too as an addition or alternative to username/password authentication.
So, have far have this SOAP project going?
I have been asked to build Java application that will do a lot of moodle's functionality. Rather than building the application from scratch, I think I will just use what Moodle has provide us, and scale from there.
Since I'm using Java, I'll need Moodle to be exposed as a web service. So I'd be happy to help you completing this project.
Rudi
I have been asked to build Java application that will do a lot of moodle's functionality. Rather than building the application from scratch, I think I will just use what Moodle has provide us, and scale from there.
Since I'm using Java, I'll need Moodle to be exposed as a web service. So I'd be happy to help you completing this project.
Rudi
I have been most reading/thinking/talking to people. However, I should now have some time to work on this actively and hope to have some basic functionality in the very near future. I have come around to the idea that the initial interface (at least) should be in XML-RPC. What sort of functionality are you looking for Moodle to expose in the first instance? My initial efforts were going to be concentrated on administrative functions - eg, users, categories, courses etc.
Yes, remote control of Moodle will be useful.
However, here is a specific example that I know people want: exposing the sort of information we have in Moodle blocks on a user-by-user basis.
For example, call the web service with username authentication info, and get back a list of "My courses" or "Upcoming events" (either as a HMTL snippet or an XML structure).
These simple things allows portal-like things to mix bits of Moodle in them.
However, here is a specific example that I know people want: exposing the sort of information we have in Moodle blocks on a user-by-user basis.
For example, call the web service with username authentication info, and get back a list of "My courses" or "Upcoming events" (either as a HMTL snippet or an XML structure).
These simple things allows portal-like things to mix bits of Moodle in them.
Yes - very good point.
I had already been asked about calendar events, in the same vein. Most of this will be reasonably easy to do.
I had already been asked about calendar events, in the same vein. Most of this will be reasonably easy to do.
Sorry for not engaging here earlier, I missed the discussion being started.
This is an important issue, one I want to see implemented really well from the outset.
Shouldn't it be possible to abstract the protocol entirely and support everything?
I am imagining a core script that simply accepts and publishes data via a set of class-based plugins that implement each protocol (SOAP, XML-RPC etc). The core script also handles authentication and security.
One bit of data accepted would be a variable called "module" (eg "mod/forum" or "moodle" for core services) that directs where this core script can find a web service function (in lib.php for example) to accept the entire dataset and deal with it according the data present, returning a response to the core script and thus to the calling party.
I think this model would eventually allow WSRP and things like that should people want them, without too much reimplementation.
This is an important issue, one I want to see implemented really well from the outset.
Shouldn't it be possible to abstract the protocol entirely and support everything?
I am imagining a core script that simply accepts and publishes data via a set of class-based plugins that implement each protocol (SOAP, XML-RPC etc). The core script also handles authentication and security.
One bit of data accepted would be a variable called "module" (eg "mod/forum" or "moodle" for core services) that directs where this core script can find a web service function (in lib.php for example) to accept the entire dataset and deal with it according the data present, returning a response to the core script and thus to the calling party.
- Request comes to core scripts from caller
- Core script analyses input, detects protocol
- Core script passes input to appropriate protocol plugin to extract data
- Core script performs appropriate authentications/security
- Core script passes data to appropriate Moodle module to process
- Moodle module passes back output
- Core script uses protocol plugin to format data and then sends it back to the caller.
I think this model would eventually allow WSRP and things like that should people want them, without too much reimplementation.
Shouldn't it be possible to abstract the protocol entirely and support everything?
definitely, yes. It was more a matter of picking a protocol to start with. I think xml-rpc is by far the simplest, but I would want to ensure that others can be added.
I do like your idea of passing by the module name - I hadn't really thought that far, but that is simple and elegant.
Absolutely right. I have some experience developing SOAP services (and consuming them of course) and would be interested in working on a SOAP implementation team.
One comment ... there are design implications of choosing one implementation method up front to just get something working that will likely make it very difficult to implement services using other methods down the road. That is why it is worth the time spent up front to think about a design that separates the functionality within Moodle of web services from the implementation method(s).
- curt
One comment ... there are design implications of choosing one implementation method up front to just get something working that will likely make it very difficult to implement services using other methods down the road. That is why it is worth the time spent up front to think about a design that separates the functionality within Moodle of web services from the implementation method(s).
- curt
hi,
what is about webservices in moodle?
The API is not very useful?! is it?
I'm searching for exporting scorm via webservices, is it possible?
thanks
what is about webservices in moodle?
The API is not very useful?! is it?
I'm searching for exporting scorm via webservices, is it possible?
thanks
Okay. We're joining the fray.
We are adding some web services interfaces to Moodle so that an external SIS can access Moodle data.
Specifically,
1) Manage user data - send and retrieve the information,
2) Manage course enrolments - add/remove teachers and students,
3) Course management - create new courses based on templates,
4) Gradebook info - extract grades information from Moodle.
I haven't made a firm decision as to SOAP or XML-RPC. It seems to me that the consensus has been to use XML-RPC rather than SOAP. I don't think XML-RPC is needed for its real purpose (to run remote applications - we won't be), but if it makes sense to use from a complexity standpoint, then maybe we should. What do you think?
mike
We are adding some web services interfaces to Moodle so that an external SIS can access Moodle data.
Specifically,
1) Manage user data - send and retrieve the information,
2) Manage course enrolments - add/remove teachers and students,
3) Course management - create new courses based on templates,
4) Gradebook info - extract grades information from Moodle.
I haven't made a firm decision as to SOAP or XML-RPC. It seems to me that the consensus has been to use XML-RPC rather than SOAP. I don't think XML-RPC is needed for its real purpose (to run remote applications - we won't be), but if it makes sense to use from a complexity standpoint, then maybe we should. What do you think?
mike
XML-RPC is simpler, older, and the basis for the original SOAP.
SOAP has since (we are talking 5 or so years) grown more complex,
and has also gained more momentum as an "industry trend".
While I have little doubt that one can skin this cat with either
XML-RPC or SOAP, I have a pretty clear sense that SOAP will
be the superior choice due to more overall support in the industry.
To make an analogy from networking, XML-RPC is like IPX and
SOAP is like IP. IPX was a simplified version of IP that could be
more easily implemented (in DOS).
What's more important, in my opinion, is to get a standards based
data model. That's above the choice of XML-RPC or SOAP.
What's being done with the IMS Enterprise is a reasonable choice,
I suggest you try to leaverage the work of Dan Stowell on the
IMS Enterprise enrollment plugin.
All 4 of the deliverables that you note here Mike, I can point to
as things being done at University of Wisconsin with IMS Enterprise
today. We are currently working on a new version of the courses
creation, using an "auto-grouping" feature that takes advantage
of lecture/lab/discussion and cross-listed courses. This is based
on work done at Ohio State. Why not take advantage of this work,
I'm happy to share.
SOAP has since (we are talking 5 or so years) grown more complex,
and has also gained more momentum as an "industry trend".
While I have little doubt that one can skin this cat with either
XML-RPC or SOAP, I have a pretty clear sense that SOAP will
be the superior choice due to more overall support in the industry.
To make an analogy from networking, XML-RPC is like IPX and
SOAP is like IP. IPX was a simplified version of IP that could be
more easily implemented (in DOS).
What's more important, in my opinion, is to get a standards based
data model. That's above the choice of XML-RPC or SOAP.
What's being done with the IMS Enterprise is a reasonable choice,
I suggest you try to leaverage the work of Dan Stowell on the
IMS Enterprise enrollment plugin.
All 4 of the deliverables that you note here Mike, I can point to
as things being done at University of Wisconsin with IMS Enterprise
today. We are currently working on a new version of the courses
creation, using an "auto-grouping" feature that takes advantage
of lecture/lab/discussion and cross-listed courses. This is based
on work done at Ohio State. Why not take advantage of this work,
I'm happy to share.
My inclination is/was, exactly as you, to consider SOAP is being the likely winner in the WS 'wars'. However, I think Moodle is such a winner due to its many smart decisions and most of these have been based on doing things in the simplest way possible.
As MD has said though, it is clearly important that we don't do anything to actually preclude adding (or moving to) SOAP, or whatever, in the future.
EDIT:
Forgot to mention, I am not familiar with IMS Enterprise but I'll certainly take a look.
As MD has said though, it is clearly important that we don't do anything to actually preclude adding (or moving to) SOAP, or whatever, in the future.
EDIT:
Forgot to mention, I am not familiar with IMS Enterprise but I'll certainly take a look.
It's worth being aware by the way that there are two IMS Enterprise specs IMS Enterprise and IMS Enterprise Services. IMS Enterprise is essentially an XML format for course and user data, whereas IMS Enterprise Services is a spec for a web service based around the IMS Enterprise data format. I'm currently writing a client for IMS Enterprise Services which implements the spec in spirit at least (the spec sends everything as one long XML file, whereas I'm sending the info as separate parameters). There are still decisions that you need to make about how to use the IMS Enterprise format e.g. is a group name the same a group short description, stuff like that.
You might also want to talk to the folk from the Minted project at Sussex University about what they are doing with Moodle and Dan's IMS Enterprise stuff.
You might also want to talk to the folk from the Minted project at Sussex University about what they are doing with Moodle and Dan's IMS Enterprise stuff.
Dirk -
I would be more than happy to take advantage of what you are doing. How can I find out more?
mike
I would be more than happy to take advantage of what you are doing. How can I find out more?
mike
I have a pretty clear sense that SOAP will
be the superior choice due to more overall support in the industry.
Mark Nottingham has a somewhat forceful dissenting opinion.
be the superior choice due to more overall support in the industry.
Mark Nottingham has a somewhat forceful dissenting opinion.
Hi Tony -
I looked at that link, and I'm not sure I know what the opinion was...
mike
I looked at that link, and I'm not sure I know what the opinion was...
mike
There's a big flame war going on between SOAP/XML-RPC/REST/whatever else right now.
Nottingham did a lot of SOAP work, but now seems to have given it up as a bad job, due to incompatibilities among vendors (or vampires, as he's calling them).
Me, I try to stay out of religious wars like this, although I'll admit that I'm somewhat more inclined toward REST.
Martin's request that the services be protocol agnostic would prepare Moodle to work with whichever one "wins" (assuming that there is a clear winner; I could easily see all of them continuing to exist). Who knows? Someone may come out with a web services API next week that blows all of them out of the water.
It's sort of like Richard Pryor's description of how, as a kid, he belonged to every street gang. "...no matter who was winning, I could say, 'This is my side, over here!'".
Nottingham did a lot of SOAP work, but now seems to have given it up as a bad job, due to incompatibilities among vendors (or vampires, as he's calling them).
Me, I try to stay out of religious wars like this, although I'll admit that I'm somewhat more inclined toward REST.
Martin's request that the services be protocol agnostic would prepare Moodle to work with whichever one "wins" (assuming that there is a clear winner; I could easily see all of them continuing to exist). Who knows? Someone may come out with a web services API next week that blows all of them out of the water.
It's sort of like Richard Pryor's description of how, as a kid, he belonged to every street gang. "...no matter who was winning, I could say, 'This is my side, over here!'".
Tony is rigth,
XML apis are involved in a absurd religious war .. However I'd like moodle to follow the directive KISS ( Keep it Simple Stupd ) and some tendences in this war, if its not possible I allways prefere to do something else elesewere...
My vote would be for the painest XML that we colud think of, and then we will adapt to the crap the warlords decide they want.
XML apis are involved in a absurd religious war .. However I'd like moodle to follow the directive KISS ( Keep it Simple Stupd ) and some tendences in this war, if its not possible I allways prefere to do something else elesewere...
My vote would be for the painest XML that we colud think of, and then we will adapt to the crap the warlords decide they want.
SOAP is the way no doubt about ...
The WSDL file is such a crucial point for any web service I cannot stress this enough. With Java, .NET, VB, Flash even Acrobat Livecycle designer having a well written very detailed WSDL can cut devlopment in those languages down to a fraction of the time. In a REST ful architecture the service is more intuitive from a browsing action
e.g http://moodle.cpit.ac.nz/course/student/grades
but lets not forget web services are for application to application communication a human should not be involved. Has anybody seen the fedora project:
http://fedora.info
Documentation on fedora web services:
http://www.fedora.info/definitions/1/0/api/
Now thats what I call an architecture two RESTful web services and two SOAP web services. Who said SOAP is just hype. the RESTful services are aimed at intuitive browsing of the repository collections and entities the SOAP services are aimed at application to application communication e.g. my LMS serarching the Respository, My LMS adding a object to the Respository.
I presented a sample student web service at the 2006 moodle moot in New Zealand with integrations in Java, Flash VB over Excel
http://moodlemoot.org.nz/moodle/course/view.php?id=12
All of which were made simpler because of the WSDL and associated schema. I devloped this in an afternoon it is not a hard thing to do. The main problem I found was finding a good structure to return there is no User class in moodle....... there is an object but no behaviour is associated to it e.g. getMy Courses() a good class structure needs to be put in place at the core before a web service of any kind can be devloped see uml class structure attached. This is just an example not a finished model.
The WSDL file is such a crucial point for any web service I cannot stress this enough. With Java, .NET, VB, Flash even Acrobat Livecycle designer having a well written very detailed WSDL can cut devlopment in those languages down to a fraction of the time. In a REST ful architecture the service is more intuitive from a browsing action
e.g http://moodle.cpit.ac.nz/course/student/grades
but lets not forget web services are for application to application communication a human should not be involved. Has anybody seen the fedora project:
http://fedora.info
Documentation on fedora web services:
http://www.fedora.info/definitions/1/0/api/
Now thats what I call an architecture two RESTful web services and two SOAP web services. Who said SOAP is just hype
I presented a sample student web service at the 2006 moodle moot in New Zealand with integrations in Java, Flash VB over Excel
http://moodlemoot.org.nz/moodle/course/view.php?id=12
All of which were made simpler because of the WSDL and associated schema. I devloped this in an afternoon it is not a hard thing to do. The main problem I found was finding a good structure to return there is no User class in moodle....... there is an object but no behaviour is associated to it e.g. getMy Courses() a good class structure needs to be put in place at the core before a web service of any kind can be devloped see uml class structure attached. This is just an example not a finished model.
Well, you can also say that SOAP has taken a few too many industry trends, specially the bloat-for-little-benefit trend 
In most implementations, if you can do it with XML-RPC you come out ahead of game performance-wise and implementation-time wise, mostly because debugging is straightforward: you can read the XML and it makes sense. Not so with SOAP.
Now, if you need marketing-buzz compliance, SOAP's the thing
In most implementations, if you can do it with XML-RPC you come out ahead of game performance-wise and implementation-time wise, mostly because debugging is straightforward: you can read the XML and it makes sense. Not so with SOAP.
Now, if you need marketing-buzz compliance, SOAP's the thing
IMHO opinion REST is the way to go. It's simple and works with existing Web infrastructure.
Based on your email to me privately you've seen my post last year: http://moodle.org/mod/forum/discuss.php?d=33521
It is nice to have a pretty URL but it's certainly not a requirement of a REST based service. It's just important that every resource have a unique URL and to use the proper HTTP verbs.
I'd be happy to start working on the REST-based services again if anyone is interested. I'm certainly willing to work within any common services infrastructure that is developed based on Martin's comments in this thread. Other clients are easy enough to develop as well to make integration/mashup work go quicker.
Regards,
Mark
Based on your email to me privately you've seen my post last year: http://moodle.org/mod/forum/discuss.php?d=33521
It is nice to have a pretty URL but it's certainly not a requirement of a REST based service. It's just important that every resource have a unique URL and to use the proper HTTP verbs.
I'd be happy to start working on the REST-based services again if anyone is interested. I'm certainly willing to work within any common services infrastructure that is developed based on Martin's comments in this thread. Other clients are easy enough to develop as well to make integration/mashup work go quicker.
Regards,
Mark
Again.... REST could easily be implemented. BUT....and as we discussed, I *like* REST in principle... I feel that the lack of a standard for it and the lack of any php libraries to support it is a big problem. I know that REST can readily be implemented without libraries but I feel that having libraries of functions with examples etc. makes life easier for users. I also think that the fairly comprehensive understanding of the HTTP protocol that is required in order to develop clients is likely to be restrictive. But please don't take this to mean that it's off the table... Amazon implements SOAP *and* REST as you will know.
I also think that the fairly comprehensive understanding of the HTTP protocol that is required in order to develop clients is likely to be restrictive. But please don't take this to mean that it's off the table... Amazon implements SOAP *and* REST as you will know.
According to Amazon's chief web services evangelist, via Tim O'Reilly, 85% of the usage of Amazon's web services API comes via the REST interface. Surely that's some indication that it's not too daunting for developers?
Gearing up to write SOAP or XML-RPC stuff isn't exactly trivial, either.
According to Amazon's chief web services evangelist, via Tim O'Reilly, 85% of the usage of Amazon's web services API comes via the REST interface. Surely that's some indication that it's not too daunting for developers?
Gearing up to write SOAP or XML-RPC stuff isn't exactly trivial, either.
It's one of those ... "if someone puts a gun to your head and says make a decision", things Perhaps XML-RPC *and* REST....
XML-RPC _is_ trivial.
/me hears the sounds of trumpets, and the XML-marshaling of troops for battle....
Whatever you guys come up with will be great, I'm sure.
Whatever you guys come up with will be great, I'm sure.
Yes, but it's not as good as PARP.
Not sure what PARP is... and google didn't help much, Poly ADP-Ribose Polymerase?
I am following this thread with interest... we do a lot of SIS/SMS integration work at Catalyst, and Web Services sounds great. On paper
We always go for good old batch processing -- the nature of the beast is that the integration is complex and we don't want tight coupling.
Web Services is kind-of loose coupling, except that it is not. It is in the 'protocol' sense, true. But in practice, the services become interdependent in terms of availability -- either of them is down and you are in trouble. Update user profiles at the SIS when Moodle is down (or the network link to Moodle is down, as SIS will be on the LAN and Moodle on the intarweb) and... you depend on the SIS transaction being either rolled back in the SIS or queued... or lost! Hmm!
You have to get your hands deep in the SIS code to do things transactionally against the web services, and do really robust error handling... and that doesn't happen often.
With batch processing, unsexy as it sounds, it is trivial to queue the updates if anything is not working (for later recovery), archive them, etc. And it's easy to optimise the code for large updates. Error handling is mostly trivial.
I am keen, of course, on a WS interface for Moodle, but I think it'll be interesting for automation jobs (things that we already do writing small utilities we put in /admin) but it'll help programmers who don't want to touch PHP... yes, we love them too And for a web-based SIS with Moodle for low/medium size installs, WS is good fit.
I am following this thread with interest... we do a lot of SIS/SMS integration work at Catalyst, and Web Services sounds great. On paper
Web Services is kind-of loose coupling, except that it is not. It is in the 'protocol' sense, true. But in practice, the services become interdependent in terms of availability -- either of them is down and you are in trouble. Update user profiles at the SIS when Moodle is down (or the network link to Moodle is down, as SIS will be on the LAN and Moodle on the intarweb) and... you depend on the SIS transaction being either rolled back in the SIS or queued... or lost! Hmm!
You have to get your hands deep in the SIS code to do things transactionally against the web services, and do really robust error handling... and that doesn't happen often.
With batch processing, unsexy as it sounds, it is trivial to queue the updates if anything is not working (for later recovery), archive them, etc. And it's easy to optimise the code for large updates. Error handling is mostly trivial.
I am keen, of course, on a WS interface for Moodle, but I think it'll be interesting for automation jobs (things that we already do writing small utilities we put in /admin) but it'll help programmers who don't want to touch PHP... yes, we love them too
PARP Ain't a Real Protocol.
Hmmmm.
I hadn't really considered it for automated or batch style operations. You would definitely have to make sure the client behaved based on success of the operation. I see this as more of an event driven thing.
But I see your point. If I update my address on my SIS, and then it is passed to Moodle through WS, and Moodle burps, then what? We'll have to make sure that it is covered.
mike
I hadn't really considered it for automated or batch style operations. You would definitely have to make sure the client behaved based on success of the operation. I see this as more of an event driven thing.
But I see your point. If I update my address on my SIS, and then it is passed to Moodle through WS, and Moodle burps, then what? We'll have to make sure that it is covered.
mike
Oh, we don't use WebServices for automated/batch ops. We load up a small database that sits on the same server as Moodle, and then use auth and enrolment plugins to read from there.
The whole point of WS is that you can hook them into the interactive side of the SIS... fir example the "update student details" screen of the SIS can update Moodle as well as its internal DB immediately. However, for large installations you cannot affort to be sloppy in your error handling, which must happen at the SIS.
And the problem I often face is that the SIS is closed, or now available for my team to hack on So the only way to setup a reliable (and resilient!) system is to operate in batch mode. Old, unsexy... and it just works
The whole point of WS is that you can hook them into the interactive side of the SIS... fir example the "update student details" screen of the SIS can update Moodle as well as its internal DB immediately. However, for large installations you cannot affort to be sloppy in your error handling, which must happen at the SIS.
And the problem I often face is that the SIS is closed, or now available for my team to hack on
Are you familiar with the J2EE platform? providing redundant and load balanced web services? Very stable designed for enterprise environments this can be replicated PHP style with an ESB Enterprice Service Bus one server being controller for all data transmitted through the bus routing requests to services and monitoring status see http://www.logicblaze.com/.
But really we have to look at the problem you outline here, that user details are updated in the SIS and Moodle is down so dosn't update the user in Moodle (is this what you are on about??). Why are you advocating two copies of userdata in the first place? Web services are designed to remove this duplication of data, leave the data about the student on the SIS this is where it belongs after all, let the SIS manage the data as this is what it is desgined for (and purchased for ). All that should be in Moodle is a reference to that user. This type of integration is differnt to classical "batch processing" and is what Web Services were designed around opening systems and exposing functionality to applications.
But really we have to look at the problem you outline here, that user details are updated in the SIS and Moodle is down so dosn't update the user in Moodle (is this what you are on about??). Why are you advocating two copies of userdata in the first place? Web services are designed to remove this duplication of data, leave the data about the student on the SIS this is where it belongs after all, let the SIS manage the data as this is what it is desgined for (and purchased for
HTTP been a standard for quite a long time 
As far as the data getting passed back and forth goes, I was using the XOXO (pronounced zoh zoh I believe) microformat which uses standard XHTML elements.
Client for PHP? CURL.
I do understand the concern you are expressing however.
Mark
As far as the data getting passed back and forth goes, I was using the XOXO (pronounced zoh zoh I believe) microformat which uses standard XHTML elements.
Client for PHP? CURL.
I do understand the concern you are expressing however.
Mark
CURL is only in PHP 4.3 onwards though.... Moodle supports 4.1.0 which could well be an issue. However, there are PHP only solutions.
FOr Moodle 1.6 the PHP requirement is 4.3 so this isn't an issue.
Oh! Missed that one completely. That's good
(...and I have amended the requirements in the docs wiki)
(...and I have amended the requirements in the docs wiki)
Hi guys,
I'm working with Mike on the project he mentioned already. I've been working out an authentication scheme for starting up a web services session and soliciting some feedback on the matter seems like a good idea now.
I'm basing my code around using the NuSOAP library included with 1.5.3+ but it should be easily extended to the new web services interfaces in the latest versions.
So I have a DB table that stores a client's IP address, session key, session times (start and end) and a flag for client validation purposes.
All communication back-and-forth between the client and server is based around a communications object that stores an action code, the client's session key, time, and a data parameter that holds the payload (which can be anything). The only method exposed to the client is a dispatch function which calls other functions based on what the action code of the comm object is. The dispatch also verifies authentication before calling another function.
For a client to connect, and be authenticated the protocol as follows:
1. Client creates a new comm object, with an INIT action, time, and empty key and data params.
2. Server verifies that the client's IP does not exist as a validated, current session in the database. If this is a new session, it will generate a session key (it's an md5sum) for the client, store that with the client IP and current time in the DB, and flag the client as unvalidated. An INIT comm object is sent back to the client with their session key as the data payload.
3. The client sends a comm object with a LOGIN action, their session key, and an array containing their Moodle username and password to the server. The server verifies their session key against the database and passes the username and password to the standard Moodle authenticate_user_login() function. If it's good a response object is sent back to the client, otherwise an object is sent back with a ERROR action code specifying that the username and/or password was wrong. If the login succeeds, the database is updated so the client is flagged as authenticated and their Moodle user ID is attached to the DB record.
4. Sessions have a timeout value (defined to something like 30 minutes). If a client is in the middle of a session and sends a request past the timeout value, they will have to re-enter their username / password to continue their session. This prevents old sessions from hanging around.
5. Data goes back and forth.
6. The client sends a LOGOUT comm object which sets their DB object back to a non-verified state and a further CLOSE comm object ends the session altogether.
Issues:
- I don't really like the idea of sending the username / password in the clear like that but I guess if a site wasn't forcing the usage of SSL then that's how it would be through the HTML form on the login page anyway.
- Associating the client IP with a particular session means that anyone behind any kind of NAT router could potentially hijack another user's session. I think I've got that problem solved because the initial client connect and INIT won't start a new session if there is a current session attached to the IP, it will just return an error message to the client attempting to connect. And to 'hijack' someone's session you'd have to be eavesdropping on the traffic from the other system to get their session key and hope they haven't logged out yet.
So that's what I've got so far. Any comments? Criticisms?
Again, I'm doing this in a generalised way so it could work on multiple protocols and providing access to areas of Moodle would mean just adding a new action code and providing the proper methods to the dispatch caller.
I'm working with Mike on the project he mentioned already. I've been working out an authentication scheme for starting up a web services session and soliciting some feedback on the matter seems like a good idea now.
I'm basing my code around using the NuSOAP library included with 1.5.3+ but it should be easily extended to the new web services interfaces in the latest versions.
So I have a DB table that stores a client's IP address, session key, session times (start and end) and a flag for client validation purposes.
All communication back-and-forth between the client and server is based around a communications object that stores an action code, the client's session key, time, and a data parameter that holds the payload (which can be anything). The only method exposed to the client is a dispatch function which calls other functions based on what the action code of the comm object is. The dispatch also verifies authentication before calling another function.
For a client to connect, and be authenticated the protocol as follows:
1. Client creates a new comm object, with an INIT action, time, and empty key and data params.
2. Server verifies that the client's IP does not exist as a validated, current session in the database. If this is a new session, it will generate a session key (it's an md5sum) for the client, store that with the client IP and current time in the DB, and flag the client as unvalidated. An INIT comm object is sent back to the client with their session key as the data payload.
3. The client sends a comm object with a LOGIN action, their session key, and an array containing their Moodle username and password to the server. The server verifies their session key against the database and passes the username and password to the standard Moodle authenticate_user_login() function. If it's good a response object is sent back to the client, otherwise an object is sent back with a ERROR action code specifying that the username and/or password was wrong. If the login succeeds, the database is updated so the client is flagged as authenticated and their Moodle user ID is attached to the DB record.
4. Sessions have a timeout value (defined to something like 30 minutes). If a client is in the middle of a session and sends a request past the timeout value, they will have to re-enter their username / password to continue their session. This prevents old sessions from hanging around.
5. Data goes back and forth.
6. The client sends a LOGOUT comm object which sets their DB object back to a non-verified state and a further CLOSE comm object ends the session altogether.
Issues:
- I don't really like the idea of sending the username / password in the clear like that but I guess if a site wasn't forcing the usage of SSL then that's how it would be through the HTML form on the login page anyway.
- Associating the client IP with a particular session means that anyone behind any kind of NAT router could potentially hijack another user's session. I think I've got that problem solved because the initial client connect and INIT won't start a new session if there is a current session attached to the IP, it will just return an error message to the client attempting to connect. And to 'hijack' someone's session you'd have to be eavesdropping on the traffic from the other system to get their session key and hope they haven't logged out yet.
So that's what I've got so far. Any comments? Criticisms?
Again, I'm doing this in a generalised way so it could work on multiple protocols and providing access to areas of Moodle would mean just adding a new action code and providing the proper methods to the dispatch caller.
. I think I've got that problem solved because the initial client connect and INIT won't start a new session if there is a current session attached to the IP, it will just return an error message to the client attempting to connect.
What happens if you really do need to have more than one machine using it from behind a NAT, or am I misunderstanding you here? Lots of school labs share one IP address, particularly at the K-12 level.
You're not misunderstanding me at all. That is the problem I was referring to. The project Mike and I are working on is dealing with a set of functions that is administration only so it's fairly safe to assume that only one person at a time would be needed a connection to the web services interface at a time. That's a pretty big 'maybe', however...
This wouldn't be a problem on an internal network behind a NAT router if the Moodle server is running on a machine which has an internal IP address. In that case, the person's IP address registered with the web services layer would be their internal IP and not the single, external, IP from the NAT router.
This wouldn't be a problem on an internal network behind a NAT router if the Moodle server is running on a machine which has an internal IP address. In that case, the person's IP address registered with the web services layer would be their internal IP and not the single, external, IP from the NAT router.
I guess what I'm worried about here is a situation like this:
1) During a free period, two (or more) K-12 teachers decide to take care of some grading or other administrative functions, so they fire up their NATed laptops and call up the SIS that's located on a server in the district office (most likely in a completely separate building, almost certainly not on the same NAT router).
2) Oops.
1) During a free period, two (or more) K-12 teachers decide to take care of some grading or other administrative functions, so they fire up their NATed laptops and call up the SIS that's located on a server in the district office (most likely in a completely separate building, almost certainly not on the same NAT router).
2) Oops.
I guess storing the client IP isn't really necessary. Right now, once the a client is validated, all future communications with the server will verify the session key against the database record (using trying to match the client IP address and session key). I could just ignore the IP altogether and instead use the database record ID for the particular session as the client ID along with their unique session key.
In all honesty I didn't realise the NAT router / multiple clients problem until I was typing my initial post.
The situation you outlined above could go two ways. The second is a problem that could occur because the INIT and LOGIN are two different operations: INIT sends a generated key back to the client and LOGIN verifies the username / password, which verifies the user's session and would disallow any new sessions from that IP for the duration.
1. Teacher A successfully logs into the system and starts doing some work. Teacher B tries to login and is denied because a session already exists for his IP.
2. - Teacher A attempts to login with an INIT request, receives a response with the newly generated session key. This key is associated with their IP in the database as an unverified session.
- Teacher B attempts to login with an INIT request (before the client Teacher A is running has a chance to send a LOGIN request to the server). The server sees an unvalidated session existing for this IP, deletes that record, generates a new session key for the IP, and returns that session key to Teacher B.
- Teacher A sends a LOGIN request to the server with their session key. The server checks this key against the database, they don't match and sends back an invalid session key message.
- Teacher B sends a LOGIN request to the server with their session key. The server verifies the key and tries to login.
- At this point, if the username and password are valid, Teacher B is allowed in and Teacher A can't get in until Teacher B is finished. A worse situation (requiring amazing timing and luck, but still a possibility) would be that this process would continue on indefinitely with each client's staggered INIT and LOGIN requests knocking the other's out.
This makes me realise those two operations need to be one and the same.
(Please accept my apologies for essentially 'thinking out loud' in my response to your post).
In all honesty I didn't realise the NAT router / multiple clients problem until I was typing my initial post.
The situation you outlined above could go two ways. The second is a problem that could occur because the INIT and LOGIN are two different operations: INIT sends a generated key back to the client and LOGIN verifies the username / password, which verifies the user's session and would disallow any new sessions from that IP for the duration.
1. Teacher A successfully logs into the system and starts doing some work. Teacher B tries to login and is denied because a session already exists for his IP.
2. - Teacher A attempts to login with an INIT request, receives a response with the newly generated session key. This key is associated with their IP in the database as an unverified session.
- Teacher B attempts to login with an INIT request (before the client Teacher A is running has a chance to send a LOGIN request to the server). The server sees an unvalidated session existing for this IP, deletes that record, generates a new session key for the IP, and returns that session key to Teacher B.
- Teacher A sends a LOGIN request to the server with their session key. The server checks this key against the database, they don't match and sends back an invalid session key message.
- Teacher B sends a LOGIN request to the server with their session key. The server verifies the key and tries to login.
- At this point, if the username and password are valid, Teacher B is allowed in and Teacher A can't get in until Teacher B is finished. A worse situation (requiring amazing timing and luck, but still a possibility) would be that this process would continue on indefinitely with each client's staggered INIT and LOGIN requests knocking the other's out.
This makes me realise those two operations need to be one and the same.
(Please accept my apologies for essentially 'thinking out loud' in my response to your post).
Hi Justin, I've been following this thread with interest. I'm just wondering why you all decided against using something like pubcookie or shibboleth?
Hi Michael -
Its an interesting idea. Would these work with web services though?
mike
Its an interesting idea. Would these work with web services though?
mike
They should. I guess it depends on what the server system has running and is setup to use. I'm basing my work around username / password authentication as it's what I haven't worked with any of the other authentication methods. But this could be extended to use other authentication methods for granting access to Moodle functions.
My main authentication concern(s) is more centered around client authentication so that once you verify credentials for access to Moodle data on the server you want to make sure further communications are REALLY still coming from the same location and that someone can't hijack a session to gain access to the Moodle server.
My main authentication concern(s) is more centered around client authentication so that once you verify credentials for access to Moodle data on the server you want to make sure further communications are REALLY still coming from the same location and that someone can't hijack a session to gain access to the Moodle server.
A worse situation (requiring amazing timing and luck, but still a possibility) would be that this process would continue on indefinitely with each client's staggered INIT and LOGIN requests knocking the other's out.
Oooh... yeah, nasty race condition there. You could hack around it by doing something like having the clients wait a random amount of time before reattempting the connect (like the way collision handling works in Ethernet) but you'd still be stuck with having only one client connected per IP.
Yeah, probably best to avoid using the IP altogether, unless you can figure out a way to avoid the mutual exclusion problem.
Oooh... yeah, nasty race condition there. You could hack around it by doing something like having the clients wait a random amount of time before reattempting the connect (like the way collision handling works in Ethernet) but you'd still be stuck with having only one client connected per IP.
Yeah, probably best to avoid using the IP altogether, unless you can figure out a way to avoid the mutual exclusion problem.
I removed the dependance on IP address for sessions and placed it, instead, on Moodle user ID. So if I login to the server with my web services app using my Moodle account, the user ID is stored in my session record and if someone else attempts to login with the same username / password, they'll receive an error message. This is also useful for checking access levels with standard Moodle functions so that teachers can't change information in course records but admin's can, etc.
My paranoia is rather high about web services. A good idea is to just force the usage of an SSL connection for all web services access because SSL has good authentication and encryption methods which takes care of a couple problems:
1. Someone just sniffing your information and getting course / user /etc. information either being updated or received from the Moodle server.
2. A man-in-the-middle attack where someone could just setup a fake client/server in between your web services application and the access layer running on the Moodle server. This fake client/server setup could just accept and record all your traffic as well as passing it to the actual intended destination server and passing it's responses back to the web services app.
My paranoia is rather high about web services. A good idea is to just force the usage of an SSL connection for all web services access because SSL has good authentication and encryption methods which takes care of a couple problems:
1. Someone just sniffing your information and getting course / user /etc. information either being updated or received from the Moodle server.
2. A man-in-the-middle attack where someone could just setup a fake client/server in between your web services application and the access layer running on the Moodle server. This fake client/server setup could just accept and record all your traffic as well as passing it to the actual intended destination server and passing it's responses back to the web services app.
...things have come unstuck a bit with PHP5 and the advent of the SOAP wrappers and lib/soaplib.php. This is another thing that put me off SOAP. It would be a job in itself to extend and fully document those wrappers.
My head is kind of in the sand on that aspect. I'm writing my code with PHP4 and using Moodle 1.5.3+ as my base. I'm kind of afraid to run my code in PHP5 as it's all class-based. I don't have a terrible desire to watch everything crumble to pieces.
i'd like to share with you our experience in developing a webservice interface for moodle. our code concentrates on functionality for the quiz module, but it's of course not limited to that.
we're using xml-rpc, because it's simpler, and because we knew it better than soap when we started to develop this interface. we're using pear xml-rpc for the server. on the client side, currently, flash is used to create a cross-browser comaptible user interface for the published functionality.
you can take a look at the (slightly outdated) documentation for our interface.
behind the scenes, in the code that really interacts with moodle, there's quite a bit of custom code. this is because
as far as character encoding goes, it works well with out setup. everything is transmitted in utf8. the data is re-encoded as necessary at the entry/exit points.
my experiences with soap vs. xml-rpc in php:
they are both easy to create clients for. hooking up the server code to underlying functions is also easy enough in both of them. what's difficult in soap is the wsdl, which is used to let the client know what methods are available, what parameters of what type the methods accept, and what the methods return. it's not easy to generate wsdl from php code, because php code is not strongly typed. it's also not easy to write a wsdl document by hand. there are some tools that help in generating wsdl from php, such as this one, which work more often than not - but i haven't found anything yet that works without a hitch. in fact, wsdl is not required - but when it exists, it makes the client side code much easier (see here for example code with/without wsdl). i've found xml-rpc easier to use and to debug, and would personally only choose to use soap if there was a real need for it.
we're using xml-rpc, because it's simpler, and because we knew it better than soap when we started to develop this interface. we're using pear xml-rpc for the server. on the client side, currently, flash is used to create a cross-browser comaptible user interface for the published functionality.
you can take a look at the (slightly outdated) documentation for our interface.
behind the scenes, in the code that really interacts with moodle, there's quite a bit of custom code. this is because
- the standard moodle code often outputs html when an error is encountered, which interferes with the xml-rpc response
- we are doing some custom things for which there was no existing moodle code
as far as character encoding goes, it works well with out setup. everything is transmitted in utf8. the data is re-encoded as necessary at the entry/exit points.
my experiences with soap vs. xml-rpc in php:
they are both easy to create clients for. hooking up the server code to underlying functions is also easy enough in both of them. what's difficult in soap is the wsdl, which is used to let the client know what methods are available, what parameters of what type the methods accept, and what the methods return. it's not easy to generate wsdl from php code, because php code is not strongly typed. it's also not easy to write a wsdl document by hand. there are some tools that help in generating wsdl from php, such as this one, which work more often than not - but i haven't found anything yet that works without a hitch. in fact, wsdl is not required - but when it exists, it makes the client side code much easier (see here for example code with/without wsdl). i've found xml-rpc easier to use and to debug, and would personally only choose to use soap if there was a real need for it.
Hey Brian -
Thanks for your experiences and observations.
mike
Thanks for your experiences and observations.
mike
The HTML output has been a problem for me as well.
I'm trying to do as much sanity checking on my input to the server before I call a Moodle function so as not to produce any error messages.
Another idea would be to wrap your Moodle function calls (anything that could possibly create HTML output on error) inside an output buffer. If the buffer has contents then you know an error ocurred and could examine the contents of that buffer to figure out what the error was.
I'm not sure what kind of performance impact that would have, though. The amount of HTML generated by a Moodle error message is usually pretty small so the buffer wouldn't be very large. And if your functions aren't generating error messages, it would be empty after every call anyway.
I'm trying to do as much sanity checking on my input to the server before I call a Moodle function so as not to produce any error messages.
Another idea would be to wrap your Moodle function calls (anything that could possibly create HTML output on error) inside an output buffer. If the buffer has contents then you know an error ocurred and could examine the contents of that buffer to figure out what the error was.
I'm not sure what kind of performance impact that would have, though. The amount of HTML generated by a Moodle error message is usually pretty small so the buffer wouldn't be very large. And if your functions aren't generating error messages, it would be empty after every call anyway.
What's the status with using PEAR libraries in Moodle? I was under the impression that they were 'not allowed' as it is difficult to guarantee the availability of the required library. I was thinking of using the Useful Inc Library and simply adding it to the Moodle distro?!?
You are right, we cannot know whether the PEAR library is in place, and whether it is the right version. So we include it with Moodle like we do with AdoDB and other bits and pieces.
Except that GPL and PEAR libraries have a small tangle. The GPL doesn't allow 'additional restrictions' and the PEAR libraries use the PHP license, which is very free (it's a BSD-like license) but add a restriction that your program cannot be called 'PHPSomething'. It is a trademark protection clause.
So distributing GPL and PHP-licensed code as part of the same program is a bit of a problem. What we've done with some PEAR code we are using is ask the developer whether he'd be happy to dual-license it, which he was.
The other option is to get MartinD to add a clarification to the GPL that covers moodle, indicating that, as the licensor he makes an exception to that GPL clause when it comes to trademark protection clauses. (The linux kernel has a similar clarification about binary drivers, iirc).
The silly thing about this whole affair is that the trademark protection clause in the PHP license is useless -- in any country where the license is enforceable, there _is_ a trademark law already, and it protects the PHP trademark in a clearer and more effective way. No need for the PHP license to reinforce trademark law, tax law or the road code
Except that GPL and PEAR libraries have a small tangle. The GPL doesn't allow 'additional restrictions' and the PEAR libraries use the PHP license, which is very free (it's a BSD-like license) but add a restriction that your program cannot be called 'PHPSomething'. It is a trademark protection clause.
So distributing GPL and PHP-licensed code as part of the same program is a bit of a problem. What we've done with some PEAR code we are using is ask the developer whether he'd be happy to dual-license it, which he was.
The other option is to get MartinD to add a clarification to the GPL that covers moodle, indicating that, as the licensor he makes an exception to that GPL clause when it comes to trademark protection clauses. (The linux kernel has a similar clarification about binary drivers, iirc).
The silly thing about this whole affair is that the trademark protection clause in the PHP license is useless -- in any country where the license is enforceable, there _is_ a trademark law already, and it protects the PHP trademark in a clearer and more effective way. No need for the PHP license to reinforce trademark law, tax law or the road code
I got permissions for the ones we have in Moodle now. Each future inclusion needs to either be GPL-compatible (they vary) or negotiated individually.
I don't want to mess with Moodle's GPL license.
I don't want to mess with Moodle's GPL license.
Hi!
Just wanted to state I really appreciate the idea of moodle webservices to be exposed to other systems! That would rather be the solution to my actual problems
Making as much of moodles functionality reachable via a webservice as possible, will significantly increase the overall value! I just think of the benefits of an automated user management and course enrolment via WS out of SAP!
Keep up the work and let us know of your efforts!
greetz, Andreas
Just wanted to state I really appreciate the idea of moodle webservices to be exposed to other systems! That would rather be the solution to my actual problems
Making as much of moodles functionality reachable via a webservice as possible, will significantly increase the overall value! I just think of the benefits of an automated user management and course enrolment via WS out of SAP!
Keep up the work and let us know of your efforts!
greetz, Andreas
Ok ... here is some more concrete stuff.
The idea is to have a single (sort of) dispatcher, so all calls to the web service would go to a single URL, for example:
http://moodle.org/ws/xmlrpc.php
My thinking here is that if we wanted to implement SOAP, only the basic dispatcher needs to change and thus the URL would become (say):
Of course, for economy, the dispatchers will descend from a common class that will provide all the common functionality.
The single point dispatcher will, I feel, make ongoing changes much easier. For example if we decide to add some sort of key-pair trust relationships it would be localised to the dispatcher(s)
The dispatcher will take a number of parameters for authentication, identification of the operation to perform and the actual data. Viz...
- Username
- Password
- Path to resource - e.g., blocks/online_users
- Operation - e.g., 'list'
- data required by function
The path to resource would be the actual path to the place that will deal with the data (general admin functions could be 'admin' or 'moodle' perhaps). This would expect to find a suitable method within the lib.php file. If this does not exist, an error is returned.The Operation describes the particular function for that moodle block/module. So the above example returns a list of online users (it wouldn't need any data).
Some more random thoughts:
- Could have a 'help' operation that returns the instructions for the specified service
- It would be nice to have some simple/elegant way to 'register' web service functions so that a complete list can be returned. Can't quite see how to do this at the moment though...
- I would have an administration setting to (as a default) force all web services to be read-only unless the admin decides otherwise.
Thoughts?
Here's one web service that should fit with this model, Tools Interoperbility Framework. This is an IMS specification for launching external tools
and getting results of assessments back, which does use WS as the interface
technology. See http://imsglobal.org/ti for the specification and
http://bug.sakaiproject.org/confluence/pages/viewpage.action?pageId=9668 for a PPT that gives a pretty good overview.
This was demonstrated at the Alt-i-Lab 2005 in Sheffield UK, my team hacked
together a Moodle and a local tool we built called ConceptTutor. Hack is
the operative term, we put bailing wires on the web services parts, using
an associated Tomcat/Axis server to do the actual work.
I'm currently working on a proof-of-concept for Alt-i-Lab 2007, one that was
first shown at the May 2006 Internet2 Member meeting, which will include
Shibboleth, TIF and ACCLIP (accessible learner information profile). You can
see the work-in-progress at http://atsosxdev.doit.wisc.edu/a4ashibmoodle.doit.wisc.edu
For this demo, I created http://atsosxdev.doit.wisc.edu/a4ashibmoodle.doit.wisc.edu/tif/toolproxy.php
not wanting to be quite so hackish. This is attached for Howard and others perusal. Pretty simple and just waiting to be wired into a better Moodle
WS framework. Note that it's using it's own copy of nusoap.
and getting results of assessments back, which does use WS as the interface
technology. See http://imsglobal.org/ti for the specification and
http://bug.sakaiproject.org/confluence/pages/viewpage.action?pageId=9668 for a PPT that gives a pretty good overview.
This was demonstrated at the Alt-i-Lab 2005 in Sheffield UK, my team hacked
together a Moodle and a local tool we built called ConceptTutor. Hack is
the operative term
an associated Tomcat/Axis server to do the actual work.
I'm currently working on a proof-of-concept for Alt-i-Lab 2007, one that was
first shown at the May 2006 Internet2 Member meeting, which will include
Shibboleth, TIF and ACCLIP (accessible learner information profile). You can
see the work-in-progress at http://atsosxdev.doit.wisc.edu/a4ashibmoodle.doit.wisc.edu
For this demo, I created http://atsosxdev.doit.wisc.edu/a4ashibmoodle.doit.wisc.edu/tif/toolproxy.php
not wanting to be quite so hackish. This is attached for Howard and others perusal. Pretty simple and just waiting to be wired into a better Moodle
WS framework. Note that it's using it's own copy of nusoap.
Hi Howard,
Do you think it would be useful to have an alternative authentication method for some embedded java applets / flash movies and possibly other apps.
Imagine if a user is logged into a course and in a module / filter / block there is a flash movie / java applet embedded, it would be nice if :
- in the php generating the page we could call a function that generates a new random session key (using md5 or something) and stores the key in a db table along with info about the user and the module they are accessing. Since the php script knows who the user is their is no need for the user to provide a username and password again of course. This web services session key would be time limited and old keys could be cleaned up by a cron or whatever. A session key would be generated by php code in the module / filter / block only when needed.
- the ws session key could be passed into the Flash movie or java applet through html parameter tags in the html page that embeds the movie / applet.
- when the flash movie/java applet or other app tries to access the ws it passes not a username and password to the web service but the session key. The session key is checked to see if it has not expired and the ws retrieves data about the user and the module they are accessing and checks data about who can access a service (access permissions working in a similar way to the roles and permissions architeciture that is being worked on??) Every time a web service is called the session key is passed with the ws request.
- Their will be a web service that can close a session but if the session closing function is not called the session key record will be cleaned up automatically anyway.
The above is what the Flash module does to communicate with Moodle.
In the case that a system accesses a web service without a session key it sends a username and password to the web service on the first ws request and along with the returned data a session key is passed and then on the next ws request the session key is passed instead of the username and password.
I wonder if security wise it would be possible to hyperlink from Moodle to other apps that will access web services with a ws session key in the url linking to the app or passed as a POST variable via a form.
Jamie
Jamie,
It's entirely possible that I have misunderstood you, for which I apologise in advance. However, I'm not sure we are quite on the same page
My thinking is that web services are/should be completely stateless, so they don't use the/a session infrastructure. In the general case, of course, the client will not be code in the Moodle system at all.
A web service that can open, close and maintain sessions is a considerable jump forward from what I am planning and, to be honest, I'm unconvinced that it is a necessary one. BUT.. I do take your point that it might be useful for things like embedded applets to be able to connect to their 'own' Moodle sites more transparently. One for the 'to think about' list perhaps?
It's entirely possible that I have misunderstood you, for which I apologise in advance. However, I'm not sure we are quite on the same page
My thinking is that web services are/should be completely stateless, so they don't use the/a session infrastructure. In the general case, of course, the client will not be code in the Moodle system at all.
A web service that can open, close and maintain sessions is a considerable jump forward from what I am planning and, to be honest, I'm unconvinced that it is a necessary one. BUT.. I do take your point that it might be useful for things like embedded applets to be able to connect to their 'own' Moodle sites more transparently. One for the 'to think about' list perhaps?
Hi Howard,
Thanks for your quick response to my post.
Let me try and explain my concept a little more clearly.
What I meant by 'session' and 'session key' is just that you have a key valid for a number of web service requests. I was not referring to a php session and I agree that web services should be stateless apart from perhaps providing a way for a web service client to send authentication information just once and thereafter send a key. Or where possible for the client to be passed a key for an authenticated session that has been opened already by Moodle.
I mean session as just a set of web service requests from the same client over a limited period of time and session key as a key that a client is passed after they authenticate correctly or is passed to the client program on start up. The key thereafter is used to prove to Moodle who the web service request is coming from.
I think it is a useful idea to consider so that in the case of a web service request from an entirely different system then there could be some way of opening such a 'session' with a username and probably encrypted password (as part of the same http transaction as the first web service request?) and then subsequent requests would send just the 'session' key. This would be more secure than sending the username with each transaction.
This is what Justin is talking about doing above in this forum here.
In the case of an embedded object such as a Flash movie or java applet or where you can pass a key to an outside application from the php application then the key could be generated by php which has already authenticated the user and knows who they are. The key could be passed into the Flash movie or java applet or outside application and passed back to Moodle.
As much as possible isn't it better to be passing around keys that are valid for a limited amount of time rather than sending passwords and usernames back and forth. And also this would allow a Flash / Java applet to interact with Moodle through web services without the user having to provide the Flash movie / java applet / ajax application with his/her username and password.
Thanks,
Jamie
Thanks for your quick response to my post.
Let me try and explain my concept a little more clearly.
What I meant by 'session' and 'session key' is just that you have a key valid for a number of web service requests. I was not referring to a php session and I agree that web services should be stateless apart from perhaps providing a way for a web service client to send authentication information just once and thereafter send a key. Or where possible for the client to be passed a key for an authenticated session that has been opened already by Moodle.
I mean session as just a set of web service requests from the same client over a limited period of time and session key as a key that a client is passed after they authenticate correctly or is passed to the client program on start up. The key thereafter is used to prove to Moodle who the web service request is coming from.
I think it is a useful idea to consider so that in the case of a web service request from an entirely different system then there could be some way of opening such a 'session' with a username and probably encrypted password (as part of the same http transaction as the first web service request?) and then subsequent requests would send just the 'session' key. This would be more secure than sending the username with each transaction.
This is what Justin is talking about doing above in this forum here.
In the case of an embedded object such as a Flash movie or java applet or where you can pass a key to an outside application from the php application then the key could be generated by php which has already authenticated the user and knows who they are. The key could be passed into the Flash movie or java applet or outside application and passed back to Moodle.
As much as possible isn't it better to be passing around keys that are valid for a limited amount of time rather than sending passwords and usernames back and forth. And also this would allow a Flash / Java applet to interact with Moodle through web services without the user having to provide the Flash movie / java applet / ajax application with his/her username and password.
Thanks,
Jamie
Ahhhh.... got it now!
I think this is a very good point. I would want it to be an optional mode of operation though, rather than a client being forced to obtain a key.
I think this is a very good point. I would want it to be an optional mode of operation though, rather than a client being forced to obtain a key.
Hi All,
Does anyone have any detail on the current status of the ws development?
I notice there is now a wiki page with an outline of the intended functionality.
I need access to moodle from .NET (to create MS SharePoint Portal Server WebParts for moodle administration)
So far i have used either the .NET MySql DataAdapter, for simple read-only db queries; or copying chunks of existing moodle code into custom PHP scripts which accept relevant GET/POST data from the calling .NET WebPart.
Main features I need so far are:
Get user's courses
Get user's assignments
Create course with enroled students
Is there any CVS/preview/alpha code that would be suitable for these tasks yet, or would i be better off waiting and carrying on using my hacked scripts in the mean time?
Any advice gratefully accepted
Does anyone have any detail on the current status of the ws development?
I notice there is now a wiki page with an outline of the intended functionality.
I need access to moodle from .NET (to create MS SharePoint Portal Server WebParts for moodle administration)
So far i have used either the .NET MySql DataAdapter, for simple read-only db queries; or copying chunks of existing moodle code into custom PHP scripts which accept relevant GET/POST data from the calling .NET WebPart.
Main features I need so far are:
Get user's courses
Get user's assignments
Create course with enroled students
Is there any CVS/preview/alpha code that would be suitable for these tasks yet, or would i be better off waiting and carrying on using my hacked scripts in the mean time?
Any advice gratefully accepted
This is a "me too!" post.
I'm a Perl programmer who's looking at all the functionality that's already in Moodle, but got stuck trying to access it all through our firewall. SOAP seems the perfect answer. The Perl module SOAP::Lite handles both SOAP and XML-RPC, so I'm agnostic in that respect.
I could really use a Web Service that:
handles login and security
gives me a list of available courses
adds, disables, deletes users
joins users to courses
The SOAP::Lite will handle cookies (to my understanding) and WSDLs, but not necessarily both.
I'm a Perl programmer who's looking at all the functionality that's already in Moodle, but got stuck trying to access it all through our firewall. SOAP seems the perfect answer. The Perl module SOAP::Lite handles both SOAP and XML-RPC, so I'm agnostic in that respect.
I could really use a Web Service that:
handles login and security
gives me a list of available courses
adds, disables, deletes users
joins users to courses
The SOAP::Lite will handle cookies (to my understanding) and WSDLs, but not necessarily both.
Very much onto exactly what you are asking for. The initial development will definitely be XML-RPC and it will all be bundled into a single directory that you can drop into 1.6 to try out. I have quite a lot working and am writing some example clients.
Soon, soon - just around the next corner
Soon, soon - just around the next corner
It is a stupid question but will it allow to see courses from a distant moodle ?
For example you want to share a course between two moodle but you want it stored in only one moodle and see it in both places like being owned by both...
I'm not sur i'm really clear...
For example you want to share a course between two moodle but you want it stored in only one moodle and see it in both places like being owned by both...
I'm not sur i'm really clear...
Amelie, why are you interested in doing that?
I ask because I am currently looking for funding (a bid is being considered at the moment by JISC) to do a project based around something very similar;
Using web services & moodle to allow lecturers/teachers at different institutions to collaborate and share material by connecting individual courses together that are hosted on different moodles. So 5 teachers at different institutions that teach the same course or subject and can share documents, quizzes etc and communicate with each other.
Is this similar to what you have in mind?
I ask because I am currently looking for funding (a bid is being considered at the moment by JISC) to do a project based around something very similar;
Using web services & moodle to allow lecturers/teachers at different institutions to collaborate and share material by connecting individual courses together that are hosted on different moodles. So 5 teachers at different institutions that teach the same course or subject and can share documents, quizzes etc and communicate with each other.
Is this similar to what you have in mind?
Yes that's what I have in mind, to be able to share ressources between moodles.
We are setting up a federation between different moodle using shibboleth to manage authentication but it's not totally satisfying...
Yes definitely. This kind of thing is firmly on the Moodle Roadmap.
You can get a flavour here... http://docs.moodle.org/en/Community_hub
You can get a flavour here... http://docs.moodle.org/en/Community_hub
Any news?
We may be forced to start attempting to create our own in time for September, which obviously is far from ideal.
Would love to have an update!
We may be forced to start attempting to create our own in time for September, which obviously is far from ideal.
Would love to have an update!
you can use SMS Service provider to send sms.