I think that web services are the way to go for integration between Moodle and other e-learning software. So I was happy to see Martin's announcement that LAMS will be integrated into Moodle via a web service interface.
The quiz module recently went the same way and is using web services to render and score question types that Moodle can not render natively, like advanced mathematical questions that need to be marked by a computer algebra system.
One practical technical issue that arises is that Moodle only requires PHP 4 but native support for SOAP was introduced only with PHP 5. There is a library called nusoap that provides SOAP functionality also in PHP 4. Martin just has added it to lib/nusoap/.
Ideally Moodle will want to use the native SOAP functions when it is running under PHP5 and nusoap when it is running PHP4. To achieve that Alex has written an extra library that gives the same API to both the native SOAP functions and nusoap. You can find it in mod/quiz/questiontypes/rqp/unisoap.php. It would naturally belong into the lib directory as well but only if other SOAP developers find it useful. Perhaps we can discuss that in this thread.
I didn't even know about those before I added lib/nusoap and I'm all for having one single, standard, flexible library so by all means add/alter things in that central directory!
Hi,
great!! Having RPC (SOAP, XML-RPC...) client/server possibilities "built into" Moodle is, really a nice idea.
Using Moodle like a web service client is pretty easy. Without being a RPC expert at all, some time ago I wrote a simple block (http://cvs.sourceforge.net/viewcvs.py/moodle/contrib/block_googleapi_search/) that make requests against Google's SOAP API to embed search results into Moodle. Simple and effective!
(note: I don't remember why, but I found some problems with the nusoap library and PHP5 so I had to slightly change it to be used both under PHP4 and PHP5 without problems. The modified library is inside the contrib code too). So, developing a web service client for moodle it's a possibility since a long time ago (having a central library available will help, sure, but it isn't mandatory at all).
But using Moodle like a web services server imply some serious consideratios because we are talking about a new (great, cool, amazing..) door to Moodle. Things like authentication, authorisation ... must be resolved an specified before start, sure! And I'm not really sure if making available the full Moodle's API to remote services is the correct approach. In my opinion there are two (three) different ways to achieve this (Moodle becoming a web services server):
- Create an Moodle's standard plugin system (similar to modules, blocks...). Once plugged, those Moodle's Web Services (MWS) will have full access to Moodle's API, DB... and every MWS service has the responsibility of use such API correctly, make authentication ... and so on. Moodle simply will receive requests, dispatch it to the proper plugged MWS and all the work will be done by the service, being its responsibility to build the response and sent it back to the client. The main benefit for this will be the customisation level and the unlimited access from any MWS to the full server structure. Everyone can develop its own MWS to suit his exact and particular needs. Cons are that, the same type of request will be generate different responses, depending of the MWS being used and, as moodle's core changes continuosly, maintaing all those MWS working can be really difficult. Another con for this is that such plugin system will be RPC-implementeation dependent, i.e. we have to decide about SOAP, XML-RPC...
- Define one central and abstract MWS, capable of accept a well defined number of requests. The number of available methods (and their specifications) will grow continuosly. Think that this abstract MWS won't be accessible outside Moodle at all. Over it (and exposed outside Moodle) we can have as many implemented MWS interfaces (wrappers) as we want (SOAP, XML-RPC..., one for each supported protocol). Such interfaces will receive the request, make authentication, validate the request and 'launch' the needed method in the abstract, private MWS. The main benefits for this approach is that code executed is completely central and Moodle decides what it exposes to the world. This will allow to make the same type of request from different PRC systems, obtaining a coherent response always. Things like authentication, validation of requests, structure of allowed requests and responses will be defined centrally. And maintaining such code should be really easier. Cons are that developers will lose some capabilities (because they are limited about what they can do (available methods are defined by Moodle abstract MWS) and, perhaps it will require more work at start.
- One third alternative is a mix of the above. Thinking (in my opinion), that the second alternative is better in the long term because it's a more organised way to do the things (code is centralised, methods available are standard and maintained centrally, authentication and validation is controlled) I assume that this is the selected approach. Then, adding plugin possibilities to build your own real WS protocols (the top layer, SOAP, XML-RPC, RMI,REST... services for example) and to add (in a plugin architecture too) available methods to the internal MWS (the bottom layer). Having these two layers in a plug-in architecture will allow 3rd parts to build practically everything (both RPC protocols (standard or propietary) and requested methods). All they have to do is to follow Moodle's architecture for those two layers.
Oh, that is becoming really a looong post. I was only trying to expose my opinion about how Moodle could offer (serve) webservices in a organised and correct way (from my perspective). Discussions about what WS implementation to use should be, in my opinion, simply out of the scope of MWS. Building the desired protocol to support MWS it should be, in theory, relatively easy, and it will provide the possibility to access to the same info from different services in a consistent way. Only a WS wrapper (to receive requests, authenticate, validate, communicate with the internal MWS and to send the response back to the client) need to be developed to suit each different RPC protocol.
Does it sound easy? Sure it isn't!!
My 1.5 cents...ciao
great!! Having RPC (SOAP, XML-RPC...) client/server possibilities "built into" Moodle is, really a nice idea.
Using Moodle like a web service client is pretty easy. Without being a RPC expert at all, some time ago I wrote a simple block (http://cvs.sourceforge.net/viewcvs.py/moodle/contrib/block_googleapi_search/) that make requests against Google's SOAP API to embed search results into Moodle. Simple and effective!
But using Moodle like a web services server imply some serious consideratios because we are talking about a new (great, cool, amazing..) door to Moodle. Things like authentication, authorisation ... must be resolved an specified before start, sure! And I'm not really sure if making available the full Moodle's API to remote services is the correct approach. In my opinion there are two (three) different ways to achieve this (Moodle becoming a web services server):
- Create an Moodle's standard plugin system (similar to modules, blocks...). Once plugged, those Moodle's Web Services (MWS) will have full access to Moodle's API, DB... and every MWS service has the responsibility of use such API correctly, make authentication ... and so on. Moodle simply will receive requests, dispatch it to the proper plugged MWS and all the work will be done by the service, being its responsibility to build the response and sent it back to the client. The main benefit for this will be the customisation level and the unlimited access from any MWS to the full server structure. Everyone can develop its own MWS to suit his exact and particular needs. Cons are that, the same type of request will be generate different responses, depending of the MWS being used and, as moodle's core changes continuosly, maintaing all those MWS working can be really difficult. Another con for this is that such plugin system will be RPC-implementeation dependent, i.e. we have to decide about SOAP, XML-RPC...
- Define one central and abstract MWS, capable of accept a well defined number of requests. The number of available methods (and their specifications) will grow continuosly. Think that this abstract MWS won't be accessible outside Moodle at all. Over it (and exposed outside Moodle) we can have as many implemented MWS interfaces (wrappers) as we want (SOAP, XML-RPC..., one for each supported protocol). Such interfaces will receive the request, make authentication, validate the request and 'launch' the needed method in the abstract, private MWS. The main benefits for this approach is that code executed is completely central and Moodle decides what it exposes to the world. This will allow to make the same type of request from different PRC systems, obtaining a coherent response always. Things like authentication, validation of requests, structure of allowed requests and responses will be defined centrally. And maintaining such code should be really easier. Cons are that developers will lose some capabilities (because they are limited about what they can do (available methods are defined by Moodle abstract MWS) and, perhaps it will require more work at start.
- One third alternative is a mix of the above. Thinking (in my opinion), that the second alternative is better in the long term because it's a more organised way to do the things (code is centralised, methods available are standard and maintained centrally, authentication and validation is controlled) I assume that this is the selected approach. Then, adding plugin possibilities to build your own real WS protocols (the top layer, SOAP, XML-RPC, RMI,REST... services for example) and to add (in a plugin architecture too) available methods to the internal MWS (the bottom layer). Having these two layers in a plug-in architecture will allow 3rd parts to build practically everything (both RPC protocols (standard or propietary) and requested methods). All they have to do is to follow Moodle's architecture for those two layers.
Oh, that is becoming really a looong post. I was only trying to expose my opinion about how Moodle could offer (serve) webservices in a organised and correct way (from my perspective). Discussions about what WS implementation to use should be, in my opinion, simply out of the scope of MWS. Building the desired protocol to support MWS it should be, in theory, relatively easy, and it will provide the possibility to access to the same info from different services in a consistent way. Only a WS wrapper (to receive requests, authenticate, validate, communicate with the internal MWS and to send the response back to the client) need to be developed to suit each different RPC protocol.
Does it sound easy? Sure it isn't!!
My 1.5 cents...ciao
My gut feel about the core, though (without having tried anything yet) is to make it pass requests to the bottom layer with as little interference as possible. It would be annoying as a module/plugin writer to need extensions to the core everytime you wanted to extend web services to do something innovative in your module.
Abstracting the core MWS from the top-layer protocols is a great idea, Eloy - we must definitely do that. Perhaps something already exists out there that we can use.
I agree with you, webservices ara a good way to go for integration betwen Moodle and whatever you want... Even moodle modules themselves could be implemented as webservices. This would give scallability to moodle. Meaning... if you have a very loaded server, and - for instance - the quizz module use the 20% of the server... you could implement the quizz module as a webservice and install it in another machine... Its a simple a and cheap way to scale your server... no clusters , no multiprocessors.
I implemented last year a experience in this line with the spellchecher. The spell check aspell program can be very resource demanding, so I changed the html editor spellcheck function to call a webservice mounted in another server... It works fine ( is someone wants it I'll send the package.)
But in my opinion SOAP is a very heavy protocol... it wraps the xml data with tons of redundant information... data that fills your bandwidth and has to be generated and parsed (its a recursive parser, so a lot of memory consumed in the stack). An alternative is XML-RPC wich works fine with PHP, and is ligther... Maybe it does not support all the functionalities of SOAP, but before getting married with SOAP we should think if we really need it, or if XML-RPC can be enouth.
Cheers
Marc
I implemented last year a experience in this line with the spellchecher. The spell check aspell program can be very resource demanding, so I changed the html editor spellcheck function to call a webservice mounted in another server... It works fine ( is someone wants it I'll send the package.)
But in my opinion SOAP is a very heavy protocol... it wraps the xml data with tons of redundant information... data that fills your bandwidth and has to be generated and parsed (its a recursive parser, so a lot of memory consumed in the stack). An alternative is XML-RPC wich works fine with PHP, and is ligther... Maybe it does not support all the functionalities of SOAP, but before getting married with SOAP we should think if we really need it, or if XML-RPC can be enouth.
Cheers
Marc
One of the problem with webservices is speed (converting data to XML and vice versa). Is SOAP a mandatory component of webservices and if so can not be replaced by XML-RPC?
Thanks!
This is really simply a question of what protocol the web service uses that Moodle wants to connect to. So if the web service uses XML-RPC then Moodle needs to use XML-RPC and if the web service uses SOAP then Moodle needs to use SOAP. Both protocols use XML format for the messages.
So, Gustav, you're saying the SOAP and XML-RPC protocols can be equally speedy? (note: I just now see ML says XML-RPC is faster). We just choose one. I also wonder if some purposes for using LAMS will not need the highest real-time speed. For exchanging units or courses, it is an occassional operation, isn't it?
No, no, I didn't say anything about speed. I wouldn't be qualified either.
It's probably useful to clarify that we're talking about two things here:
(The LAMS work will require both consuming and providing)
- Moodle as a consumer of web services like the quiz module is doing (could be both SOAP or XML-RPC like Gustav mentions, so we still need a core SOAP library for consistency).
- Moodle as a provider of web services (most likely XML-RPC judging from recent comments).
(The LAMS work will require both consuming and providing)
We are beginning to use Axis (Apache Java WS) as a way to bring
WS into Moodle. The attached diagram shows how we are looking
to do this for a tool called eTEACH. We've used this approach on
the desktop in other projects and now are moving it to the web.
Axis can run within the same Apache http, though it does need
to fire up a Java servlet using Tomcat. There's a number of details
to work out, but in principle we can make this look seemless to
the "tools" in Moodle.
Here's another place where we have Moodle using AXIS
http://croquetlinux.doit.wisc.edu/moodle
login as student1/student1
go into BioChem 517
down in lecture 3, click on "What is Oxidative Phosphoralation?"
This launches into an externally hosted tool, using Web Services.
Go thru this, it's a mini-tutorial on this "concept". At the end,
do Check. This is an embedded assessment. Do some answers and then
do Report.
Back comes the result, in Web Service.
This is a demo of the new IMS Tools Interoperability.
That does use Web Services to provide for interoperability of tools.
We haven't made changes in Moodle to do this, though we did have
to figure out how the "quiz" stores it's results and mimic that.
This is going to be part of the Alt-I-Lab Demo at Sheffield, UK in June 2005,
see http://www.imsglobal.org/altilab. This is still an early version
of what looks to be a very important new specification for eLearning.
The reason we are using Axis is that we couldn't get the WS in PHP 5 to
work well enough. I have every expection that PHP will get to the same point, it's just a question of time (and who does the heavy lifting). Since IBM is working with Zend on PHP, perhaps IBM will do this. We took about 2 days
to try this, but then gave up and went to Axis.
We see the same dynamic in Flash, btw. The WS are ok, but not great.
If we keep it simple, things work fine. We expect this to get better with new versions of Flash, maybe Flash 8 will nail it.
WS into Moodle. The attached diagram shows how we are looking
to do this for a tool called eTEACH. We've used this approach on
the desktop in other projects and now are moving it to the web.
Axis can run within the same Apache http, though it does need
to fire up a Java servlet using Tomcat. There's a number of details
to work out, but in principle we can make this look seemless to
the "tools" in Moodle.
Here's another place where we have Moodle using AXIS
http://croquetlinux.doit.wisc.edu/moodle
login as student1/student1
go into BioChem 517
down in lecture 3, click on "What is Oxidative Phosphoralation?"
This launches into an externally hosted tool, using Web Services.
Go thru this, it's a mini-tutorial on this "concept". At the end,
do Check. This is an embedded assessment. Do some answers and then
do Report.
Back comes the result, in Web Service.
This is a demo of the new IMS Tools Interoperability.
That does use Web Services to provide for interoperability of tools.
We haven't made changes in Moodle to do this, though we did have
to figure out how the "quiz" stores it's results and mimic that.
This is going to be part of the Alt-I-Lab Demo at Sheffield, UK in June 2005,
see http://www.imsglobal.org/altilab. This is still an early version
of what looks to be a very important new specification for eLearning.
The reason we are using Axis is that we couldn't get the WS in PHP 5 to
work well enough. I have every expection that PHP will get to the same point, it's just a question of time (and who does the heavy lifting). Since IBM is working with Zend on PHP, perhaps IBM will do this. We took about 2 days
to try this, but then gave up and went to Axis.
We see the same dynamic in Flash, btw. The WS are ok, but not great.
If we keep it simple, things work fine. We expect this to get better with new versions of Flash, maybe Flash 8 will nail it.
"We haven't made changes in Moodle to do this, though we did have
to figure out how the "quiz" stores it's results and mimic that."
In Moodle 1.5 under PHP5 this will no longer be necessary, the quiz module can connect directly to web services for rendering and scoring of questions via the Remote Question Protocol. You can see this in action on the Serving Maths website. This is using the SOAP in PHP5 successfully. In future it will also allow XML-RPC as an alternative to SOAP.
to figure out how the "quiz" stores it's results and mimic that."
In Moodle 1.5 under PHP5 this will no longer be necessary, the quiz module can connect directly to web services for rendering and scoring of questions via the Remote Question Protocol. You can see this in action on the Serving Maths website. This is using the SOAP in PHP5 successfully. In future it will also allow XML-RPC as an alternative to SOAP.
I'd like to talk to someone from this project about "bringing it into IMS".
This fits nicely with Tools Interoperability work.
And we are very interested in bringing it into IMS 
. To learn more about the Remote Question Protocol take a look at http://mantis.york.ac.uk/rqp/. It's a Moodle site so there is a good discussion forum there. And call me (gustavdelius) or Niall Barr (niallbarr) on Skype if you want to discuss things in person.
. To learn more about the Remote Question Protocol take a look at http://mantis.york.ac.uk/rqp/. It's a Moodle site so there is a good discussion forum there. And call me (gustavdelius) or Niall Barr (niallbarr) on Skype if you want to discuss things in person.
... the quiz module can connect directly to web services for rendering and scoring of questions via the Remote Question Protocol. ...
What about the other side of coin, can moodle act as the web-service to dish out questions from quiz module to RQP clients?
You're right - I had assumed SOAP would be best without really considering XML-RPC. I like the idea of keeping things as simple as possible so will definitely examine XML-RPC first.
http://weblog.masukomi.org/writings/xml-rpc_vs_soap.htm
http://weblog.masukomi.org/writings/xml-rpc_vs_soap.htm
XML-RPC beats the pants off SOAP any day.
In most situations where you'd expect to actually pass heavyweight rich objects around, you want a language-specific facility, like RMI. So, in my book, SOAP has a narrow focus, and is rather top-heavy.
<aol>me too ;-)</aol>
In most situations where you'd expect to actually pass heavyweight rich objects around, you want a language-specific facility, like RMI. So, in my book, SOAP has a narrow focus, and is rather top-heavy.
<aol>me too ;-)</aol>
What maters of a protocol is what for is designed... XML-RPC is designed to make Remote Procedure Calls when server and client kwnos the logic of the service and what's at stake... so for the moodle intents is ideal.
SOAP is intended to allow clients and servers to dinamically meet themselves without theis programmers kwno what the frell they-re doing.. this is done by the use of laaaaarge midleware layers and providing the usefull datta with wraps lots and lots and loots of unirrellevant information. All of this is generated in a very laarge xml stream wich must be squeezed in your bandwidth, parsed and stored in a object model... to make it work you'll need a beautifull (and of course expensive) websphere server ( or a another J2EE toy ) or a .NET server... In a beautifull apache with PHP what do you get... a lot of trouble because memory stacks rise in the SOAP parse , PHP crashes and everybody is unhappy when the number of access frell your server. The you can run to your boss and tell him that we need to upgrade to J2EE or .NET cause LAMP is not scalable enought...
As Martin said if moodle has to act as a client it MUST use whatever protocol says the server... If moodle has to act as a server XML-RPC is a possible choice, in anycase the best option would be the less heavy possible ... afterwards we always can develop filers or proxies for XML-whatever for ocmpatibility...
Cheers
SOAP is intended to allow clients and servers to dinamically meet themselves without theis programmers kwno what the frell they-re doing.. this is done by the use of laaaaarge midleware layers and providing the usefull datta with wraps lots and lots and loots of unirrellevant information. All of this is generated in a very laarge xml stream wich must be squeezed in your bandwidth, parsed and stored in a object model... to make it work you'll need a beautifull (and of course expensive) websphere server ( or a another J2EE toy ) or a .NET server... In a beautifull apache with PHP what do you get... a lot of trouble because memory stacks rise in the SOAP parse , PHP crashes and everybody is unhappy when the number of access frell your server. The you can run to your boss and tell him that we need to upgrade to J2EE or .NET cause LAMP is not scalable enought...
As Martin said if moodle has to act as a client it MUST use whatever protocol says the server... If moodle has to act as a server XML-RPC is a possible choice, in anycase the best option would be the less heavy possible ... afterwards we always can develop filers or proxies for XML-whatever for ocmpatibility...
Cheers
First off, I agree that SOAP can be much more complicated and heavy over the wire than XML-RPC. However, it doesn't have to be. It just provides the ability for complex user-defined types and document, rather than RPC, models. As long as your aren't doing the SOAP stuff in PHP, you'll be fine. The modern C libs for SOAP and XML handling are rather fast.
You mentioned that SOAP provides the ability for two systems to agree on data types/structures. However, this is in the realm of WSDL. The SOAP spec does not automatically provide it. In the case of a typeless language like PHP, WSDL can be a godsend. It frees you from having to defines data types during your procedure call.
If anyone assumes that SOAP is always complicated and heavy, I invite you to look over the Google SOAP API. It's easy to use and amazingly light weight.
Let me just say that I am not saying SOAP is better than XML-RPC, REST, etc. They have their places. However, shouldn't we stick with a spec that is both appropriate and expected. SOAP fits both.
~Rob
You mentioned that SOAP provides the ability for two systems to agree on data types/structures. However, this is in the realm of WSDL. The SOAP spec does not automatically provide it. In the case of a typeless language like PHP, WSDL can be a godsend. It frees you from having to defines data types during your procedure call.
If anyone assumes that SOAP is always complicated and heavy, I invite you to look over the Google SOAP API. It's easy to use and amazingly light weight.
Let me just say that I am not saying SOAP is better than XML-RPC, REST, etc. They have their places. However, shouldn't we stick with a spec that is both appropriate and expected. SOAP fits both.
~Rob
I've been doing some mucking around in this area and SOAP so far has been successful. I've set up a central question bank that allows users to download category structures, create questions download questions and finally submit questions to the question bank. You can see the service at http://www.liquidboundaries.com.au/questionBank/questionBank.php.
I've got a plugin to moodle 1.4 which I'm mucking around with at the moment - It's semi complete and you can submit questions including the images with relative ease. If you want to have a look at how it works send me an email and I'll set you up on my moodle site.
This sounds interesting. I would like to try it. How do you envisage it being used? I tried to set up a SOAP thingy on my site to allow on-site access to a Japanese - English dictionary.
Tim
Tim,
Ideally the way it would be used is that people submit questions to the question bank under existing question bank categories. These questions are then validated to ensure they meet a minimum quality standard and potentially attached to new categories. Once this is done they are then made available to everyone else to download and use.
I'll set you up tonight AEST so you can go on and have a play tomorrow. There's one little bug that needs fixing before I can open it up.
Cheers
Andrew
Thanks Andrew.
So Repositories, leaning on things like this: http://www.loc.gov/standards/sru/srw/index.html are included also?
My concern is the 'learning curve' with SOAP. With a community of developers (in many cases software development is not their day job) I think we have an obligation to keep everything as simple and straightforward as possible. I fear that with that in mind, SOAP may be more than we need. XML-RPC is, IMHO, a lot easier to get to grips with.
The quiz module recently went the same way and is using web services to render and score question types that Moodle can not render natively, like advanced mathematical questions that need to be marked by a computer algebra system.
Need to know more about quiz modules web service support. Please point me to the right resources.
Thanks
Tahir
Hello,
I developped a full implementation of WebServices based on the Moodle database. You can try it if you want, it is SOAP 1.1 and SOAP 1.2 compliant so you can easily consume the webservices from NuSOAP/PHP, .NET, JAVA,....
http://www.pagsoft.com/MoodleWebServices
I've done 3 implementations : Moodle 1.5.4, Moodle 1.6.1, and the latest 1.7
Each database table is represented by a WebService Endpoint. For instance, table « User »’s information is accessible by the User web service (ex://localhost/MoodleWebServices/User.asmx).
2 – The webservices define an information object which reflects the table schema. This object is used with the corresponding web service endpoint.
3 – For each web service endpoints, the following methods are available:
- endpoint.Add(object info)
- endpoint.Delete(object info)
- endpoint.Update(object info)
- endpoint.GetByPK(pk) for each PK in the table
- endpoint.GetAll
Have a good day!