The big problem/opportunity I am running into is that a lot of the operations required are not available as any kind of library function. For example even though the code is trivial, new categories are created by a database call in course/index.php. Much more complicated a new course is created or edited (and a mass of checks on the entry data) is all performed 'in-line' in the course folder too. So it goes on...
What I would like to do is to refactor these functions into a library. I am supposing moodlelib.php, but possibly other developers have a different/better opinion on that, as maybe this will be the remote control api talked about in the roadmap - that always seemed *very* closely related to web services to me. So perhaps a new library file (moodleapi.php or somesuch) is better.
The blocks situation is slightly nastier and makes me want to growl about separating business logic and the view The class provides a method 'get_content' which regrettably gets the data (we want for web services) and generates all the output intertwined. I really need to split this up, but my proposal is to provide two new methods - probably get_data() and view_data() to split the functionality. The default method for get_content() then becomes a call to the above two functions, ie, all existing blocks still work fine (but not with web services) until we get around to refactoring them.
I would like to get on with some of this urgently. I am hoping to for 1.7, but some of it is obviously a bit drastic from a testing point of view.
As ever, comments and/or abuse appreciated!
For now just a very simple and secure interface for:
- accepting the messages,
- passing them to the relevant library file to a standard function there,
- passing back the results,
would be a great start and would not impact all the other refactoring we're currently doing (Roles and DB Schema)
We can worry about the nuts and bolts of individual commands later.
Understood, but, in most cases there is no standard library function (unless I am missing something), no library function for creating a user, no library function for creating a new course.... etc., etc., Without these basic functions, the web services api would be nowhere near as useful.
It is no problem to reproduce this functionality (which in many cases is going to be lengthy with all the necessary checks), but it does seem a bit wasteful. Either way, does it make sense to make these into some sort of api library, which would simplify the path to refactor these functions once they have had some time to be used and tested? Two birds with one stone as it were.
The blocks thing is more of a showstopper and I know that you are keen to expose that data - I was trying to think of a low risk way of extending the functionality.
What happened to 1.8?
The existence of library with standard functions is a must especially when dealing with large data, and it's importance is out of question according to my humble opinion.
For the purpose of the web service - how about more object approach:
I suggest creation of class – this class will has data properties filled by the existent functions (or it’s own wrapper methods on later stage), than the data holded in the class could easily be serialized with SOAP (if the object is executed remotely and of course in case that a transportation of the result set is needed) by the web service.
Recently I've coded generic data helper class (which is useful when dealing with hierarchical data) available at here.
In addition - for the block case it could be an elegant solution to implement 2 interfaces:
- IData - for data retrieval from various data sources (db, xml, custom data objects, etc.).
- IRender - for rendering data to various targets - like xml, html, etc.
I hope some of this to get in work…
What I meant by a standard external request processing function like forum_external(), moodle_external(), block_external() in relevant libraries.
These could accept some sort of standard object as a parameter (produced by the web services API), and then put it through basically a big switch statement which calls relevant old and sometimes new local functions to do the work. A layer if you like.
This sort of approach doesn't impact the current API at all, and can possibly be migrated later to a more thorough refactoring.
- If you think you can abstract a few things into functions, without breaking anything, in small well tested patches, I think we can fold them in.
- In terms of secure WebServices, one of the outcomes of the moodle network / community hub project is a messaging layer based on WS, with optional encryption/signing. This messaging layer can probably do what you want.
What I would like to do is to refactor these functions into a library.
I'm all for moving major functionalities of Moodle into a library (or multiple libraries) so that everyone's code can make use of it. Also, by building a more overarching API, more technologies and possibilities become available like Hijax.
Very good idea to start implementing some such functions, in the cautious way that Juliette (hi!) suggests.
* Build an external api library.. completely new, nothing (yet) depends on it, with stuff like 'create a user', 'create a course', 'create a catregory'....
* This can be called directly as an api for this sort of stuff and/or by the myriad of web services projects that seem to be going on.
* Aiming for the hard-core basics of this to be in place for 1.7?
Question: should this be a completely new library file (which no existing code will require/include anyway), or should it be in an existing library. My inclination, is to go for new library - lib/moodleapi.php - as if it gets broken, nothing else does.
Seems like the low risk route, and refactoring of these functions can be done later.
PS: I'd like an opinion on my small alteration to the blocks parent class to enable the business logic and view to be separated. Again, note that this would not affect existing blocks.
I fully support the idea of a central "core api". So this API could not only be used for remoting, but would in general allow for a clear separation of business logic and presentation layer.
I talk about kind of a (business) delegate pattern here. For those of us not too deep into software engineering: the (business) delegate pattern in short is a way to structure an application or system to loosely couple presentation and logic. That way parts of the application can easily be exchanged or extended/refactored without the need to "touch" lots of modules. When applying a delegate pattern you have one (or at least a limited number) of central apis containing functions which can be called by presentation layer (or remoting services). The logic for these functions is not contained in the API itself, it just "delegates" the call and afterwards returns the result to the initial caller.
What would this mean for Moodle?
The hundreds of functions now spread all over Moodle stay where they are. This way existing legacy code remains compatible. We only introduce a "delegator" - an API containing only the most relevant core functions I would suggest.
The functions contained in the delegator would have to be clearly defined in means of input and output parameters. They should STRICTLY NOT fire any presentation layer code or raise errors, print messages or any stuff like that. They should ALWAYS return silently indicating errors etc. via return parameters (or exceptions if we once fully switch to PHP5).
These functions in the central API could then be safely used by any remoting architecture (like SOAP, XML-RPC etc) or simply by any developer just looking for "standard tasks". That way we would open Moodle up a lot, allowing for easier and faster development cycles!
I can only talk for myself, but when I was starting to work with Moodle I was kind of struck. For my thesis I developed a mobile rich client which should access Moodle functionality. For a newbie developer it is kind of horror finding the correct "core" functions spread all over the libraries. Not to mention that not all of them are documented too clear. And what was worse for me - a large number does not seperate logic from presentation. So the only way for me for the moment was to "reproduce" the business logic to meet my requirements.
Most of the major functionalities are already in libraries ...
Yes of course, sorry. I was referring to ones that were not already in the standard libraries like the ones Howard was mentioning.
I would be happy to help test anything, and potentially help develop as I have a few years of PHP under my belt but however am still quite new to the Moodle Way.
The ldap enrolment is too restrictive for us unfortunately (and our MIS data is virtually useless for direct relationships of users/groups/membership)
I know we can sync the ldap directory periodically but it has been requested that we avoid this approach to avoid problems we've had in the past with other systems/VLEs whereby the userlist would always be at least 'n' minutes behind the 'authoritative' source - AD/Active Directory.
I manually added a record to the user table (matching an existing user from AD) and that seemed to work fine. Obviously I need to be able to do this programatically, and would prefer to use higher level moodle code than just attacking the database directly.
So far I have taken two approaches:
First I started looking at the ldap auth code to see if it was possible to add a single user by username, but it appears i can only do this from Moodle>LDAP rather than LDAP>Moodle (without syncing the whole DN).
Currently, I am scanning /user/edit.php for the necessary code but have got very very lost and can't even find where it creates the db record. (Roll on Moodle API!)
Any advice gratefully accepted
Hi,<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
I too am developing a standard API for calling Moodle functions, which will then be called from various places including a web service. I am focussing specifically on creating courses and resources rather than users, and am developing functions that will “wrap” the existing functions, in order to make the job of the calling program simpler.
Unfortunately I will not be able to post my code in the public domain in its entirety, as it is interwoven with code that was developed for internal use only, and I do not have sufficient time to write a completely generic API class to post to moodle.org. However, I can advise and assist with the development of a library of generic functions, that could appear in a future Moodle release.
Here is a very rough list of the sort of functions I am developing and their functionality – these will be methods of a library class.
Creates a course and fixes sort order.
i. Passed an object representing a course record (with properties representing the required fields).
ii. Validates that the required properties are present, defaults certain values (timecreated, timemodified).
iii. Defaults sortorder field prior to record insertion and calls fix_course_sortorder before and after insert to ensure that sortorder field contains valid values.
iv. Calls addslashes on the database fields and inserts record into the course table.
v. Returns the ID of the new course.
Provides a simple method to create an HTML resource, and links it to a specified section in the course page.
i. Passed as arguments: course, section (section number within page rather than unique ID), name, alltext (the HTML to display), summary, windowpopup, windowpage (these last two govern whether new window should be opened or display in same page).
ii. Looks up the section number. Can also create a new section with the next consecutive section number if a dummy section number is passed.
iii. Calls resource_add_instance from mod/resource/lib.php
iv. Calls add_module_to_section (see below).
v. Returns the new resource ID number.
Provides a simple method to commit a file to Moodle, and links it to a specified section in the course page.
i. Passed as arguments: course, section (section number within page rather than unique ID), name, sourcefile (the name and path of the resource file), summary, windowpopup, windowpage (these last two govern whether new window should be opened or display in same page)..
ii. Internal operation is similar to add_resource_html except that it also copies the file to the correct location within the moodledata file store.
iii. Returns the new resource ID number.
Provides a simple method to add a label to a section in a Moodle course.
i. Passed as arguments: course, section (section number within page), content, headingfont (governs whether label should be in bold font as for a heading or normal text format).
ii. Calls label_add_instance from lib.php.
iii. Calls add_module_to_section.
iv. Returns the label ID number.
This function is called from the add_resource_html, add_resource_file and add_label functions to add a label, file link or html page to the correct section on a page.
i. Passed as arguments: course, module type, module instance no. (unique ID), section (section number within course), summary text, visible (if a new section is to be generated, governs whether or not it will be visible to students).
ii. Ensures that the course exists (otherwise throws exception).
iii. If passed a dummy value for section, (-1) it calls Create_Section to create a new section, after the highest-numbered existing one within the specified course, and add the module link or label to that section (this feature is merely a time-saver).
iv. It calls Moodle library functions add_course_module and add_mod_to_section to create the module record and link it to the correct section.
v. It also sets the module’s visible status, dependant upon that of the overall section. (Note that an option to override this could be added to the parameters).
vi. It calls rebuild_course_cache.
i. Passed as arguments: course, section (i.e.section number within course), summary, visible (whether or not visible to students).
ii. Inserts a record into course_sections with the correct details.
iii. Returns the new section’s unique ID number.
i. Passed as arguments: course, section (i.e.section number within course), noexception (boolean – whether to throw exception).
ii. If the section exists, it returns the section ID number (the unique ID).
iii. If section does not exist, it either returns -1 or throws an exception dependant upon value of noexception.
Where data is inserted into the database, addslashes() is called as a matter of course, but clean_param() is not used, as it is assumed that these functions will always be called from other code which will determine whether or not the inbound data requires sweeping for scripts etc.
It would also be desirable to develop “update” versions of these functions to allow the courses and objects within them to be updated without deleting them completely, but that is less of a priority for myself at the moment.
For live, automatic import of authentication and authorization data, course set up, etc, wouldn't LDAP be a good way to access user and course data from outside Moodle?
And while there were functions, particularly in terms of defining roles, that I found just after writing direct sql inserts, there's no one place to look them up and see what they do, nor in fact do they all work without munging the includes (see MDL-8278 in the tracker for example).
I've created a forum for discussion, support and flames:
We'd greatly appreciate feedback from all interested Moodle developers.
I have just checked in our work on Web Services to contrib. This is a working implementation using SOAP. We also utilize WSDL to make it easier for clients to access the API.
The release is in 'contrib/ws' and contains a document describing the API, and how to use it. It contains methods for managing users, courses and grades. We welcome any comments.
As I've mentioned before, we've also developed a web service, using xml-rpc. Our development focused on quizzes and questions. We've been using this in production for quite a while on 1.4 and 1.5 moodles. There was some hesitation expressed at integrating our library, because of possible licensing issues based on the PEAR library classes used.
For a new project, we're going to need to do quite a bit with web services for Moodle 1.7+ - again focusing on quizzes and questions, but not limited to that. Are you open to our adding code to your web services code? With the idea that we begin to build a library of functionality?
We would definitely be open to adding code. In fact the design of our code is such that adding another transport type (like XML-RPC) is relatively easy because the core functionality of the service calls (i.e. get_course()) is extensible into a server sub class that contains any transport implementation details.
Our example uses SOAP via the NuSOAP library included with Moodle but it would be fairly easy to A) extend what we have to use XML-RPC and B) add new functionality to the core service class which could then be specially handled (if necessary) in any extended transport classes.
The project planning is still being done, so we're not touching any code yet. Probably in January/February we'll get seriously into coding.
Currently we're leaning towards SOAP. Some of the services that look like they will need to be implemented:
quizzes (export imsmanifest & receive answers)
export a collection of learning resources (activities, resources: again with ims)
logging of activities
reporting (who did what when etc.)
Having the plumbing in place frees you up to implement the services
Hi Martin and Justin,
I'm wondering what the status of contrib/ws is. We're working on a large project now, and part of that will require us to expose a good deal of Moodle actions ( login, logout, create/edit question, create/edit quiz, create forum, etc.) as web services.
So, is anybody actively working on this code? It seems that the current version is not made for 1.7; there are errors when trying to run the database update code; the error message says that one should be using the new xml-db stuff.
Anyway, we would like to use it as a basis for our development, and to offer our code back to the community. And we'd of course like to work in close collaboration with whoever else is working on it, so that nobody goes around reinventing wheels. Personally, I'm leaning towards using rest instead of soap, because of speed and ease issues.
It would seem to me to be a good idea to break the actual calls to the moodle code out from the server classes; there's no need for soap and xmlrpc and rest implementation to all duplicate code. It would also make sense to me to create library files to group the code that's actually accessing moodle (e.g. wslib.php for general things, wsuserlib.php, wsforum.php, wsquestion.php, etc.).
### and then a little bug report:
I tried running the test code, but I see that the service url (/ws/service.php?type=soap) fails when I call it directly:
*Fatal error*: Cannot redeclare class soapserver in */var/www/host/html/m17/ws/soapserver.class.php* on line *38*
This is because we're running php5 with it's native soap extenstion, which has a SoapServer class.
An idea would be to use a naming convention like moodleserver instead of server. This would result in names like moodlesoapserver, moodlexmlrpcserver, moodlerestserver for posts service.php?type=soap, service.php?type=xmlrpc, service.php?type=rest
Thanks for your time!
Hi Brian, I have _no_ idea where contrib/ws is or who is working on the code. A quick check of the cvs history should give you a bit more info on the vitality of the code, and who's involved. As part of moodle network [coming to 1.8] there is good XML-RPC transport mechanism that can optionally expose the moodle internals to specific hosts. This is exactly as dangerous (and as useful) as it sounds ;-) Not all moodle operations have a good API call for them. Surprisingly, create user, update user, create/update course and a few central ops are not consolidated into function calls. Before you get too nervous, there are good reasons -- and abstracting them into well designed function calls would take quite a bit of careful work to get things like error handling right. Also, bear in mind that WS is _not_ a good vehicle to try to do login/logout. You should write an auth plugin for that.
using rest instead of soap, because of speed and ease issues.
The code in 1.8 uses XML-RPC because (a) PHP includes a C implementation of XML-RPC that is fast and solid, (b) everthing under the sun can do XML-RPC, not just the fancy chic languages, and (c) it is a lot more debuggable than REST, SOAP and JSON. HOWEVER, you can implement additional transports. Have a look ;-) It is not 100% perfectly abstracted, but it is 95% abstracted. Adding additional protocols should be trivial. But I doubt rest/soap or json are different enough to warrant the effort. We put the abstraction in place to make sure we support really different protocols when we need it ;-)
that's actually accessing moodle (e.g. wslib.php for general things, wsuserlib.php, wsforum.php, wsquestion.php, etc.).
There is a bit of that in the 1.8 code -- but the separation is for security purposes. The coder of a function or a class _must_ declare the function as callable via Moodle Network facilities. It cannot happen by accident. The "promiscuous" XML-RPC mode, however, doesn't have those checks.
Thanks for your time!
No problem -- but I'd like to make this thread public and so all the Moodle WS stuff gets automagically documented and people can get involved. I get about 200 private emails per day about FAQs and it doesn't make any sense to reply to them in private. WS and MNET are FAQs for me lately, it's much smarter to have this in the General Dev Forum. cheers m
we have installed your scripts on our test system, but I think we have made something wrong.
We have made the following steps:
- copy /ws under http://mydomain/moodle/ws
- create tables webservices_clients_allow
- insert values for webservices_clients_allow
- modify username and password in ws-test.php
I insert a username and password from a moodle user there.
Is this correct ?
When I start my browser with http://mydomain/ws/ws-test.php, I get the following error:
Attempting to create a server connection...
Sent server call...
Error: HTTP Error: no data present after HTTP headersThat means for me, I cann't get a login. Any idea ?
2006-12-21 10:26:34.700723 wsdl: have 2 part(s) to serialize 2006-12-21 10:26:34.701364 wsdl: have 2 parameter(s) provided as arrayStruct to serialize 2006-12-21 10:26:34.701512 wsdl: serializing part "username" of type "http://www.w3.org/2001/XMLSchema:string" 2006-12-21 10:26:34.701641 wsdl: calling serializeType w/named param 2006-12-21 10:26:34.701779 wsdl: in serializeType: name=username, type=http://www.w3.org/2001/XMLSchema:string, use=encoded, encodingStyle=, unqualified=qualified value=string(8) "username" 2006-12-21 10:26:34.701945 wsdl: in serializeType: got a prefixed type: string, http://www.w3.org/2001/XMLSchema 2006-12-21 10:26:34.702075 wsdl: in serializeType: type namespace indicates XML Schema or SOAP Encoding type 2006-12-21 10:26:34.702244 wsdl: in getTypeDef: type=string, ns=http://www.w3.org/2001/XMLSchema 2006-12-21 10:26:34.702382 wsdl: in getTypeDef: do not have schema for namespace http://www.w3.org/2001/XMLSchema 2006-12-21 10:26:34.702523 wsdl: in serializeType: returning: username 2006-12-21 10:26:34.702652 wsdl: serializing part "password" of type "http://www.w3.org/2001/XMLSchema:string" 2006-12-21 10:26:34.702774 wsdl: calling serializeType w/named param 2006-12-21 10:26:34.702910 wsdl: in serializeType: name=password, type=http://www.w3.org/2001/XMLSchema:string, use=encoded, encodingStyle=, unqualified=qualified value=string(8) "password" 2006-12-21 10:26:34.703057 wsdl: in serializeType: got a prefixed type: string, http://www.w3.org/2001/XMLSchema 2006-12-21 10:26:34.703182 wsdl: in serializeType: type namespace indicates XML Schema or SOAP Encoding type 2006-12-21 10:26:34.703326 wsdl: in getTypeDef: type=string, ns=http://www.w3.org/2001/XMLSchema 2006-12-21 10:26:34.703458 wsdl: in getTypeDef: do not have schema for namespace http://www.w3.org/2001/XMLSchema 2006-12-21 10:26:34.703595 wsdl: in serializeType: returning: password2006-12-21 10:26:34.703722 wsdl: serializeRPCParameters returning: usernamepassword 2006-12-21 10:26:34.703880 soap_proxy_1547403706: wrapping RPC request with encoded method element 2006-12-21 10:26:34.704039 soap_proxy_1547403706: In serializeEnvelope length=190 body (max 1000 characters)=usernamepassword style=rpc use=encoded encodingStyle=http://schemas.xmlsoap.org/soap/encoding/ 2006-12-21 10:26:34.704161 soap_proxy_1547403706: headers: bool(false) 2006-12-21 10:26:34.704293 soap_proxy_1547403706: namespaces:The username and password are not being parsed into XML properly. Any help with this issue would be greatly appreciated.
I tried running this also under moodle 1.7.1, using php5.
I had these problems:
- our php5 is compiled with the soap extension, so theres a conflict with the built-into-php SoapServer class.
However the PHP5 issue probably still exists in the latest code as it stands. I've had zero success using the wrapper library for SOAP that exists in Moodle (as of 1.6?), /lib/soaplib.php. It's supposed to easily support using NuSOAP on PHP4 and the built-in class in PHP5. I've got an easy fix for this and supporting both PHP4 and 5 within the code I have so that's not a huge problem.
Right now I can't access Contrib for some reason so I need to get that problem solved before I can make any of this new code readily available to anyone here.
If anyone finds this useful, please send us some feedback.
However, if you want to get up and running quick here's what you need to do:
- Create a password file. The supplied .htaccess expects to find this file at /etc/htpass To create a password file, use the following command
htpasswd -c /etc/htpass username
- Rename the file htaccess to .htaccess
I hope that helps.
I am trying to use a Java client to consume the Moodle Web Service, creating stubs and classes with WSDL2Java , from the Axis Library.
I have a problem with the generation of complex types as GetUserInput, GetCourseInput, GetGradesInput and EnrolStudentInputs, that are normally arrays of string.
But WSDL2Java generate some classes that extends java.lang.Object, which is a weird syntax, and I can't create or use those complex types to invoque the corresponding web service methods.
Am I the only one who had this problem ?
If someone has an answer or a piece of answer, it would be very helpful...
There were some compatibility issues using the built-in PHP5 SOAP functions so I ended up forcing the use of NuSOAP. Because the methods being exposed in the WSDL are class methods they must all be prefixed with the class name to be properly executed after being received. The names that NuSOAP exposes have conflicts with .NET (or some MS *Studio thing), I think. I do have a non-OO service structure that doesn't require the method names to be prefixed with the class name. I'll e-mail a zip file to you containing this which you can try and see if it solves your problem.
I think the problem comes froms the WSDL, because Java classes are builded from it.
That is the weird point, there are no mistakes in the WSDL syntax (I suppose ).
For the moment, I work on getuser method.
The complex type si now managed by WSDL2Java (specifying an array of string).
Now I have a numberFormatException, occuring while calling the method with an array of string in parameter.
The proposed 1.5 version is now accepted by many clients written in languages such as Java with WSDL2Java, PHP5 with WSDL2php, php4 with nusoap, python, .NET (Visual Studio) and even soapUI.
Current restriction is that Moodle's server MUST be running php5+php_soap
See this discussion http://moodle.org/mod/forum/discuss.php?d=67947
Jason Noble wrote "If anyone is still looking for a solution for this problem, my company has published our XML-RPC module."
Jason, I have a couple of question about the SML-RPC module...
If I use CreateAccount, I receive SUCCESS, and if I try CreateAccount again (with same info), I get FAIL: USER ALREADY EXISTS, which is what I should get. If I login to Moodle, I can confirm as well that the new user has been created.
But if I try DisableAccount, I get the following error: FAIL: USER DOES NOT EXIST. Since the user does exist, I'm not sure why I'm getting this error:
Here is the code I'm using for CREATE (which works fine), based on the test file provided:<?phpinclude("../config.php");include("xmlrpcutils/utils.php");$host = $CFG->dbhost;$port = 80;$uri = $CFG->wwwroot."/xmlrpc/xmlrpc.php";$method = "CreateAccount";$m_user['username']='testuser';$m_user['firstname']='John';$m_user['lastname']='Doe';$m_user['email']='email@example.com';$m_user['timemodified'] = time();$m_user['password'] = md5('password');$m_user['confirmed']=1;$callspec = array('method' => $method,'host' => $host,'port' => $port,'uri' => $uri,'user' => 'username','pass' => 'password','secure' => false,'debug' => true,'args' => $m_user);$result = xu_rpc_http_concise($callspec);print_r($result);?>
Here is the code I'm using for DISABLE. All I've done is change the METHOD to DisableAccount, and removed all $m_user variables except for the USERNAME.<?phpinclude("../config.php");include("xmlrpcutils/utils.php");$host = $CFG->dbhost;$port = 80;$uri = $CFG->wwwroot."/xmlrpc/xmlrpc.php";$method = "DisableAccount";$m_user['username']='testuser';$callspec = array('method' => $method,'host' => $host,'port' => $port,'uri' => $uri,'user' => 'username','pass' => 'password','secure' => false,'debug' => true,'args' => $m_user);$result = xu_rpc_http_concise($callspec);print_r($result);?>
Any idea why it's not working? Do I need to pass other variables along besides the username?
Looking at the code in the XMLRPC.php file, it appears that the DisableAccountById does the same thing Moodle would do if you selected Delete User within User Admin. By that I mean it sets DELETED=1 , Changes the USERNAME=EMAIL ADDRESS, and sets the EMAIL=""
When looking at the code for DisableAccount, it does not appear to do all that. It only appears to set DELETED=1, but not perform the other changes.
I am not an expert in php, so I might be missing something obvious, but just thought I'd mention it.
EDIT: I'm using Moodle 1.6.4
I'm trying to use your module with a 1.8 beta installation of Moodle
on a Win2003/IIS6/SQL Server 2005 box... however... I get an
empty/blank screen, even on the test method...
if it matters, the Moodle instance is not installed in a "/moodle"