Proposal: Web Services / API / Refactoring Opportunity

Proposal: Web Services / API / Refactoring Opportunity

by Howard Miller -
Number of replies: 41
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
I have been working (very slowly) on a simple web services framework mostly (to start with) for remote administration (new user, new course etc.) and recovery of data from blocks etc. That's my starting point anyway.

The big problem/opportunity I am running into is that a lot of the operations required are not available as any kind of library function. For example even though the code is trivial, new categories are created by a database call in course/index.php. Much more complicated a new course is created or edited (and a mass of checks on the entry data) is all performed 'in-line' in the course folder too. So it goes on...

What I would like to do is to refactor these functions into a library. I am supposing moodlelib.php, but possibly other developers have a different/better opinion on that, as maybe this will be the remote control api talked about in the roadmap - that always seemed *very* closely related to web services to me. So perhaps a new library file (moodleapi.php or somesuch) is better.

The blocks situation is slightly nastier and makes me want to growl about separating business logic and the view tongueout The class provides a method 'get_content' which regrettably gets the data (we want for web services) and generates all the output intertwined. I really need to split this up, but my proposal is to provide two new methods - probably get_data() and view_data() to split the functionality. The default method for get_content() then becomes a call to the above two functions, ie, all existing blocks still work fine (but not with web services) until we get around to refactoring them.

I would like to get on with some of this urgently. I am hoping to for 1.7, but some of it is obviously a bit drastic from a testing point of view.

As ever, comments and/or abuse appreciated!
Average of ratings: -
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Totally do not even consider any such refactoring in 1.7. smile 1.9 perhaps!

For now just a very simple and secure interface for:
  1. accepting the messages,
  2. passing them to the relevant library file to a standard function there,
  3. passing back the results,


would be a great start and would not impact all the other refactoring we're currently doing (Roles and DB Schema)

We can worry about the nuts and bolts of individual commands later.
In reply to Martin Dougiamas

Re: Proposal: Web Services / API / Refactoring Opportunity

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Hi Martin,

Understood, but, in most cases there is no standard library function (unless I am missing something), no library function for creating a user, no library function for creating a new course.... etc., etc., Without these basic functions, the web services api would be nowhere near as useful.

It is no problem to reproduce this functionality (which in many cases is going to be lengthy with all the necessary checks), but it does seem a bit wasteful. Either way, does it make sense to make these into some sort of api library, which would simplify the path to refactor these functions once they have had some time to be used and tested? Two birds with one stone as it were.

The blocks thing is more of a showstopper and I know that you are keen to expose that data - I was trying to think of a low risk way of extending the functionality.

What happened to 1.8? tongueout
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Petar Atanasov -
Hi Howard,

The existence of library with standard functions is a must especially when dealing with large data, and it's importance is out of question according to my humble opinion.

For the purpose of the web service - how about more object approach:

I suggest creation of class – this class will has data properties filled by the existent functions (or it’s own wrapper methods on later stage), than the data holded in the class could easily be serialized with SOAP (if the object is executed remotely and of course in case that a transportation of the result set is needed) by the web service.
Recently I've coded generic data helper class (which is useful when dealing with hierarchical data) available at here.

In addition - for the block case it could be an elegant solution to implement 2 interfaces:
  1. IData - for data retrieval from various data sources (db, xml, custom data objects, etc.).
  2. IRender - for rendering data to various targets - like xml, html, etc.

I hope some of this to get in work…

Regards
Petar
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I agree that refactoring is attractive (isn't it always?) BUT it's a really bad time to break Moodle 100% (I'm trying to get 1.7 out on time with roles and xml schema and then a 1.8 with accessibility).

What I meant by a standard external request processing function like forum_external(), moodle_external(), block_external() in relevant libraries.

These could accept some sort of standard object as a parameter (produced by the web services API), and then put it through basically a big switch statement which calls relevant old and sometimes new local functions to do the work. A layer if you like.

This sort of approach doesn't impact the current API at all, and can possibly be migrated later to a more thorough refactoring.
In reply to Martin Dougiamas

Re: Proposal: Web Services / API / Refactoring Opportunity

by Martín Langhoff -
+1 from here on "no big refactor please". However,

- If you think you can abstract a few things into functions, without breaking anything, in small well tested patches, I think we can fold them in.

- In terms of secure WebServices, one of the outcomes of the moodle network / community hub project is a messaging layer based on WS, with optional encryption/signing. This messaging layer can probably do what you want.

Cheers!
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Mark Nielsen -
Hi Howard,

What I would like to do is to refactor these functions into a library.

I'm all for moving major functionalities of Moodle into a library (or multiple libraries) so that everyone's code can make use of it.  Also, by building a more overarching API, more technologies and possibilities become available like Hijax.

Cheers,
Mark
In reply to Mark Nielsen

Re: Proposal: Web Services / API / Refactoring Opportunity

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Most of the major functionalities are already in libraries ...
In reply to Martin Dougiamas

Re: Proposal: Web Services / API / Refactoring Opportunity

by Juliette Culver -
I'd have found functions for creating users and courses really useful for the work I did on IMS Enterprise Services, though adding in these functions wouldn't mean that existing code would need to be changed to use them for the next release. They could just be used by new code for the time being so that there's no danger of anything existing breaking and to give a chance for any bugs to be weeded out first. 
In reply to Juliette Culver

Re: Proposal: Web Services / API / Refactoring Opportunity

by Dan Stowell -
Yes, I agree. When I started the IMS Enterprise importer I was absolutely stunned to discover there were no generic functions for create_user(), create_course(), etc - presumably because when Moodle started, there was only ever one script which needed to do these things.

Very good idea to start implementing some such functions, in the cautious way that Juliette (hi!) suggests.
In reply to Juliette Culver

Re: Proposal: Web Services / API / Refactoring Opportunity

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Ok... it's just that when I start to see reproducing significantly large chunks of code then alarm bells start to ring, however....

Modified proposal...

* Build an external api library.. completely new, nothing (yet) depends on it, with stuff like 'create a user', 'create a course', 'create a catregory'....
* This can be called directly as an api for this sort of stuff and/or by the myriad of web services projects that seem to be going on.
* Aiming for the hard-core basics of this to be in place for 1.7?

Question: should this be a completely new library file (which no existing code will require/include anyway), or should it be in an existing library. My inclination, is to go for new library - lib/moodleapi.php - as if it gets broken, nothing else does.

Seems like the low risk route, and refactoring of these functions can be done later.

PS: I'd like an opinion on my small alteration to the blocks parent class to enable the business logic and view to be separated. Again, note that this would not affect existing blocks.
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Andreas Forstinger -
Hi!

I fully support the idea of a central "core api". So this API could not only be used for remoting, but would in general allow for a clear separation of business logic and presentation layer.

I talk about kind of a (business) delegate pattern here. For those of us not too deep into software engineering: the (business) delegate pattern in short is a way to structure an application or system to loosely couple presentation and logic. That way parts of the application can easily be exchanged or extended/refactored without the need to "touch" lots of modules. When applying a delegate pattern you have one (or at least a limited number) of central apis containing functions which can be called by presentation layer (or remoting services). The logic for these functions is not contained in the API itself, it just "delegates" the call and afterwards returns the result to the initial caller.

What would this mean for Moodle?

The hundreds of functions now spread all over Moodle stay where they are. This way existing legacy code remains compatible. We only introduce a "delegator" - an API containing only the most relevant core functions I would suggest.
The functions contained in the delegator would have to be clearly defined in means of input and output parameters. They should STRICTLY NOT fire any presentation layer code or raise errors, print messages or any stuff like that. They should ALWAYS return silently indicating errors etc. via return parameters (or exceptions if we once fully switch to PHP5).

These functions in the central API could then be safely used by any remoting architecture (like SOAP, XML-RPC etc) or simply by any developer just looking for "standard tasks". That way we would open Moodle up a lot, allowing for easier and faster development cycles!

I can only talk for myself, but when I was starting to work with Moodle I was kind of struck. For my thesis I developed a mobile rich client which should access Moodle functionality. For a newbie developer it is kind of horror finding the correct "core" functions spread all over the libraries. Not to mention that not all of them are documented too clear. And what was worse for me - a large number does not seperate logic from presentation. So the only way for me for the moment was to "reproduce" the business logic to meet my requirements.

Regards,
Andreas





In reply to Martin Dougiamas

Re: Proposal: Web Services / API / Refactoring Opportunity

by Mark Nielsen -
Hi Martin,

Most of the major functionalities are already in libraries ...

Yes of course, sorry.  I was referring to ones that were not already in the standard libraries like the ones Howard was mentioning.

Cheers,
Mark
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Alastair Hole -
This sounds ideal, I have a need for this already too. My recent efforts have been trying to weed out the code necessary for creating a user (so we can create users from our network account creation / enrolment kiosk script rather than doing a batch from ldap evey x minutes) - not as simple as I first thought!

I would be happy to help test anything, and potentially help develop as I have a few years of PHP under my belt but however am still quite new to the Moodle Way.
In reply to Alastair Hole

Re: Proposal: Web Services / API / Refactoring Opportunity

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Have you looked at auth/enrolment plugings? 
In reply to Tim Hunt

Re: Proposal: Web Services / API / Refactoring Opportunity

by Alastair Hole -
We are currently using the ldap authentication plugin (with AD/ActiveDirectory).
The ldap enrolment is too restrictive for us unfortunately (and our MIS data is virtually useless for direct relationships of users/groups/membership)

I know we can sync the ldap directory periodically but it has been requested that we avoid this approach to avoid problems we've had in the past with other systems/VLEs whereby the userlist would always be at least 'n' minutes behind the 'authoritative' source - AD/Active Directory.

I manually added a record to the user table (matching an existing user from AD) and that seemed to work fine. Obviously I need to be able to do this programatically, and would prefer to use higher level moodle code than just attacking the database directly.

So far I have taken two approaches:

First I started looking at the ldap auth code to see if it was possible to add a single user by username, but it appears i can only do this from Moodle>LDAP rather than LDAP>Moodle (without syncing the whole DN).

Currently, I am scanning /user/edit.php for the necessary code but have got very very lost and can't even find where it creates the db record. (Roll on Moodle API!)

Any advice gratefully accepted smile
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Gareth Morgan -

Hi,<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

 

I too am developing a standard API for calling Moodle functions, which will then be called from various places including a web service.  I am focussing specifically on  creating courses and resources rather than users, and am developing functions that will “wrap” the existing functions, in order to make the job of the calling program simpler.

 

Unfortunately I will not be able to post my code in the public domain in its entirety, as it is interwoven with code that was developed for internal use only, and I do not have sufficient time to write a completely generic API class to post to moodle.org.  However, I can advise and assist with the development of a library of generic functions, that could appear in a future Moodle release.

 

Here is a very rough list of the sort of functions I am developing and their functionality – these will be methods of a library class.

 

  1. init_generic_course

Creates a course and fixes sort order.

i.                   Passed an object representing a course record (with properties representing the required fields).

ii.                 Validates that the required properties are present, defaults certain values (timecreated, timemodified).

iii.               Defaults sortorder field prior to record insertion and calls fix_course_sortorder before and after insert to ensure that sortorder field contains valid values.

iv.                Calls addslashes on the database fields and inserts record into the course table.

v.                  Returns the ID of the new course.

 

  1. add_resource_html

Provides a simple method to create an HTML resource, and links it to a specified section in the course page.

i.                   Passed as arguments:  course, section (section number within page rather than unique ID), name, alltext (the HTML to display), summary, windowpopup, windowpage (these last two govern whether new window should be opened or display in same page).

ii.                 Looks up the section number.  Can also create a new section with the next consecutive section number if a dummy section number is passed.

iii.               Calls resource_add_instance from mod/resource/lib.php

iv.                Calls add_module_to_section (see below).

v.                  Returns the new resource ID number.

 

  1. add_resource_file

Provides a simple method to commit a file to Moodle, and links it to a specified section in the course page.

i.                   Passed as arguments:  course, section (section number within page rather than unique ID), name, sourcefile (the name and path of the resource file), summary, windowpopup, windowpage (these last two govern whether new window should be opened or display in same page)..

ii.                 Internal operation is similar to add_resource_html except that it also copies the file to the correct location within the moodledata file store.

iii.               Returns the new resource ID number.

 

  1. add_label

Provides a simple method to add a label to a section in a Moodle course.

i.                   Passed as arguments:  course, section (section number within page), content, headingfont    (governs whether label should be in bold font as for a heading or normal text format).

ii.                 Calls label_add_instance from lib.php.

iii.               Calls add_module_to_section.

iv.                Returns the label ID number.

 

  1. add_module_to_section

This function is called from the add_resource_html, add_resource_file and add_label functions to add a label, file link or html page to the correct section on a page.

i.                   Passed as arguments:  course, module type, module instance no. (unique ID), section (section number within course), summary text, visible (if a new section is to be generated, governs whether or not it will be visible to students).

ii.                 Ensures that the course exists (otherwise throws exception).

iii.               If passed a dummy value for section, (-1) it  calls Create_Section to create a new section, after the highest-numbered existing one within the specified course, and add the module link or label to that section  (this feature is merely a time-saver).

iv.                It calls Moodle library functions add_course_module and add_mod_to_section to create the module record and link it to the correct section.

v.                  It also sets the module’s visible status, dependant upon that of the overall section.  (Note that an option to override this could be added to the parameters).

vi.                It calls rebuild_course_cache.

 

  1. create_section

i.                   Passed as arguments:  course, section (i.e.section number within course), summary, visible (whether or not visible to students).

ii.                 Inserts a record into course_sections with the correct details.

iii.               Returns the new section’s unique ID number.

 

  1. check_section_valid

i.                   Passed as arguments:  course, section (i.e.section number within course), noexception (boolean – whether to throw exception).

ii.                 If the section exists, it returns the section ID number (the unique ID).

iii.               If section does not exist, it either returns -1 or throws an exception dependant upon value of noexception.

 

Where data is inserted into the database, addslashes() is called as a matter of course, but clean_param() is not used, as it is assumed that these functions will always be called from other code which will determine whether or not the inbound data requires sweeping for scripts etc.

 

It would also be desirable to develop “update” versions of these functions to allow the courses and objects within them to be updated without deleting them completely, but that is less of a priority for myself at the moment.

 

Thanks,

 

Gareth Morgan

 

In reply to Gareth Morgan

Re: Proposal: Web Services / API / Refactoring Opportunity

by Mark Dennehy -
This idea of a generic function library for moodle is a darn good one. I just finished a custom access->moodle migration utility that had to go off and create users and courses and categories for a 1.7 install (not as much fun as you'd imagine). Having to add users by writing sql to directly add rows to the moodle database is not good practise. This is the second bespoke project I've done where I'm accessing moodle data from outside the moodle framework and the lack of such a generic library is a very painful thing.
In reply to Mark Dennehy

Re: Proposal: Web Services / API / Refactoring Opportunity

by Michael Penney -
Hi Mark, while I'm all for Web Services, I can't figure out why you would insert data directly into the database rather than using on of the built in methods for importing users, enrollments, auto-creating courses, etc. (batch, External db, IMS Enterprise, or LDAP)?

For live, automatic import of authentication and authorization data, course set up, etc, wouldn't LDAP be a good way to access user and course data from outside Moodle?
In reply to Michael Penney

Re: Proposal: Web Services / API / Refactoring Opportunity

by Mark Dennehy -
Hi Mark, while I'm all for Web Services, I can't figure out why you would insert data directly into the database rather than using on of the built in methods for importing users, enrollments, auto-creating courses, etc. (batch, External db, IMS Enterprise, or LDAP)?

Because we were taking a large existing Access database in a wholly different format and inserting it into a Moodle setup. And we couldn't use LDAP as this setup didn't have (and couldn't have, because of its configuration) LDAP.

And while there were functions, particularly in terms of defining roles, that I found just after writing direct sql inserts, there's no one place to look them up and see what they do, nor in fact do they all work without munging the includes (see MDL-8278 in the tracker for example).
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Donal McMullan -
Can I hijack this thread to plug the recent code we've released to provide an inter-Moodle communication layer? As part of the Community Hub project, the code allows for encrypted communication between Moodles, using good healthy protocols like XML-RPC, XMLDSIG (for digital signatures) and XMLENC (for encapsulation of encryption, keys, etc).

I've created a forum for discussion, support and flames:
http://moodle.org/mod/forum/discuss.php?d=57228

We'd greatly appreciate feedback from all interested Moodle developers.
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers
Hello -

I have just checked in our work on Web Services to contrib. This is a working implementation using SOAP. We also utilize WSDL to make it easier for clients to access the API.

The release is in 'contrib/ws' and contains a document describing the API, and how to use it. It contains methods for managing users, courses and grades. We welcome any comments.

mike
In reply to Mike Churchward

Re: Proposal: Web Services / API / Refactoring Opportunity

by Brian King -
Hello Mike,

As I've mentioned before, we've also developed a web service, using xml-rpc. Our development focused on quizzes and questions. We've been using this in production for quite a while on 1.4 and 1.5 moodles. There was some hesitation expressed at integrating our library, because of possible licensing issues based on the PEAR library classes used.

For a new project, we're going to need to do quite a bit with web services for Moodle 1.7+ - again focusing on quizzes and questions, but not limited to that. Are you open to our adding code to your web services code? With the idea that we begin to build a library of functionality?

Brian

In reply to Brian King

Re: Proposal: Web Services / API / Refactoring Opportunity

by Justin Filip -
Hi Brian,

We would definitely be open to adding code. In fact the design of our code is such that adding another transport type (like XML-RPC) is relatively easy because the core functionality of the service calls (i.e. get_course()) is extensible into a server sub class that contains any transport implementation details.

Our example uses SOAP via the NuSOAP library included with Moodle but it would be fairly easy to A) extend what we have to use XML-RPC and B) add new functionality to the core service class which could then be specially handled (if necessary) in any extended transport classes.
In reply to Justin Filip

Re: Proposal: Web Services / API / Refactoring Opportunity

by Brian King -
Hi Justin,

The project planning is still being done, so we're not touching any code yet. Probably in January/February we'll get seriously into coding.

Currently we're leaning towards SOAP. Some of the services that look like they will need to be implemented:
quizzes (export imsmanifest & receive answers)
export a collection of learning resources (activities, resources: again with ims)
glossary
logging of activities
reporting (who did what when etc.)
...

In reply to Brian King

Re: Proposal: Web Services / API / Refactoring Opportunity

by Martín Langhoff -
Hi Brian! Have you had a look at the Web Services layer that we are building into Moodle as part of the Moodle Network development? I certainly do hope that it's useful for stuff like this.

Having the plumbing in place frees you up to implement the services wink
In reply to Justin Filip

Re: Proposal: Web Services / API / Refactoring Opportunity

by Brian King -
I'm posting here an email exchange between Martin Langhoff and myself; Martin rightly suggested that this should better be posted in the developer forum.


Hi Martin and Justin,
I'm wondering what the status of contrib/ws is. We're working on a large project now, and part of that will require us to expose a good deal of Moodle actions ( login, logout, create/edit question, create/edit quiz, create forum, etc.) as web services.

So, is anybody actively working on this code? It seems that the current version is not made for 1.7; there are errors when trying to run the database update code; the error message says that one should be using the new xml-db stuff.

Anyway, we would like to use it as a basis for our development, and to offer our code back to the community. And we'd of course like to work in close collaboration with whoever else is working on it, so that nobody goes around reinventing wheels. Personally, I'm leaning towards using rest instead of soap, because of speed and ease issues.

It would seem to me to be a good idea to break the actual calls to the moodle code out from the server classes; there's no need for soap and xmlrpc and rest implementation to all duplicate code. It would also make sense to me to create library files to group the code that's actually accessing moodle (e.g. wslib.php for general things, wsuserlib.php, wsforum.php, wsquestion.php, etc.).

### and then a little bug report:
I tried running the test code, but I see that the service url (/ws/service.php?type=soap) fails when I call it directly:
*Fatal error*: Cannot redeclare class soapserver in */var/www/host/html/m17/ws/soapserver.class.php* on line *38*

This is because we're running php5 with it's native soap extenstion, which has a SoapServer class.

An idea would be to use a naming convention like moodleserver instead of server. This would result in names like moodlesoapserver, moodlexmlrpcserver, moodlerestserver for posts service.php?type=soap, service.php?type=xmlrpc, service.php?type=rest
###

Thanks for your time!
Best regards,
Brian

Hi Brian,

I have _no_ idea where contrib/ws is or who is working on the code. A
quick check of the cvs history should give you a bit more info on the
vitality of the code, and who's involved.

As part of moodle network [coming to 1.8] there is good XML-RPC
transport mechanism that can optionally expose the moodle internals to
specific hosts. This is exactly as dangerous (and as useful) as it
sounds  ;-) 

Not all moodle operations have a good API call for them. Surprisingly,
create user, update user, create/update course and a few central ops are
not consolidated into function calls. Before you get too nervous, there
are good reasons -- and abstracting them into well designed function
calls would take quite a bit of careful work to get things like error
handling right.

Also, bear in mind that WS is _not_ a good vehicle to try to do
login/logout. You should write an auth plugin for that.

> using rest instead of soap, because of speed and ease issues.
The code in 1.8 uses XML-RPC because (a) PHP includes a C implementation
of XML-RPC that is fast and solid, (b) everthing under the sun can do
XML-RPC, not just the fancy chic languages, and (c) it is a lot more
debuggable than REST, SOAP and JSON.

HOWEVER, you can implement additional transports. Have a look  ;-) 
It is not 100% perfectly abstracted, but it is 95% abstracted. Adding
additional protocols should be trivial. But I doubt rest/soap or json
are different enough to warrant the effort. We put the abstraction in
place to make sure we support really different protocols when we need it  ;-) 

> that's actually accessing moodle (e.g. wslib.php for general things,
> wsuserlib.php, wsforum.php, wsquestion.php, etc.).
There is a bit of that in the 1.8 code -- but the separation is for
security purposes. The coder of a function or a class _must_ declare the
function as callable via Moodle Network facilities. It cannot happen by
accident. The "promiscuous" XML-RPC mode, however, doesn't have those
checks.

> Thanks for your time!
No problem -- but I'd like to make this thread public and so all the
Moodle WS stuff gets automagically documented and people can get
involved. I get about 200 private emails per day about FAQs and it
doesn't make any sense to reply to them in private. WS and MNET are FAQs
for me lately, it's much smarter to have this in the General Dev Forum.

cheers


m

In reply to Mike Churchward

Re: Proposal: Web Services / API / Refactoring Opportunity

by André Krüger -
Picture of Core developers
Hello Mike,

we have installed your scripts on our test system, but I think we have made something wrong.
We have made the following steps:
- copy /ws under http://mydomain/moodle/ws
- create tables webservices_clients_allow
webservices_sessions
- insert values for webservices_clients_allow
- modify username and password in ws-test.php
I insert a username and password from a moodle user there.
Is this correct ?

When I start my browser with http://mydomain/ws/ws-test.php, I get the following error:

Attempting to create a server connection...

Client connected...

Login:

Sent server call...

Error: HTTP Error: no data present after HTTP headers

That means for me, I cann't get a login. Any idea ?

Thanks, André
In reply to André Krüger

Re: Proposal: Web Services / API / Refactoring Opportunity

by Jason Noble -
I am also having this problem. I am new to SOAP, so I'm not too sure about how all this stuff works, but I took a look at the debugging info and I appears that wsdl is not properly parsing the username and password. Here's the output that makes me think that is the case:

2006-12-21 10:26:34.700723 wsdl: have 2 part(s) to serialize
2006-12-21 10:26:34.701364 wsdl: have 2 parameter(s) provided as arrayStruct to serialize
2006-12-21 10:26:34.701512 wsdl: serializing part "username" of type "http://www.w3.org/2001/XMLSchema:string"
2006-12-21 10:26:34.701641 wsdl: calling serializeType w/named param
2006-12-21 10:26:34.701779 wsdl: in serializeType: name=username, type=http://www.w3.org/2001/XMLSchema:string, use=encoded, encodingStyle=, unqualified=qualified
value=string(8) "username"
2006-12-21 10:26:34.701945 wsdl: in serializeType: got a prefixed type: string, http://www.w3.org/2001/XMLSchema
2006-12-21 10:26:34.702075 wsdl: in serializeType: type namespace indicates XML Schema or SOAP Encoding type
2006-12-21 10:26:34.702244 wsdl: in getTypeDef: type=string, ns=http://www.w3.org/2001/XMLSchema
2006-12-21 10:26:34.702382 wsdl: in getTypeDef: do not have schema for namespace http://www.w3.org/2001/XMLSchema
2006-12-21 10:26:34.702523 wsdl: in serializeType: returning: username
2006-12-21 10:26:34.702652 wsdl: serializing part "password" of type "http://www.w3.org/2001/XMLSchema:string"
2006-12-21 10:26:34.702774 wsdl: calling serializeType w/named param
2006-12-21 10:26:34.702910 wsdl: in serializeType: name=password, type=http://www.w3.org/2001/XMLSchema:string, use=encoded, encodingStyle=, unqualified=qualified
value=string(8) "password"
2006-12-21 10:26:34.703057 wsdl: in serializeType: got a prefixed type: string, http://www.w3.org/2001/XMLSchema
2006-12-21 10:26:34.703182 wsdl: in serializeType: type namespace indicates XML Schema or SOAP Encoding type
2006-12-21 10:26:34.703326 wsdl: in getTypeDef: type=string, ns=http://www.w3.org/2001/XMLSchema
2006-12-21 10:26:34.703458 wsdl: in getTypeDef: do not have schema for namespace http://www.w3.org/2001/XMLSchema
2006-12-21 10:26:34.703595 wsdl: in serializeType: returning: password
2006-12-21 10:26:34.703722 wsdl: serializeRPCParameters returning: usernamepassword
2006-12-21 10:26:34.703880 soap_proxy_1547403706: wrapping RPC request with encoded method element
2006-12-21 10:26:34.704039 soap_proxy_1547403706: In serializeEnvelope length=190 body (max 1000 characters)=usernamepassword style=rpc use=encoded encodingStyle=http://schemas.xmlsoap.org/soap/encoding/
2006-12-21 10:26:34.704161 soap_proxy_1547403706: headers:
bool(false)
2006-12-21 10:26:34.704293 soap_proxy_1547403706: namespaces:
The username and password are not being parsed into XML properly. Any help with this issue would be greatly appreciated.
In reply to André Krüger

Re: Proposal: Web Services / API / Refactoring Opportunity

by Brian King -
Hi André,

I tried running this also under moodle 1.7.1, using php5.
I had these problems:
- our php5 is compiled with the soap extension, so theres a conflict with the built-into-php SoapServer class.
- changing the naming convention so that the nusoap server and everything that references it is named moodlesoapserver instead of soapserver gets around that.
- the database update code is not working; it needs to be rewritten in xml-db style for moodle 1.7+

In reply to Brian King

Re: Proposal: Web Services / API / Refactoring Opportunity

by Justin Filip -
This code was initially written under Moodle 1.5 and I'm aware of it's PHP5 incompatibilities. The code that exists in Contrib is pretty old and has since been quite modified but hasn't been accordingly updated there. One of the updates is that everything is being defined within a WSDL file to make writing a client for this service very easy.

However the PHP5 issue probably still exists in the latest code as it stands. I've had zero success using the wrapper library for SOAP that exists in Moodle (as of 1.6?), /lib/soaplib.php. It's supposed to easily support using NuSOAP on PHP4 and the built-in class in PHP5. I've got an easy fix for this and supporting both PHP4 and 5 within the code I have so that's not a huge problem.

Right now I can't access Contrib for some reason so I need to get that problem solved before I can make any of this new code readily available to anyone here.
In reply to Howard Miller

Re: Proposal: Web Services / API / Refactoring Opportunity

by Jason Noble -
If anyone is still looking for a solution for this problem, my company has published our XML-RPC module. It provides the ability to create and delete accounts, enroll and unenroll students, enter quiz answers and get quiz feedback and several others. These seem to be the most useful functions to call remotely. The module is released under the GNU license so you are free to modify it. The Moodle page for it is here: http://moodle.org/mod/data/view.php?d=13&rid=708 or you can go directly to the download page here: https://secure.linuxbox.com/tiki/tiki-index.php?page=moodle_xmlrpc
If anyone finds this useful, please send us some feedback.
In reply to Jason Noble

Re: Proposal: Web Services / API / Refactoring Opportunity

by Mel Smith -
Hi, I have downloaded the zip and am a bit confused how to use it. I get to step for but not sure how to setup authentication.
In reply to Mel Smith

Re: Proposal: Web Services / API / Refactoring Opportunity

by Jason Noble -
As the readme says, authentication and encryption are expected to be handled by the webserver. If you are using Apache, the documentation on configuring BasicAuth can be found here: http://httpd.apache.org/docs/2.0/howto/auth.html

However, if you want to get up and running quick here's what you need to do:
  1. Create a password file. The supplied .htaccess expects to find this file at /etc/htpass To create a password file, use the following command
    htpasswd -c /etc/htpass username
  2. Rename the file htaccess to .htaccess
The xmlrpc directory should now be restricted by Apache. Test this by trying to access http://hostname/moodle/xmlrpc/test_xmlrpc.php You should get a Forbidden error. If you do, BasicAuth is setup correctly and working. If not, you will need to troubleshoot your Apache configuration.

I hope that helps.
In reply to Jason Noble

Re: Proposal: Web Services / API / Refactoring Opportunity

by Frédéric Mathiot -
Hi
I am trying to use a Java client to consume the Moodle Web Service, creating stubs and classes with WSDL2Java , from the Axis Library.
I have a problem with the generation of complex types as GetUserInput, GetCourseInput, GetGradesInput and EnrolStudentInputs, that are normally arrays of string.
But WSDL2Java generate some classes that extends java.lang.Object[], which is a weird syntax, and I can't create or use those complex types to invoque the corresponding web service methods.
Am I the only one who had this problem ?
If someone has an answer or a piece of answer, it would be very helpful...sourire
In reply to Frédéric Mathiot

Re: Proposal: Web Services / API / Refactoring Opportunity

by Justin Filip -
The problem could be related to the object-oriented structure of the service end in Moodle. It was designed to be extensible so that you could extend the base methods (like get_grades()) to use another transport protocol (i.e. XML-RPC or REST).

There were some compatibility issues using the built-in PHP5 SOAP functions so I ended up forcing the use of NuSOAP. Because the methods being exposed in the WSDL are class methods they must all be prefixed with the class name to be properly executed after being received. The names that NuSOAP exposes have conflicts with .NET (or some MS *Studio thing), I think. I do have a non-OO service structure that doesn't require the method names to be prefixed with the class name. I'll e-mail a zip file to you containing this which you can try and see if it solves your problem.
In reply to Justin Filip

Re: Proposal: Web Services / API / Refactoring Opportunity

by Frédéric Mathiot -
Hmmm, it didn't solve the problem.
I think the problem comes froms the WSDL, because Java classes are builded from it.
That is the weird point, there are no mistakes in the WSDL syntax (I suppose smile).



In reply to Frédéric Mathiot

Re: Proposal: Web Services / API / Refactoring Opportunity

by Catalin Maican -

Hi,

Could I also have the code? It might be related to this post.

Thanks,

Catalin

In reply to Frédéric Mathiot

Re: Proposal: Web Services / API / Refactoring Opportunity

by Frédéric Mathiot -
I may have fixed a part of the probleme. Adding "[]" to 'xsd:string' complex type specification (line 528, of soapserver.class.php).
For the moment, I work on getuser method.
The complex type si now managed by WSDL2Java (specifying an array of string).

Now I have a numberFormatException, occuring while calling the method with an array of string in parameter.
In reply to Frédéric Mathiot

Re: Proposal: Web Services / API / Refactoring Opportunity

by Patrick Pollet -
Hello,

The proposed 1.5 version is now accepted by many clients written in languages such as Java with WSDL2Java, PHP5 with WSDL2php, php4 with nusoap, python, .NET (Visual Studio) and even soapUI.

Current restriction is that Moodle's server MUST be running php5+php_soap

See this discussion http://moodle.org/mod/forum/discuss.php?d=67947

Enjoy .
In reply to Jason Noble

Re: Proposal: Web Services / API / Refactoring Opportunity

by Deleted user -

Jason Noble wrote "If anyone is still looking for a solution for this problem, my company has published our XML-RPC module."

Jason, I have a couple of question about the SML-RPC module...

QUESTION #1

If I use CreateAccount, I receive SUCCESS, and if I try CreateAccount again (with same info), I get FAIL: USER ALREADY EXISTS, which is what I should get. If I login to Moodle, I can confirm as well that the new user has been created.

But if I try DisableAccount, I get the following error: FAIL: USER DOES NOT EXIST. Since the user does exist, I'm not sure why I'm getting this error:

Here is the code I'm using for CREATE (which works fine), based on the test file provided:

<?php
include("../config.php");
include("xmlrpcutils/utils.php");
$host = $CFG->dbhost;
$port = 80;
$uri = $CFG->wwwroot."/xmlrpc/xmlrpc.php";
$method = "CreateAccount";
$m_user['username']='testuser';
$m_user['firstname']='John';
$m_user['lastname']='Doe';
$m_user['email']='johndoe@mydomain.com';
$m_user['timemodified'] = time();
$m_user['password'] = md5('password');
$m_user['confirmed']=1;
$callspec = array(
'method' => $method,
'host' => $host,
'port' => $port,
'uri' => $uri,
'user' => 'username',
'pass' => 'password',
'secure' => false,
'debug' => true,
'args' => $m_user);
$result = xu_rpc_http_concise($callspec);
print_r($result);
?>

Here is the code I'm using for DISABLE. All I've done is change the METHOD to DisableAccount, and removed all $m_user variables except for the USERNAME.

<?php
include("../config.php");
include("xmlrpcutils/utils.php");
$host = $CFG->dbhost;
$port = 80;
$uri = $CFG->wwwroot."/xmlrpc/xmlrpc.php";
$method = "DisableAccount";
$m_user['username']='testuser';
$callspec = array(
'method' => $method,
'host' => $host,
'port' => $port,
'uri' => $uri,
'user' => 'username',
'pass' => 'password',
'secure' => false,
'debug' => true,
'args' => $m_user);
$result = xu_rpc_http_concise($callspec);
print_r($result);
?>

Any idea why it's not working? Do I need to pass other variables along besides the username?

QUESTION #2

Looking at the code in the XMLRPC.php file, it appears that the DisableAccountById does the same thing Moodle would do if you selected Delete User within User Admin. By that I mean it sets DELETED=1 , Changes the USERNAME=EMAIL ADDRESS, and sets the EMAIL=""

When looking at the code for DisableAccount, it does not appear to do all that. It only appears to set DELETED=1, but not perform the other changes.

I am not an expert in php, so I might be missing something obvious, but just thought I'd mention it.

THanks

EDIT: I'm using Moodle 1.6.4

In reply to Jason Noble

Re: Proposal: Web Services / API / Refactoring Opportunity

by Catalin Maican -
Hi,

I'm trying to use your module with a 1.8 beta installation of Moodle
on a Win2003/IIS6/SQL Server 2005 box... however... I get an
empty/blank screen, even on the test method...

if it matters, the Moodle instance is not installed in a "/moodle"
virtual directory.

Thanks,
Catalin