Moodle 2.0: create a Java Client consumming Moodle web service

Moodle 2.0: create a Java Client consumming Moodle web service

by Jérôme Mouneyrac -
Number of replies: 25
Here is a topic for helping to create a Java client for Moodle 2.0

The Soap server used by Moodle is the Zend_soap server. It is, in the 1.8 version, not really friendly with Java and .Net client generator.

At this moment I didn't success to consume anything from the Zend_soap server, but I did success to generate a client from Netbeans.

I'm going to explain how to reach the point where you can generate a Java client then I'll explain why I'm blocked and I cannot call the web service function. It's going to be a bit long.


***********************************************************
First update to the latest Zend framework into your_moodle_folder/lib/zend/ (if you copy all framework it works, of course strictly in a environment that you'll end up to dump).
Then edit webservice/soap/lib.php:

[code php]
$autodiscover = new Zend_Soap_AutoDiscover();
//the following is important, because Java doesn't want to generate the client if it's 'use' => 'encoded'
$autodiscover->setOperationBodyStyle(
array('use' => 'literal',
'namespace' => $CFG->wwwroot)
);
$autodiscover->setBindingStyle(
array('style' => 'rpc')
);
$autodiscover->setClass('ws_authentication');
$autodiscover->handle();
[/code]

Then in Netbeans:
1. Create an empty basic Java project
2. On the project title, right click, select create web service client
3. Choose JAX-RPC style and enter the WSDL address (your_moodle/webservice/soap/server.php?wsdl)
4. Click on finish, some classes should be generated
5. click right in the Main.java, Insert Code ... call web service operation ... select get_token

At this moment you have the following line:
_ws_authenticationPort.get_token(/* Enter params */);

I didn't find yet how to create the params in Java. The params should be an Array type (it's a generated class).
Is the WSDL correct and are the classes well generated by Netbeans? How to use them ?


I'll keep you inform. Keep in touch with this topic smile

Average of ratings: -
In reply to Jérôme Mouneyrac

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Roger Robins -

Hi Jerome,

I've tried changing the settings to lib.php as you described above, and managed to get the code generators in both NetBean and Oracle JDeveloper to generate a Web Service Proxy. The code I used in NetBean loks somerthing like:

  au.edu.oten.Array ar = new au.edu.oten.Array();
        try { // This code block invokes the Ws_authenticationPort:get_token operation on web service
            javax.xml.soap.SOAPFactory sf = javax.xml.soap.SOAPFactory.newInstance();
            javax.xml.soap.SOAPElement seName = sf.createElement("roger.robins");
            javax.xml.soap.SOAPElement sePass = sf.createElement("<password>");
            javax.xml.soap.SOAPElement soe[] = { seName, sePass };
            ar.set_any(soe);
            au.edu.oten.Ws_authenticationService ws_authenticationService = new au.edu.oten.Ws_authenticationService_Impl();

            au.edu.oten.Ws_authenticationPort _ws_authenticationPort = ws_authenticationService.getWs_authenticationPort();
            int token = _ws_authenticationPort.get_token(ar);
            System.out.println("Token: "+token);

Unfortunately I'm getting a SOAPFaultException:

SEVERE: null
javax.xml.rpc.soap.SOAPFaultException: Unknown error
        at com.sun.xml.rpc.client.StreamingSender._raiseFault(StreamingSender.java:528)
        at com.sun.xml.rpc.client.StreamingSender._send(StreamingSender.java:307)
        at au.edu.oten.Ws_authenticationPort_Stub.get_token(Ws_authenticationPort_Stub.java:69)
        at integration2.Main.main(Main.java:29)

I get the same result using the JDeveloper Web Service Client Code.

Any ideas?

In reply to Roger Robins

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Roger Robins -

The packet returned from the Moodle server looks like:

HTTP/1.1 500 Internal Service Error
Date: Fri, 08  May 2009 04:19: 55 GMT
Server:  Apache/2.2.11 (U nix) PHP/5.2.9
X-Powered-By: PH P/5.2.9
Expires : Thu, 19 Nov 19 81 08:52:00 GMT
Cache-Control:  no-store, no-cac he, must-revalidate, post-check= 0, pre-check=0
Pragma: no-cache
Content-Type:  text/xml; charse t=utf-8
Content -Length: 273
Connection: close
Set-Cookie: MoodleSession=aa7be 24263bcc3d6bd6d7 e8d30d1f7e0; path=/
<?xml version="1.0" encod ing="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
   <SOAP-ENV:Body>
      <SOAP-ENV:Fault>
         <faultcode>Receiver</faultcode>
         <faultstring>Unknown error</faultstring>
      </SOAP-ENV:Fault>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>.

I think the Internet Error 500 might be telling us something.

In reply to Roger Robins

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Roger Robins -

I've tracked the Unknown Error down to a Moodle Exception raised by get_token method of lib.php. The real error is "wrongusernamepassword" which is hard coded in lib.php. However, by the time I enter get_token (server side code) I've already lost the username/password parameters. This causes the Moodle Exception to trigger.

The packet the Soap client is sending is:

<?xml version="1.0" encoding="UTF-8"?>

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:enc="http://schemas.xmlsoap.org/soap/encoding/"

xmlns:ns0="http://otfapdev2.oten.tafensw.edu.au:8888/moodle">

<env:Body>

<ns0:get_token>

<params><roger.robins /><pwd /></params>

</ns0:get_token>

</env:Body>

</env:Envelope>

Can anyone tell me if the client is sending the right Soap packet?

Any ideas would be greatly appreciated.

B.T.W. I get the same error; whether using a java soap client or a local php soap client.

In reply to Roger Robins

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Jérôme Mouneyrac -
Same for me here, I used your JAVA code to generate the params in the Java client, the params are not received by the zend_soap server. I'll have a better look tomorrow with a nusoap client.
In reply to Jérôme Mouneyrac

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Jérôme Mouneyrac -
About the unknow error it's because I forgot to put:

$soap->registerFaultException('moodle_exception');
//put it in webservice/soap/lib.php after the first $soap = new Zend_Soap_Server(...);
//I'll commit the fix tomorrow for that
In reply to Jérôme Mouneyrac

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Jérôme Mouneyrac -
My last investigations were:

I changed the get_token() param type in the phpdoc from "array" to "object" and adapted the function+the client. In this case the PHP client works but I can not generate the Java client anymore (modeler error: invalid entity name: "struct" (in namespace: "http://www.w3.org/2001/XMLSchema"))

I also tried "class" type as suggested by someone in the zend nabble mailing list but without success yet. Neither with the PHP client, neither with a Java client.

Apparently we can also modify the WSDL on the fly with Zend_soap_wsdl, it's also a thing to dig into for this SOAP server...

I'll keep you update...
In reply to Jérôme Mouneyrac

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Roger Robins -

Hi Jerome,

Is there any update on what seem to be several SOAP related issues?

Thanks,

Roger

In reply to Roger Robins

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Jérôme Mouneyrac -
Hi Roger,
I come back with bad news, my work on the JAVA/.Net compatible SOAP server is in stand-by from now. It's been too much time consumming for me. I'll post here when/if I go back to work on it.

The current SOAP server can be used with Zend PHP client.
In reply to Jérôme Mouneyrac

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Nanaji Jonnadula -
Hi

Can you please tell me to how to import latest Zend framework into your_moodle_folder/lib/zend/ (

I downloaded ZendFramework-1.9.6

and kept at

/var/www/html/moodle/lib/zend

there already one Zend folder is there.


Can you please explain this.

Appreciate.

Thanks
Nanaji

In reply to Jérôme Mouneyrac

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

FYI

Recently I've created a Library for accessing the REST web services of Moodle 2.0. Not complete yet as it does not support the File web services, but working on it.

http://moodlerestjava.sourceforge.net

Bill

In reply to Bill Antonia

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by snadly Mohamed Amine -

hi

first of all,thanks for the library

could you tell me exactly how to use it in order to interact with moodle from a java application?

In reply to snadly Mohamed Amine

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

Hi Snadly,

First set up your authentication, I generally use a token but you can use username and password pair. The code for token is:

String wstoken = "somelongstringofgobbledegook";
String moodleServer = "http://moodlewebserver/moodle/webservice/rest/server.php";
MoodleRestWebService.init(moodleServer, wstoken);

Once this is set up then you can use the various other classes to create parameters and execute calls to the Moodle server. If you use username/password pair then you have to connect to simpleserver.php instead.

An example:

package testrest;

import java.io.UnsupportedEncodingException;
import java.util.logging.*;
import net.beaconhillcott.moodlerest.*;
public class Main {

public static void main(String[] args) {
try {
String wstoken = "somelongstringofgobbledegook";
String testMoodle = "http://moodlewebserver/moodle/webservice/rest/server.php";
long courseId=798;
MoodleRestWebService.init(testMoodle, wstoken);
MoodleCourse course = MoodleRestCourse.getCourseFromId(courseId);
System.out.println("Course "+course.getFullname());
MoodleGroup[] groups = MoodleRestGroup.getGroupsFromCourseId(courseId);
for (int i=0;i<groups.length;i++) {
System.out.println("  Group "+groups[i].getName());
MoodleCourseUser[] courseUsers = MoodleRestEnrol.getEnrolledUsers(courseId, "mod/chat:chat", groups[i].getId(), true);
for (int j=0;j<courseUsers.length;j++) {
MoodleUser user = MoodleRestUser.getUserById(courseUsers[j].getUserId());
System.out.println("    "+user.getFirstname()+" "+user.getLastname());
}
}
} catch (MoodleRestException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
} catch (UnsupportedEncodingException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
}
}

Sorry the indentation got lost! Anyway, the above code retrieves all users of a course with the id of 798 who have the "mod/chat:chat" capability. the boolean parameter at the end of MoodleRestEnrol.getEnrolledUsers as yet I'm not sure what it does, if set to false I get the same response. If the group id is 0 then you get all enrolled users on a course whether they are in a group or not.

Hope this helps as an example.

Regards

In reply to Bill Antonia

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by snadly Mohamed Amine -

hi bill

thanks a lot for the exemple , worked like a charm for me on android

have you extended your library? if so , please post the latest version, so the

moodlers can  use it. Actually I am a new java developper and I don't have

much

experience .And As I see , you are a genius.You facilitated my life with your library

My email : aminesnadly@yahoo.fr

greetings from Tunisia smile

 

In reply to snadly Mohamed Amine

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by snadly Mohamed Amine -

Hi again,

Sorry man to disturb you

but I am willing to build an android application that authenticate to moodle using the (username/password) pair and moodle return a unique token for each session so i can control the user access.

Then i get the list of the courses of the logged user. and download whatever course of the list

the user can manage his profile of course

and finally logout.

I'am confused about what class to use in each step. I didn't figure out  the mapping between my UI and  your classes.

Can you give me a hint please : Autntification ---->get the courses list------>download a course from the list----->manage the profile---->logout.

Thanks and best regards

In reply to snadly Mohamed Amine

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

Hi Snadly,

From as I understand this you are trying to allow users to manage their course profiles using an Android app.

One major problem that currently there is little profile control, if any, you can do through the default web services. To do this type of change extra web services on the server would need to be created. Also, if you enable an end user read/write access through web services, you enable it globally across the whole site for those particular web services features not just for individual courses so there is a security problem here.

I'm not trying to put you off but as I say, at present the standard web services currently available are extremely limited and are best used by adminstrators or for transfering data between systems.

Sorry for the bad news.

Regards

In reply to snadly Mohamed Amine

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

By the way.

The current download is up to date in terms of the REST protocol. I have restructured the library slightly since so I can include SOAP calls but at present it is at an early stage as I have been busy with other things such as extending the website for the archery club I belong to (written in Java and using Glassfish).

Regards

Bill

In reply to Bill Antonia

Getting SAXParserException "Already Seen Doctype" with GetAllUsersById example

by Yunus Luniwala -

Hi Bill,

First of all great cheers for you to build such a useful library to deal with Moodle. I am creating one project in which I am using Moodle WS in it.

As a demo project, I used your "GetAllUsersById" example. But unfortunately getting the SAXParserException : "Already Seen Doctype". Before getting this exception, I get the whole page of my moodle server as a content in the console.

I am not able to figure out the things which are going wrong.

So please tell me the correct path to resolve this as soon as possible.

Waiting for your reply.

Regards.

Yunus

In reply to snadly Mohamed Amine

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Yunus Luniwala -

Hi Amin,

Seems like you have perfectly run your android application as depicted from the above post.

I am facing problem in running the sample program written above.

Can you share the things for succesfful run of above program?

Thanks

In reply to Yunus Luniwala

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

Hi Yunus

Hmmmmmm............

It could be: because Moodle does not add "Entity" definitions to the XML stream I have had to inject into the stream a set of Entities at the point before where the main set of data is read. What might be happening is two lines containing "!DOCTYPE" are being read, one from the injected data and one from elsewhere. If you look in the source of the library there is a file called "EntityInjection.xml". In there you will find one source of "!DOCTYPE" or, if you are using a later version of the library which I don't think you are, then the code to inject the file is commented out so the file is no longer used but the contents is still added to the buffer as a complete string (line 344 MoodleCallRestWebService.java). If your site does not use characters which have to be converted then the injection does not need to be used therefore removing one of the "!DOCTYPE". Where your second "!DOCTYPE" is I cannot tell.

In the version of the library you can download from www.beaconhillcott.net, line 348 in MoodleCallRestWebService.java is commented out "//System.out.println(line);" if you uncomment this and run your code you might be able to find the second "!DOCTYPE".

I do see you are referencing Moodle 2.0 in your subject, is this the version you are using? My testing was against versions 2.1 and 2.2. I don't know if this is significant. The version to download from the URL above has extra calls than you can use in 2.0 ie you can only use the moodle_  wsfunctions so you would have to switch the library into legacy mode, otherwise the default is to use the core_ wsfunctions.

Regards

Bill

In reply to Bill Antonia

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

As a second thought, I wonder if there was already an error which had given a "!DOCTYPE" within the xml message sent back by Moodle? The "Entity" injection would provide a second "!DOCTYPE" and therefore may cause an XML parse error due to not having any "RESPONSE" elements present.

Just a thought.

To see if you can test your REST call, I'll try and work out a URL GET and post it later.

In reply to Bill Antonia

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

http://webserver.domain.com/webservice/rest/server.php?wstoken=securitytokenforcall&wsfunction=moodle_user_get_users_by_id&userids[0]=1&userids[1]=2

Replace the server and domain name (webserver.domain.com), add any subdirectory path to your Moodle installation and add your token for the authentication (securitytokenforcall). An array of ids for each user to retrieve the details of is added to the end of the parameter list, if more, just append on the end separated by an &. Note: increase each array element by 1. The two in the example above should be Guest and Admin. Once altered for your Moodle server, test it using a browser as the URL, parameters and all. You should get an XML response if it is correct.

I've just tested a theory, if a 404 error occurs (Object not found) then the page returned will have a "!DOCTYPE" already, however the library at present does not check for this type of error and will carry on regardless with the injection and XML parse. Might be an idea to check the URL you are using is correct otherwise a 404 error could occur and may produce the error you are experiencing.

Regards

Bill

In reply to Bill Antonia

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Yunus Luniwala -

Hi Bill,

Thank you very much for your elaborative explanation. I am using latest version of MoodleRest.jar (0.1.5) downloaded from SourceForge net. And as you suggested, I searched around the System.out.print(line); , but found nothing. Moreover I made MoodleCallRestWebService.setLegacy(true);. But its also not working. As I tried to run the below line with server,domain name and ws-token, I got Runtime exception. This exception is also listed below

MoodleCourse course = MoodleRestCourse.getCourseFromId(courseId);

Exception:

05-29 15:56:28.193: E/AndroidRuntime(912): java.lang.RuntimeException: Unable to start activity ComponentInfo{test.rest/test.rest.MoodleGetUsersByIdAndroidExampleActivity}: java.lang.NullPointerException
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1647)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1663)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.app.ActivityThread.access$1500(ActivityThread.java:117)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:931)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.os.Handler.dispatchMessage(Handler.java:99)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.os.Looper.loop(Looper.java:123)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.app.ActivityThread.main(ActivityThread.java:3683)
05-29 15:56:28.193: E/AndroidRuntime(912):     at java.lang.reflect.Method.invokeNative(Native Method)
05-29 15:56:28.193: E/AndroidRuntime(912):     at java.lang.reflect.Method.invoke(Method.java:507)
05-29 15:56:28.193: E/AndroidRuntime(912):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)
05-29 15:56:28.193: E/AndroidRuntime(912):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)
05-29 15:56:28.193: E/AndroidRuntime(912):     at dalvik.system.NativeStart.main(Native Method)
05-29 15:56:28.193: E/AndroidRuntime(912): Caused by: java.lang.NullPointerException
05-29 15:56:28.193: E/AndroidRuntime(912):     at java.io.Reader.<init>(Reader.java:65)
05-29 15:56:28.193: E/AndroidRuntime(912):     at java.io.InputStreamReader.<init>(InputStreamReader.java:122)
05-29 15:56:28.193: E/AndroidRuntime(912):     at java.io.InputStreamReader.<init>(InputStreamReader.java:59)
05-29 15:56:28.193: E/AndroidRuntime(912):     at net.beaconhillcott.moodlerest.MoodleCallRestWebService.call(MoodleCallRestWebService.java:253)
05-29 15:56:28.193: E/AndroidRuntime(912):     at net.beaconhillcott.moodlerest.MoodleRestCourse.getCoursesById(MoodleRestCourse.java:110)
05-29 15:56:28.193: E/AndroidRuntime(912):     at net.beaconhillcott.moodlerest.MoodleRestCourse.getCourseFromId(MoodleRestCourse.java:83)
05-29 15:56:28.193: E/AndroidRuntime(912):     at test.rest.MoodleGetUsersByIdAndroidExampleActivity.onCreate(MoodleGetUsersByIdAndroidExampleActivity.java:37)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1047)
05-29 15:56:28.193: E/AndroidRuntime(912):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1611)
05-29 15:56:28.193: E/AndroidRuntime(912):     ... 11 more

In reply to Yunus Luniwala

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

Hi Yunus,

The line numbers are for the version I'm working on, I'll have a look at 0.1.5 and see what can be changed to debug your issue. By the way, I've just relised the title is Jerome's discussion. What version of Moodle are you using? If it's 2.1 or 2.2 then you don't have to switch to legacy mode.

Version 0.1.5 uses a file to inject the "Entity" statements. It appears the file "net/beaconhillcott/moodlerest/EntityInjection.xml" is not being opened as a resource and therefore variable "resource" is null which makes InputStreamReader throw a wobbly!

If you download the version from http://www.beaconhillcott.net, this newer version gets around the problem of not finding this file by appending the complete contents as a string in the same position of the data stream instead of being read from a file. The alternative is to update the lines in 0.1.5 where the entities are injected with the code from the latest version. I've attached a modified version of MoodleCallRestWebService.java in 0.1.5 for you to swap if you like.

Keep in mind though, if you do use the latest full version downloadable from the site above, it is subject to change and does have extra calls which are non-functional with Moodle 2.2 and below, I've looked in the tracker and have written code against those wsfunctions which are planned for Moodle 2.3, so they are untested.

Regards

Bill

In reply to Bill Antonia

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Yunus Luniwala -

HI Bill,

Thanks for your descriptive answer. As I told you earlier, I am using Moodle 2.0 version, and also disabled legacy mode in my demo application.

I applied your tricks that you mentioned above. Also I am using MoodleRest 0.1.5 version. As I replaced my MoodleCallRestWebService.java file with the file given by you. So the "Already Seen Doctype" error has been removed. smile

I am very glad that you assisted me till now.

But as I debugged my application throughly, I got a parser exception, that states that ArrayIndexOutOfBoundException. I am using GetAllUsersById example. Below is the line, which causing the exception to occur. I hereby also attaching the my project's sample file. Can you please guide me where I am going wrong?

Line causing exception :

MoodleUser[] user= (MoodleUser[])MoodleRestUser.getUsersById(users);

for (int i=0; i<users.length; i++, System.out.println("***************************************************"))
        printUser(user[i]);

Exception:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
    at moodlegetusersbyidexample.MoodleGetUsersByIdExample.main(MoodleGetUsersByIdExample.java:32)
I know that I am missing one small point to make it work, but not able to figure it out sad

Thank you very much smile

Regards

In reply to Yunus Luniwala

Re: Moodle 2.0: create a Java Client consumming Moodle web service

by Bill Antonia -

Hi Yunus,

I assume you have a modifiied line with your own server and token for:

MoodleCallRestWebService.init("http://servername/moodle/webservice/rest/server.php", "verylongtokenstring");

However if you are using Moodle 2.0 as you say, you do need to switch the library into legacy mode. This is because the default is to call the "core_" wsfunctions. By switching to legacy mode, all the calls use the "moodle_" wsfunctions within Moodle. Moodle 2.0 does not have the "core_" wsfunctions so your error might be being caused by this issue.

You need to add:

MoodleCallRestWebService.setLegacy(true);

before you call any methods which will access the Moodle server. This only has to be added once as it is a static attribute of the class.

Regards

Bill