Moodle for mobile

Debugging remote add--ons

 
Me !
Debugging remote add--ons
Plugin developers
Hi All,

I'm working on enabling one of my activity modules for MoodleMobile. I've got the development environment set up and working with MoodleMobile running in Chromium, this successfully connects to my Moodle test systems. This wasn't too painful since I've already written a few Android apps from scratch, one of which was done with Cordova.

I've got a basic module set up and have put in the appropriate zip file and db/mobile.php entry. When I start up MoodleMobile, I can see from my webserver log that the zip file is downloaded, so clearly the set up to tell MoodleMobile that the remote addon is available is working and I get Registered addon messages for course content, prefetch and link handler in the browser console.

However, that's the extent of what I can get to work. The icon which I defined in the handler getController method isn't used (I still see the jigsaw piece icon for an unsupported module) and when I click on the activity I get the "You can still use it using your devices browser" message, although the earlier part about the mod not being supported and contacting your sysadmin is now missing, which suggests that the module is being partially detected at this point causing that message to be hidden. There is not nothing else which relates to the module, so I can only assume that something critical is missing, but I'm at a loss as to what that could be since I've largely copied the page module code verbatim (with appropriate adjustments to the mod name.

So the question is, how do validate the basic structure of the plugin so I can work out what is wrong, or trigger an error message that gives me a clue,

Thanks in anticipation!
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

good job getting this far! Let's see if we can fix this problem.

The service responsible of handling the activities is $mmCourseDelegate, so that's one of the places to look. I'd add some more console.log in $mmCourseDelegate#updateContentHandlers and $mmCourseDelegate#updateContentHandler to make sure your activity is enabled (added to enabledHandlers). 

If the activity is not added to enabledHandlers then it probably means that the isEnabled function in your activity is returning false, you should check why is that. Maybe some missing WebService?

If the activity is added to enabledHandlers, then check that the function $mmCourseDelegate#getContentHandlerControllerFor is able to retrieve the handler (maybe the name is not the same?).

If you want to have access to enabledHandlers from the browser console you can just do something like this in the service:

window.enabledHandlers = enabledHandlers;

That way you'll have a global variable that can be accessed via the console.

I hope this helps you!

Dani

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Hi Dani,

Thanks for your help. I've added in a load of extra logging as you suggested to $mmCourseDelegate.

With the extra debugging, I can see that all of the internal mods are processed by the updateContentHandler method prior to the point where my module is registered. Looking at the log output I see the following immediately after the 3 Registered log entries for my addon:

$mmEvents: Event remote_addons_loaded triggered.
$mmCoursePrefetchDelegate: Updating prefetch handlers for current site.

Since my addon contains a content handler, I would also have expected the "Updating content handlers for current site." log entry to be sent, followed by a repeat of the earlier log entires for all the handlers, but with my addon included. However, that doesn't happen. I tried disabling the prefetch handler in my addon config and this caused the content handler registration to be re-run (a minor triumph). So the immediate problem is that my prefetch handler is causing a problem, which might be because I've not written the webservice yet. I'm going to leave the prefetch code for now, since I wasn't going to look at this as the first task.

However, my addon still fails to activate and the updateContentHandler method never calls the isEnabled method on my addon. As far as I can tell, it fails on this line:

handlerInfo.instance = $mmUtil.resolveObject(handlerInfo.handler, true);

The console.log prior to this line is displayed, but the one immediately after it is never printed. Digging into the $mmUtil.resolveObject method, I added some more logging output. The method gets to this line and silently fails, with none of the following logging output being generated:

resolved = $injector.get(toInject[0]);

I tried sticking a try-catch around this to see if there was an exception that was being suppressed somewhere higher up, but got nothing. I haven't as yet been able to identify where $injector comes from, so that's as far as I can get trying to debug this. Any advice gratefully received!
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

that's weird. Can you send me the main.js and the handlers.js files in your plugin? You can send them to mobile@moodle.com. I'll check if I see why is it failing.

Cheers,

Dani

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Actually I think I've found the initial problem, I was using the manual packaging procedure described here:

https://docs.moodle.org/dev/Moodle_Mobile_Remote_add-ons

and I hadn't fully appreciated the step which said "place all the JavaScript code of your addon in a single file named addon.js. This file needs to be in your addon's root folder." I didn't realise that it meant to duplicate absolutely everything, I thought it just related to the content of main.js, so I was making addon.js as a copy of main.js.

Which begs the question as to why this duplication step is necessary, it seems like a redundant thing to do, when all the original js files are included in the final zip as well but seemingly not used. Is this forced upon you by the angular.js framework, or was it a deliberate decision for reasons I have yet to understand?

The module now seems to be starting up. I've still got some issues, which might be the subject of a further post, but I'm also getting meaningful error messages from the console so I'm going to work through them first.

Thanks again, hopefully I'll now have something working soon!
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

I'm glad you found the problem!

Placing all the code in a single file was a deliberate decision we made. It has 2 reasons:

  1. The app only needs to execute 1 file that has a fixed name, there's no need to iterate over all the files in the ZIP file and execute them.
  2. We can control the order of the file content, so the module declaration (main.js) is at the start of the file. Otherwise we'd have to look for the main.js (it could have a different name!) and load it first.

Please notice we have a gulp task to automatically build the remote addon, and this process already creates this addon.js file. Please see:

https://docs.moodle.org/dev/Moodle_Mobile_Remote_add-ons#Automatic_packaging

Kind regards,

Dani

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Yes, I'm now using the gulp task to do this.

I see your point about addon.js, but it's making debugging a real pain, since the line numbers I get are in addon.js rather than the actual line number from the real file. So I have to extract the addon.js from the zip file, find the line and then search for it in the actual source code. Is there any way of persuading MoodleMobile to work with the real files and ignore addon.js in a some kind of debug mode?

Also, if the other js files aren't used, why are they still included in the addon.js by the gulp task? Surely they should be excluded.
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

to debug the addon we recommend you to test it in the development environment, before building the addon. We always try to debug it in Chromium because it's faster, but some issues can only be reproduced in a real device. If the error only happens after the addon is built then there is no other choice as checking the addon.js file, I'm sorry.

Regarding not including the JS files, you're right. I created an issue:

https://tracker.moodle.org/browse/MOBILE-2241

Thanks,

Dani

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Re addons.js, I can see that some developers may prefer to split there code up, but I like to keep things as simple and direct as possible. Is there any reason why I shouldn't simply put all my code into addons.js to start with and dispense with the gulp build process since the only thing it does from my point of view is to complicate debugging, something I'm already finding to be quite a painful process.

I've got Chromium set up all ready, it helps a bit, but I'm finding MoodleMobile addon development to be very difficult to get to grips with. I've written an Android apps from scratch using Cordova as well as having done Apps directly using straight java code so I'm not unfamiliar with phone app development. I think my main complaint is that it's very difficult to identify where problems are if you haven't set something up right. That will probably get easier as I learn more, but the learning curve is extremely steep compared what I'm used to when learning something new.
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

if an addon is simple then it might be easier to put all the code in the addon.js file. We like to keep it in different files because it's easier to maintain and work with, at least for us. 

For example, mod/assign has thousands of lines of code. If someone has to implement a complex module like this one, it will be way easier to work with if the files are split instead of having a single file with thousands of lines.

Please take into account that the gulp task does more things besides building the addon.js file. It also compiles the sass files into a css file, and treats the language files to add the required prefix.

We know it's hard to start with MoodleMobile development, specially because there are a lot of different libraries and frameworks involved: cordova, angular, ionic, npm, bower, gulp, etc. Our gulp tasks were created because it is a must to be able to keep code split into separate files, and we don't want to include hundreds of files into the Mobile app (and in case of remote addons this becomes even harder for the reasons I explained in a previous post). Ionic 3 by default also has a build task to compile everything into a single file, so it's not just us.

Kind regards,

Dani

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
In which case a "debug" mode where plugins can be run "raw" so that you can see the real line numbers and source of the errors seems like an essential addition. Other software which has this kind of consolidation for performance will usually have such a mode to make development easier so that you can directly trace errors to the original source.
 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
My other suggestion is that you need to give more consideration to finding ways to lower the barriers inherent in writing a remote addon. One of the joys of working with Moodle is that for the most part plugin development has quite a low barrier of entry, but it doesn't feel that way with MoodleMobile.

The basic build environment and dev tools needed aren't much of a issue (at least for me), it's not so different to stuff I've dealt with before. I can live with debugging issues caused by code consolidation for addon.js, it's irritating and costs extra time but I know how to work around it.

What I'm finding difficult is getting my head around how MoodleMobile "hangs together", the execution paths are not at all clear to me and this leads to debugging even simple problems being very time consuming. I'm not saying the design is in any way wrong, it's just feels really alien compared to what I'm used to both in Moodle and with the other mobile development work that I've done and right now there just isn't enough documentation explaining how things work to be able to get past that barrier. It feels like you need a very deep understanding of MoodleMobile in order to write addons, far deeper than is necessary for regular Moodle plugins.

I've followed the existing tutorials, but they give me very little handle on what to do when something doesn't work as expected or how to go beyond the very specific examples. If there is more out there, please point me at it, because my extensive searches have failed to find it. I have delved into the MoodleMobile source code, but there aren't enough comments in it really help.

For me, if you want to encourage existing plugin developers to support the app, then you need to find ways to make it easier for them to do so, since I suspect many of the developers who write Moodle plugins will hit the same roadblock which I feel that I'm hitting. I can get past this if I choose to invest the time, but the question I'm now having to ask myself is if it's worth the effort to do so. I'd like to add mobile app support, but cost of adding it is far higher than I initially anticipated.

This is not a complaint or a request for a major design change, just a point for discussion which I hope will inform future development decision.
 
Average of ratings: -
Picture of Juan Leyva
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developersTesters

Hi Tim,

yes, we are aware that developing remote add-ons it's not easy at all. 

We are working on several ways to try to improve it, we are targeting Moodle 3.5 for a solution (something like developing server-side PHP code so app pages can be rendered on the server)

I'll publish in the forum soon about our plans to the new Moodle Mobile app (updated to Ionic3)

Juan

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Thanks Juan, my second comment above was written before seeing this. I'll look forward to the improvements. My secondary concern is that this seems to be a bit of an afterthought, when it should have been part of the initial design process.

On thing that would be really handy right now would be a better tutorial. The existing tutorial doesn't cover what I suspect most developers are going to want to do, which is to add Mobile support to an existing activity module. The choice of a notes module which seems to be divorced from a specific activity is rather curious.

Trying to decypher what's going for an activity module on by looking at the code of an existing activity module is far from easy.

A stripped down "fill in the blanks template" which contains the minimum quantity of code with plenty of comments telling you what each bit does would be really handy.
 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
I'm having another look at this to see if I can make a "breakthrough" and make something work. After adding extensive debug logging messages, I've traced a possible reason why the remote addon still fails to do anything.

In the index controller fetchContent method (copied more or less verbatim from the Page module), there is the following call:

promises.push($mmaModHelixmedia.getPageHtml(module.contents, module.id).then ( .....

The problem seems to be that module.contents is undefined, this causes the getPageHtml to throw an exception. This implies a failure earlier on in the code where the following call is executed:

// Download content. This function also loads module contents if needed.
return $mmaModHelixmediaPrefetchHandler.download(module, courseId)

As far as I can tell neither of the methods in the webservice php code (which is a copy of the code from the page module) is actually called when the download is invoked, although by looking at the logs and by using wireshark to analyse the interaction with the server there does seem to be a call to "core_course_get_contents" when I click on an instance of my module in MoodleMobile, but nothing else. Any light which you can shed on this would be much appreciated.
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

when a Page is opened in the app, we always downloads all its files so they'r available offline. The list of files to download is inside module.contents.

For example, in one of the pages in my local site, module.contents has a list like this (I removed some fields to make it more readable):

[
{"filename":"6479325377_ba1fa998bc_b.jpg","filepath":"/","filesize":53459,"fileurl":"SITE/webservice/pluginfile.php/106/mod_page/content/12/6479325377_ba1fa998bc_b.jpg?forcedownload=1"},
{"filename":"2016010.pdf","filepath":"/","filesize":79169,"fileurl":"SITE/webservice/pluginfile.php/106/mod_page/content/12/2016010.pdf?forcedownload=1"},
{"filename":"index.html","filepath":"/","filesize":0,"fileurl":"SITE/webservice/pluginfile.php/106/mod_page/content/index.html?forcedownload=1"}
]

This module.contents is returned by the WebService you mentioned: core_course_get_contents. If you're planning to make your module work like page, you should check why the WebService isn't returning the list of files. You can find the implementation of the WebService in here. You should look at this part.

If you're planning to debug the WebService, please notice that you can call it via CURL or even performing a GET request using the browser, it's usually faster than using the app.

Please let us know if you have any problem or you don't manage to make this work smile

Kind regards,

Dani

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Picking this up again, I've had a major conference to run and other more pressing matters over the past month...

All I need is the data from the database tables associated with the module instance in order to construct the page I want locally, there shouldn't be any files returned to cache. From what I can tell the method in my external.php is called (but only when the cached version expires, how do I turn that cache off?) and it returns what looks like the correct data. I've also wiresharked the API call and the JSON that's sent looks correct to me when I compare it to the definition I wrote into export.php, but for whatever reason MoodleMobile seems to be incapable of reading this data and passing it back to my code.

So I'm still stuck at the point where the MoodleMobile code inexplicably fails with an undefined error. This is what the fetchContent method looks like:

function fetchContent(refresh) {
var downloadFailed = false;
// Download content. This function also loads module contents if needed.
return $mmaModMyModulePrefetchHandler.download(module, courseId).catch(function() {
// Mark download as failed but go on since the main files could have been downloaded.
downloadFailed = true;
if (!module.contents.length) {
// Try to load module contents for offline usage.
return $mmCourse.loadModuleContents(module, courseId);
}
}).then(function() {
var promises = [];
var getPagePromise;
// Get the module to get the latest title and description. Data should've been updated in download.
if ($scope.canGetPage) {
getPagePromise = $mmaModMyModule.getPageData(courseId, module.id);
} else {
getPagePromise = $mmCourse.getModule(module.id, courseId);
}
promises.push(getPagePromise.then(function(mod){
$scope.title = mod.name;
$scope.description = mod.intro || mod.description;
}).catch(function(error){
console.log("!!!! error: "+error);
// Ignore errors.
}));
// Get the page HTML.
console.log(module.contents+" "+module.id);
promises.push($mmaModMyModule.getPageHtml(module.contents, module.id).then(function(content) {
// All data obtained, now fill the context menu.
$mmCourseHelper.fillContextMenu($scope, module, courseId, refresh, mmaModMyModuleComponent);
$scope.content = content;
if (downloadFailed && $mmApp.isOnline()) {
// We could load the main file but the download failed. Show error message.
$mmUtil.showErrorModal('mm.core.errordownloadingsomefiles', true);
}
}));
return $q.all(promises);
}).catch(function(error) {
console.log("**** error: "+error);
$mmUtil.showErrorModalDefault(error, 'mma.mod_mymodule.errorwhileloadingthepage', true);
return $q.reject();
}).finally(function() {
$scope.loaded = true;
$scope.refreshIcon = 'ion-refresh';
});
}

When the code is run, the two console.log statements in the exception catches simply give a message reporting that an undefined error has been thrown. Without a meaningful error message or in-depth knowledge of the largely undocumented MoodleMobile code, tracing the source of the problem is virtually impossible.
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

the page resource is supposed to work by using module.contents files (the page HTML is returned in this list of files). If your custom addon doesn't use it, then you need to adapt your addon's code so it's not used. The page automatically downloads the file when it's opened (that's why it calls "$mmaModMyModulePrefetchHandler.download").

I guess you should remove this call for now, and if you need it later then you can add it back once it's adapted to your needs. Also, since your plugin will always include the right WebService, you don't need the "if ($scope.canGetPage)" (the page module needs it because old Moodles don't have the right WebService.

Finally, you should adapt the function "$mmaModMyModule.getPageHtml(module.contents, module.id)" so it uses the right data instead of module.contents, calling a WebService if needed. 

There are several ways to don't use cache:

  1. Instead of calling .read('my_webservice'), call .write. Please notice that the returned data won't be available in offline.
  2. Invalidate the data in Pull To Refresh. This is done in the "doRefresh" function of the controller, that calls "$mmaModPage.invalidateContent". When you invalidate a WebService call, the next request will ignore cache and try to fetch the data again.
  3. The .read('my_webservice') function accepts a "preSets" param (3rd param). If you include "getFromCache: 0" in these preSets, the app will ignore cached data and try to fetch it again.

Kind regards,

Dani

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Dani, thanks for your continued help.

I did try generating the contents fields in the external.php code using a placeholder value, but that had no impact on the crash, so I assumed this wasn't the problem. A few questions:

1 - Does the 'module' object which I'm seeing in controller code directly represent what came back from the initial webservices API call?
2 - What actually gets called when $mmaModMyModulePrefetchHandler.download is invoked? I don't see any further activity in wireshark after the initial module data is retrieved and if I'm understanding what you've said correctly the module object has already been populated by the time this method is reached. So I would have expected this to trigger a further call to moodle to actually get the content, but there isn't one. I could still generate something on the server side, rather than doing it locally, but it seemed to me that generating the HTML server side and sending it to the app negates the point of having an app, I would expect the app to simply get the raw data and generate it's own page using it's own internal templates.
3 - If I don't use $mmaModMyModulePrefetchHandler.download, then what is the fetchContent(refresh) method in the controller supposed to return? eg What is the bare minimum code that I could put into fetchContent (and getPageHtml) to locally generate something that would display a field or fields contained in the module object which has already been pulled down? I've struggled to find good minimalist examples of how things are supposed to work.
4 - I'm not worried about making the activity work offline, that's not part of the current spec and may not be desirable in any case (we have not taken that decision, that's for a future release if we do it at all). However, I'm really confused as to why a method called write() would perform the same action as read(), but without the caching. I appreciate that might be what it does, but on the face of it the naming is illogical.
 
Average of ratings: -
Picture of Dani Palou
Re: Debugging remote add--ons
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Tim,

  1. This 'module' object comes from the WebService "core_course_get_contents". This WebService is called to display the list of activities in a section, and then the module is passed as a parameter to your controller when your activity is opened.
  2. I guess your prefetch handler is just a copy of the page one. This prefetch handler is a "subclass" created with $mmPrefetchFactory (javascript ES5 doesn't have classes, so we use a factory to emulate the subclass behaviour). Since your prefetch handler doesn't override the "download" method, it's calling the default one that is inside $mmPrefetchFactory. You can see the function in here. In the list of activities we usually don't load the contents of all the modules to improve the performance of the app and decrease the data usage, that's why we call "loadContents" that loads them if needed. After that, the app will try to download all the files in the activity (module.contents and description files). For page activity the page content is already in module.contents, that's why we don't perform any other WS call. If you need to perform some extra calls then you should override the download and prefetch function in your prefetch handler.
  3. The minimum code that you need depends on your WebServices and your templates smile In this function you should call all the WebServices to get the data you need to render the page. So if your activity requires to call a WebService to get the page contents, then you should call it in here (you can replace $mmaModPage.getPageData and $mmaModPage.getPageHtml with your own functions).

    All Moodle core resources rely on module.contents, but you say you don't, so maybe you should look at an activity instead. mod/lti is one of the simplest activities since it doesn't have prefetch and offline data. If you don't want

    So for example you could do something like:

    fetchCotent(refresh) {
        return $mmaMyModule.getPageHtml(module.id).then(function(html) { // WS call.
            $scope.content = html;
        }).catch(function() {
            // Show error message.
        });
    }

  4. Our way of communicating with Moodle is through POST WebService calls, so whatever you do in the end it will always be a POST WS call, it doesn't matter if you only want to get data or you want to modify it too. Modifying data usually won't use cache, that's why we decided these read/write names. In any case, you can always call "read" and pass the PreSets to don't use cache and the other way around smile

 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Thanks for the extra information Dani, I'm going to have another bash and see if I can finally persuade this to work.

To be honest I still can't get my head around the logic of the read/write naming, I'd have used something like "fetch" and "fetch_nocache", but in the end it doesn't matter that much now that I have a clear explanation of what the calls actually do.
 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Posting this as a quick FYI in case anybody else has this problem. I've just spent rather longer than I would liked to trace a parse error in my external.php file, the reason it took me so long is because the class loading mechanism for the webservices API will tell you that there is an error, but you can't get a line number out of it, this information is hidden even with full debugging turned on.

The problem occurs at the point inthe externallib.php function external_function_info where the initial class load takes place, the exception is generated on the class_exists method call, but all it does it report the nature of the parse fail. The exception itself is then reported to the user without any kind of stack trace (even when debugging is on), so all you actually see is a parse error message. It required some detective work printing out the underlying stack traces to even get to the point where I found that the exception originated from the class_exists call externallib.php. In the end the only way I was able to trace the actual line number of the parse error was by including the external.php in the view.php page of my module and loading that, this generated a proper stack trace with the actual source of the error.

The PHP class_exists method is somewhat to blame here, but it seems to me that there needs to be an enhancement so that a proper stack trace is generated from the PHP when a parse error is thrown by this method.
 
Average of ratings: -
Me !
Re: Debugging remote add--ons
Plugin developers
Actually, this is worse than I thought, errors during execution of the external.php code are also suppressed, I've just had another one where all I get is a notice with the text of the error message, no line numbers and the underlying stack trace doesn't lead back to my code. All you see is:

PHP Notice: Undefined property: Error::$errorcode in /home/httpd/mdl33.dev1.autotrain.org/html/webservice/rest/locallib.php on line 153, referer: http://192.168.1.2:8100/

Fortunately in this case the text of the error message gave a strong indication as to the actual source of the error, since it referred to a mis-spelt method call which I can search for. Is there some kind of "webservices debugging" option I'm missing here that would give a proper stack trace? Suppressing error messages like this is just one element making MoodleMobile development more difficult than it needs to be.
 
Average of ratings: -