Moodle for mobile

Problems with core_message_get_message

 
Picture of Howard Miller
Problems with core_message_get_message
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

We've been having some quite significant load problems with certain users making massive numbers of calls the the API function core_message_get_message from the app.  We're talking over 14,000 calls from a single user in one 20 minute period. 

I'm at a bit of a loss. Is this a bug or are we being hacked in some way?

 
Average of ratings: -
Picture of Howard Miller
Re: Problems with core_message_get_message
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

We've found multiple instances of this. The same web service function being hammered - several a second by different users in different places. It's not just the messaging function but that's the usual one. Here's a short example, but even for this one there are pages and pages like this...



 
Average of ratings: -
Picture of Ken Task
Re: Problems with core_message_get_message
Particularly helpful Moodlers

There's a userid in that table ... all the same in your clip.   Find out what devices that account has used to access.  Could be some calendar app (not in the Moodle app) but Googles? that's trying to sync?

Had something like that on a school Moodle and finally tracked down to a single user using multiple devices (iPhone, IPad, laptop) and attempting to sync Google Calendar to Moodles.

No new events are showing up in the calendar are there?

Guess.

'spirit of sharing', Ken

 
Average of ratings: -
Picture of Howard Miller
Re: Problems with core_message_get_message
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

Ok... good point. I'll check the apache logs to see what it says it is. I assumed it was the app but, you're right, it's possible it isn't.

In this case they are calling the web service to get messages so I doubt it's the calendar but it could be something else. Nobody has every been very interested in the calendar around here so I'd be surprised if it was anything to do with that anyway. 

Oddly, these are nearly always students in halls of residence (does that translate outside of the UK?). It's not always the same student either. However, that's rather my point. We're finding single users absolutely hammering the web service api for periods of time with exactly the same call. 

 
Average of ratings: -
Picture of Ken Task
Re: Problems with core_message_get_message
Particularly helpful Moodlers

'Halls of Residence' does translate across 'pond' - dorms.

So residents are what ... 19-22 years of 'experience'?  Of generation Z?  Am sure some computer sci/IT types and students with 'an interest in tech' (Wonder if any supplement their student income with electronic copies of whatever?   Knew a fellow student who paid his semester fees by 'dumpster diving' at the end of every semester ... had old semester exams and finals from a variety of departments).

Web services to me has always been stealthy ... and potentially a hole if not setup properly with a user just for web services use, a token, and access controls to web service functions.  Am not claiming to be an expert at mobile ... far from it ... but I'm willing to bet those 'residents' are far ahead of me.   Since Moodle is open source ... anyone can inspect and possibly attempt  to build their own app for their personally rooted smart phone ... could they not?

Not saying web services is in-secure either ... might be something of annoyance even though it's hammering away at the server that traffic isn't dragging performance down that much, is it?

Anyhoo!   Good luck tracking back and figuring out what's up! smile

'spirit of sharing;, Ken



 
Average of ratings: -
Picture of Howard Miller
Re: Problems with core_message_get_message
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

Don't think it didn't occur to me!

However, I did a few background checks and so far all "arts" subject students. This doesn't exclude them from being mad, KBG-backed hackers but I think it makes it less likely. I'm not fond of conspiracy theories. These things tend to have a more mundane explanation. I just haven't found it yet...

 
Average of ratings: -
Picture of Dani Palou
Re: Problems with core_message_get_message
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Howard,

that WebService is used in the following cases:

  • Viewing notifications in the app. The WebService is called twice: one for read notifications and another one for unread notifications.
  • If your site is older than 3.2, this WS is also used to calculate the number of unread notifications (to show the badge in the app). This is called in several cases: when the app is started, when some notifications are marked as read, or every certain time (10 minutes in mobile, 1 minute in desktop).
  • Viewing the list of message conversations in the app. This will be improved on Moodle 3.6, but right now we need to call the WebService 4 times to be able to build the list of discussions, since we need different parameters to retrieve read/unread messages and to retrieve sent/received messages.
  • Viewing a discussion. This calls the WS several times too.
  • As before, if your site is older than 3.2 this WS is also used to calculate the number of unread messages (to show the badge in the app). The cases are similar to the ones for notifications.

So it could happen that, if the process to check unread notifications & messages are executed at the same time as viewing the list of discussions and viewing a discussion, you could have like 10 requests in a row, but not 30! Of course, the user could have the same account in different devices, but it could be strange that all requests happen at the same time (unless he has a lot of devices, but that's unlikely).

You should check the parameters of the requests, maybe they give you some clues on what is going on. IMO the most important ones are:

  • type: whether to retrieve notifications, messages or both.
  • useridto and useridfrom: the user sending and receiving the messages.
  • read: whether the messages are read or unread.

In Moodle 3.6, messaging is being refactored so hopefully we'll be able to reduce the number of WebService calls.

Cheers,

Dani


 
Average of ratings: -
Picture of Mei Ling
Re: Problems with core_message_get_message
 

Hi Dani Palou,

We found out it take times to send message out with the mobile app, and most of the time the message can't send out successfully and it will keep retrying, hence duplicate message and email sent out. 

I have tested by sending message to 6 users, 5 of them got this problem and 1 user no problem.

This is what i mean, the message sent so many times and if fail to send, it will have a clocking icon next to the message. 



After i go out from the message sending page, and click 'Danny' again from the Message list, it prompt me this error :



If send to this user, i don't face any problem.


I'm not sure if the issue is because of the Moodle Mobile App or the Moodle Setup in our University. Appreciate if you can enlighten me. In my university, we have 6 apps servers and 3 databases. Each two apps servers connect to 1 database server


Looking anyone can reply on my post.

Thank you

Regards,

Mei Ling

 

 
Average of ratings: -
Picture of Dani Palou
Re: Problems with core_message_get_message
Core developersMoodle HQParticularly helpful MoodlersPlugin developers

Hi Mei Ling,

in the first screenshot there are a lot of messages with the text "1". Did you manually write all of them or it was written only once and then it was automatically sent several times by the app?

From what you're saying, my guess is that your server is too slow to handle the requests, so they time out. The app has a 30 seconds timeout, so if your server doesn't answer in that time then the request is cancelled.

I recommend you to follow this guide to debug the WS calls:

https://docs.moodle.org/dev/Moodle_Mobile_debugging_WS_requests

With the Chrome DevTools you should be able to see which network requests are done to the server when using messaging, and how long does the server take to answer.

Kind regards,

Dani

 
Average of ratings: -