Moodle server hacked with a number of added files.

Moodle server hacked with a number of added files.

av Francesco Garganese -
Antall svar: 9

Hello from Italy,
we are an NGO working in the Civil Protection sector.
We've had a Moodle server in production for over seven years, which give us a huge help in training volunteers. Until ten days ago, it had no problems.

When the issue we're about to describe occurred, the installed version was 5.0.3 on LAMP with a standard theme and customcert as the only added plugin.
Suddenly, the site returned a 500 error after authenticating any user, then went invisible, redirecting traffic to an Indonesian gambling page.

After reinstalling everything three times with the same result after a couple of days, two days ago we decided to isolate the Moodle instance on a new separate server (Ubuntu Server 24.04) and upgrade Moodle to the latest version, successfully.
After about 24 hours, the issue returned.
We restored everything, but we need to resolve the situation in a final way.

We attach the result of
 diff [unzip Moodle dir] [hacked Moodle dir]
 which highlights the files present on the hacked installation.

Can you help us? 🙏

Gjennomsnittlig vurdering: -
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av Visvanath Ratnaweera -
Bilde av Particularly helpful Moodlers Bilde av Translators
If that moodle/public/ scared you, first read https://docs.moodle.org/501/en/Upgrading#Code_directories_restructure.

But that is a confusion you added to the mix by trying to upgrade a dying site. A more stable approach is to get the dying site on its feets first.

You say, the (old) site started throwing 500 error. You restored it twice, every time stops with 500 error. I don't think that proves the site is hacked. 500 is a generic error message for many things. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/500.

The good news is that you seem to have a good backup. Start again, increase the Debugging to its maximum and get detailed error messages from Moodle.
Gjennomsnittlig vurdering:Useful (1)
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av Ken Task -
Bilde av Particularly helpful Moodlers

Don't think you've been hacked ... but have fallen 'victum' to upgrading to 5.1!!! trist

The text file you shared shows /public/ paths which is only present in 5.1.

Please see:

https://docs.moodle.org/501/en/Upgrading

'SoS', Ken

Gjennomsnittlig vurdering: -
Som svar til Ken Task

Moodle server hacked with a number of added files.

av Ken Task -
Bilde av Particularly helpful Moodlers

In looking at the .txt file you provided, I did see a couple of things:

Only in production.moodle/public/local: moodle_webshell

If one is not careful as to which accounts are set to admin levels, moodle_webshell could be mistakenly used to replace the front page (index.php file at code root)!

The other thing ... 

Are all the addons installed compatible with 5.1? or are the plugin code directories missing?

For example:

Only in production.moodle/public/repository: alfresco

https://moodle.org/mod/forum/discuss.php?d=446729

And ...

Only in production.moodle/public/media/player: realplayer

https://github.com/moodlehq/moodle-media_realplayer

Realplayer media player for Moodle 3.2 and above

Plugin is left for legacy reasons since it was included in Moodle 3.1 and earlier. Moodle HQ will not actively maintain it but will consider pull requests.

'SoS', Ken

Gjennomsnittlig vurdering: -
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av Howard Miller -
Bilde av Core developers Bilde av Documentation writers Bilde av Particularly helpful Moodlers Bilde av Peer reviewers Bilde av Plugin developers
So which version are you running in production now? Do you still have the problem. If it can be avoided, don't upgrade a site with a mysterious problem - the risk of making it even worse is ever present. Expecially (as has been pointed out) with 5.1 thrown into the mix.
Gjennomsnittlig vurdering: -
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av Francesco Garganese -
Thanks to everyone who's answered so far, but we haven't catch the problem.
We are not worried about upgrading to 5.1, our site operate as well and users use courses without problems... until the error 500 shows up!
After that, we restore the clean installation from the officlal release and all returns to operate normally... until the next time.
The update to version 5.1+ (Build: 20251121) was (also) done, because we hoped to fix these intrusions.

The real problem is inside the diff file, which I invite you to examine line by line.
The question is: in what way "someone" is installing software without our action?
We found:
- local/moodle_webshell
- mod/lanciau
- theme/lanciau
- lib/mlbackend/lmswithaipython → does not correspond to the official mlbackend_python backend (the standard one uses the moodle-mlbackend-python directory and package)
- payment/gateway/duitku → payment gateway from an Indonesian provider (official plugin, but typically only used in Indonesia)!

Someone is entering in our Moodle installation and adds what we've listed and the rest you see in the diff result file.
We already changed: phisical server, administrator passwords and db password without solve the issue.
Gjennomsnittlig vurdering: -
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av James Steerpike -
Bilde av Particularly helpful Moodlers

A link to a gambling site is a pretty clear indication of a a problem. Someone has changed something so how are they doing it? On a LAMP server, someone had root access. 

Moodle has three parts - code, database and files in the moodledata folder.  First, back up everything including the database.

I would then create a clean LAMP server with new Moodle code, the moodledata files and db transferred over. I would look very carefully at security, starting with removing password and root access, allowing it only through ssh keys. Keep Linux and moodle up to date with security patches, lock down ports with a firewall and check audit logs.

Gjennomsnittlig vurdering: -
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av Ken Task -
Bilde av Particularly helpful Moodlers

I did look at your txt file ... line by line ... and commented on one that has the potential to replace the front page of your moodle.

Add the following line to your config.php file
$CFG->disableupdateautodeploy = true;
Reduce attack surface .... shutdown ftp or any other deamon that has ability to upload files.
Set sshd port to a high port in the range: 49152 to 65535 not well known port 22.
The only ports opened to the server are 80 and 443.

Manually remove files you don't expect to find.

Set ownerships/permissions to root:root with only the ability to read.

Run a clamscan on all files in moodle code ... it might find injections in .php files.

When running a git pull, if git sees any file that is new or whose byte size doesn't match what it knows, git will whoopie and show you the files in question.

Sent you a PM. ... please respond there to the requested info!

'SoS', Ken

Gjennomsnittlig vurdering: -
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av Visvanath Ratnaweera -
Bilde av Particularly helpful Moodlers Bilde av Translators

 The question is: in what way "someone" is installing software without our action?

I would say, that depends on what your server is. Pl. provide the information requested in Before you post... read this...

For a start, one could say, if your Moodle code directory tree is writable to the web server, that is a good start (for the hacker).

Talking about Indonesian, I wonder whether there was a theme whose name is a dirty word.

If a site is hacked, won't reinstalling it the same way lead to the same result? Are you sure the backup you reinstall is clean? Depending on the answer your options may be reasonable or next to impossible.

Gjennomsnittlig vurdering: -
Som svar til Francesco Garganese

Moodle server hacked with a number of added files.

av Justin Hunt -
Bilde av Core developers Bilde av Particularly helpful Moodlers Bilde av Plugin developers

The hackers clearly have a backdoor into the system. I will just suggest a few things that come to mind, in case it helps.

  • Your clean install is not clean
    Are you double sure the code was all new, and nothing was copied over?

  • Files or scripts in the data directory
    If a script was sneakily added to the data directory AND permissions on the data directory were not tight enough its possible a hacker could run a script on your server. After you did the clean install presumably the data dir was just re-attached to the clean install. Thats understandable but also potentially where they are getting in. Are the permissions as stated here: https://docs.moodle.org/501/en/Security_recommendations. And just check permissions generally against those recommendations.

  • Your credentials have been leaked
    If the site administrator password has been learned, of course they could just login and do stuff. Did you change it when you did the clean install? Also check to see if you have any "extra" site administrators that were added. And that the moodle permissions were not part of the hack, in particular permissions to install plugins. Because those will be stored in the database.   

    I would also check other credentials: e.g AWS access credentials or CPanel credentials. Though given the nature of the hack, it seems they are actually getting in via Moodle because all the hacks are installing Moodle plugins. 
Gjennomsnittlig vurdering:Useful (1)