Alain Raap

For who's interested, I've also got my bash backup script working with our newer Moodle version (3.5.4+).

Our development environment: Redhat 7, Moodle 3.5.4+, Apache 2.4, PHP 7.1.8, Mariadb 10.1.29, Redis Caching Server 3.2.12 

My bash script I wrote and that I scheduled in the Linux cron for Apache user:

#!/bin/bash
# ------------------------------------------------------------------------------
# Script: backup_moodle_courses.sh
#
# - Backup a list of courses with admin/cli/backup.php
#
# Writer:       Alain Raap
# Date:         24-01-2019
# ------------------------------------------------------------------------------
exec > >(tee -i /var/log/backup_moodle_courses.log)
exec 2>&1

# ------------------------------------------------------------------------------
# Set parameters
# ------------------------------------------------------------------------------
SCRIPT="backup_moodle_courses"
MOODLE_ROOT="/var/www/moodle"
MOODLE_DATA="/moodledata"
COURSES="${MOODLE_ROOT}/admin/cli/courses.txt"
BACKUP="${MOODLE_ROOT}/admin/cli/backup.php"
PHP_CMD="/usr/local/bin/php -d log_errors=1 -d error_reporting=E_ALL -d display_errors=0 -d html_errors=0 -d memory_limit=2048M"
MUC="${MOODLE_DATA}/muc/config.php"
BACKUP_PATH="${MOODLE_DATA}/backups/"

echo "${SCRIPT}: Start backup script"

# ------------------------------------------------------------------------------
# Change MUC (Remove Moodle-redis store)
# ------------------------------------------------------------------------------
echo "${SCRIPT}: Remove Moodle-redis store from muc/config.php"
sed -i -e "0,/'store' => 'Moodle-redis'/s/'store' => 'Moodle-redis'/'store' => 'default_application'/" $MUC
sed -i -e "0,/'store' => 'Moodle-redis'/s/'store' => 'Moodle-redis'/'store' => 'default_session'/" $MUC

# ------------------------------------------------------------------------------
# Read input file with course-id's to backup
# ------------------------------------------------------------------------------
while read COURSE_LINE ; do
   ID="$(echo ${COURSE_LINE} | cut -d',' -f1)";
   NAME="$(echo ${COURSE_LINE} | cut -d',' -f2)";
   echo "${SCRIPT}: Backup of course: " $ID " - " $NAME;
   echo "$PHP_CMD $BACKUP --courseid=${ID} --destination=${BACKUP_PATH}"
   $PHP_CMD $BACKUP --courseid=${ID} --destination=${BACKUP_PATH}
done < ${COURSES}

# ------------------------------------------------------------------------------
# Change MUC (Restore Moodle-redis store)
# ------------------------------------------------------------------------------
echo "${SCRIPT}: Restore Moodle-redis store in muc/config.php"
sed -i -e "s/'store' => 'default_application'/'store' => 'Moodle-redis'/" $MUC
sed -i -e "s/'store' => 'default_session'/'store' => 'Moodle-redis'/" $MUC

echo "${SCRIPT}: End backup script"
exit 0

The inputfile for the bash script courses.txt (id and fullname of courses)

1001,course 1

1002,course 2

1003,course 3

Add this line in the crontab of your database server (runs on Saturday night 0:00 hrs):

0 0 * * 6 /var/www/moodle/admin/cli/backup_moodle_courses.sh > /dev/null

I've been working on a little POC to move the backup process to our data/database server and run it on this server (without Apache) webserver. Ken suggested this option and I spent some time on investigating this subject. Finally I've got it working now in our Redhat environment.
What I did was install php and an instance of Moodle (same version as on our website), wrote a little bash script that I run in the Linux cron. I took the course that didn't backup via the Moodle cron anymore and managed to make a backup that took 3 hours now instead of the 7 hours before. We use an NFS share for the Moodledata on the DB server.

Problem I encountered was the Caching store of Memcache that prevented the backup.php script to run. I managed to solve this in my backup bash script (temporary disabled/enabled the Memcache caching store in muc/config.php)

Our environment: Redhat 7, Moodle 3.1.5+, Apache 2.4, PHP 5.6, Mariadb 10.1.29 

My bash script I wrote and that I scheduled in the Linux cron for Apache user:

#!/bin/bash
# ------------------------------------------------------------------------------
# Script: backup_moodle_courses.sh
#
# - Backup a list of courses with admin/cli/backup.php
#
# Writer:       Alain Raap
# Date:         24-01-2019
# ------------------------------------------------------------------------------
exec > >(tee -i /var/log/backup_moodle_courses.log)
exec 2>&1

# ------------------------------------------------------------------------------
# Set parameters
# ------------------------------------------------------------------------------
SCRIPT="backup_moodle_courses"

MOODLE_ROOT="/var/www/moodle"
MOODLE_DATA="/moodledata"
COURSES="${MOODLE_ROOT}/admin/cli/courses.txt"
COMMAND="${MOODLE_ROOT}/admin/cli/backup.php"
PHP="/usr/bin/php -d log_errors=1 -d error_reporting=E_ALL -d display_errors=0 -d html_errors=0 -d memory_limit=2048M"
MUC="${MOODLE_DATA}/muc/config.php"
BACKUP_PATH="${MOODLE_DATA}/backups/"

echo "${SCRIPT}: Start backup script"

# ------------------------------------------------------------------------------
# Change MUC (Remove Moodle-memcache store)
# ------------------------------------------------------------------------------
echo "${SCRIPT}: Remove Moodle-memcache store from muc/config.php"
sed -i -e "s/'store' => 'Moodle-memcache'/'store' => 'default_application'/" $MUC

# ------------------------------------------------------------------------------
# Read input file with course-id's to backup
# ------------------------------------------------------------------------------
while read COURSE_LINE ; do
   ID="$(echo ${COURSE_LINE} | cut -d',' -f1)";
   NAME="$(echo ${COURSE_LINE} | cut -d',' -f2)";
   echo "${SCRIPT}: Backup of course: " $ID " - " $NAME;
   echo "$PHP $COMMAND --courseid=${ID} --destination=${BACKUP_PATH}"
   $PHP $COMMAND --courseid=${ID} --destination=${BACKUP_PATH}
done < ${COURSES}

# ------------------------------------------------------------------------------
# Change MUC (Restore Moodle-memcache store)
# ------------------------------------------------------------------------------
echo "${SCRIPT}: Restore Moodle-memcache store in muc/config.php"
sed -i -e "s/'store' => 'default_application'/'store' => 'Moodle-memcache'/" $MUC

echo "${SCRIPT}: End backup script"
exit 0

The inputfile for the bash script courses.txt (id and fullname of courses)

1001,course 1

1002,course 2

1003,course 3

@Ken I just tested this option with one of our 'headache' courses, normally this backup took 7 hours to finish, on the database server with a Moodle instance and only PHP it took only 3 hours! So that's worth to take a look at
Main difference I think is that the backup script didn't have to write to an NFS share and that it runs on the same server as the data of Moodle. We have our webserver in a DMZ and DB server in BZ, connected via NFS share.

@Ken I also tried this option, to make a backup with a Moodle instance on the database server. It looked like it wasn't much faster, and I also see that writing the course-files to the filesystem is slow in my opinion. What is your speed when writing to disk in the temporary folder of Moodle? And did you tweak any configuration settings for this backup configuration (database or PHP)?

Just a tip before you try to configure/test this, if you're using a caching server (like Memcache or Redis) that's not installed on your database server, remove the caching server in the plugin configuration settings (Plugins -> Caching -> configuration in Admin). This will prevent errors in your PHP error log. Otherwise the backup script won't work.

I think you need to thoroughly know Apache before removing any of these modules. If you follow the manual to install Apache for Moodle as Ken mentioned (https://docs.moodle.org/36/en/Apache) I think you'll already be able to run Moodle without problems. Further tuning and configuration depends on number of concurrent users and performance and security requirements. Hope this helps you to make a start.