Hi Rob,

I'm not familiar with Moodle on Windows Azure, but running the (automated) backup is a processor intensive task indeed.
We are running our Moodle instance on Redhat Linux + MariaDB database+data on a separate server with 8 CPU's and now
96GB memory for the database. We also had problems to tune this process and with a lot of courses, it takes a lot of time.
What is the number of courses backup-ed?
Things to look at are the temporary backup folder (fast enough FS?), load balancing and caching (do you use a caching server
on your Ubuntu OS?), just to name a few.
Ken Task wrote some good articles about the backup proces here in the forums, just have a look at them I can only say.
Maybe this thread can give you some insight, I had problems myself with the automated backup:
https://moodle.org/mod/forum/discuss.php?d=383012

The one moodle instance primarily used as a quiz machine is heavy processing ... at any time ... students taking or even during backups.   So you might look into how those quizzes are setup ... all questions on one page?  5 questions per page?  Could be you have some quiz related tables that are growing large.  Larger tables = longer processing.

Here is a query I found (didn't create) that shows largest tables:

SELECT CONCAT(table_schema, '.', table_name), CONCAT(ROUND(table_rows / 1000000, 2), 'M') rows, CONCAT(ROUND(data_length / ( 1024 * 1024 * 1024 ), 2), 'G') DATA, CONCAT(ROUND(index_length / ( 1024 * 1024 * 1024 ), 2), 'G') idx, CONCAT(ROUND(( data_length + index_length ) / ( 1024 * 1024 * 1024 ), 2), 'G') total_size, ROUND(index_length / data_length, 2) idxfrac FROM information_schema.TABLES ORDER BY data_length + index_length DESC LIMIT 10;

On a K12 site 'smaller' than yours but does a lot of quizzes throughout the site:

Also, when was the last time the DB for that instance was checked with some tool like MySQLTuner?

You didn't mention if the DB server is dedicated or does each of your servers 'do it all' ... code + DB + files?

What does the 'top' of 'top' look like?

'spirit of sharing', Ken

Back to OP's problem ...

There's a couple of things I've discovered on 'large' course backups (large here doesn't necessarily mean many files ... but 'large' processing ... like with Quizzes):

PHP max_input_vars increased from default ... which I think is 1000 to something larger ... 2 or 3 thousand.

And the unloading of Zend php xcode.   See anything related to that in php -m at the bottom for Zend extensions.   Not really needed to run Moodle and it had some restrictions that affected Moodle.

'spirit of sharing', Ken