It turned out to be a setting on our load balancer (Redirect Rewrite was set to Matching) so it was ignoring the headers. Working now.
Installing and upgrading help
Site Administration forms redirecting to https
Keskustelu on lukittu, joten et voi enää vastata siihen.