Removing it from the config is what we have done. But we have some that are in the config and some that are not, and it would be nice if things were more consistent. We will sometimes lose the sticky blocks and then have to put them back. I don't know if that happens when dashboards are reset or how they disappear.
It appears maybe the only solution is to write a one-off script to retroactively add blocks to already-created courses if we decide to stop using the sticky block method.