I've been thinking that if we are going to document this, trying to document it on the wiki is going to be very error prone.
Instead, we shoud document all the semantics of each cache (as much as possible) in the cache.php file where it is defined. If it is documented there, integrators can insist on docs before changes are integrated. We can also display the information on the cache admin screen.
I guess a good way to caputre the semantics is to use event-based cache invalidation (rather than manual cache invalidation code) whereve possible.
I also think the best solution would be for us to find a way for Moodle to reliably do purging of local caches in a load-balanced scenario. I know that it a hard problem, but if we can solve that hard problem, it just deals with the issue for all admins.