Hi Emma,
I still don't know the root case for this (I'm still going through the code), but I've noticed that the profile fields names that appear in the new settings screen are all lower case (e.g., field_updatelocal_profile_field_state). On the other hand, the same setting in your database query is mixed case (i.e., field_updatelocal_profile_field_State.
If my understanding of the code flow is right, at some point we use array_key_exists() to check for the setting name, to see if it already exists as a configured setting. Given that array_key_exists() compares keys in a case-sensitive way, the upgrade code thinks you don't have the given setting already configured. And so it adds the setting to the new (unconfigured) settings list.
I still can't answer a couple of questions. First, why did it work before and now it doesn't? Second, why aren't the "new" settings stored correctly (i.e., lowercased) in the database once you click on "Save changes" on that screen?
I highly suspect (but can't confirm) that this issue is a bad interaction between the custom profile fields code (especially, allowing mixed-cased custom profiles shortnames) and the LDAP plugin (that expects all attribute mapping settings names to be lowercase. In fact, there have been several issues in the past (see e.g., MDL-49189 and MDL-57043, which is still open and afecting other places besides LDAP).
Emma, if you rename all your custom profile fields shortnames to lowercase (and change the settings names in the database accordingly), does the issue still persist?
Saludos.
Iñaki.
