You may have noticed that currently adding a User to the edsvaSecondaryOwners Attribute, the corresponding value gets propagated to the msExchCoManagedByLink attribute.
But removing a User from edsvaSecondaryOwners attribute doesn't lead to the removal of the corresponding value from the msExchCoManagedByLink attribute.
This is a product defect (TF00456718).
This will be fixed on a future release of the product. If you require this immediately corrected, please contact support for a hotfix referencing the defect ID TF00456718.