There is currently no way to configure this timeout.
Currently the LDAP timeout is set to 5 seconds
CAM will wait the 5 seconds to perform both the TCP connection and the LDAP Bind of the administrator user’s credentials, before trying the next configured server.
The following ER has been raised. At the time of this writing there is no ETA (March 13th, 2019).
ER 798827: Make the LDAP timeout configurable