Symptom
CI job build-maven (ubuntu-latest, 26) (PR #701, which only touches Windows MSI packaging) failed with a single test hang out of 31829 tests:
[ERROR] PasswordPolicyTestCase.testResetWithLastLoginTime:4252 » ThreadTimeout
Method org.opends.server.core.PasswordPolicyTestCase.testResetWithLastLoginTime() didn't finish within the time-out 600000
The client side of the test hung for 600 s in LDAPModify.run(...) — the bind as uid=test.user with the new password never got a response.
Root cause
The thread dump captured in the job log shows the smoking gun — server worker thread spinning RUNNABLE (all other workers idle):
"Worker Thread 12" daemon prio=5 Id=85 RUNNABLE
at org.opends.server.types.AttributeBuilder$1.hasNext(AttributeBuilder.java:1007)
at org.opends.server.api.ClientConnection.getPrivileges(ClientConnection.java:1086)
at org.opends.server.api.ClientConnection.updatePrivileges(ClientConnection.java:1145)
at org.opends.server.api.ClientConnection.updateAuthenticationInfo(ClientConnection.java:883)
at org.opends.server.core.AuthenticatedUsers.doPostResponse(AuthenticatedUsers.java:223)
at org.opends.server.core.PluginConfigManager.invokePostResponseModifyPlugins(PluginConfigManager.java:3759)
...
at org.opends.server.core.PasswordPolicyState.finalizeStateAfterBind(PasswordPolicyState.java:2822)
at org.opends.server.workflowelement.localbackend.LocalBackendBindOperation.processLocalBind(LocalBackendBindOperation.java:168)
This is a regression from 51ed0a4 "Remove per-bind global lock contention in AuthenticatedUsers (#660)".
The bind triggers an internal modify (last-login-time update), and its post-response handler iterates the live ConcurrentHashMap.newKeySet() of connections registered for the user DN:
Set<ClientConnection> connectionSet = userMap.get(oldEntry.getName()); // live view
for (ClientConnection conn : connectionSet) {
conn.updateAuthenticationInfo(oldEntry, newEntry);
}But updateAuthenticationInfo → setAuthenticationInfo calls authenticatedUsers.remove(dn, this) followed by put(dn, this) — it removes and re-adds the connection to the very set being iterated.
ConcurrentHashMap appends the re-inserted node to the tail of its bin's chain, while the removed node keeps its next pointer. With ≥2 connections in the set this becomes a deterministic treadmill: the iterator processes C1 → C1 is re-appended at the tail → the iterator walks to the tail and sees C1 again → remove+put again → forever. The thread stays RUNNABLE, burning CPU in getPrivileges (the heaviest part of each iteration) — exactly what the dump shows.
Before #660 both protections against this existed: the loop iterated a snapshot iterator (CopyOnWriteArraySet) and ran under a reentrant write lock. Both were removed in one commit.
Why it is flaky
The infinite loop needs ≥2 connections registered under the same user DN (leftovers from the preceding ~77 test methods of the class whose sockets had not yet been closed/deregistered), and the re-appended node has to land in a bin the iterator has not passed yet. Hence the rare reproduction on a single runner (macOS/Windows Java 26 jobs passed).
The delete/modifyDN paths of doPostResponse are not affected: their sets are detached from the map first via removeSubtree.
Suggested fix
In doPostResponse(PostResponseModifyOperation) iterate over a snapshot, e.g.:
for (ClientConnection conn : connectionSet.toArray(new ClientConnection[0])) {
conn.updateAuthenticationInfo(oldEntry, newEntry);
}This restores the pre-#660 copy-on-write semantics while keeping registration lock-free.
Symptom
CI job
build-maven (ubuntu-latest, 26)(PR #701, which only touches Windows MSI packaging) failed with a single test hang out of 31829 tests:The client side of the test hung for 600 s in
LDAPModify.run(...)— the bind asuid=test.userwith the new password never got a response.Root cause
The thread dump captured in the job log shows the smoking gun — server worker thread spinning
RUNNABLE(all other workers idle):This is a regression from 51ed0a4 "Remove per-bind global lock contention in AuthenticatedUsers (#660)".
The bind triggers an internal modify (last-login-time update), and its post-response handler iterates the live
ConcurrentHashMap.newKeySet()of connections registered for the user DN:But
updateAuthenticationInfo→setAuthenticationInfocallsauthenticatedUsers.remove(dn, this)followed byput(dn, this)— it removes and re-adds the connection to the very set being iterated.ConcurrentHashMapappends the re-inserted node to the tail of its bin's chain, while the removed node keeps itsnextpointer. With ≥2 connections in the set this becomes a deterministic treadmill: the iterator processes C1 → C1 is re-appended at the tail → the iterator walks to the tail and sees C1 again → remove+put again → forever. The thread staysRUNNABLE, burning CPU ingetPrivileges(the heaviest part of each iteration) — exactly what the dump shows.Before #660 both protections against this existed: the loop iterated a snapshot iterator (
CopyOnWriteArraySet) and ran under a reentrant write lock. Both were removed in one commit.Why it is flaky
The infinite loop needs ≥2 connections registered under the same user DN (leftovers from the preceding ~77 test methods of the class whose sockets had not yet been closed/deregistered), and the re-appended node has to land in a bin the iterator has not passed yet. Hence the rare reproduction on a single runner (macOS/Windows Java 26 jobs passed).
The delete/modifyDN paths of
doPostResponseare not affected: their sets are detached from the map first viaremoveSubtree.Suggested fix
In
doPostResponse(PostResponseModifyOperation)iterate over a snapshot, e.g.:This restores the pre-#660 copy-on-write semantics while keeping registration lock-free.