Skip to content

Use ConcurrentHashMap for per-route configurable state - #4378

Open
Frankle97 wants to merge 1 commit into
spring-cloud:mainfrom
Frankle97:fix/stateful-configurable-concurrent-map
Open

Frankle97 wants to merge 1 commit into
spring-cloud:mainfrom
Frankle97:fix/stateful-configurable-concurrent-map

Conversation

@Frankle97

@Frankle97 Frankle97 commented Nov 18, 2026 •

Copy link
Copy Markdown

Problem

AbstractStatefulConfigurable stores per-route configs (used by RedisRateLimiter and Bucket4jRateLimiter) in a plain HashMap. It is written while routes are built (AbstractRateLimiter#onApplicationEvent, GatewayFilterSpec.RequestRateLimiterSpec#rateLimiter) and read on every request. If two route builds fill the map at the same time, entries can be silently lost during a resize, and RedisRateLimiter#loadConfiguration then throws No Configuration found for route ... for every request on that route until the next refresh.

Background

CachingRouteLocator rebuilds routes on every RefreshRoutesEvent, on the thread that published it, and only updateCache() is synchronized. These events come from several independent sources: the startup ContextRefreshedEvent, discovery heartbeats and InstanceRegisteredEvent, RefreshScopeRefreshedEvent, /actuator/gateway/refresh, or the application itself (e.g. a scheduled reload of routes from a database). When two of them overlap, route builds, and the writes to this map, run on two threads at once. The map is most exposed right after startup, while it is still empty and resizes as it is filled.

We ran into this when an application-triggered refresh overlapped the startup refresh: on one instance, every rate-limited route kept failing until the next refresh.

Solution

Back the map with a ConcurrentHashMap. Existing callers never use null keys or values (keys are route ids, values are newConfig() instances), so behavior is otherwise unchanged. AbstractStatefulConfigurableTests populates the map from several threads and checks that no route id is lost.

AbstractStatefulConfigurable kept per-route configs in a HashMap. The
map is written while routes are built (AbstractRateLimiter on
FilterArgsEvent, GatewayFilterSpec.RequestRateLimiterSpec in the Java
DSL) and read on every request (RedisRateLimiter#loadConfiguration,
Bucket4jRateLimiter#loadConfiguration).

CachingRouteLocator only synchronizes updateCache(), so route builds
can run concurrently, e.g. when an application-triggered refresh
overlaps the refresh on startup. Concurrent puts into an empty HashMap
can silently drop entries while it resizes. A dropped entry makes
loadConfiguration() throw "No Configuration found for route" for every
request on that route until the next refresh. When the route also has
a circuit breaker, the exception is counted as a failure and opens the
breaker even though the backend is healthy.

Use a ConcurrentHashMap instead. Keys are route ids (never null:
CompositeRouteDefinitionLocator assigns one and Route requires one)
and values are newConfig() instances, so the null restriction of
ConcurrentHashMap does not affect existing callers.

Signed-off-by: Jaeyeob Jung <61595394+Frankle97@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants