Skip to content
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
136e381
Introduce DataSourceResolver for multi-datasource support in JDBC per…
Subham-KRLX Mar 9, 2026
207aa4c
Address review comments on DataSourceResolver PR
Subham-KRLX Mar 9, 2026
e024bea
Final review nits on DataSourceResolver PR
Subham-KRLX Mar 9, 2026
96e7ff0
Address final nit: use import for RealmContext
Subham-KRLX Mar 9, 2026
9a07e6e
Final formatting and naming consistency cleanup
Subham-KRLX Mar 10, 2026
75e4096
Fix InactiveBeanException in non-JDBC profiles via lazy DataSource re…
Subham-KRLX Mar 11, 2026
fccb254
Address review comments: tighten scope, use plain injection and @Defa…
Subham-KRLX Mar 12, 2026
9c22635
Address review comments: tighten scope, use @Identifier and producer …
Subham-KRLX Mar 13, 2026
1e00328
Address final nits: Rename identifier to default and move config to J…
Subham-KRLX Mar 13, 2026
493c386
Refine DataSourceResolver configuration wiring per review
Subham-KRLX Mar 16, 2026
6311fbe
Address final CDI producer wiring nits
Subham-KRLX Mar 18, 2026
a36e745
Reinstate StoreType, update config docs, and finalize CDI wiring
Subham-KRLX Mar 23, 2026
118e046
Address review feedback: Resolve separate DataSources for METASTORE, …
Subham-KRLX Mar 24, 2026
dc8ffda
Address further review feedback: Revert unrelated changes, move CDI p…
Subham-KRLX Mar 25, 2026
7eb2304
Finalize PR #3960: split v4 DDL, fix license headers, and move CDI pr…
Subham-KRLX Mar 31, 2026
7ceaa69
Introduce DataSourceResolver for multi-datasource support in JDBC per…
Subham-KRLX Apr 1, 2026
9df42be
Refine Multi-DataSource support: selective bootstrap, independent ver…
Subham-KRLX Apr 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.polaris.persistence.relational.jdbc;

import javax.sql.DataSource;

/**
* Service to resolve the correct {@link DataSource} for a given realm and store type. This enables
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this Javadoc is describing the longer-term shape more than what the PR actually does today. Right now this is a metastore-routing foundation, not a system-wide metadata/metrics/events routing layer. I wonder if tightening the wording to match the current behavior would make the contract less confusing for future implementors/readers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback @flyingImer I completely agree. I have pushed an update that narrows the scope of this first step to strictly cover METASTORE routing, completely removing METRICS and EVENTS from StoreType and updating the Javadoc to reflect this tighter scope. I also removed the unnecessary runtime ambiguity by injecting DataSourceResolver directly instead of Instance. Finally, I added @DefaultBean to DefaultDataSourceResolver so downstream users can cleanly override the routing logic without ambiguous CDI resolution errors.

* isolating different workloads (e.g., entity metadata vs metrics vs events) into different
* physical databases or connection pools.
*/
public interface DataSourceResolver {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this contract is getting a bit ahead of the implementation. This PR only wires METASTORE, but the SPI already publishes METRICS and EVENTS as if those routing modes were part of the current supported surface. Would it make sense to keep the first step narrower and add new StoreType values only once the corresponding paths are actually routed through this resolver?

/** The type of store representing the workload pattern. */
enum StoreType {
METASTORE,
METRICS,
EVENTS
}

/**
* Resolves the DataSource for a given realm and store type.
*
* @param realmId the realm identifier
* @param storeType the type of store (e.g., main, metrics, events)
* @return the resolved DataSource
*/
DataSource resolve(
org.apache.polaris.core.context.RealmContext realmContext, StoreType storeType);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: import

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dimas-b I have pushed an update to resolve the CI failures. It turned out to be a CDI injection issue with the @Identifier annotation that I have now resolved.

}
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,15 @@ public class JdbcMetaStoreManagerFactory implements MetaStoreManagerFactory {
final Map<String, Supplier<BasePersistence>> sessionSupplierMap = new HashMap<>();

@Inject Clock clock;

@Inject PolarisDiagnostics diagnostics;

@Inject PolarisStorageIntegrationProvider storageIntegrationProvider;
@Inject Instance<DataSource> dataSource;

@Inject Instance<DataSourceResolver> dataSourceResolver;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this injected as Instance<DataSourceResolver> instead of a plain DataSourceResolver? In the current code we unconditionally call .get(), so it seems we are paying the runtime ambiguity cost without really using dynamic selection. Unless we expect multiple qualified resolvers here, would direct injection give us a cleaner and safer contract?


@Inject RelationalJdbcConfiguration relationalJdbcConfiguration;

@Inject RealmConfig realmConfig;

protected JdbcMetaStoreManagerFactory() {}
Expand Down Expand Up @@ -121,10 +126,12 @@ private void initializeForRealm(
metaStoreManagerMap.put(realmId, metaStoreManager);
}

public DatasourceOperations getDatasourceOperations() {
public DatasourceOperations getDatasourceOperations(
RealmContext realmContext, DataSourceResolver.StoreType storeType) {
DatasourceOperations databaseOperations;
try {
databaseOperations = new DatasourceOperations(dataSource.get(), relationalJdbcConfiguration);
DataSource resolvedDs = dataSourceResolver.get().resolve(realmContext, storeType);
databaseOperations = new DatasourceOperations(resolvedDs, relationalJdbcConfiguration);
} catch (SQLException sqlException) {
throw new RuntimeException(sqlException);
}
Expand Down Expand Up @@ -154,7 +161,8 @@ public synchronized Map<String, PrincipalSecretsResult> bootstrapRealms(
for (String realm : bootstrapOptions.realms()) {
RealmContext realmContext = () -> realm;
if (!metaStoreManagerMap.containsKey(realm)) {
DatasourceOperations datasourceOperations = getDatasourceOperations();
DatasourceOperations datasourceOperations =
getDatasourceOperations(realmContext, DataSourceResolver.StoreType.METASTORE);
int currentSchemaVersion =
JdbcBasePersistenceImpl.loadSchemaVersion(datasourceOperations, true);
int requestedSchemaVersion = JdbcBootstrapUtils.getRequestedSchemaVersion(bootstrapOptions);
Expand Down Expand Up @@ -219,7 +227,8 @@ public Map<String, BaseResult> purgeRealms(Iterable<String> realms) {
public synchronized PolarisMetaStoreManager getOrCreateMetaStoreManager(
RealmContext realmContext) {
if (!metaStoreManagerMap.containsKey(realmContext.getRealmIdentifier())) {
DatasourceOperations datasourceOperations = getDatasourceOperations();
DatasourceOperations datasourceOperations =
getDatasourceOperations(realmContext, DataSourceResolver.StoreType.METASTORE);
initializeForRealm(datasourceOperations, realmContext, null);
checkPolarisServiceBootstrappedForRealm(realmContext);
}
Expand All @@ -229,7 +238,8 @@ public synchronized PolarisMetaStoreManager getOrCreateMetaStoreManager(
@Override
public synchronized BasePersistence getOrCreateSession(RealmContext realmContext) {
if (!sessionSupplierMap.containsKey(realmContext.getRealmIdentifier())) {
DatasourceOperations datasourceOperations = getDatasourceOperations();
DatasourceOperations datasourceOperations =
getDatasourceOperations(realmContext, DataSourceResolver.StoreType.METASTORE);
initializeForRealm(datasourceOperations, realmContext, null);
}
checkPolarisServiceBootstrappedForRealm(realmContext);
Expand Down
1 change: 1 addition & 0 deletions runtime/common/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ plugins {

dependencies {
compileOnly(libs.smallrye.config.core)
implementation(project(":polaris-core"))
implementation(project(":polaris-relational-jdbc"))
implementation(platform(libs.quarkus.amazon.services.bom))
implementation("io.quarkiverse.amazonservices:quarkus-amazon-rds")
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.polaris.quarkus.common.config.jdbc;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import javax.sql.DataSource;
import org.apache.polaris.core.context.RealmContext;
import org.apache.polaris.persistence.relational.jdbc.DataSourceResolver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* Default implementation of {@link DataSourceResolver} that routes all realms and store types to a
* single default {@link DataSource}. This serves as both the production default and the base for
* multi-datasource extensions.
*/
@ApplicationScoped
public class DefaultDataSourceResolver implements DataSourceResolver {

private static final Logger LOGGER = LoggerFactory.getLogger(DefaultDataSourceResolver.class);

private final DataSource defaultDataSource;

@Inject
public DefaultDataSourceResolver(DataSource defaultDataSource) {
this.defaultDataSource = defaultDataSource;
}

@Override
public DataSource resolve(RealmContext realmContext, StoreType storeType) {
LOGGER.debug(
"Using default DataSource for realm '{}' and store '{}'",
realmContext.getRealmIdentifier(),
storeType);
return defaultDataSource;
}
}
Loading