Support region, endpoint, and cross-region access from Iceberg REST catalog vended credentials by kaveti · Pull Request #27922 · trinodb/trino

kaveti · 2026-01-14T18:57:50Z

Support vended credentials from Iceberg REST catalog

Description

Adds support for client.region, s3.endpoint, s3.cross-region-access-enabled,
and s3.path-style-access from Iceberg REST catalog vended S3 credentials.

Also fixes ConnectorIdentity.extraCredentials handling so existing extra credentials
are preserved when vended S3 properties are applied.

Problem

When using vended credentials with Iceberg REST catalog, several S3 client settings
from the catalog response were being ignored by the native S3 filesystem. This meant:

Tables in different regions would connect using whatever region was set at catalog startup
Custom S3 endpoints (like MinIO or regional endpoints) specified by the REST catalog were ignored
Cross-region access settings from vended credentials had no effect
Path-style access settings from vended credentials had no effect, which is required for some S3-compatible endpoints
Pre-signed URI generation did not consistently use mapping-specific path-style behavior
Applying vended S3 properties could overwrite existing ConnectorIdentity.extraCredentials instead of merging them

Solution

This extends the existing S3 security mapping mechanism to support vended credentials
and related S3 connectivity properties:

Converted S3SecurityMappingProvider to an interface, with DefaultS3SecurityMappingProvider
preserving the existing file/URI-based security mapping behavior
Created VendedCredentialsS3SecurityMappingProvider in the Iceberg plugin that extracts
vended credentials and properties from ConnectorIdentity and returns an S3SecurityMappingResult
Extended S3SecurityMappingResult to carry optional pathStyleAccess
S3FileSystemModule now routes through S3FileSystemLoader with an optional
S3SecurityMappingProvider, which defaults to empty when no mapping is configured
IcebergRestCatalogFileSystemFactory populates ConnectorIdentity.extraCredentials with the
vended S3 properties from the Iceberg REST catalog response and merges them with any existing
extra credentials instead of overwriting them
IcebergVendedCredentialsModule binds VendedCredentialsS3SecurityMappingProvider when
iceberg.rest-catalog.vended-credentials-enabled=true
S3FileSystemLoader now applies mapping-aware region, endpoint, cross-region access,
and path-style access when creating S3 clients
S3FileSystemUtils now applies the same mapping-aware settings when creating S3 presigners,
so pre-signed URIs use the correct endpoint/addressing mode
Boolean vended properties are parsed strictly, including s3.cross-region-access-enabled
and s3.path-style-access
Client caching in S3FileSystemLoader reuses S3Client instances per unique credential
configuration, including path-style access

Precedence order:

Vended properties from the REST catalog, when S3 session credentials are vended for the table
Catalog-level static config as a fallback when the REST catalog does not provide a value
SDK defaults (when nothing is specified)

S3 FileIO properties not currently considered

The Iceberg S3 FileIO specification
defines additional properties that an Iceberg REST catalog could potentially vend, including
but not limited to:

Encryption: s3.sse.type, s3.sse.key, s3.sse.md5 (server-side encryption)
Access control: s3.acl, s3.access-grants.enabled, s3.access-grants.fallback-to-iam
Access points: s3.access-points.*
Tags: s3.write.tags.*, s3.delete.tags.*
Client behavior: s3.remote-signing-enabled

This PR focuses on the properties that affect S3 client connectivity — credentials,
region, endpoint, cross-region access, and path-style access — as these are the values
required to connect correctly to multi-region or S3-compatible REST-catalog-managed storage.

Support for additional vended properties can be added incrementally via the
S3SecurityMappingProvider extension point introduced here.

Example

When the REST catalog returns:

{
  "config": {
    "client.region": "eu-west-1",
    "s3.endpoint": "https://s3.eu-west-1.amazonaws.com",
    "s3.cross-region-access-enabled": "true",
    "s3.path-style-access": "true",
    "s3.access-key-id": "...",
    "s3.secret-access-key": "...",
    "s3.session-token": "..."
  }
}

Description

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things

cla-bot · 2026-01-14T18:57:54Z