Skip to content

Add NEAREST join support #28937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
martint merged 2 commits into from
Apr 14, 2026
Merged

Add NEAREST join support #28937

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bf58801
Add NEAREST join support
martint Mar 30, 2026
9920a45
Simplify checks for "is join ON TRUE"
martint Apr 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion core/trino-grammar/src/main/antlr4/io/trino/grammar/sql/SqlBase.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,10 @@ relationPrimary
)?
((ERROR | EMPTY) ON ERROR)?
')' #jsonTable
| NEAREST '('
FROM relation
(WHERE where=booleanExpression)?
MATCH match=booleanExpression ')' #nearest
Comment on lines +484 to +487
Copy link
Copy Markdown
Member

@findepi findepi Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not NEAREST (relation) MATCH match?

This would make for a simpler grammar that's more familiar to users. Otherwise there will be questions why I cannot SELECT before the FROM or, why i can have WHERE but not HAVING, etc.

Copy link
Copy Markdown
Member Author

@martint martint Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need a WHERE clause to link the left and right sides of the join. I don't think that's more familiar to users. NEAREST is nothing but a simplified (syntactic sugar for) form of:

LATERAL (
    SELECT *
    FROM <relation>
    WHERE <condition> AND left.match_column <op> right.match_column
    ORDER BY <match column>
    LIMIT 1
)

If users want to do something more complicated, they can always use the explicit form.

Copy link
Copy Markdown
Member

@findepi findepi Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need a WHERE clause to link the left and right sides of the join.

If we're thinking about NEAREST as a special case of a join, we could use a syntax that describes that.

if we're thinking about NEAREST MATCH as a way to find the best match, then WHERE clause is no special.
The lateral subquery could be just values, some aggregation or something else. The MATCH is special, the WHERE is not. I don't see why grammar would make WHERE special.

Copy link
Copy Markdown
Member Author

@martint martint Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NEAREST is not a special kind of join. It is a table-like relation, similar in spirit to UNNEST or JSON_TABLE, whose job is to return the row from its input that is closest to some anchor expression from the left side. In that model, MATCH is special because it defines what "closest" means. WHERE is not special, but it's what's needed for the function to constrain the candidate set based on some row on the left side of the join.

;

jsonTableColumn
Expand Down Expand Up @@ -1052,7 +1056,7 @@ nonReserved
| KEEP | KEY | KEYS
| LANGUAGE | LAST | LATERAL | LEADING | LEAVE | LEVEL | LIMIT | LOCAL | LOGICAL | LOOP
| MAP | MATCH | MATCHED | MATCHES | MATCH_RECOGNIZE | MATERIALIZED | MEASURES | MERGE | MINUTE | MONTH
| NESTED | NEXT | NFC | NFD | NFKC | NFKD | NO | NONE | NULLIF | NULLS
| NESTED | NEXT | NFC | NFD | NEAREST | NFKC | NFKD | NO | NONE | NULLIF | NULLS
| OBJECT | OF | OFFSET | OMIT | ONE | ONLY | OPTION | ORDINALITY | OUTPUT | OVER | OVERFLOW
| PARTITION | PARTITIONS | PASSING | PAST | PATH | PATTERN | PER | PERIOD | PERMUTE | PLAN | POSITION | PRECEDING | PRECISION | PRIVILEGES | PROPERTIES | PRUNE
| QUOTES
Expand Down Expand Up @@ -1232,6 +1236,7 @@ MONTH: 'MONTH';
NATURAL: 'NATURAL';
NESTED: 'NESTED';
NEXT: 'NEXT';
NEAREST: 'NEAREST';
NFC : 'NFC';
NFD : 'NFD';
NFKC : 'NFKC';
Expand Down
1 change: 1 addition & 0 deletions core/trino-grammar/src/test/java/io/trino/grammar/sql/TestSqlKeywords.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ public void test()
"MONTH",
"NATURAL",
"NESTED",
"NEAREST",
"NEXT",
"NFC",
"NFD",
Expand Down
24 changes: 24 additions & 0 deletions core/trino-main/src/main/java/io/trino/sql/analyzer/Analysis.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
import io.trino.sql.analyzer.PatternRecognitionAnalysis.PatternInputAnalysis;
import io.trino.sql.planner.PartitioningHandle;
import io.trino.sql.tree.AllColumns;
import io.trino.sql.tree.ComparisonExpression;
import io.trino.sql.tree.DataType;
import io.trino.sql.tree.ExistsPredicate;
import io.trino.sql.tree.Expression;
Expand All @@ -73,6 +74,7 @@
import io.trino.sql.tree.JsonTableColumnDefinition;
import io.trino.sql.tree.LambdaArgumentDeclaration;
import io.trino.sql.tree.MeasureDefinition;
import io.trino.sql.tree.Nearest;
import io.trino.sql.tree.Node;
import io.trino.sql.tree.NodeRef;
import io.trino.sql.tree.Offset;
Expand Down Expand Up @@ -242,6 +244,7 @@ public class Analysis
private final Map<NodeRef<Table>, Map<ColumnHandle, Expression>> defaultColumnValues = new LinkedHashMap<>();

private final Map<NodeRef<Unnest>, UnnestAnalysis> unnestAnalysis = new LinkedHashMap<>();
private final Map<NodeRef<Nearest>, NearestAnalysis> nearestAnalysis = new LinkedHashMap<>();
private Optional<Create> create = Optional.empty();
private Optional<Insert> insert = Optional.empty();
private Optional<RefreshMaterializedViewAnalysis> refreshMaterializedView = Optional.empty();
Expand Down Expand Up @@ -1001,6 +1004,16 @@ public UnnestAnalysis getUnnest(Unnest node)
return unnestAnalysis.get(NodeRef.of(node));
}

public void setNearest(Nearest node, NearestAnalysis analysis)
{
nearestAnalysis.put(NodeRef.of(node), analysis);
}

public NearestAnalysis getNearest(Nearest node)
{
return nearestAnalysis.get(NodeRef.of(node));
}

public void addTableColumnReferences(AccessControl accessControl, Identity identity, Multimap<QualifiedObjectName, String> tableColumnMap)
{
AccessControlInfo accessControlInfo = new AccessControlInfo(accessControl, identity);
Expand Down Expand Up @@ -2609,6 +2622,17 @@ public record JsonTableAnalysis(
}
}

public record NearestAnalysis(
ComparisonExpression.Operator operator,
Expression candidateExpression)
{
public NearestAnalysis
{
requireNonNull(operator, "operator is null");
requireNonNull(candidateExpression, "candidateExpression is null");
}
}

public record CorrespondingAnalysis(List<Integer> indexes, List<Field> fields)
{
public CorrespondingAnalysis
Expand Down
172 changes: 153 additions & 19 deletions core/trino-main/src/main/java/io/trino/sql/analyzer/StatementAnalyzer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@
import io.trino.sql.analyzer.Analysis.GroupingSetAnalysis;
import io.trino.sql.analyzer.Analysis.JsonTableAnalysis;
import io.trino.sql.analyzer.Analysis.MergeAnalysis;
import io.trino.sql.analyzer.Analysis.NearestAnalysis;
import io.trino.sql.analyzer.Analysis.ResolvedWindow;
import io.trino.sql.analyzer.Analysis.SelectExpression;
import io.trino.sql.analyzer.Analysis.SourceColumn;
Expand All @@ -136,6 +137,7 @@
import io.trino.sql.tree.ColumnDefinition;
import io.trino.sql.tree.Comment;
import io.trino.sql.tree.Commit;
import io.trino.sql.tree.ComparisonExpression;
import io.trino.sql.tree.Corresponding;
import io.trino.sql.tree.CreateCatalog;
import io.trino.sql.tree.CreateMaterializedView;
Expand Down Expand Up @@ -198,6 +200,7 @@
import io.trino.sql.tree.MergeInsert;
import io.trino.sql.tree.MergeUpdate;
import io.trino.sql.tree.NaturalJoin;
import io.trino.sql.tree.Nearest;
import io.trino.sql.tree.NestedColumns;
import io.trino.sql.tree.Node;
import io.trino.sql.tree.NodeLocation;
Expand Down Expand Up @@ -414,6 +417,7 @@
import static io.trino.sql.analyzer.ExpressionTreeUtils.extractWindowMeasures;
import static io.trino.sql.analyzer.Scope.BasisType.TABLE;
import static io.trino.sql.analyzer.ScopeReferenceExtractor.getReferencesToScope;
import static io.trino.sql.analyzer.ScopeReferenceExtractor.hasReferencesToScope;
import static io.trino.sql.analyzer.SemanticExceptions.semanticException;
import static io.trino.sql.analyzer.TypeSignatureProvider.fromTypes;
import static io.trino.sql.analyzer.TypeSignatureTranslator.toTypeSignature;
Expand Down Expand Up @@ -1702,6 +1706,62 @@ protected Scope visitLateral(Lateral node, Optional<Scope> scope)
return createAndAssignScope(node, scope, queryScope.getRelationType());
}

@Override
protected Scope visitNearest(Nearest node, Optional<Scope> scope)
{
if (scope.isEmpty()) {
throw semanticException(NOT_SUPPORTED, node, "NEAREST is only supported on the right side of CROSS JOIN, INNER JOIN, LEFT JOIN, or an implicit join");
}

Scope leftScope = scope.orElseThrow();
if (leftScope.getRelationType().getAllFieldCount() == 0) {
throw semanticException(NOT_SUPPORTED, node, "NEAREST is only supported on the right side of CROSS JOIN, INNER JOIN, LEFT JOIN, or an implicit join");
}

// NEAREST is treated as a lateral relation by visitJoin(), but in the current implementation only the
// top-level WHERE and MATCH clauses may correlate with the left side of the join.
// The inner FROM relation is analyzed against the query boundary so it behaves like an
// uncorrelated relation body.
//
// TODO: If NEAREST is extended to allow correlation inside FROM <relation>, analyze the
// FROM relation against Optional.of(leftScope) instead, similar to LATERAL.
Scope sourceScope = process(node.getRelation(), Optional.of(leftScope.getQueryBoundaryScope()));

// Re-wrap the analyzed FROM relation in a scope whose parent is the left side of the
// join. This allows WHERE and MATCH to reference both the FROM relation fields and
// the left-side fields while keeping the inner FROM relation itself uncorrelated.
Scope nearestScope = createAndAssignScope(node, scope, sourceScope.getRelationType());

node.getWhere().ifPresent(where -> {
verifyNoAggregateWindowOrGroupingFunctions(session, functionResolver, accessControl, where, "NEAREST WHERE clause");
ExpressionAnalysis whereAnalysis = analyzeExpression(where, nearestScope, CorrelationSupport.ALLOWED);
Type whereType = whereAnalysis.getType(where);
if (!whereType.equals(BOOLEAN)) {
if (!whereType.equals(UNKNOWN)) {
throw semanticException(TYPE_MISMATCH, where, "NEAREST WHERE clause must evaluate to a boolean: actual type %s", whereType);
}
analysis.addCoercion(where, BOOLEAN);
}
verifyNoCorrelatedSubqueries(where, leftScope, sourceScope, "NEAREST WHERE clause");
analysis.recordSubqueries(node, whereAnalysis);
});

verifyNoAggregateWindowOrGroupingFunctions(session, functionResolver, accessControl, node.getMatch(), "NEAREST MATCH clause");
ExpressionAnalysis matchAnalysis = analyzeExpression(node.getMatch(), nearestScope, CorrelationSupport.ALLOWED);
Type matchType = matchAnalysis.getType(node.getMatch());
if (!matchType.equals(BOOLEAN)) {
if (!matchType.equals(UNKNOWN)) {
throw semanticException(TYPE_MISMATCH, node.getMatch(), "NEAREST MATCH clause must evaluate to a boolean: actual type %s", matchType);
}
analysis.addCoercion(node.getMatch(), BOOLEAN);
}
verifyNoCorrelatedSubqueries(node.getMatch(), leftScope, sourceScope, "NEAREST MATCH clause");
analysis.recordSubqueries(node, matchAnalysis);
analysis.setNearest(node, analyzeNearestMatch(node, nearestScope, leftScope));

return nearestScope;
}

@Override
protected Scope visitTableFunctionInvocation(TableFunctionInvocation node, Optional<Scope> scope)
{
Expand Down Expand Up @@ -3443,32 +3503,43 @@ protected Scope visitJoin(Join node, Optional<Scope> scope)
});
}
if (isUnnestRelation(node.getRight())) {
if (criteria != null) {
if (!(criteria instanceof JoinOn joinOn) || !joinOn.getExpression().equals(TRUE_LITERAL)) {
throw semanticException(
NOT_SUPPORTED,
criteria instanceof JoinOn joinOn ? joinOn.getExpression() : node,
"%s JOIN involving UNNEST is only supported with condition ON TRUE",
node.getType().name());
}
if (criteria != null && !isJoinOnTrue(criteria)) {
throw semanticException(
NOT_SUPPORTED,
getJoinErrorLocation(node, criteria),
"%s JOIN involving UNNEST is only supported with condition ON TRUE",
node.getType().name());
}
}
else if (isJsonTable(node.getRight())) {
if (criteria != null) {
if (!(criteria instanceof JoinOn joinOn) || !joinOn.getExpression().equals(TRUE_LITERAL)) {
throw semanticException(
NOT_SUPPORTED,
criteria instanceof JoinOn joinOn ? joinOn.getExpression() : node,
"%s JOIN involving JSON_TABLE is only supported with condition ON TRUE",
node.getType().name());
}
if (criteria != null && !isJoinOnTrue(criteria)) {
throw semanticException(
NOT_SUPPORTED,
getJoinErrorLocation(node, criteria),
"%s JOIN involving JSON_TABLE is only supported with condition ON TRUE",
node.getType().name());
}
}
else if (isNearestRelation(node.getRight())) {
if (criteria instanceof JoinUsing) {
throw semanticException(NOT_SUPPORTED, node, "JOIN USING involving NEAREST is not supported");
}
if ((node.getType() == Join.Type.INNER || node.getType() == LEFT) && !isJoinOnTrue(criteria)) {
throw semanticException(
NOT_SUPPORTED,
getJoinErrorLocation(node, criteria),
"%s JOIN involving NEAREST is only supported with condition ON TRUE",
node.getType().name());
}
if (node.getType() != Join.Type.CROSS && node.getType() != Join.Type.IMPLICIT && node.getType() != Join.Type.INNER && node.getType() != LEFT) {
throw semanticException(NOT_SUPPORTED, node, "%s JOIN involving NEAREST is not supported", node.getType().name());
}
}
else if (node.getType() == FULL) {
if (!(criteria instanceof JoinOn joinOn) || !joinOn.getExpression().equals(TRUE_LITERAL)) {
if (!isJoinOnTrue(criteria)) {
throw semanticException(
NOT_SUPPORTED,
criteria instanceof JoinOn joinOn ? joinOn.getExpression() : node,
getJoinErrorLocation(node, criteria),
"FULL JOIN involving LATERAL relation is only supported with condition ON TRUE");
}
}
Expand Down Expand Up @@ -4047,7 +4118,7 @@ private boolean isLateralRelation(Relation node)
if (node instanceof AliasedRelation aliasedRelation) {
return isLateralRelation(aliasedRelation.getRelation());
}
return node instanceof Unnest || node instanceof Lateral || node instanceof JsonTable;
return node instanceof Unnest || node instanceof Lateral || node instanceof JsonTable || node instanceof Nearest;
}

private boolean isUnnestRelation(Relation node)
Expand All @@ -4066,6 +4137,69 @@ private boolean isJsonTable(Relation node)
return node instanceof JsonTable;
}

private boolean isNearestRelation(Relation node)
{
if (node instanceof AliasedRelation aliasedRelation) {
return isNearestRelation(aliasedRelation.getRelation());
}
return node instanceof Nearest;
}
Comment thread
martint marked this conversation as resolved.

private static boolean isJoinOnTrue(JoinCriteria criteria)
{
return criteria instanceof JoinOn joinOn && joinOn.getExpression().equals(TRUE_LITERAL);
}

private static Node getJoinErrorLocation(Join join, JoinCriteria criteria)
{
if (criteria instanceof JoinOn joinOn) {
return joinOn.getExpression();
}
return join;
}

private void verifyNoCorrelatedSubqueries(Expression expression, Scope leftScope, Scope sourceScope, String description)
{
for (SubqueryExpression subquery : extractExpressions(ImmutableList.of(expression), SubqueryExpression.class)) {
if (hasReferencesToScope(subquery, analysis, leftScope) || hasReferencesToScope(subquery, analysis, sourceScope)) {
throw semanticException(UNSUPPORTED_SUBQUERY, subquery, "Correlated subqueries are not supported in %s", description);
}
}
}

private NearestAnalysis analyzeNearestMatch(Nearest node, Scope nearestScope, Scope leftScope)
{
if (!(node.getMatch() instanceof ComparisonExpression comparison)) {
throw semanticException(NOT_SUPPORTED, node.getMatch(), "NEAREST MATCH clause must be a comparison expression");
}
Comment thread
martint marked this conversation as resolved.

// MATCH is analyzed in nearestScope, which exposes fields from the FROM relation locally
// and the left join input through the parent scope.
boolean leftReferencesFromRelation = hasReferencesToScope(comparison.getLeft(), analysis, nearestScope);
boolean rightReferencesFromRelation = hasReferencesToScope(comparison.getRight(), analysis, nearestScope);
boolean leftReferencesOuterRelation = hasReferencesToScope(comparison.getLeft(), analysis, leftScope);
boolean rightReferencesOuterRelation = hasReferencesToScope(comparison.getRight(), analysis, leftScope);
if (leftReferencesFromRelation == rightReferencesFromRelation) {
throw semanticException(NOT_SUPPORTED, node.getMatch(), "NEAREST MATCH clause must compare one FROM relation expression with one non-FROM expression");
}

if ((leftReferencesFromRelation && leftReferencesOuterRelation) || (rightReferencesFromRelation && rightReferencesOuterRelation)) {
throw semanticException(NOT_SUPPORTED, node.getMatch(), "NEAREST MATCH clause must keep FROM relation and non-FROM expressions on opposite sides");
}

Expression candidateExpression = leftReferencesFromRelation ? comparison.getLeft() : comparison.getRight();

ComparisonExpression.Operator operator = leftReferencesFromRelation ? comparison.getOperator() : comparison.getOperator().flip();
if (operator != ComparisonExpression.Operator.LESS_THAN &&
operator != ComparisonExpression.Operator.LESS_THAN_OR_EQUAL &&
operator != ComparisonExpression.Operator.GREATER_THAN &&
operator != ComparisonExpression.Operator.GREATER_THAN_OR_EQUAL) {
throw semanticException(NOT_SUPPORTED, node.getMatch(), "NEAREST MATCH clause must use <, <=, >, or >=");
}

return new NearestAnalysis(operator, candidateExpression);
}

@Override
protected Scope visitValues(Values node, Optional<Scope> scope)
{
Expand Down
Loading