Skip to content

feat: bump datafusion to 53#8107

Merged
waynexia merged 7 commits into
mainfrom
bump-datafusion
May 14, 2026
Merged

feat: bump datafusion to 53#8107
waynexia merged 7 commits into
mainfrom
bump-datafusion

Conversation

@waynexia
Copy link
Copy Markdown
Member

Signed-off-by: Ruihang Xia waynestxia@gmail.comI hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

Bump

  • datafusion to 53.1
  • arrow to 58.3
  • parquet to 58.3

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.

waynexia added 3 commits May 13, 2026 10:53
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@waynexia waynexia requested review from a team, discord9, evenyag and v0y4g3r as code owners May 13, 2026 07:03
@github-actions github-actions Bot added docs-not-required This change does not impact docs. size/XL labels May 13, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 372d707839

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/query/src/part_sort.rs Outdated
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request upgrades core dependencies to Arrow 58.3 and DataFusion 53.1.0, while refactoring the OpendalStore compatibility layer to support updated object_store traits like get_ranges and delete_stream. A major overhaul of PartSortExec replaces the previous TopK integration with manual buffer management and dynamic filter pushdown. Feedback identifies a potential compilation error in part_sort.rs caused by unstable Rust if-let chains and suggests optimizing the TopK compaction logic to reduce redundant sorting and improve computational complexity.

Comment thread src/query/src/part_sort.rs
Comment thread src/query/src/part_sort.rs Outdated
@waynexia waynexia requested a review from MichaelScofield May 13, 2026 08:56
discord9

This comment was marked as resolved.

discord9

This comment was marked as resolved.

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@waynexia
Copy link
Copy Markdown
Member Author

@codex review

@waynexia waynexia enabled auto-merge May 14, 2026 03:44
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dcf97128be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/query/src/part_sort.rs
Comment thread src/query/src/part_sort.rs
@waynexia waynexia added this pull request to the merge queue May 14, 2026
Merged via the queue into main with commit 3b3f5d6 May 14, 2026
48 checks passed
@waynexia waynexia deleted the bump-datafusion branch May 14, 2026 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs. size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants