Iceberg core code by unidevel · Pull Request #1836 · IBM/velox

unidevel · 2026-03-23T22:12:36Z

Fix the rebase issue in PR 425

Steps to resolve conflicts

Prepare remote and local branch(only once)

git remote add ibm git@github.com:IBM/velox.git
git fetch ibm
git checkout -b ibm-iceberg-new ibm/ibm-iceberg-new

Fetch latest changes and update

git fetch ibm

# checkout and reset(in case someone else changed it)
git checkout ibm-iceberg-new
git reset --hard ibm/ibm-iceberg-new

# get the commit sha
git log --oneline | head -1

Get the commit sha from output:

1629285ce Iceberg core code

Get the staging branch from conflict message and reset to it, e.g.

Rebase your branch with staging/staging-rebase and fix the conflict. If the rebase item is a PR, you can change the base branch to this staging branch.

# Now keep in the branch ibm-iceberg-new
git reset --hard ibm/staging/staging-rebase

Cherry-pick and resolve conflict

git cherry-pick <commit sha from step 1>

# resolve conflict
# will continue cherry-pick after testing in step 5

Save the script below in the project root and run it for testing:

#!/bin/sh
export EXTRA_CMAKE_FLAGS=" \
  -DVELOX_BUILD_TESTING=ON \
  -DVELOX_ENABLE_GEO=ON \
  -DVELOX_ENABLE_S3=ON \
  -DVELOX_ENABLE_PARQUET=ON \
  -DVELOX_MONO_LIBRARY=OFF \
  -DVELOX_ENABLE_SPARK_FUNCTIONS=ON \
  -DCMAKE_EXPORT_COMPILE_COMMANDS=1"
make clean

TESTS=" \
  velox_functions_iceberg_test \
  velox_hive_iceberg_test \
  velox_hive_connector_test \
  velox_hive_iceberg_dwrf_insert_test \
  velox_hive_iceberg_equality_delete_test \
  velox_hive_iceberg_deletion_vector_writer_test \
  velox_hive_iceberg_deletion_vector_test \
  velox_parquet_e2e_filter_test \
  velox_hive_iceberg_insert_test"

TESTS_REGEX=$(echo "$TESTS" | xargs | tr ' ' '|')

cmake -B "_build/release"
cmake --build _build/release --target $TESTS -j16

cd _build/release
ctest -V -R $TESTS_REGEX

Force push updated PR branch and base branch back

# after conflict resolved
git add -u .
git cherry-pick --continue

git push ibm  ibm-iceberg-new --force
git push ibm HEAD~1: ibm-iceberg-base --force

Add alchemy comments to this PR

Find the timestamp from conflict message like:

On a PR: alchemy merge @2026-05-15T06:38:01Z
On an issue: alchemy link [updated comma-separated commit SHAs for this issue] @2026-05-15T06:38:01Z

Copy the command like alchemy merge @2026-05-15T06:38:01Z and paste it as a new comment in this PR

Then follow the steps below to do the test.

unidevel · 2026-03-23T22:14:10Z

alchemy rebase @2026-03-23T21:37:29Z

prestodb-ci · 2026-03-23T22:14:12Z

Invalid alchemy verb: rebase

unidevel · 2026-03-23T22:14:33Z

alchemy merge @2026-03-23T21:37:29Z

prestodb-ci · 2026-03-23T22:14:35Z

alchemy link 4f10953 @2026-03-23T21:37:29Z

prestodb-ci · 2026-03-23T22:14:40Z

Added new rebase item:

ID 1205 [velox/ibm/1836] "Iceberg core code" first introduced at 2026-03-23T21:37:29Z owned by @PingLiuPing with commits 4f10953 - added at 2026-03-23T21:37:29Z by @prestodb-ci via Iceberg core code #1836 (comment), still effective

prestodb-ci · 2026-03-24T15:59:24Z

Failed to cherry-pick commit 4f10953 in rebase request #1841:

exit status 1
error: could not apply 4f10953ed... Iceberg core code

Auto-merging velox/connectors/hive/iceberg/CMakeLists.txt
Auto-merging velox/connectors/hive/iceberg/IcebergParquetStatsCollector.h
CONFLICT (content): Merge conflict in velox/connectors/hive/iceberg/IcebergParquetStatsCollector.h

Please:

Rebase your branch with staging/staging-e154cab1a-rebase and fix the conflict. If the rebase item is a PR, you can change the base branch to this staging branch.
Comment on this issue:

alchemy link [updated comma-separated commit SHAs for this issue] @2026-03-24T15:53:57Z

Re-open Rebase branch staging-e154cab1a-rebase with staging-e154cab1a-head (e154cab) #1841 to retry the cherry-pick.

unidevel · 2026-03-24T16:05:01Z

alchemy link 83bb267 @2026-03-24T15:53:57Z

prestodb-ci · 2026-03-24T16:05:06Z

The following unexpired item was removed at 2026-03-24T15:53:57Z by @unidevel via #1836 (comment):

ID 1205 [velox/ibm/1836] "Iceberg core code" first introduced at 2025-03-15T02:40:35Z owned by @PingLiuPing with commits 4f10953 - added at 2026-03-23T21:37:29Z by @prestodb-ci via Iceberg core code #1836 (comment), still effective

Added new rebase item:

ID 1209 [velox/ibm/1836] "Iceberg core code" first introduced at 2025-03-15T02:40:35Z owned by @PingLiuPing with commits 83bb267 - added at 2026-03-24T15:53:57Z by @unidevel via Iceberg core code #1836 (comment), still effective

prestodb-ci · 2026-03-26T08:00:08Z

Failed to cherry-pick commit 83bb267 in rebase request #1850:

exit status 1
error: could not apply 83bb26744... Iceberg core code

Auto-merging velox/docs/configs.rst
Auto-merging velox/dwio/common/Options.h
Auto-merging velox/dwio/parquet/tests/writer/ParquetWriterTest.cpp
Auto-merging velox/dwio/parquet/writer/Writer.cpp
CONFLICT (content): Merge conflict in velox/dwio/parquet/writer/Writer.cpp
Auto-merging velox/dwio/parquet/writer/Writer.h

Please:

Rebase your branch with staging/staging-ad018364c-rebase and fix the conflict. If the rebase item is a PR, you can change the base branch to this staging branch.
Comment on this issue:

alchemy link [updated comma-separated commit SHAs for this issue] @2026-03-26T07:55:58Z

Re-open Rebase branch staging-ad018364c-rebase with staging-ad018364c-head (ad01836) #1850 to retry the cherry-pick.

mohsaka · 2026-05-21T16:10:14Z

Removed UnicodeUtil and tests and closed OSS PR:
facebookincubator#17407

It is covered by
facebookincubator#17388

velox/dwio/parquet/writer/arrow/StringTruncation.cpp
velox/dwio/parquet/writer/arrow/tests/StringTruncationTest.cpp

mohsaka · 2026-05-21T16:12:36Z

alchemy merge @2026-05-20T22:38:38Z

prestodb-ci · 2026-05-21T16:12:38Z

alchemy link 10fd819 @2026-05-20T22:38:38Z

prestodb-ci · 2026-05-21T16:14:09Z

The following unexpired item was removed at 2026-05-20T22:38:38Z by @prestodb-ci via #1836 (comment):

ID 1452 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: c4ccb58
- Added: 2026-05-15T22:38:38Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

Added new rebase item:

ID 1480 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: 10fd819
- Added: 2026-05-20T22:38:38Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

mohsaka · 2026-05-21T17:48:16Z

Remove common namespace from Filter.cpp

diff --git a/velox/type/Filter.cpp b/velox/type/Filter.cpp
index 139dbbc3c..7a978705e 100644
--- a/velox/type/Filter.cpp
+++ b/velox/type/Filter.cpp
@@ -1110,7 +1110,7 @@ std::unique_ptr<Filter> createBigintValuesFilter(
 std::unique_ptr<Filter> createBigintValues(
     const std::vector<int64_t>& values,
     bool nullAllowed) {
-  return common::createBigintValuesFilter(values, nullAllowed, false);
+  return createBigintValuesFilter(values, nullAllowed, false);
 }
 
 std::unique_ptr<Filter> createHugeintValues(
@@ -1126,7 +1126,7 @@ std::unique_ptr<Filter> createHugeintValues(
 std::unique_ptr<Filter> createNegatedBigintValues(
     const std::vector<int64_t>& values,
     bool nullAllowed) {
-  return common::createBigintValuesFilter(values, nullAllowed, true);
+  return createBigintValuesFilter(values, nullAllowed, true);
 }

mohsaka · 2026-05-21T18:49:52Z

Timestamp PR opened here:
facebookincubator#17582

mohsaka · 2026-05-21T20:49:33Z

alchemy merge @2026-05-21T01:38:38Z

prestodb-ci · 2026-05-21T20:49:36Z

alchemy link 58e1704 @2026-05-21T01:38:38Z

prestodb-ci · 2026-05-21T20:49:39Z

The following unexpired item was removed at 2026-05-21T01:38:38Z by @prestodb-ci via #1836 (comment):

ID 1480 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: 10fd819
- Added: 2026-05-20T22:38:38Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

Added new rebase item:

ID 1485 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: 58e1704
- Added: 2026-05-21T01:38:38Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

mohsaka · 2026-05-21T22:03:42Z

Removed unnecessary streaming test change.

--- a/velox/exec/benchmarks/StreamingAggregationBenchmark.cpp
+++ b/velox/exec/benchmarks/StreamingAggregationBenchmark.cpp
@@ -84,9 +84,9 @@ class StreamingAggregationBenchmark : public VectorTestBase {
         std::to_string(params.numGroups));
 
     folly::addBenchmark(__FILE__, name, [plan = &test->plan]() {
-      std::shared_ptr<Task> task;
-      exec::test::AssertQueryBuilder(*plan).serialExecution(true).countResults(
-          task);
+      exec::test::AssertQueryBuilder(*plan)
+          .serialExecution(true)
+          .countResults();
       return 1;
     });

mohsaka · 2026-05-21T22:33:30Z

alchemy merge @2026-05-21T012:38:38Z

prestodb-ci · 2026-05-21T22:33:33Z

alchemy link 9593c94 @2026-05-21T012:38:38Z

prestodb-ci · 2026-05-21T22:33:36Z

Invalid timestamp 2026-05-21T012:38:38Z: parsing time "2026-05-21T012:38:38Z": month out of range

🫙 Alchemy usage

`alchemy link`

Track the current issue or PR as a rebase item with the specified commits.

alchemy link <comma-separated commit SHAs> [@timestamp] [#group]

@timestamp (optional): Accepts @2006-01-02T15:04:05Z07:00 or @01022006. Defaults to the author time of the first commit.
#group (optional): Rebase group this item belongs to.
- On a PR: defaults to the first group bound to the PR's base branch via branch-rebase.groups, or default when no binding.
- On a non-PR issue: always default.

`alchemy merge`

PR-only. Shortcut for alchemy link with all the PR's commits.

alchemy merge [@timestamp] [#group]

`alchemy close`

Close a rebase item linked to the current issue, scoped to one group.

alchemy close [@timestamp] [#group]

Without #group: closes the item in default.
With #group: closes the item in that group.
Re-close with a corrected @timestamp is supported.
Auto-closes the GitHub issue only when no other group still has active items.

`alchemy group list`

Post a reply comment listing every rebase group on this (repo, remote) with the creator and creation time.

`alchemy group create <name>`

Create an empty rebase group. Name charset: A-Za-z0-9._/-, max 64 chars.

`alchemy group create <name> from <source>`

Create a new group and duplicate every active item from <source> into it. The commenter is recorded as the duplicated items' added_by_user.

`alchemy sync`

Sync the open/closed state of this issue or PR with an upstream issue or PR.

alchemy sync [URL of the issue or PR to sync with | null]

`alchemy help`

Post this command reference as a comment.

mohsaka · 2026-05-21T22:33:51Z

alchemy merge @2026-05-21T03:38:38Z

prestodb-ci · 2026-05-21T22:33:53Z

alchemy link 9593c94 @2026-05-21T03:38:38Z

prestodb-ci · 2026-05-21T22:33:57Z

The following unexpired item was removed at 2026-05-21T03:38:38Z by @prestodb-ci via #1836 (comment):

ID 1485 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: 58e1704
- Added: 2026-05-21T01:38:38Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

Added new rebase item:

ID 1486 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: 9593c94
- Added: 2026-05-21T03:38:38Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

prestodb-ci · 2026-05-22T01:51:42Z

Failed to cherry-pick commit 9593c94 in rebase request #2053:

exit status 1
error: could not apply 9593c942c9... Iceberg core code

Auto-merging velox/connectors/hive/iceberg/IcebergDataSink.cpp
CONFLICT (content): Merge conflict in velox/connectors/hive/iceberg/IcebergDataSink.cpp
Auto-merging velox/connectors/hive/iceberg/PartitionSpec.cpp
CONFLICT (content): Merge conflict in velox/connectors/hive/iceberg/PartitionSpec.cpp
Auto-merging velox/docs/configs.rst
Auto-merging velox/dwio/common/Options.h

Please:

Reset your branch with staging/staging-rebase and fix the conflict. If the rebase item is a PR, you can change the base branch to this staging branch.
Comment on this issue with the updated rebase item:
- On a PR: alchemy merge @2026-05-22T01:46:41Z
- On an issue: alchemy link [updated comma-separated commit SHAs for this issue] @2026-05-22T01:46:41Z
Re-open Rebase branch staging-rebase (8e4c413) with staging-rebase-head (54d3510) #2053 to retry the cherry-pick.

Co-authored-by: Li Zhou <unidevel@hotmail.com> Co-authored-by: mohsaka <135669458+mohsaka@users.noreply.github.com>

mohsaka · 2026-05-22T03:36:12Z

alchemy merge @2026-05-22T01:46:41Z

prestodb-ci · 2026-05-22T03:36:14Z

alchemy link 42dd5b5 @2026-05-22T01:46:41Z

prestodb-ci · 2026-05-22T03:36:18Z

The following unexpired item was removed at 2026-05-22T01:46:41Z by @prestodb-ci via #1836 (comment):

ID 1486 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: 9593c94
- Added: 2026-05-21T03:38:38Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

Added new rebase item:

ID 1487 velox/ibm/1836 — "Iceberg core code"
- Group: default
- First introduced: 2025-03-15T02:40:35Z by @PingLiuPing
- Commits: 42dd5b5
- Added: 2026-05-22T01:46:41Z by @prestodb-ci via Iceberg core code #1836 (comment)
- Status: still effective

unidevel requested a review from majetideepak as a code owner March 23, 2026 22:12

prestodb-ci added Merged rebase-item labels Mar 23, 2026

prestodb-ci added the rebase-conflict label Mar 24, 2026

prestodb-ci mentioned this pull request Mar 24, 2026

Rebase branch staging-e154cab1a-rebase with staging-e154cab1a-head (e154cab) #1841

Closed

1 task

unidevel force-pushed the ibm-iceberg-new branch 2 times, most recently from 4f10953 to 83bb267 Compare March 24, 2026 16:03

prestodb-ci removed the rebase-conflict label Mar 24, 2026

prestodb-ci added the rebase-conflict label Mar 26, 2026

prestodb-ci mentioned this pull request Mar 26, 2026

Rebase branch staging-ad018364c-rebase with staging-ad018364c-head (ad01836) #1850

Closed

1 task

unidevel force-pushed the ibm-iceberg-new branch from 83bb267 to d8b1672 Compare March 26, 2026 10:35

This was referenced May 20, 2026

Rebase branch staging-rebase (8fd95bd) with staging-rebase-head (4b070b7) #2049

Closed

Rebase branch staging-rebase (efa0df7) with staging-rebase-head (f145f40) #2050

Closed

Rebase branch staging-rebase (1deac50) with staging-rebase-head (a2cac01) #2051

Closed

prestodb-ci mentioned this pull request May 21, 2026

Rebase branch staging-rebase (fd9b498) with staging-rebase-head (ca8ed06) #2052

Closed

1 task

This was referenced May 21, 2026

feat: Refactor and comment cleanup to improve code readability and add input validation facebookincubator/velox#17586

Open

feat: Add skipTrailingZeroSeconds option to Timestamp formatting facebookincubator/velox#17582

Open

prestodb-ci mentioned this pull request May 22, 2026

Rebase branch staging-rebase (8e4c413) with staging-rebase-head (54d3510) #2053

Closed

1 task

Iceberg core code

42dd5b5

Co-authored-by: Li Zhou <unidevel@hotmail.com> Co-authored-by: mohsaka <135669458+mohsaka@users.noreply.github.com>

This was referenced May 22, 2026

Rebase branch staging-rebase (4bfbf69) with staging-rebase-head (b1026ad) #2054

Closed

Rebase branch staging-rebase (cd0cb4e) with staging-rebase-head (559fc32) #2055

Closed

Conversation

unidevel commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Steps to resolve conflicts

Uh oh!

unidevel commented Mar 23, 2026

Uh oh!

prestodb-ci commented Mar 23, 2026

Uh oh!

unidevel commented Mar 23, 2026

Uh oh!

prestodb-ci commented Mar 23, 2026

Uh oh!

prestodb-ci commented Mar 23, 2026

Uh oh!

prestodb-ci commented Mar 24, 2026

Uh oh!

unidevel commented Mar 24, 2026

Uh oh!

prestodb-ci commented Mar 24, 2026

Uh oh!

prestodb-ci commented Mar 26, 2026

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

🫙 Alchemy usage

alchemy link

alchemy merge

alchemy close

alchemy group list

alchemy group create <name>

alchemy group create <name> from <source>

alchemy sync

alchemy help

Uh oh!

mohsaka commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

Uh oh!

prestodb-ci commented May 21, 2026

Uh oh!

prestodb-ci commented May 22, 2026

Uh oh!

mohsaka commented May 22, 2026

Uh oh!

prestodb-ci commented May 22, 2026

Uh oh!

prestodb-ci commented May 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

unidevel commented Mar 23, 2026 •

edited

Loading

`alchemy link`

`alchemy merge`

`alchemy close`

`alchemy group list`

`alchemy group create <name>`

`alchemy group create <name> from <source>`

`alchemy sync`

`alchemy help`