feat(test-cli): Add support for testing block building via simulator by fselmo · Pull Request #2679 · ethereum/execution-specs

fselmo · 2026-04-14T19:23:15Z

🗒️ Description

Adds a hive-based simulator that verifies EL clients build correct blocks via testing_buildBlockV1

For each payload in a blockchain_test_engine fixture, the test sends the expected transactions and block attributes to the client's block building endpoint. The resulting block is validated against the fixture's expected output (state_root, receipts_root, etc). The fixture block is then imported via engine_newPayloadVX to advance the chain for subsequent blocks.

gas_limit is validated a bit different for now, since testing_buildBlockV1 doesn't accept a gas limit parameter. The client picks its own within the EIP-1559 adjustment range so we validate it is within that range until we can actually account for the gas limit. For this reason, block_hash is also excluded from comparison as it depends on gas_limit.

Also looking for recs on the naming (uv run build-block). We could go with build by itself but it feels vague to me. That said, I can be easily convinced here 😄.

🔗 Related Issues or PRs

closes #2560

✅ Checklist

All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
```
just static
```
All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
All: Considered updating the online docs in the ./docs/ directory.
All: Set appropriate labels for the changes (only maintainers can apply labels).

Cute Animal Picture

- Test build building via ``testing_buildBlockV1``, validating the built block, and re-consuming against the client

codecov · 2026-04-14T19:32:49Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.26%. Comparing base (6672cfe) to head (795a4f9).
⚠️ Report is 7 commits behind head on forks/amsterdam.

Additional details and impacted files

@@               Coverage Diff                @@
##           forks/amsterdam    #2679   +/-   ##
================================================
  Coverage            86.25%   86.26%           
================================================
  Files                  599      599           
  Lines                37032    37038    +6     
  Branches              3795     3795           
================================================
+ Hits                 31943    31949    +6     
  Misses                4525     4525           
  Partials               564      564

Flag	Coverage Δ
unittests	`86.26% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Wire up slotnum for `testing_buildBlockV1` for `bal-devnet-3` branch We are experimenting testing block building through hive via EELS (PR [here](ethereum/execution-specs#2679)). This is the only change needed to test against `bal-devnet-3` - missing slotnum.

marioevz

Looks great! Just two comments to try to improve maintainability.

fselmo · 2026-04-15T23:13:49Z

@marioevz lmk if the last commit here is what you meant. I re-ran against geth bal-devnet-3 with this cherry-picked there with some minor updates that are needed (slotnum, etc) and it's still running smooth 👍🏼

LouisTsai-Csie · 2026-04-16T15:37:38Z

+                    "-p",
+                    "execution_testing.cli.pytest_commands.plugins.consume.simulators.build_block.conftest",
+                ]
+            )


This is not important but maybe we could have something like this for the entire if-else comparison:

command_name = self.command_name simulator_command_list = ["engine", "enginex", ...] if command_name in simulator_command_list: modified_args.extend( [ "-p", f"execution_testing.cli.pytest_commands.plugins.consume.simulators.{command_name}.conftest", ] ) else: raise ValueError(f"Unknown command name: {self.command_name}")

i would like to continue on reviewing the remaining part of the PR!

LouisTsai-Csie

Thank you @fselmo for this feature! I leave some comment but they are all style-related issue.

For test_via_build.py, i think test_blockchain_via_build is too large as a single function and has too much nested logic. Could we split it into smaller helpers?

My proposal:

test_blockchain_via_build
-> bootstrap_engine_at_genesis: handle genesis block FCU, verify genesis hash
-> Extract valid payload and calls build_validate_and_advance for each payload

build_validate_and_advance
-> build block
-> validate block response and gas limit
-> proceed the chain with newPayload
-> update the chain with FCU

edit: two more things for the new command

Could we add some docs for the new command?
Should we PR to Hive for the new simulator?

LouisTsai-Csie · 2026-05-07T07:04:47Z

+    return PytestCommand(
+        config_file="pytest-consume.ini",
+        argument_processors=[
+            HelpFlagsProcessor("build_block"),


Suggested change

HelpFlagsProcessor("build_block"),

HelpFlagsProcessor("consume"),

The --help command is currently not working, this is the error message after i tried:

ERROR: usage: build-block [options] [file_or_dir] [file_or_dir] [...] build-block: error: unrecognized arguments: --build_block-help inifile: /Users/caijiacheng/Documents/Ethereum/eels-worktree/wt-eip-7778/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-consume.ini rootdir: /Users/caijiacheng/Documents/Ethereum/eels-worktree/wt-eip-7778

After we register build_block in HelpFlagsProcessor, it would be processed to --build_block-help command (see process_args in HelpFlagsProcessor).

But there is no such command registered under helper.py, only --consume-help, --fill-help and others.

We either register --build_block-help or update to HelpFlagsProcessor("consume").

LouisTsai-Csie · 2026-05-07T07:12:16Z

+@click.command(
+    name="build-block",
+    context_settings={
+        "help_option_names": ["-h", "--help"],


Suggested change

"help_option_names": ["-h", "--help"],

Do we need this? I checked common_pytest_options and found there is built-in help decorator:

# check common_pytest_options function func = click.option( "-h", "--help", "help_flag", is_flag=True, default=False, expose_value=True, help="Show help message.", )(func)

LouisTsai-Csie · 2026-05-07T07:14:51Z

+if TYPE_CHECKING:
+    from execution_testing.rpc.rpc_types import PayloadAttributes
+


Suggested change

if TYPE_CHECKING:

from execution_testing.rpc.rpc_types import PayloadAttributes

I think we dont need this? as you import PayloadAttributes in get_payload_attributes function, so there should be no effect of circular import.

LouisTsai-Csie · 2026-05-07T07:20:14Z

+        parent_beacon_block_root = (
+            self.params[2] if len(self.params) >= 3 else None
+        )


Suggested change

parent_beacon_block_root = (

self.params[2] if len(self.params) >= 3 else None

)

parent_beacon_block_root = (

self.params[2] if self.forkchoice_updated_version >= 3 else None

)

I think this is much clear. So now we know that the parent_beacon_block_root only exists starting from version 3.

LouisTsai-Csie · 2026-05-07T07:24:26Z

+                max_attempts=30,
+                wait_fixed=1.0,


Suggested change

max_attempts=30,

wait_fixed=1.0,

Removed since they are default value

LouisTsai-Csie · 2026-05-07T07:28:41Z

+            expected = genesis_header.block_hash
+            got = genesis_block["hash"]
+            logger.fail(
+                f"Genesis block hash mismatch. "
+                f"Expected: {expected}, Got: {got}"
+            )


Suggested change

expected = genesis_header.block_hash

got = genesis_block["hash"]

logger.fail(

f"Genesis block hash mismatch. "

f"Expected: {expected}, Got: {got}"

)

I notice this comparison logic is covered in GenesisBlockMismatchExceptionError:

# inside GenesisBlockMismatchExceptionError message = ( "Genesis block hash mismatch.\n\n" f"Expected: {expected_header.block_hash}\n" f" Got: {got_genesis_block['hash']}." )

danceratopz

Really nice!! 🔥

Nice review by @LouisTsai-Csie. In addition, I added one comment below about adding execution request verification for forks >= Prague.

About the naming, imo, I don't think this merits a new top-level command namespace and it that this rather belongs under consume, please see the comment below.

Docs

Also, could you please add some docs for this? Assuming you rename under consume, we could add it to:

docs/running_tests/running.md
docs/consume/simulators.md?

If we don't rename, then it will be trickier to find a spot to document this, but I think this demonstrates that it belongs to the consume family? But it could go in a new section docs/running_tests/build_block/index.md. But I'm strongly in favor of not adding a new top-level entry-point for this functionality.

System Test

Last request, could you add an e2e test in https://github.com/ethereum/execution-specs/blob/c9d8be0101c6649e060950317e14fb4f515f70e9/.github/workflows/hive-consume.yaml ? It should be pretty quick!

danceratopz · 2026-05-07T14:48:46Z

+                    expected=expected_payload,
+                    parent_gas_limit=parent_gas_limit,
+                )
+


We should check the execution requests here for >= v4. I.e., that build_response.execution_requests == payload.params[3].

Potential follow-up: It would also be really nice to test build_response.blobs_bundle, but we don't include commitments/proofs in any fixture format yet. We just have these tests that are execute mode only:

execution-specs/tests/osaka/eip7594_peerdas/test_get_blobs.py

Lines 352 to 367 in c9d8be0

@pytest.mark.parametrize_by_fork(

"txs_blobs",

generate_valid_blob_tests,

)

@pytest.mark.exception_test

@pytest.mark.valid_from("Cancun")

def test_get_blobs(

blobs_test: BlobsTestFiller,

pre: Alloc,

txs: List[NetworkWrappedTransaction | Transaction],

) -> None:

"""

Test valid blob combinations where one or more txs in the block serialized

version contain a full blob (network version) tx.

"""

blobs_test(pre=pre, txs=txs)

danceratopz · 2026-05-07T15:26:45Z

+@click.command(
+    name="build-block",


This command does use the testing_buildBlockV1 endpoint, but I think it's rather close to a consume simulator (it still consumes a fixture) so I wondered if it should be a consume sub-command?

I.e., something like uv run consume build-block.

Then the simulator in hive would be eels/consume-build-block.

Imo, when we add a command to fill tests using the testing_buildBlockV1 in the future, that command should be the new build or build-block command. Wdyt?

feat(test-cli): Add support for testing block building via simulator

aa4fd31

- Test build building via ``testing_buildBlockV1``, validating the built block, and re-consuming against the client

fselmo added C-feat Category: an improvement or new feature A-test-cli Area: execution_testing.cli labels Apr 14, 2026

fselmo marked this pull request as ready for review April 14, 2026 19:26

chore: pretty diff BALs on errors

679ff43

fselmo mentioned this pull request Apr 14, 2026

eth/catalyst: wire up slotnum for testing_buildBlockV1 ethereum/go-ethereum#34721

Merged

LouisTsai-Csie self-requested a review April 15, 2026 06:48

marioevz reviewed Apr 15, 2026

View reviewed changes

Comment thread ...ion_testing/cli/pytest_commands/plugins/consume/simulators/simulator_logic/test_via_build.py Outdated

Comment thread ...ion_testing/cli/pytest_commands/plugins/consume/simulators/simulator_logic/test_via_build.py Outdated

fselmo added a commit to fselmo/execution-specs that referenced this pull request Apr 15, 2026

refactor(test): cleanup based on comments on PR ethereum#2679

5852e2d

refactor(test): cleanup based on comments on PR ethereum#2679

795a4f9

fselmo force-pushed the feat/block-building-tests branch from 5852e2d to 795a4f9 Compare April 15, 2026 23:03

LouisTsai-Csie reviewed Apr 16, 2026

View reviewed changes

LouisTsai-Csie requested changes May 7, 2026

View reviewed changes

danceratopz requested changes May 7, 2026

View reviewed changes

	HelpFlagsProcessor("build_block"),
	HelpFlagsProcessor("consume"),

		if TYPE_CHECKING:
		from execution_testing.rpc.rpc_types import PayloadAttributes

	@pytest.mark.parametrize_by_fork(
	"txs_blobs",
	generate_valid_blob_tests,
	)
	@pytest.mark.exception_test
	@pytest.mark.valid_from("Cancun")
	def test_get_blobs(
	blobs_test: BlobsTestFiller,
	pre: Alloc,
	txs: List[NetworkWrappedTransaction \| Transaction],
	) -> None:
	"""
	Test valid blob combinations where one or more txs in the block serialized
	version contain a full blob (network version) tx.
	"""
	blobs_test(pre=pre, txs=txs)

		@click.command(
		name="build-block",

Conversation

fselmo commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🗒️ Description

🔗 Related Issues or PRs

✅ Checklist

Cute Animal Picture

Uh oh!

codecov Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

marioevz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fselmo commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LouisTsai-Csie left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danceratopz left a comment

Choose a reason for hiding this comment

Docs

System Test

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fselmo commented Apr 14, 2026 •

edited

Loading

codecov Bot commented Apr 14, 2026 •

edited

Loading

fselmo commented Apr 15, 2026 •

edited

Loading

LouisTsai-Csie left a comment •

edited

Loading