Merge pull request #1040 from ethereum/eip-blob-schedule

feat(forks,specs,fixtures): add eip-7840 blob schedule config

Author: marioevz

docs/CHANGELOG.md

@@ -80,6 +80,7 @@ Release tarball changes:
 - ✨ Allow verification of the transaction receipt on executed test transactions ([#1068](https://github.com/ethereum/execution-spec-tests/pull/1068)).
 - ✨ Modify `valid_at_transition_to` marker to add keyword arguments `subsequent_transitions` and `until` to fill a test using multiple transition forks ([#1081](https://github.com/ethereum/execution-spec-tests/pull/1081)).
 - 🐞 fix(consume): use `"HIVE_CHECK_LIVE_PORT"` to signal hive to wait for port 8551 (Engine API port) instead of the 8545 port when running `consume engine` ([#1095](https://github.com/ethereum/execution-spec-tests/pull/1095)).
+- ✨ `state_test`, `blockchain_test` and `blockchain_test_engine` fixtures now contain the `blobSchedule` from [EIP-7840](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-7840.md), only for tests filled for Cancun and Prague forks ([#1040](https://github.com/ethereum/execution-spec-tests/pull/1040)).
 
 ### 🔧 EVM Tools
 
@@ -107,6 +108,7 @@ Release tarball changes:
 - The EOF fixture format contained in `eof_tests` may now contain multiple exceptions in the `"exception"` field in the form of a pipe (`|`) separated string ([#759](https://github.com/ethereum/execution-spec-tests/pull/759)).
 - Remove redundant tests within stable and develop fixture releases, moving them to a separate legacy release ([#788](https://github.com/ethereum/execution-spec-tests/pull/788)).
 - Ruff now replaces Flake8, Isort and Black resulting in significant changes to the entire code base including its usage ([#922](https://github.com/ethereum/execution-spec-tests/pull/922)).
+- `state_test`, `blockchain_test` and `blockchain_test_engine` fixtures now contain a `config` field, which contains an object that contains a `blobSchedule` field. On the `blockchain_test` and `blockchain_test_engine` fixtures, the object also contains a duplicate of the `network` root field. The root's `network` field will be eventually deprecated ([#1040](https://github.com/ethereum/execution-spec-tests/pull/1040)).
 
 ## [v3.0.0](https://github.com/ethereum/execution-spec-tests/releases/tag/v3.0.0) - 2024-07-22
 

docs/consuming_tests/blockchain_test.md

@@ -54,8 +54,12 @@ For each [`Fixture`](#fixture) test object in the JSON fixture file, perform the
 
 #### - `network`: [`Fork`](./common_types.md#fork)
 
+##### TO BE DEPRECATED
+
 Fork configuration for the test.
 
+This field is going to be replaced by the value contained in `config.network`.
+
 #### - `pre`: [`Alloc`](./common_types.md#alloc-mappingaddressaccount)
 
 Starting account allocation for the test. State root calculated from this allocation must match the one in the genesis block.
@@ -84,6 +88,20 @@ Account allocation for verification after all the blocks have been processed.
 
 Deprecated: Seal engine used to mine the blocks.
 
+#### - `config`: [`FixtureConfig`](#fixtureconfig)
+
+Chain configuration object to be applied to the client running the blockchain test.
+
+### `FixtureConfig`
+
+#### - `network`: [`Fork`](./common_types.md#fork)
+
+Fork configuration for the test. It is guaranteed that this field contains the same value as the root field `network`.
+
+#### - `blobSchedule`: [`BlobSchedule`](./common_types.md#blobschedule-mappingforkforkblobschedule)
+
+Optional; present from Cancun on. Maps forks to their blob schedule configurations as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840).
+
 ### `FixtureHeader`
 
 #### - `parentHash`: [`Hash`](./common_types.md#hash)

docs/consuming_tests/blockchain_test_engine.md

@@ -45,8 +45,12 @@ For each [`HiveFixture`](#hivefixture) test object in the JSON fixture file, per
 
 #### - `network`: [`Fork`](./common_types.md#fork)
 
+##### TO BE DEPRECATED
+
 Fork configuration for the test.
 
+This field is going to be replaced by the value contained in `config.network`.
+
 #### - `genesisBlockHeader`: [`FixtureHeader`](./blockchain_test.md#fixtureheader)
 
 Genesis block header.
@@ -63,10 +67,28 @@ Version of the `engine_forkchoiceUpdatedVX` directive to use to set the head of
 
 Starting account allocation for the test. State root calculated from this allocation must match the one in the genesis block.
 
+#### - `lastblockhash`: [`Hash`](./common_types.md#hash)
+
+Hash of the last valid block, or the genesis block hash if the list of blocks is empty, or contains a single invalid block.
+
 #### - `post`: [`Alloc`](./common_types.md#alloc-mappingaddressaccount)
 
 Account allocation for verification after all the blocks have been processed.
 
+#### - `config`: [`FixtureConfig`](#fixtureconfig)
+
+Chain configuration object to be applied to the client running the blockchain engine test.
+
+### `FixtureConfig`
+
+#### - `network`: [`Fork`](./common_types.md#fork)
+
+Fork configuration for the test. It is guaranteed that this field contains the same value as the root field `network`.
+
+#### - `blobSchedule`: [`BlobSchedule`](./common_types.md#blobschedule-mappingforkforkblobschedule)
+
+Optional; present from Cancun on. Maps forks to their blob schedule configurations as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840).
+
 ### `FixtureEngineNewPayload`
 
 #### - `executionPayload`: [`FixtureExecutionPayload`](#fixtureexecutionpayload)

docs/consuming_tests/common_types.md

@@ -82,6 +82,26 @@ Storage of the account.
 
 State allocation represented as a JSON object, where the keys are the addresses of the accounts, and the values are the accounts.
 
+### `BlobSchedule`: [`Mapping`](#mapping)`[`[`Fork`](#fork)`,`[`ForkBlobSchedule`](#forkblobschedule)`]`
+
+Maps forks to blob schedule configurations as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840).
+
+### `ForkBlobSchedule`
+
+A fork blob schedule as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840) as a JSON dictionary with the following entries:
+
+#### - `target`: [`ZeroPaddedHexNumber`](#zeropaddedhexnumber)
+
+The target blob count for a block.
+
+#### - `max`: [`ZeroPaddedHexNumber`](#zeropaddedhexnumber)
+
+The maximum possible blob count for a block.
+
+#### - `base_fee_update_fraction`: [`ZeroPaddedHexNumber`](#zeropaddedhexnumber)
+
+The blob base fee update fraction (adjusts the responsiveness of blob gas pricing per fork).
+
 ## Fork
 
 Fork type is represented as a JSON string that can be set to one of the following values:

docs/consuming_tests/state_test.md

@@ -60,6 +60,18 @@ Transaction to be executed.
 
 Mapping of lists of post for verification per fork, where each element represents a single possible outcome of the transaction execution after being applied to the `pre`.
 
+#### - `config`: [`FixtureConfig`](#fixtureconfig)
+
+Chain configuration object.
+
+### `FixtureConfig`
+
+At the moment this object can contain only the `blobSchedule` that is necessary to apply the correct cap to the maximum amount of blobs that a transaction can have. Otherwise, in forks prior to Cancun for example, it will be an empty JSON object.
+
+#### - `blobSchedule`: [`BlobSchedule`](./common_types.md#blobschedule-mappingforkforkblobschedule)
+
+Optional; present from Cancun on. Maps forks to their blob schedule configurations as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840).
+
 ### `FixtureEnvironment`
 
 #### - `currentCoinbase`: [`Address`](./common_types.md#address)

src/ethereum_clis/clis/besu.py

@@ -11,6 +11,7 @@
 
 import requests
 
+from ethereum_test_base_types import BlobSchedule
 from ethereum_test_exceptions import (
     EOFException,
     ExceptionMapper,
@@ -101,8 +102,9 @@ def evaluate(
         txs: List[Transaction],
         env: Environment,
         fork: Fork,
-        chain_id: int = 1,
-        reward: int = 0,
+        chain_id: int,
+        reward: int,
+        blob_schedule: BlobSchedule | None = None,
         eips: Optional[List[int]] = None,
         debug_output_path: str = "",
         state_test: bool = False,

src/ethereum_clis/tests/test_execution_specs.py

@@ -176,6 +176,9 @@ def test_evm_t8n(
             txs=txs,
             env=env,
             fork=Berlin,
+            chain_id=1,
+            reward=0,
+            blob_schedule=Berlin.blob_schedule(),
         )
         assert to_json(t8n_output.alloc) == expected.get("alloc")
         if isinstance(t8n, ExecutionSpecsTransitionTool):

src/ethereum_clis/transition_tool.py

@@ -8,7 +8,7 @@
 import textwrap
 import time
 from abc import abstractmethod
-from dataclasses import dataclass, field
+from dataclasses import dataclass
 from pathlib import Path
 from typing import Any, Dict, List, Mapping, Optional, Type
 from urllib.parse import urlencode
@@ -17,14 +17,21 @@
 from requests.exceptions import ConnectionError as RequestsConnectionError
 from requests_unixsocket import Session  # type: ignore
 
+from ethereum_test_base_types import BlobSchedule
 from ethereum_test_exceptions import ExceptionMapper
 from ethereum_test_fixtures import FixtureFormat, FixtureVerifier
 from ethereum_test_forks import Fork
 from ethereum_test_types import Alloc, Environment, Transaction
 
 from .ethereum_cli import EthereumCLI
 from .file_utils import dump_files_to_directory, write_json_file
-from .types import TransactionReceipt, TransitionToolInput, TransitionToolOutput
+from .types import (
+    TransactionReceipt,
+    TransitionToolContext,
+    TransitionToolInput,
+    TransitionToolOutput,
+    TransitionToolRequest,
+)
 
 model_dump_config: Mapping = {"by_alias": True, "exclude_none": True}
 
@@ -130,9 +137,10 @@ class TransitionToolData:
         txs: List[Transaction]
         env: Environment
         fork_name: str
-        chain_id: int = field(default=1)
-        reward: int = field(default=0)
-        state_test: bool = field(default=False)
+        chain_id: int
+        reward: int
+        blob_schedule: BlobSchedule | None
+        state_test: bool
 
         def to_input(self) -> TransitionToolInput:
             """Convert the data to a TransactionToolInput object."""
@@ -142,6 +150,18 @@ def to_input(self) -> TransitionToolInput:
                 env=self.env,
             )
 
+        def get_request_data(self) -> TransitionToolRequest:
+            """Convert the data to a TransitionToolRequest object."""
+            return TransitionToolRequest(
+                state=TransitionToolContext(
+                    fork=self.fork_name,
+                    chain_id=self.chain_id,
+                    reward=self.reward,
+                    blob_schedule=self.blob_schedule,
+                ),
+                input=self.to_input(),
+            )
+
     def _evaluate_filesystem(
         self,
         *,
@@ -298,37 +318,29 @@ def _evaluate_server(
         timeout: int,
     ) -> TransitionToolOutput:
         """Execute the transition tool sending inputs and outputs via a server."""
-        input_contents = t8n_data.to_input()
-        input_json = input_contents.model_dump(mode="json", **model_dump_config)
-        post_data = {
-            "state": {
-                "fork": t8n_data.fork_name,
-                "chainid": t8n_data.chain_id,
-                "reward": t8n_data.reward,
-            },
-            "input": input_json,
-        }
+        request_data = t8n_data.get_request_data()
+        request_data_json = request_data.model_dump(mode="json", **model_dump_config)
 
         if debug_output_path:
             request_info = (
                 f"Server URL: {self.server_url}\n\n"
-                f"Request Data:\n{json.dumps(post_data, indent=2)}\n"
+                f"Request Data:\n{json.dumps(request_data_json, indent=2)}\n"
             )
             dump_files_to_directory(
                 debug_output_path,
                 {
-                    "input/alloc.json": input_contents.alloc,
-                    "input/env.json": input_contents.env,
+                    "input/alloc.json": request_data.input.alloc,
+                    "input/env.json": request_data.input.env,
                     "input/txs.json": [
                         tx.model_dump(mode="json", **model_dump_config)
-                        for tx in input_contents.txs
+                        for tx in request_data.input.txs
                     ],
                     "request_info.txt": request_info,
                 },
             )
 
         response = self._server_post(
-            data=post_data, url_args=self._generate_post_args(t8n_data), timeout=timeout
+            data=request_data_json, url_args=self._generate_post_args(t8n_data), timeout=timeout
         )
         output: TransitionToolOutput = TransitionToolOutput.model_validate(response.json())
 
@@ -465,8 +477,9 @@ def evaluate(
         txs: List[Transaction],
         env: Environment,
         fork: Fork,
-        chain_id: int = 1,
-        reward: int = 0,
+        chain_id: int,
+        reward: int,
+        blob_schedule: BlobSchedule | None,
         eips: Optional[List[int]] = None,
         debug_output_path: str = "",
         state_test: bool = False,
@@ -493,6 +506,7 @@ def evaluate(
             fork_name=fork_name,
             chain_id=chain_id,
             reward=reward,
+            blob_schedule=blob_schedule,
             state_test=state_test,
         )
 

src/ethereum_clis/types.py

@@ -4,7 +4,7 @@
 
 from pydantic import Field
 
-from ethereum_test_base_types import Bloom, Bytes, CamelModel, Hash, HexNumber
+from ethereum_test_base_types import BlobSchedule, Bloom, Bytes, CamelModel, Hash, HexNumber
 from ethereum_test_types import Alloc, Environment, Transaction, TransactionReceipt
 
 
@@ -52,3 +52,19 @@ class TransitionToolOutput(CamelModel):
     alloc: Alloc
     result: Result
     body: Bytes | None = None
+
+
+class TransitionToolContext(CamelModel):
+    """Transition tool context."""
+
+    fork: str
+    chain_id: int = Field(..., alias="chainid")
+    reward: int
+    blob_schedule: BlobSchedule | None
+
+
+class TransitionToolRequest(CamelModel):
+    """Transition tool server request data."""
+
+    state: TransitionToolContext
+    input: TransitionToolInput

src/ethereum_test_base_types/__init__.py

@@ -16,7 +16,15 @@
     Wei,
     ZeroPaddedHexNumber,
 )
-from .composite_types import AccessList, Account, Alloc, Storage, StorageRootType
+from .composite_types import (
+    AccessList,
+    Account,
+    Alloc,
+    BlobSchedule,
+    ForkBlobSchedule,
+    Storage,
+    StorageRootType,
+)
 from .constants import (
     AddrAA,
     AddrBB,
@@ -39,16 +47,18 @@
     "AddrBB",
     "Address",
     "Alloc",
+    "BlobSchedule",
     "Bloom",
     "BLSPublicKey",
     "BLSSignature",
     "Bytes",
     "CamelModel",
     "EmptyOmmersRoot",
     "EmptyTrieRoot",
-    "FixedSizeBytes",
     "EthereumTestBaseModel",
     "EthereumTestRootModel",
+    "FixedSizeBytes",
+    "ForkBlobSchedule",
     "Hash",
     "HashInt",
     "HeaderNonce",