feat: chain orchestrator #185

frisitano · 2025-07-02T19:06:49Z

closes: #182

Copilot

Pull Request Overview

This PR implements a new ChainOrchestrator to replace the previous indexer, integrates it throughout the node, watcher, network, and engine, and updates tests and database migrations accordingly.

Introduces ChainOrchestrator in place of Indexer, refactors RollupNodeManager to consume orchestrator events instead of indexer events.
Adds Synced notifications to L1Watcher and updates engine driver to handle optimistic sync via ChainOrchestrator.
Refactors configuration (ScrollRollupNodeConfig), network manager, and database migrations; adjusts tests to cover the new orchestrator flows.

Reviewed Changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
crates/indexer/src/lib.rs	Rename `Indexer` to `ChainOrchestrator` and overhaul API flows
crates/manager/src/manager/mod.rs	Replace indexer usage with `ChainOrchestrator` in node manager
crates/node/src/args.rs	Instantiate `ChainOrchestrator` in `ScrollRollupNodeConfig`
crates/watcher/src/lib.rs	Add `Synced` variant and `is_synced` flag to `L1Watcher`
crates/scroll-wire/src/protocol/proto.rs	Adjust doc comment for `NewBlock::new`
crates/node/tests/e2e.rs	Add/revise reorg and sync end-to-end tests
crates/watcher/tests/reorg.rs	Update tests to skip `Synced` notifications
crates/database/db/src/operations.rs	Extend DB ops with `L1MessageStart` and block-and-batch queries
crates/database/migration/src/migration_info.rs	Add `genesis_hash()` to migrations and insert genesis blocks
crates/network/src/manager.rs	Wire up eth-wire listener and dispatch chain-orchestrator events
crates/engine/src/driver.rs	Support `ChainImport` and `OptimisticSync` futures in engine driver

Comments suppressed due to low confidence (2)

crates/scroll-wire/src/protocol/proto.rs:33

The doc comment uses "blocks" (plural) but the constructor takes a single block; change to "block" for accuracy.

    /// Returns a [`NewBlock`] instance with the provided signature and blocks.

crates/node/tests/e2e.rs:95

The follower_can_reorg test has no assertions; either add meaningful checks or remove the empty test to maintain coverage.

async fn follower_can_reorg() -> eyre::Result<()> {

Copilot · 2025-07-16T14:15:04Z

crates/watcher/tests/reorg.rs

@@ -91,15 +91,19 @@ async fn test_should_detect_reorg() -> eyre::Result<()> {
            continue
        }

+        // skip the `L1Notification::Synced` notifications
+        let mut notification = l1_watcher.recv().await.unwrap();
+        if matches!(notification.as_ref(), L1Notification::Synced) {


This only skips one Synced notification; consider looping (e.g. while matches!(...) { ... }) to skip all consecutive Synced messages.

Suggested change

if matches!(notification.as_ref(), L1Notification::Synced) {

while matches!(notification.as_ref(), L1Notification::Synced) {

greged93

Couple of comments and some small nits and leftover code to clean.

.github/workflows/test.yaml

greged93 · 2025-07-23T09:44:05Z

crates/chain-orchestrator/src/metrics.rs

@@ -33,7 +36,7 @@ impl IndexerItem {
    }
 }

-/// The metrics for the [`super::Indexer`].
+/// The metrics for the [`super::ChainOrchestrator`].
 #[derive(Metrics, Clone)]
 #[metrics(scope = "indexer")]
 pub struct IndexerMetrics {


nit: ChainOrchestratorMetrics

greged93 · 2025-07-23T10:18:28Z

crates/chain-orchestrator/src/lib.rs

+        let mut received_chain_headers = vec![received_block.header.clone()];
+        let mut received_header_tail = received_block.header.clone();
+
+        // We should never have a re-org that is deeper than the current safe head.


why up to the safe head and not the finalized head?

This becomes a protocol design question. This assumes that L1 is the source of truth and batches posted to L1 should never be reorged. If so, then we would never expect a reorg deeper than the safe head. If there is a L1 reorg or a batch revert, then the safe head would change and deeper reorg would be allowed. If we allowed safe blocks to be reorged then a rogue sequencer could override batches posted to L1, would we want to allow this?

no you are right. I'm wondering if "safe" should be committed on L1 + L1 finalized, because currently "safe" in DB is just committed on L1 I believe.

I think this is worthy of open discourse amongst the team. In my opinion, safe should mean that it is highly probable that the block will be settled at some later time. The only way a block associated with a batch posted to L1 would not be settled is if the batch was reorged (and not included later), and given that reorgs are relatively infrequent and shallow, I think the current approach is reasonable. Furthermore, I think we can consider a block as finalized if the batch associated with the block is included in a finalzied block. Provided we work under the assumption of completeness of the proof system and permissionless submission of proofs then I think we can mark these blocks as described before any proofs are posted to L1. Lets open this discussion for the team more broadly.

crates/chain-orchestrator/src/lib.rs

greged93 · 2025-07-23T10:40:15Z

crates/chain-orchestrator/src/lib.rs

+                        received_header_tail = header;
+                    } else {
+                        return Err(ChainOrchestratorError::MissingBlockHeader {
+                            hash: current_chain_headers.front().unwrap().parent_hash,
+                        });
+                    }
+                }
+            }
+
+            // We search the in-memory chain to see if we can reconcile the block import.
+            if let Some(pos) = current_chain_headers
+                .iter()
+                .rposition(|h| h.hash_slow() == received_header_tail.parent_hash)


I'm not sure I understand this part of the code: you fetch 50 blocks from the L2 client, starting at the received block. At the end of the fetch phase, received_header_tail = received_block - 50 and current_chain_headers contains blocks from received_block to received_block - 50. But would the above branch ever work then?

I think there is a bug here, we shouldn't be updating received_header_tail = header. Let me update this and add some test cases for deep reorgs.

greged93 · 2025-07-23T15:32:21Z

crates/node/tests/sync.rs

+        signer_args: Default::default(),
+    };
+
+    // Create the chain spec for scroll dev with Euclid v2 activated and a test genesis.


nit:

Suggested change

// Create the chain spec for scroll dev with Euclid v2 activated and a test genesis.

// Create the chain spec for scroll dev with Feynman activated and a test genesis.

greged93 · 2025-07-23T15:33:45Z

crates/sequencer/tests/e2e.rs

@@ -40,7 +40,7 @@ async fn can_build_blocks() {
    reth_tracing::init_test_tracing();

    const BLOCK_BUILDING_DURATION: Duration = Duration::from_millis(0);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


greged93 · 2025-07-23T15:33:56Z

crates/sequencer/tests/e2e.rs

@@ -284,7 +281,7 @@ async fn can_build_blocks_with_finalized_l1_messages() {

    let chain_spec = SCROLL_DEV.clone();
    const BLOCK_BUILDING_DURATION: Duration = tokio::time::Duration::from_millis(0);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


greged93 · 2025-07-23T15:34:07Z

crates/sequencer/tests/e2e.rs

@@ -671,7 +674,7 @@ async fn can_build_blocks_and_exit_at_time_limit() {
    let chain_spec = SCROLL_DEV.clone();
    const MIN_TRANSACTION_GAS_COST: u64 = 21_000;
    const BLOCK_BUILDING_DURATION: Duration = Duration::from_secs(1);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


greged93 · 2025-07-23T15:35:20Z

crates/watcher/src/lib.rs

-            if self.is_synced() {
+            if self.is_synced {
                tokio::time::sleep(SLOW_SYNC_INTERVAL).await;
+            } else if self.current_block_number == self.l1_state.head {
+                // if we have synced to the head of the L1, notify the channel and set the
+                // `is_synced`` flag.
+                if let Err(L1WatcherError::SendError(_)) = self.notify(L1Notification::Synced).await
+                {
+                    tracing::warn!(target: "scroll::watcher", "L1 watcher channel closed, stopping the watcher");
+                    break;
+                }
+                self.is_synced = true;


the current logic suggests the watcher can never transition from is_synced = true -> false. Is this expected?

Good question. In the context of the RN, Synced should mean that we have synced all L1 messages required to validate messages included in unsafe L2 blocks. Given that we only include L1 messages after the corresponding L1 block has been finalized I think this should be fine provided the watcher doesn't start to lag > 2 epochs behind the safe tip then the Synced status should still remain valid. What do you think about this?

hmmm but so if we lose a provider for 12 minutes we might enter an edge case we can't exit from?

good point and given that we have had recent experiences of the L1 provider being down for longer than 12 minutes I think we should cover this case.

greged93

couple of extra comments and questions. Also I see we have a lot more unwraps in the code, are all these safe to keep?

greged93 · 2025-08-03T15:25:58Z

crates/chain-orchestrator/src/lib.rs

+        // Reverse the new chain headers to have them in the correct order.
+        received_chain_headers.reverse();


Personally think we would gain in code clarity if received_chain_headers and current_chain_headers were ordered in the same way.

greged93 · 2025-08-03T15:37:09Z

crates/chain-orchestrator/src/lib.rs

+                    let consolidated = if !*optimistic_mode.lock().await {
+                        true


we don't perform any consolidation here. Is this because in !optimistic_mode, the consolidation happens in handle_new_block?

greged93 · 2025-08-03T15:41:50Z

crates/engine/src/driver.rs

+        // Purge all pending block imports.
+        self.chain_imports.clear();


why do we purge all pending block imports?

greged93 · 2025-08-03T15:48:28Z

crates/network/src/manager.rs

+    /// Handles a new block received from the eth-wire protocol.
+    fn handle_eth_wire_block(


makes sense to have this here 👌

greged93 · 2025-08-03T15:51:08Z

crates/network/src/manager.rs

+        // Handle blocks received from the eth-wire protocol.
+        while let Some(Poll::Ready(Some(block))) =


note (not relevant for now but lets keep it in mind): at some point we might want to implement the same logic as in Reth to avoid having some components in the node monopolizing all resources.

greged93 · 2025-08-03T15:54:40Z

crates/sequencer/tests/e2e.rs

@@ -599,7 +598,7 @@ async fn can_build_blocks_and_exit_at_gas_limit() {
    let chain_spec = SCROLL_DEV.clone();
    const MIN_TRANSACTION_GAS_COST: u64 = 21_000;
    const BLOCK_BUILDING_DURATION: Duration = Duration::from_millis(250);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


frisitano added 12 commits July 2, 2025 19:44

feat: chain orchestrator

c1e5cc2

Merge branch 'main' into feat/chain-orchestrator

bbe0e8f

lint

4032fa5

lint

39802e2

docs lint

c84dc0d

fix: clone the in-memory chain instead of taking it

ea3140c

Merge branch 'main' into feat/chain-orchestrator

a28a5b2

update chain consolidation and l1 message validation

fa6952c

add missing network block error

43cc2be

update fork sync / reconcilliation to use network as opossed to database

5dfc813

Merge branch 'main' into feat/chain-orchestrator

a50558d

fix merege

19458f7

frisitano mentioned this pull request Jul 15, 2025

[ChainOrchestrator] Monitor performance of recursive syncing of blocks #199

Open

frisitano added 2 commits July 16, 2025 12:56

refactor and add test cases

4578d49

cleanup

e6248f5

frisitano marked this pull request as ready for review July 16, 2025 14:10

Merge branch 'main' into feat/chain-orchestrator

626e29b

frisitano requested a review from Copilot July 16, 2025 14:13

Copilot AI reviewed Jul 16, 2025

View reviewed changes

frisitano added 2 commits July 16, 2025 22:21

rename chain orchestrator crate

7a6ebfc

cleanup

7d35185

frisitano requested a review from greged93 July 17, 2025 17:14

frisitano added 7 commits July 18, 2025 18:36

Merge branch 'main' into feat/chain-orchestrator

74fd7b8

remove expect for sequencer in rnm

fe75ed5

sync test

f3bab0d

sync test

56a0ff3

sync test

1320f14

add error handling for missing paylaod id

0fa87fe

remove networking for L1 consolidation sync test

cecbd03

greged93 mentioned this pull request Jul 22, 2025

feat: bump reth #211

Merged

greged93 reviewed Jul 23, 2025

View reviewed changes

frisitano added 7 commits July 30, 2025 01:58

Merge branch 'main' into feat/chain-orchestrator

af4bac9

improve test coverage and fix bugs

7df1a59

lint

fa51172

fix cli NetworkArgs

e86446d

add block gap to reorg integration test

9cdbdf3

Merge branch 'main' into feat/chain-orchestrator

01a10cf

make test more robust

2f851b1

greged93 reviewed Aug 3, 2025

View reviewed changes

	if matches!(notification.as_ref(), L1Notification::Synced) {
	while matches!(notification.as_ref(), L1Notification::Synced) {

	// Create the chain spec for scroll dev with Euclid v2 activated and a test genesis.
	// Create the chain spec for scroll dev with Feynman activated and a test genesis.

		// Reverse the new chain headers to have them in the correct order.
		received_chain_headers.reverse();

		// Purge all pending block imports.
		self.chain_imports.clear();

		/// Handles a new block received from the eth-wire protocol.
		fn handle_eth_wire_block(

		// Handle blocks received from the eth-wire protocol.
		while let Some(Poll::Ready(Some(block))) =

feat: chain orchestrator #185

Are you sure you want to change the base?

feat: chain orchestrator #185

Uh oh!

Conversation

frisitano commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jul 16, 2025

Choose a reason for hiding this comment

Uh oh!

greged93 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greged93 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

frisitano commented Jul 2, 2025 •

edited

Loading