Skip to content

Conversation

eserilev
Copy link
Member

@eserilev eserilev commented Sep 2, 2025

Issue Addressed

#7950

Proposed Changes

Skip column gossip verification logic during block production as its redundant and potentially computationally expensive.

@eserilev eserilev added das Data Availability Sampling v8.0.0-rc.0 Q3 2025 release for Fusaka on Holesky labels Sep 2, 2025
@eserilev eserilev added ready-for-review The code is ready for review optimization Something to make Lighthouse run more efficiently. labels Sep 2, 2025
@jimmygchen jimmygchen self-assigned this Sep 3, 2025
Copy link

mergify bot commented Sep 4, 2025

Some required checks have failed. Could you please take a look @eserilev? 🙏

@mergify mergify bot added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Sep 4, 2025
@jimmygchen
Copy link
Member

@eserilev the sync job is failing, doesn't feel like your change would trigger this, but interestingly both fullnode and supernode are failing to sync, and stuck at SyncTransition, would you mind checking the logs to see what happened?
https://github.com/sigp/lighthouse/actions/runs/17475882997/job/49636337462?pr=7973

Err(warp_utils::reject::custom_bad_request(format!("{e:?}")))
}
}
GossipVerifiedDataColumn::new_for_block_publishing(data_column_sidecar)
Copy link
Member

@jimmygchen jimmygchen Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still need to do the following to avoid trying to publish & process them again:

  1. check the availability cache and observed cache to make sure we exclude columns that we have seen
  2. mark the column as seen in the observe cache

verify_is_first_sidecar(chain, &data_column)?;
// Check if the data column is already in the DA checker cache. This happens when data columns
// are made available through the `engine_getBlobs` method. If it exists in the cache, we know
// it has already passed the gossip checks, even though this particular instance hasn't been
// seen / published on the gossip network yet (passed the `verify_is_first_sidecar` check above).
// In this case, we should accept it for gossip propagation.
if chain
.data_availability_checker
.is_data_column_cached(&data_column.block_root(), &data_column)
{
// Observe this data column so we don't process it again.
if O::observe() {
observe_gossip_data_column(&data_column, chain)?;
}
return Err(GossipDataColumnError::PriorKnownUnpublished);
}

We've seen a race condition where the proposer node gets the columns from peers before the proposer publishes - a scenario where other nodes was able to fetch blobs from the EL and compute them faster than the proposer node.

Copy link
Member

@jimmygchen jimmygchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added a minor comment but i think we're close to merging this.

// are made available through the `engine_getBlobs` method. If it exists in the cache, we know
// it has already passed the gossip checks, even though this particular instance hasn't been
// seen / published on the gossip network yet (passed the `verify_is_first_sidecar` check above).
// In this case, we should accept it for gossip propagation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also check for is_first_sidecar here, and we can avoid acquiring the cache write lock below (in the observe fn) if not needed - the cache read & write happens in burst around this time and it's time sensitive so it'd be great to avoid this.

@jimmygchen jimmygchen removed their assignment Sep 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
das Data Availability Sampling optimization Something to make Lighthouse run more efficiently. v8.0.0-rc.0 Q3 2025 release for Fusaka on Holesky waiting-on-author The reviewer has suggested changes and awaits thier implementation.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants