Skip to content

Conversation

0xpolarzero
Copy link
Collaborator

@0xpolarzero 0xpolarzero commented May 27, 2025

1. Summary

This pull request introduces @tevm/mud, an extension that implements optimistic updates for MUD applications.

Problem Solved:

  • Eliminates UI lag in MUD games by providing instant visual feedback
  • Maintains data consistency by reconciling optimistic state with canonical blockchain state
  • Provides transaction status tracking (simulating → optimistic → confirmed/reverted)

Core Mechanism:
The plugin intercepts writeContract calls and getStorageAt requests to create a parallel optimistic execution environment:

  1. Transaction Interception: When a user calls writeContract, the plugin:

    • Executes the transaction locally using a Tevm MemoryClient
    • Broadcasts the real transaction to the network simultaneously
    • Keeps the (optimistic) transaction in the pending pool
  2. State Management: Maintains two memory clients:

    • Internal Client: Builds optimistic state incrementally during transaction processing
    • Optimistic Client: Provides the complete optimistic view for storage requests and optimistic state retrieval in the client, by replaying the optimistic transactions—the pending pool—on top of the canonical chain
  3. Storage Override: Intercepts eth_getStorageAt calls to the MUD store contract, finding the right record inside the stash to encode it into a storage value instead of canonical blockchain data (or falls back to the canonical eth_getStorageAt)

  4. Reconciliation: When transactions confirm on-chain, the plugin removes them from the pending pool, which "automatically" reconciles the optimistic state with canonical state (the optimistic tx won't need to be replayed on top of the canonical state anymore)

2. Highlight: Practical Integration Examples

Vanilla JS Example

import { createOptimisticHandler } from '@tevm/mud';
import { createBundlerClient } from 'viem/account-abstraction'
import { createStash } from '@latticexyz/stash/internal';

// Setup your MUD stash and client
const stash = createStash(mudConfig);
const client = createBundlerClient({ ... });

// Create the optimistic handler
const handler = createOptimisticHandler({
  client,
  storeAddress: '0x...', // Your MUD world address
  stash,
  loggingLevel: 'debug'
});

// Core API methods:

// Get current optimistic state (canonical + pending optimistic changes)
// same API as `stash.get()`
const optimisticState = handler.getOptimisticState();

// Get specific optimistic record
// same API as `stash.getRecord()`
const playerRecord = handler.getOptimisticRecord({
  table: PlayerTable,
  key: { playerId: '0x123...' }
});

// Get multiple optimistic records
// same API as `stash.getRecords()`
const allPlayers = handler.getOptimisticRecords({
  table: PlayerTable
});

// Subscribe to optimistic state changes (fires on both optimistic and canonical updates)
// same API as `subscribeStash()`
const unsubscribe = handler.subscribeOptimisticState({
  subscriber: (updates) => {
    console.log('State updated:', updates);
    // Update your UI here
  }
});

// Subscribe to transaction status changes
const unsubscribeTx = handler.subscribeTx({
  subscriber: (status) => {
    console.log('Transaction status:', status);
    // status.status can be: 'simulating' | 'optimistic' | 'confirmed' | 'reverted'
    // Update loading states, show toasts, etc.
  }
});

// Use the sync adapter with MUD's store-sync
const syncAdapter = handler.syncAdapter; // Pass to MUD's SyncProvider

React Example

Based on the pattern in examples/mud/packages/client/src/Providers.tsx:

import React from 'react';
import { OptimisticWrapperProvider, useOptimisticState } from '@tevm/mud/react';
import { useSessionClient } from '@latticexyz/entrykit/internal';
import { stash } from './mud/stash';

// Provider setup
function OptimisticProvider({ children }) {
  const { data: sessionClient } = useSessionClient();
  
  return (
    <OptimisticWrapperProvider
      stash={stash}
      storeAddress="0x..." // Your MUD world address
      client={sessionClient}
      sync={{ startBlock: 0n }} // default will sync and start right away
      loggingLevel="debug"
    >
      {children}
    </OptimisticWrapperProvider>
  );
}

// Component using optimistic state
function PlayerComponent({ playerId }) {
  // Get optimistic player data with automatic re-renders
  // Here as well the API is the same as stash React hooks (`useState`)
  // we use a deep equal by default
  const player = useOptimisticState(
    state => getRecord({ state, table: PlayerTable, key: { playerId } }),
  );
  
  // Transaction status monitoring
  const [txStatus, setTxStatus] = useState(null);
  const wrapper = useOptimisticWrapper();
  
  useEffect(() => {
    if (!wrapper) return;
    return wrapper.subscribeTx({
      subscriber: setTxStatus
    });
  }, [wrapper]);
  
  const movePlayer = async (x, y) => {
    // This writeContract call is automatically intercepted
    // UI updates instantly with optimistic state
    await sessionClient.writeContract({
      address: worldAddress,
      abi: worldAbi,
      functionName: 'movePlayer',
      args: [playerId, x, y]
    });
  };
  
  return (
    <div>
      <div>Player Position: {player?.x}, {player?.y}</div>
      {txStatus?.status === 'simulating' && <div>Simulating...</div>}
      {txStatus?.status === 'optimistic' && <div>Pending confirmation...</div>}
      <button onClick={() => movePlayer(10, 20)}>
        Move Player
      </button>
    </div>
  );
}

3. Detailed Optimistic Update Flow

Here's the precise step-by-step trace of the entire optimistic update process:

Phase 1: Transaction Initiation

  1. User Action: User calls client.writeContract() (e.g., moving a player in a game)
  2. Interception: mudStoreWriteRequestOverride intercepts the call in bundler-packages/mud/src/internal/decorators/mudStoreWriteRequestOverride.ts:36
  3. Transaction ID Generation: System generates unique identifier with generateTxIdentifier() for tracking
  4. Parallel Execution:
    • Real transaction broadcast initiated via originalWriteContract()
    • Optimistic simulation begins simultaneously
    • Important here, we add the unique id (a short hex) as a suffix to the data for both transactions so we can pair them back when the canonical tx is synced to the client

Phase 2: Optimistic Simulation

  1. Status Update: Transaction status set to 'simulating', subscribers notified
  2. Cache Clearing: Fork cache cleared to force fresh getStorageAt requests through interceptor (this is an ugly workaround to the request bypassing the getStorageAt procedure, which we intercept, and trying to get from the state's manager cache directly)
  3. Local Execution: memoryClient.tevmCall() executes transaction with:
    • Same parameters as real transaction
    • addToMempool: true - adds to pending transaction pool
    • blockTag: 'pending' - executes on pending block
  4. Transaction Pool Addition: Transaction added to MemoryClient's transaction pool, triggering 'txadded' event

Phase 3: Optimistic State Calculation

  1. Pool Processing: processTransactionsAndUpdateLogs() triggered by 'txadded' event
  2. Transaction Ordering: txPool.txsByPriceAndNonce() orders pending transactions
  3. VM Copy Creation: Deep copy of VM created for isolated execution
  4. Incremental Execution: For each pending transaction:
    • Storage cache cleared for store address
    • Transaction executed via vmCopy.runTx()
    • Event logs extracted and parsed
    • MUD store updates accumulated in internalLogs
  5. State View Generation: getStateView() creates optimistic state:
    • Deep clones canonical stash state
    • Applies accumulated optimistic updates
    • Returns merged optimistic state

Phase 4: UI Update Propagation

  1. Subscriber Notification: notifyStashSubscribers() notifies all optimistic state subscribers
  2. React Re-renders: Components using useOptimisticState automatically re-render with optimistic data
  3. Status Update: Transaction status updated to 'optimistic'

Phase 5: Storage Request Interception

  1. Storage Requests: When fork requests getStorageAt for store contract:
    • mudStoreGetStorageAtOverride intercepts the request
    • Checks if requested storage slot exists in optimistic state
    • Returns optimistic value or falls back to canonical storage
    • Ensures subsequent contract reads see optimistic changes

Phase 6: Network Confirmation

  1. Transaction Mining: Real transaction confirms on network
  2. Receipt Processing: waitForTransactionReceipt() detects confirmation (this one is in case it wasn't synced by the store-sync, e.g. if it reverted/failed, so we remove it from the optimistic pool; otherwise, the store-sync will hit first)
  3. Status Final Update: Transaction status updated to 'confirmed' or 'reverted'
  4. Canonical Sync: MUD's store-sync processes incoming transaction:
    • Canonical stash updated with real on-chain data
    • syncAdapter.onTx() called with transaction hash
  5. Optimistic Cleanup:
    • matchOptimisticTxCounterpart() finds corresponding optimistic transaction (using the data suffix we added to both)
    • Optimistic transaction removed from pending pool
    • 'txremoved' event triggers optimistic state recalculation (we reapply all remaining optimistic txs on top of the newly updated canonical state, to cache the optimistic logs)
  6. State Reconciliation: Optimistic state recalculated without confirmed transaction
  7. UI Reconciliation: React components re-render with final canonical state

Error Handling & Edge Cases

  • Simulation Failure: If optimistic execution fails, error logged but real transaction still broadcast
  • Transaction Revert: If real transaction reverts, optimistic transaction removed from pool
  • Race Conditions: stateUpdateCoordinator serializes optimistic updates to prevent conflicts (one of the ugly workarounds)
  • Multiple Rapid Transactions: Each gets unique identifier and processed in order

4. Limitations

Client Type Restriction

The current implementation is specifically designed for MUD projects using an EntryKit SessionClient (bundler client) and will not work with other client types. The interception logic in mudStoreWriteRequestOverride.ts:18 explicitly checks for client.type === 'bundlerClient' and only overrides the writeContract method on SessionClient instances. This should not be hard to implement for other client types, but requires additional development.

Implementation Architecture Issues

The implementation is a 'clunky' first version with several non-ideal practices:

  • Direct Method Mutation: The plugin directly overwrites/mutates the writeContract method on the provided client instance (client.writeContract = async function... in mudStoreWriteRequestOverride.ts:36), which is dangerous and can cause unexpected behaviors if other extensions depend on the original reference.

  • Dual Client Architecture: The system requires two separate MemoryClients (internalClient and optimisticClient) with different storage interceptors, which adds complexity and resource overhead.

Race Conditions and State Management

The example demonstrates that running multiple calls quickly causes the UI to flash incorrect states, highlighting probable race conditions or similar issues. Despite attempts to mitigate this with a stateUpdateCoordinator class to manage state inside the plugin, it's not sufficient:

  • Asynchronous State Updates: The optimistic state calculation and removal from pending pool being async creates windows where multiple MUD sync → stash update → remove tx from pending pool operations can fire rapidly and interfere with each other. This is what the stateUpdateCoordinator fixes, but needing it indicates the architecture has fundamental synchronization issues.

  • Memory Management: The plugin requires a WeakMap registry (handlerRegistry in useOptimisticWrapper.tsx:23) to prevent multiple writeContract wrappers and txPool subscriptions, which is described as "awful and obviously needs to be fixed."

## Description

The MUD plugin before that has several flaws, and is super slow to
execute optimistic updates, which makes the whole plugin not really
relevant.

This PR introduces multiple fixes to highly reduce the occurrence of
sync inconsistencies/race conditions, some layer of caching and
optimizations, and attempts to fix remaining issues.

Fixes:
- Before, when getting the optimistic state, for each tx ran, we would
apply the logs and notify subscribers, which would cause consumers to
request the state, hence run that tx again. This would be problematic as
multiple txs need to be processed, since each tx update would trigger
another run with all the txs so far again. The fix here is simply to run
txs and apply the logs to build up the state, and notify subscribers
only after we ran all the txs.
- Much better separation between view and mutable state. This causes the
main optimistic state getters to become sync, as we now "cache" the
state.

Two biggest pain points that make the whole plugin broken and
unoptimized respectively, are the following:
- [x] There is an architectural sync issue: we have two separate,
asynchronous event streams that should be synchronized but aren't:
- canonical sync stream: stash updates when canonical transactions are
indexed
- optimistic pool stream: txremoved events when optimistic transactions
are removed
These streams are inherently out of sync because:
- The canonical sync happens when the indexer processes blocks
- The optimistic tx removal happens when we detect the canonical tx
exists
- We don't have the canonical tx hash available when the sync event
fires
**-> For now, this one has been fixed by appending a small hex string to
the end of both the canonical and optimistic txs data so we can link
them back together later**
- [ ] Everytime we run a call locally basically, we're clearing the fork
cache for the store address, as if we didn't, it would directly retrieve
storage values from that cache and never hit our getStorageAt override
(that fetches from the synced MUD store instead). This is super bad
design, and also causes a bunch of other storage slots to be
invalidated, when these actually should stay.
- [ ] We'll probably want our fork clients to rebase frequently so it
considers the latest fork state when executing txs
Copy link
Collaborator

@roninjin10 roninjin10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't get through entire pr but left a bunch of comments.

@@ -0,0 +1,119 @@
{
"name": "@tevm/mud",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this in bundler-packages instead of packages?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought everything inside packages was related to the evm. Saw whatsabi in bundler-packages so just copied that. Maybe we can move right before merging to not break comments?

type QueuedUpdate = { type: 'optimistic'; fn: () => Promise<void> } | { type: 'canonical'; fn: () => Promise<void> }

/**
* Coordinates state updates to prevent race conditions between optimistic and canonical updates.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't need this. Needing this is a smell that we are not letting MUD stay source of truth for all data and when state transitions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is one of the atrocities I was mentioning, I just don't know how to avoid such compromise with the current design.

MUD sync is the source of truth, but applying the optimistic state & removing from the pending pool being async we eventually end up with race conditions when multiple MUD sync -> stash update -> remove tx from pending pool fire rapidly and both read and alter the state at the same time.

@@ -0,0 +1,17 @@
// React
export const deepEqual = (obj1: any, obj2: any): boolean => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit it isn't detecting cycles so this can run forever and freeze program. Generally we should avoid needing this it's a smell

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This deepEqual is used in our react hooks. Since getting the optimistic state means internally copying the original stash state and adding optimistic records on top, we have to perform deepEqual as the state reference is never the same.

We use it in the memoize commented below to avoid getting the records again for every render if the stash state (the optimistic one in our case) didn't actually change.

@roninjin10
Copy link
Collaborator

This shouldn't be in bundler-packages since it's not a bundler. extensions/* is better spot. I think I'm gonna just leave this on the branch and cherry pick stuff off of it when I circle around to looking at MUD so don' tworry about rebasing the pr or anything

@0xpolarzero
Copy link
Collaborator Author

@claude Please analyze this PR to provide some insight structured as follows:

  1. Comprehensive Summary: Begin with a detailed explanation of the pull request. Describe its purpose, the problem it solves (introducing optimistic updates for MUD), and the general mechanism it uses (intercepting transactions, simulating them with a Tevm MemoryClient, and updating the MUD stash locally).

  2. Highlight Section: Create a distinct 'Highlight' section focused on the practical application of the new @tevm/mud package. This section must include:

    • Integration Examples: Provide two clear, separate code examples for integration:
      • A Vanilla JS example demonstrating how to use the core createOptimisticHandler function directly.
      • A React example, based on the pattern in examples/mud/packages/client/src/Providers.tsx, showing how to implement and use an OptimisticWrapper provider and a corresponding useOptimisticState hook.
    • Inlined API Explanations: Within the vanilla JS example, use comments to inline the explanations for each method and property returned by the createOptimisticHandler (e.g., getOptimisticState, subscribeTx, etc.).
  3. Flow Description: Provide an extremely thorough and precise step-by-step trace of the entire optimistic update flow. Start from the moment a user initiates a writeContract call and follow it through interception, local simulation, optimistic state calculation, UI notification, and finally, the reconciliation process when the transaction is confirmed on-chain. Ensure every step in the data flow is accurately described.

  4. Limitations: Conclude with a clear and honest 'Limitations' section. You must explicitly mention the following points:

    • The current implementation is designed specifically for a MUD project using an EntryKit SessionClient (bundler client) and will not work with other client types. This should not be hard to implement tho.
    • The implementation is a 'clunky' first version because it directly overwrites/mutates the writeContract method on the provided client instance, which is not an ideal practice." A similar workaround is used for intercepting getStorageAt requests.
    • The example shows that running multiple calls quickly will get the UI to flash incorrect states, highlighting some probable race conditions, or something similar; which is what we tried to avoid with a class to manage the state inside the plugin (this too is a limitation, we should not need it), but it's not enough.

Copy link
Contributor

claude bot commented Jun 17, 2025

Claude encountered an error —— View job


I'll analyze this and get back to you.

Copy link
Contributor

claude bot commented Jun 17, 2025

Claude finished @0xpolarzero's task —— View job


PR Analysis: MUD Plugin with Optimistic Updates

Todo List:

  • Read core implementation files
  • Analyze React integration pattern
  • Write comprehensive summary
  • Create highlight section with examples
  • Document detailed flow process
  • List limitations and current issues

1. Comprehensive Summary

This pull request introduces @tevm/mud, a sophisticated plugin that implements optimistic updates for MUD (Multi-User Dungeon) applications built on Ethereum. The plugin addresses a critical UX problem in blockchain gaming: the delay between user actions and on-chain confirmation, which typically results in 1-12 second waits for UI updates.

Problem Solved:

  • Eliminates UI lag in MUD games by providing instant visual feedback
  • Maintains data consistency by reconciling optimistic state with canonical blockchain state
  • Provides transaction status tracking (simulating → optimistic → confirmed/reverted)

Core Mechanism:
The plugin intercepts writeContract calls and getStorageAt requests to create a parallel optimistic execution environment:

  1. Transaction Interception: When a user calls writeContract, the plugin:

    • Executes the transaction locally using a Tevm MemoryClient
    • Broadcasts the real transaction to the network simultaneously
    • Updates the local MUD stash with optimistic results
  2. State Management: Maintains two memory clients:

    • Internal Client: Builds optimistic state incrementally during transaction processing
    • Optimistic Client: Provides the complete optimistic view for storage requests
  3. Storage Override: Intercepts eth_getStorageAt calls to the MUD store contract, returning optimistic data instead of canonical blockchain data when available

  4. Reconciliation: When transactions confirm on-chain, the plugin removes them from the pending pool and reconciles the optimistic state with canonical state via MUD's store-sync mechanism

2. Highlight: Practical Integration Examples

Vanilla JS Example

import { createOptimisticHandler } from '@tevm/mud';
import { createMemoryClient } from '@tevm/memory-client';
import { createStash } from '@latticexyz/stash/internal';
import { http } from 'viem';

// Setup your MUD stash and client
const stash = createStash(mudConfig);
const client = createPublicClient({ transport: http('https://...') });

// Create the optimistic handler
const handler = createOptimisticHandler({
  client,
  storeAddress: '0x...', // Your MUD world address
  stash,
  loggingLevel: 'debug'
});

// Core API methods:

// Get current optimistic state (canonical + pending optimistic changes)
const optimisticState = handler.getOptimisticState();

// Get specific optimistic record with type safety
const playerRecord = handler.getOptimisticRecord({
  table: PlayerTable,
  key: { playerId: '0x123...' }
});

// Get multiple optimistic records
const allPlayers = handler.getOptimisticRecords({
  table: PlayerTable
});

// Subscribe to optimistic state changes (fires on both optimistic and canonical updates)
const unsubscribe = handler.subscribeOptimisticState({
  subscriber: (updates) => {
    console.log('State updated:', updates);
    // Update your UI here
  }
});

// Subscribe to transaction status changes
const unsubscribeTx = handler.subscribeTx({
  subscriber: (status) => {
    console.log('Transaction status:', status);
    // status.status can be: 'simulating' | 'optimistic' | 'confirmed' | 'reverted'
    // Update loading states, show toasts, etc.
  }
});

// Use the sync adapter with MUD's store-sync
const syncAdapter = handler.syncAdapter; // Pass to MUD's SyncProvider

React Example

Based on the pattern in examples/mud/packages/client/src/Providers.tsx:

import React from 'react';
import { OptimisticWrapperProvider, useOptimisticState } from '@tevm/mud/react';
import { useSessionClient } from '@latticexyz/entrykit/internal';
import { stash } from './mud/stash';

// Provider setup
function OptimisticProvider({ children }) {
  const { data: sessionClient } = useSessionClient();
  
  return (
    <OptimisticWrapperProvider
      stash={stash}
      storeAddress="0x..." // Your MUD world address
      client={sessionClient}
      sync={{ startBlock: 0n }}
      loggingLevel="debug"
    >
      {children}
    </OptimisticWrapperProvider>
  );
}

// Component using optimistic state
function PlayerComponent({ playerId }) {
  // Get optimistic player data with automatic re-renders
  const player = useOptimisticState(
    state => getRecord({ state, table: PlayerTable, key: { playerId } }),
    { isEqual: deepEqual } // Prevents unnecessary re-renders
  );
  
  // Transaction status monitoring
  const [txStatus, setTxStatus] = useState(null);
  const wrapper = useOptimisticWrapper();
  
  useEffect(() => {
    if (!wrapper) return;
    return wrapper.subscribeTx({
      subscriber: setTxStatus
    });
  }, [wrapper]);
  
  const movePlayer = async (x, y) => {
    // This writeContract call is automatically intercepted
    // UI updates instantly with optimistic state
    await sessionClient.writeContract({
      address: worldAddress,
      abi: worldAbi,
      functionName: 'movePlayer',
      args: [playerId, x, y]
    });
  };
  
  return (
    <div>
      <div>Player Position: {player?.x}, {player?.y}</div>
      {txStatus?.status === 'simulating' && <div>Simulating...</div>}
      {txStatus?.status === 'optimistic' && <div>Pending confirmation...</div>}
      <button onClick={() => movePlayer(10, 20)}>
        Move Player
      </button>
    </div>
  );
}

3. Detailed Optimistic Update Flow

Here's the precise step-by-step trace of the entire optimistic update process:

Phase 1: Transaction Initiation

  1. User Action: User calls client.writeContract() (e.g., moving a player in a game)
  2. Interception: mudStoreWriteRequestOverride intercepts the call in bundler-packages/mud/src/internal/decorators/mudStoreWriteRequestOverride.ts:36
  3. Transaction ID Generation: System generates unique identifier with generateTxIdentifier() for tracking
  4. Parallel Execution:
    • Real transaction broadcast initiated via originalWriteContract()
    • Optimistic simulation begins simultaneously

Phase 2: Optimistic Simulation

  1. Status Update: Transaction status set to 'simulating', subscribers notified
  2. Cache Clearing: Fork cache cleared to force fresh getStorageAt requests through interceptor
  3. Local Execution: memoryClient.tevmCall() executes transaction with:
    • Same parameters as real transaction
    • addToMempool: true - adds to pending transaction pool
    • blockTag: 'pending' - executes on pending block
  4. Transaction Pool Addition: Transaction added to MemoryClient's transaction pool, triggering 'txadded' event

Phase 3: Optimistic State Calculation

  1. Pool Processing: processTransactionsAndUpdateLogs() triggered by 'txadded' event
  2. Transaction Ordering: txPool.txsByPriceAndNonce() orders pending transactions
  3. VM Copy Creation: Deep copy of VM created for isolated execution
  4. Incremental Execution: For each pending transaction:
    • Storage cache cleared for store address
    • Transaction executed via vmCopy.runTx()
    • Event logs extracted and parsed
    • MUD store updates accumulated in internalLogs
  5. State View Generation: getStateView() creates optimistic state:
    • Deep clones canonical stash state
    • Applies accumulated optimistic updates
    • Returns merged optimistic state

Phase 4: UI Update Propagation

  1. Subscriber Notification: notifyStashSubscribers() notifies all optimistic state subscribers
  2. React Re-renders: Components using useOptimisticState automatically re-render with optimistic data
  3. Status Update: Transaction status updated to 'optimistic'

Phase 5: Storage Request Interception

  1. Storage Requests: When fork requests getStorageAt for store contract:
    • mudStoreGetStorageAtOverride intercepts the request
    • Checks if requested storage slot exists in optimistic state
    • Returns optimistic value or falls back to canonical storage
    • Ensures subsequent contract reads see optimistic changes

Phase 6: Network Confirmation

  1. Transaction Mining: Real transaction confirms on network
  2. Receipt Processing: waitForTransactionReceipt() detects confirmation
  3. Status Final Update: Transaction status updated to 'confirmed' or 'reverted'
  4. Canonical Sync: MUD's store-sync processes confirmed transaction:
    • Canonical stash updated with real on-chain data
    • syncAdapter.onTx() called with transaction hash
  5. Optimistic Cleanup:
    • matchOptimisticTxCounterpart() finds corresponding optimistic transaction
    • Optimistic transaction removed from pending pool
    • 'txremoved' event triggers optimistic state recalculation
  6. State Reconciliation: Optimistic state recalculated without confirmed transaction
  7. UI Reconciliation: React components re-render with final canonical state

Error Handling & Edge Cases

  • Simulation Failure: If optimistic execution fails, error logged but real transaction still broadcast
  • Transaction Revert: If real transaction reverts, optimistic transaction removed from pool
  • Race Conditions: stateUpdateCoordinator serializes optimistic updates to prevent conflicts
  • Multiple Rapid Transactions: Each gets unique identifier and processed in order

4. Limitations

Client Type Restriction

The current implementation is specifically designed for MUD projects using an EntryKit SessionClient (bundler client) and will not work with other client types. The interception logic in mudStoreWriteRequestOverride.ts:18 explicitly checks for client.type === 'bundlerClient' and only overrides the writeContract method on SessionClient instances. This should not be hard to implement for other client types, but requires additional development.

Implementation Architecture Issues

The implementation is a 'clunky' first version with several non-ideal practices:

  • Direct Method Mutation: The plugin directly overwrites/mutates the writeContract method on the provided client instance (client.writeContract = async function... in mudStoreWriteRequestOverride.ts:36), which is dangerous and can cause unexpected behaviors if other extensions depend on the original reference.

  • Transport Request Override: Similar workaround used for intercepting getStorageAt requests where the original transport.request function is replaced rather than properly proxied.

  • Dual Client Architecture: The system requires two separate MemoryClients (internalClient and optimisticClient) with different storage interceptors, which adds complexity and resource overhead.

Race Conditions and State Management

The example demonstrates that running multiple calls quickly causes the UI to flash incorrect states, highlighting probable race conditions or similar issues. Despite attempts to mitigate this with a stateUpdateCoordinator class to manage state inside the plugin, it's not sufficient:

  • Asynchronous State Updates: The optimistic state calculation and removal from pending pool being async creates windows where multiple MUD sync → stash update → remove tx from pending pool operations can fire rapidly and interfere with each other.

  • State Management Complexity: The need for a stateUpdateCoordinator (which is acknowledged as a limitation that "we should not need") indicates the architecture has fundamental synchronization issues.

  • Memory Management: The plugin requires a WeakMap registry (handlerRegistry in useOptimisticWrapper.tsx:23) to prevent multiple writeContract wrappers and txPool subscriptions, which is described as "awful and obviously needs to be fixed."


Copy link
Contributor

claude bot commented Jun 23, 2025

Claude finished @0xpolarzero's task —— View job


PR Review: MUD Plugin with Optimistic Updates

Todo List:

  • Read core implementation files
  • Analyze React integration patterns
  • Review test coverage and quality
  • Check security and performance concerns
  • Provide comprehensive code review feedback

Code Review Summary

I've conducted a thorough review of the MUD plugin implementation. This is an innovative and sophisticated approach to solving optimistic updates in blockchain gaming, but it has several architectural concerns and areas for improvement.

🎯 Overall Assessment

Strengths:

  • Innovative approach to optimistic updates in MUD applications
  • Comprehensive test coverage with well-structured test files
  • Thoughtful error handling and logging throughout
  • Clear separation of concerns between optimistic and canonical state
  • Well-documented API design that mirrors MUD's stash patterns

Concerns:

  • Architectural complexity requiring significant workarounds
  • Direct method mutation creating potential instability
  • Race condition management requiring additional coordination layer
  • Memory management concerns with dual client architecture

🔍 Detailed Code Quality Analysis

1. Architecture & Design Issues

Direct Method Mutation (mudStoreWriteRequestOverride.ts:36)

client.writeContract = async function interceptedWriteContract(args) {

This is a dangerous pattern that mutates the client object directly. This can:

  • Break other extensions that depend on the original reference
  • Create debugging difficulties when multiple extensions interact
  • Lead to unpredictable behavior in complex application setups

Recommendation: Use a Proxy pattern instead to intercept calls without mutation.

Dual Client Architecture
The need for two separate MemoryClients (internalClient and optimisticClient) with different storage interceptors adds unnecessary complexity and resource overhead. This indicates architectural issues that should be addressed.

2. State Management & Race Conditions

State Update Coordinator (stateUpdateCoordinator.ts)
The need for this coordinator is itself a code smell, as noted in the comments. The async nature of optimistic state calculation creates windows where multiple operations can interfere:

// Race condition: when canonical sync removes optimistic txs from the pool...
// but before applying the canonical state to stash, a pending optimistic 
// update could run and see empty pool + old canonical state

Memory Management Issues (useOptimisticWrapper.tsx:23)

// This is awful and obviously needs to be fixed as soon as there is a better solution
const handlerRegistry = new WeakMap<Client | SessionClient, CreateOptimisticHandlerResult<StoreConfig>>()

The WeakMap registry to prevent multiple wrappers indicates fundamental architectural problems.

3. Performance Concerns

Fork Cache Clearing (mudStoreWriteRequestOverride.ts:56)

// TODO: this should definitely be avoided
// clear the fork cache so it doesn't try to read `getStorageAt` from it
vmCopy.stateManager._baseState.forkCache.storage.clearStorage(createAddressFromString(storeAddress))

This aggressive cache clearing on every transaction will hurt performance significantly and accessing private _baseState properties is brittle.

Deep Cloning on Every State Access (createOptimisticHandler.ts:115)

records: structuredClone(stash.get().records),

structuredClone is expensive for large state objects and happens on every optimistic state access.

4. Security & Robustness

Error Handling in Storage Override (mudStoreGetStorageAtOverride.ts)
The try-catch block covers too much code (lines 55-76), making it unclear what specific errors are being handled. Error cases should be more granular.

Type Safety Issues

// @ts-expect-error - Type 'unknown' is not assignable to type '_returnType'.
return async function interceptedRequest(requestArgs: any, options: any)

The @ts-expect-error and any types indicate type safety issues that should be resolved.

5. Code Quality Issues

Recursive Function Without Protection (deepEqual.ts:13)

for (const key of keys1) {
    if (!keys2.includes(key) || !deepEqual(obj1[key], obj2[key])) return false
}

The recursive deepEqual function lacks:

  • Cycle detection (can cause infinite loops)
  • Stack overflow protection for deep objects
  • Performance optimization for large objects

Inefficient Array Operations (deepEqual.ts:13)
Using keys2.includes(key) inside a loop has O(n²) complexity. Should use a Set for O(1) lookups.


🔒 Security Analysis

Low Risk Issues:

  • No direct security vulnerabilities found
  • Proper input validation in storage interceptors
  • Safe handling of blockchain data

Architectural Security:

  • Method mutation could be exploited by malicious extensions
  • State consistency depends on proper coordination (potential for state corruption)

🚀 Performance Analysis

Major Performance Impacts:

  1. Cache Clearing: Clearing fork cache on every transaction eliminates performance benefits
  2. Deep Cloning: State cloning on every access scales poorly with state size
  3. Dual Clients: Resource overhead of maintaining two separate EVM instances
  4. Deep Equality: O(n²) complexity in object comparison utilities

Memory Usage:

  • Two complete MemoryClient instances per handler
  • Potential memory leaks if cleanup isn't properly called
  • Large state objects being cloned frequently

🧪 Test Coverage Assessment

Strengths:

  • Comprehensive test files for core functionality
  • Good use of beforeEach setup with realistic test data
  • Tests cover both success and error scenarios
  • React hooks have dedicated test coverage

Gaps:

  • Many tests marked as it.todo indicating incomplete coverage
  • Limited stress testing for race conditions
  • No performance benchmarks for large state objects
  • Missing tests for cleanup and memory management

📋 Recommendations

High Priority (Architectural)

  1. Replace Method Mutation with Proxy Pattern
  2. Redesign to Eliminate Dual Client Architecture
  3. Remove State Update Coordinator Dependency
  4. Implement Proper Cache Invalidation Strategy

Medium Priority (Performance)

  1. Optimize State Cloning (use structural sharing or copy-on-write)
  2. Improve DeepEqual Implementation (add cycle detection, use Sets)
  3. Add Performance Monitoring for large state operations

Low Priority (Quality)

  1. Fix TypeScript Errors and remove @ts-expect-error
  2. Complete TODO Tests and add stress testing
  3. Improve Error Handling Granularity
  4. Add Documentation for complex flows

💭 Final Thoughts

This is an impressive technical achievement that solves a real problem in blockchain UX. However, the current implementation relies on several workarounds that indicate architectural issues. The core concept is sound, but the execution needs refinement to be production-ready.

The most critical issue is the architectural complexity requiring multiple workarounds (state coordinator, WeakMap registry, dual clients). A redesign focusing on a cleaner integration pattern would significantly improve maintainability and reliability.

The implementation shows deep understanding of both MUD and Tevm, but prioritizing architectural simplicity over feature completeness would yield better long-term results.


@vercel vercel bot temporarily deployed to Preview – node June 23, 2025 11:15 Inactive
Copy link
Contributor

claude bot commented Jun 23, 2025

Claude finished @0xpolarzero's task —— View job


PR Review: MUD Plugin with Optimistic Updates

Todo List:

  • Read repository guidelines (CLAUDE.md)
  • Analyze core implementation files
  • Review React integration patterns
  • Examine test coverage and quality
  • Check security and performance concerns
  • Provide comprehensive code review feedback

Code Review Summary

I've conducted a thorough review of the MUD plugin implementation. This is an innovative and sophisticated approach to solving optimistic updates in blockchain gaming, but it has several architectural concerns and areas for improvement.

🎯 Overall Assessment

Strengths:

  • Innovative approach to optimistic updates in MUD applications
  • Comprehensive test coverage with well-structured test files
  • Thoughtful error handling and logging throughout
  • Clear separation of concerns between optimistic and canonical state
  • Well-documented API design that mirrors MUD's stash patterns

Concerns:

  • Architectural complexity requiring significant workarounds
  • Direct method mutation creating potential instability
  • Race condition management requiring additional coordination layer
  • Memory management concerns with dual client architecture

🔍 Detailed Code Quality Analysis

1. Architecture & Design Issues

Direct Method Mutation (mudStoreWriteRequestOverride.ts:36)

client.writeContract = async function interceptedWriteContract(args) {

This is a dangerous pattern that mutates the client object directly. This can:

  • Break other extensions that depend on the original reference
  • Create debugging difficulties when multiple extensions interact
  • Lead to unpredictable behavior in complex application setups

Recommendation: Use a Proxy pattern instead to intercept calls without mutation.

Dual Client Architecture
The need for two separate MemoryClients (internalClient and optimisticClient) with different storage interceptors adds unnecessary complexity and resource overhead. This indicates architectural issues that should be addressed.

2. State Management & Race Conditions

State Update Coordinator (stateUpdateCoordinator.ts)
The need for this coordinator is itself a code smell, as noted in the comments. The async nature of optimistic state calculation creates windows where multiple operations can interfere:

// Race condition: when canonical sync removes optimistic txs from the pool...
// but before applying the canonical state to stash, a pending optimistic 
// update could run and see empty pool + old canonical state

Memory Management Issues (useOptimisticWrapper.tsx:23)

// This is awful and obviously needs to be fixed as soon as there is a better solution
const handlerRegistry = new WeakMap<Client | SessionClient, CreateOptimisticHandlerResult<StoreConfig>>()

The WeakMap registry to prevent multiple wrappers indicates fundamental architectural problems.

3. Performance Concerns

Fork Cache Clearing (mudStoreWriteRequestOverride.ts:56)

// TODO: this should definitely be avoided
// clear the fork cache so it doesn't try to read `getStorageAt` from it
vmCopy.stateManager._baseState.forkCache.storage.clearStorage(createAddressFromString(storeAddress))

This aggressive cache clearing on every transaction will hurt performance significantly and accessing private _baseState properties is brittle.

Deep Cloning on Every State Access (createOptimisticHandler.ts:115)

records: structuredClone(stash.get().records),

structuredClone is expensive for large state objects and happens on every optimistic state access.

4. Security & Robustness

Error Handling in Storage Override (mudStoreGetStorageAtOverride.ts)
The try-catch block covers too much code (lines 55-76), making it unclear what specific errors are being handled. Error cases should be more granular.

Type Safety Issues

// @ts-expect-error - Type 'unknown' is not assignable to type '_returnType'.
return async function interceptedRequest(requestArgs: any, options: any)

The @ts-expect-error and any types indicate type safety issues that should be resolved.

5. Code Quality Issues

Recursive Function Without Protection (deepEqual.ts:13)

for (const key of keys1) {
    if (!keys2.includes(key) || !deepEqual(obj1[key], obj2[key])) return false
}

The recursive deepEqual function lacks:

  • Cycle detection (can cause infinite loops)
  • Stack overflow protection for deep objects
  • Performance optimization for large objects

Inefficient Array Operations (deepEqual.ts:13)
Using keys2.includes(key) inside a loop has O(n²) complexity. Should use a Set for O(1) lookups.


🔒 Security Analysis

Low Risk Issues:

  • No direct security vulnerabilities found
  • Proper input validation in storage interceptors
  • Safe handling of blockchain data

Architectural Security:

  • Method mutation could be exploited by malicious extensions
  • State consistency depends on proper coordination (potential for state corruption)

🚀 Performance Analysis

Major Performance Impacts:

  1. Cache Clearing: Clearing fork cache on every transaction eliminates performance benefits
  2. Deep Cloning: State cloning on every access scales poorly with state size
  3. Dual Clients: Resource overhead of maintaining two separate EVM instances
  4. Deep Equality: O(n²) complexity in object comparison utilities

Memory Usage:

  • Two complete MemoryClient instances per handler
  • Potential memory leaks if cleanup isn't properly called
  • Large state objects being cloned frequently

🧪 Test Coverage Assessment

Strengths:

  • Comprehensive test files for core functionality
  • Good use of beforeEach setup with realistic test data
  • Tests cover both success and error scenarios
  • React hooks have dedicated test coverage

Gaps:

  • Many tests marked as it.todo indicating incomplete coverage
  • Limited stress testing for race conditions
  • No performance benchmarks for large state objects
  • Missing tests for cleanup and memory management

📋 Recommendations

High Priority (Architectural)

  1. Replace Method Mutation with Proxy Pattern
  2. Redesign to Eliminate Dual Client Architecture
  3. Remove State Update Coordinator Dependency
  4. Implement Proper Cache Invalidation Strategy

Medium Priority (Performance)

  1. Optimize State Cloning (use structural sharing or copy-on-write)
  2. Improve DeepEqual Implementation (add cycle detection, use Sets)
  3. Add Performance Monitoring for large state operations

Low Priority (Quality)

  1. Fix TypeScript Errors and remove @ts-expect-error
  2. Complete TODO Tests and add stress testing
  3. Improve Error Handling Granularity
  4. Add Documentation for complex flows

💭 Final Thoughts

This is an impressive technical achievement that solves a real problem in blockchain UX. However, the current implementation relies on several workarounds that indicate architectural issues. The core concept is sound, but the execution needs refinement to be production-ready.

The most critical issue is the architectural complexity requiring multiple workarounds (state coordinator, WeakMap registry, dual clients). A redesign focusing on a cleaner integration pattern would significantly improve maintainability and reliability.

The implementation shows deep understanding of both MUD and Tevm, but prioritizing architectural simplicity over feature completeness would yield better long-term results.


const tableState = stash._.state.records[table.namespaceLabel]?.[table.label]
const encodedKey = encodeKey({ table, key })
const prevRecord = tableState?.[encodedKey]
const currentRecord = value == null ? undefined : tableState?.[encodedKey]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be mistaken, but wondering why this is emitting the previous value rather than the new update that came in?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, it should absolutely not do that... this might be a reason for some of the issues

@vercel vercel bot temporarily deployed to Preview – node July 11, 2025 11:40 Inactive
Copy link
Contributor

claude bot commented Jul 11, 2025

Claude finished @0xpolarzero's task —— View job


PR Code Review: MUD Plugin Implementation

Todo List:

  • Read repository guidelines (CLAUDE.md)
  • Examine core implementation files
  • Analyze React integration patterns
  • Review test coverage and quality
  • Assess security and performance concerns
  • Provide comprehensive code review feedback

I've conducted a thorough code review of this MUD plugin implementation. This is an innovative and sophisticated approach to solving optimistic updates in blockchain gaming, but it has several architectural concerns and areas for improvement.

🎯 Overall Assessment

Strengths:

  • Innovative approach to optimistic updates in MUD applications
  • Comprehensive test coverage with well-structured test files
  • Thoughtful error handling and logging throughout
  • Clear separation of concerns between optimistic and canonical state
  • Well-documented API design that mirrors MUD's stash patterns

Concerns:

  • Architectural complexity requiring significant workarounds
  • Direct method mutation creating potential instability
  • Race condition management requiring additional coordination layer
  • Memory management concerns with dual client architecture

🔍 Detailed Code Quality Analysis

1. Architecture & Design Issues

Direct Method Mutation (mudStoreWriteRequestOverride.ts:36)

client.writeContract = async function interceptedWriteContract(args) {

This is a dangerous pattern that mutates the client object directly. This can:

  • Break other extensions that depend on the original reference
  • Create debugging difficulties when multiple extensions interact
  • Lead to unpredictable behavior in complex application setups

Recommendation: Use a Proxy pattern instead to intercept calls without mutation.

Dual Client Architecture
The need for two separate MemoryClients (internalClient and optimisticClient) with different storage interceptors adds unnecessary complexity and resource overhead. This indicates architectural issues that should be addressed.

2. State Management & Race Conditions

State Update Coordinator (stateUpdateCoordinator.ts)
The need for this coordinator is itself a code smell, as noted in the comments. The async nature of optimistic state calculation creates windows where multiple operations can interfere:

// Race condition: when canonical sync removes optimistic txs from the pool...
// but before applying the canonical state to stash, a pending optimistic 
// update could run and see empty pool + old canonical state

Memory Management Issues (useOptimisticWrapper.tsx:23)

// This is awful and obviously needs to be fixed as soon as there is a better solution
const handlerRegistry = new WeakMap<Client | SessionClient, CreateOptimisticHandlerResult<StoreConfig>>()

The WeakMap registry to prevent multiple wrappers indicates fundamental architectural problems.

3. Performance Concerns

Fork Cache Clearing (mudStoreWriteRequestOverride.ts:56)

// TODO: this should definitely be avoided
// clear the fork cache so it doesn't try to read `getStorageAt` from it
vmCopy.stateManager._baseState.forkCache.storage.clearStorage(createAddressFromString(storeAddress))

This aggressive cache clearing on every transaction will hurt performance significantly and accessing private _baseState properties is brittle.

Deep Cloning on Every State Access (createOptimisticHandler.ts:115)

records: structuredClone(stash.get().records),

structuredClone is expensive for large state objects and happens on every optimistic state access.

4. Security & Robustness

Error Handling in Storage Override (mudStoreGetStorageAtOverride.ts)
The try-catch block covers too much code (lines 55-76), making it unclear what specific errors are being handled. Error cases should be more granular.

Type Safety Issues

// @ts-expect-error - Type 'unknown' is not assignable to type '_returnType'.
return async function interceptedRequest(requestArgs: any, options: any)

The @ts-expect-error and any types indicate type safety issues that should be resolved.

5. Code Quality Issues

Recursive Function Without Protection (deepEqual.ts:13)

for (const key of keys1) {
    if (!keys2.includes(key) || !deepEqual(obj1[key], obj2[key])) return false
}

The recursive deepEqual function lacks:

  • Cycle detection (can cause infinite loops)
  • Stack overflow protection for deep objects
  • Performance optimization for large objects

Inefficient Array Operations (deepEqual.ts:13)
Using keys2.includes(key) inside a loop has O(n²) complexity. Should use a Set for O(1) lookups.


🔒 Security Analysis

Low Risk Issues:

  • No direct security vulnerabilities found
  • Proper input validation in storage interceptors
  • Safe handling of blockchain data

Architectural Security:

  • Method mutation could be exploited by malicious extensions
  • State consistency depends on proper coordination (potential for state corruption)

🚀 Performance Analysis

Major Performance Impacts:

  1. Cache Clearing: Clearing fork cache on every transaction eliminates performance benefits
  2. Deep Cloning: State cloning on every access scales poorly with state size
  3. Dual Clients: Resource overhead of maintaining two separate EVM instances
  4. Deep Equality: O(n²) complexity in object comparison utilities

Memory Usage:

  • Two complete MemoryClient instances per handler
  • Potential memory leaks if cleanup isn't properly called
  • Large state objects being cloned frequently

🧪 Test Coverage Assessment

Strengths:

  • Comprehensive test files for core functionality
  • Good use of beforeEach setup with realistic test data
  • Tests cover both success and error scenarios
  • React hooks have dedicated test coverage

Gaps:

  • Many tests marked as it.todo indicating incomplete coverage
  • Limited stress testing for race conditions
  • No performance benchmarks for large state objects
  • Missing tests for cleanup and memory management

📋 Recommendations

High Priority (Architectural)

  1. Replace Method Mutation with Proxy Pattern
  2. Redesign to Eliminate Dual Client Architecture
  3. Remove State Update Coordinator Dependency
  4. Implement Proper Cache Invalidation Strategy

Medium Priority (Performance)

  1. Optimize State Cloning (use structural sharing or copy-on-write)
  2. Improve DeepEqual Implementation (add cycle detection, use Sets)
  3. Add Performance Monitoring for large state operations

Low Priority (Quality)

  1. Fix TypeScript Errors and remove @ts-expect-error
  2. Complete TODO Tests and add stress testing
  3. Improve Error Handling Granularity
  4. Add Documentation for complex flows

💭 Final Thoughts

This is an impressive technical achievement that solves a real problem in blockchain UX. However, the current implementation relies on several workarounds that indicate architectural issues. The core concept is sound, but the execution needs refinement to be production-ready.

The most critical issue is the architectural complexity requiring multiple workarounds (state coordinator, WeakMap registry, dual clients). A redesign focusing on a cleaner integration pattern would significantly improve maintainability and reliability.

The implementation shows deep understanding of both MUD and Tevm, but prioritizing architectural simplicity over feature completeness would yield better long-term results.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants