|
| 1 | +# Direct Indexer Testing with Multiple Senders |
| 2 | + |
| 3 | +This guide explains how to set up multiple test senders for direct indexer testing, bypassing the gateway to test various sender scenarios and edge cases. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +**Direct testing approach:** |
| 8 | +- Create receipts with different sender private keys |
| 9 | +- Send signed receipts directly to indexer endpoints |
| 10 | +- Test trusted vs untrusted senders, balance limits, deny list behavior, etc. |
| 11 | + |
| 12 | +**Why this works:** |
| 13 | +- Gateway signature verification happens **off-chain** by the indexer service |
| 14 | +- RAV signature verification happens **on-chain** by the TAPVerifier contract |
| 15 | +- We can simulate different senders without multiple gateway instances |
| 16 | + |
| 17 | +## Signature Scheme & Address Generation |
| 18 | + |
| 19 | +**The system uses standard Ethereum cryptography:** |
| 20 | + |
| 21 | +- **Signature Algorithm**: **ECDSA with secp256k1** curve (same as Ethereum) |
| 22 | +- **Address Derivation**: **Keccak-256 hash** of public key (standard Ethereum addresses) |
| 23 | +- **Receipt Signing**: **EIP-712 structured data signing** for TAP receipts |
| 24 | +- **Key Format**: **32-byte private keys** (standard Ethereum private keys) |
| 25 | + |
| 26 | +### Key Implementation Details: |
| 27 | + |
| 28 | +```rust |
| 29 | +// From gateway_palaver/src/main.rs:74-76 |
| 30 | +let receipt_signer = PrivateKeySigner::from_bytes(&conf.receipts.signer) |
| 31 | + .expect("failed to prepare receipt signer"); |
| 32 | +let signer_address = receipt_signer.address(); // Standard Ethereum address derivation |
| 33 | +``` |
| 34 | + |
| 35 | +```rust |
| 36 | +// From gateway_palaver/src/receipts.rs:170-176 |
| 37 | +let v2_domain = Eip712Domain { |
| 38 | + name: Some("TAP".into()), |
| 39 | + version: Some("2".into()), |
| 40 | + chain_id: Some(chain_id), |
| 41 | + verifying_contract: Some(verifying_contract), |
| 42 | + salt: None, |
| 43 | +}; |
| 44 | +``` |
| 45 | + |
| 46 | +**Address generation follows Ethereum standards:** |
| 47 | +1. Private key (32 bytes) → Public key (secp256k1) |
| 48 | +2. Public key → Keccak-256 hash → Take last 20 bytes → Ethereum address |
| 49 | + |
| 50 | +**This means you can use any Ethereum wallet/tooling to generate test keys!** |
| 51 | + |
| 52 | +### ⚠️ **Important: Independent Keys for Security** |
| 53 | + |
| 54 | +**DO NOT** use mnemonic-derived keys for different senders! Even though they have different addresses, they share the same seed entropy, which is a security risk. |
| 55 | + |
| 56 | +**Instead, use completely independent private keys:** |
| 57 | + |
| 58 | +```rust |
| 59 | +// Account 0 (existing gateway - from hardhat mnemonic) |
| 60 | +pub const ACCOUNT0_SECRET: &str = "ac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"; |
| 61 | +pub const ACCOUNT0_ADDRESS: &str = "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266"; |
| 62 | + |
| 63 | +// Test Sender 1 (independent random key) |
| 64 | +pub const TEST_SENDER_1_SECRET: &str = "7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6"; |
| 65 | +pub const TEST_SENDER_1_ADDRESS: &str = "0x976EA74026E726554dB657fA54763abd0C3a0aa9"; |
| 66 | + |
| 67 | +// Test Sender 2 (independent random key) |
| 68 | +pub const TEST_SENDER_2_SECRET: &str = "8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba"; |
| 69 | +pub const TEST_SENDER_2_ADDRESS: &str = "0x14dC79964da2C08b23698B3D3cc7Ca32193d9955"; |
| 70 | + |
| 71 | +// Untrusted Sender (independent random key - not in trusted_senders) |
| 72 | +pub const UNTRUSTED_SENDER_SECRET: &str = "92db14e403b83dfe3df233f83dfa3a0d7096f21ca9b0d6d6b8d88b2b4ec1564e"; |
| 73 | +pub const UNTRUSTED_SENDER_ADDRESS: &str = "0x23618e81E3f5cdF7f54C3d65f7FBc0aBf5B21E8f"; |
| 74 | +``` |
| 75 | + |
| 76 | +### Generate Independent Keys: |
| 77 | + |
| 78 | +```bash |
| 79 | +# Generate completely independent random keys for testing: |
| 80 | +cast wallet new |
| 81 | +# Output: |
| 82 | +# Address: 0x976EA74026E726554dB657fA54763abd0C3a0aa9 |
| 83 | +# Private key: 0x7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6 |
| 84 | + |
| 85 | +cast wallet new |
| 86 | +# Output: |
| 87 | +# Address: 0x14dC79964da2C08b23698B3D3cc7Ca32193d9955 |
| 88 | +# Private key: 0x8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba |
| 89 | + |
| 90 | +# Verify address derivation: |
| 91 | +cast wallet address --private-key 0x7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6 |
| 92 | +# Output: 0x976EA74026E726554dB657fA54763abd0C3a0aa9 ✅ |
| 93 | +``` |
| 94 | + |
| 95 | +**Why independent keys matter:** |
| 96 | +- Each key has **completely different entropy** |
| 97 | +- **No shared seed material** between test senders |
| 98 | +- **Realistic testing** - real-world senders don't share keys |
| 99 | +- **Better security practices** - follows principle of key isolation |
| 100 | + |
| 101 | +## Requirements |
| 102 | + |
| 103 | +### 1. On-Chain: Escrow Funding |
| 104 | + |
| 105 | +Each test sender must have GRT deposited in the TAP escrow contract: |
| 106 | + |
| 107 | +```solidity |
| 108 | +// Escrow.sol maintains this mapping: |
| 109 | +mapping(address sender => mapping(address receiver => EscrowAccount)) escrowAccounts; |
| 110 | +``` |
| 111 | + |
| 112 | +**Required steps per sender:** |
| 113 | +1. Transfer GRT tokens to sender address |
| 114 | +2. Approve TAP escrow contract to spend GRT |
| 115 | +3. Deposit GRT to escrow for the indexer (receiver) |
| 116 | + |
| 117 | +### 2. Off-Chain: Indexer Configuration |
| 118 | + |
| 119 | +Update indexer configuration to recognize new senders: |
| 120 | + |
| 121 | +```toml |
| 122 | +[tap] |
| 123 | +trusted_senders = [ |
| 124 | + "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266", # Original (Account 0) |
| 125 | + "0x70997970C51812dc3A010C7d01b50e0d17dc79C8", # Test Sender 1 (Account 1) |
| 126 | + "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC" # Test Sender 2 (Account 2) |
| 127 | +] |
| 128 | + |
| 129 | +[tap.sender_aggregator_endpoints] |
| 130 | +"0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266" = "http://tap-aggregator:7610" |
| 131 | +"0x70997970C51812dc3A010C7d01b50e0d17dc79C8" = "http://tap-aggregator:7610" |
| 132 | +"0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC" = "http://tap-aggregator:7610" |
| 133 | +``` |
| 134 | + |
| 135 | +## Setup Script |
| 136 | + |
| 137 | +Create `setup_test_senders.sh`: |
| 138 | + |
| 139 | +```bash |
| 140 | +#!/bin/bash |
| 141 | +# Load environment from local network |
| 142 | +source ../contrib/local-network/.env |
| 143 | + |
| 144 | +# Get contract addresses |
| 145 | +GRAPH_TOKEN=$(jq -r '."1337".L2GraphToken.address' ../contrib/local-network/horizon.json) |
| 146 | +TAP_ESCROW_V1=$(jq -r '."1337".TAPEscrow.address' ../contrib/local-network/tap-contracts.json) |
| 147 | + |
| 148 | +# Test senders (independent random keys - NOT mnemonic-derived) |
| 149 | +SENDERS=( |
| 150 | + "0x976EA74026E726554dB657fA54763abd0C3a0aa9:7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6" |
| 151 | + "0x14dC79964da2C08b23698B3D3cc7Ca32193d9955:8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba" |
| 152 | +) |
| 153 | + |
| 154 | +AMOUNT="10000000000000000000" # 10 GRT per sender |
| 155 | + |
| 156 | +echo "Setting up test senders for direct indexer testing..." |
| 157 | + |
| 158 | +for sender_info in "${SENDERS[@]}"; do |
| 159 | + SENDER_ADDR="${sender_info%:*}" |
| 160 | + SENDER_KEY="${sender_info#*:}" |
| 161 | + |
| 162 | + echo "📝 Setting up sender: $SENDER_ADDR" |
| 163 | + |
| 164 | + # 1. Transfer GRT to sender |
| 165 | + echo " 💰 Transferring GRT..." |
| 166 | + docker exec chain cast send \ |
| 167 | + --rpc-url http://localhost:8545 \ |
| 168 | + --private-key $ACCOUNT0_SECRET \ |
| 169 | + $GRAPH_TOKEN "transfer(address,uint256)" $SENDER_ADDR "20000000000000000000" |
| 170 | + |
| 171 | + # 2. Approve escrow contract |
| 172 | + echo " ✅ Approving escrow..." |
| 173 | + docker exec chain cast send \ |
| 174 | + --rpc-url http://localhost:8545 \ |
| 175 | + --private-key $SENDER_KEY \ |
| 176 | + $GRAPH_TOKEN "approve(address,uint256)" $TAP_ESCROW_V1 $AMOUNT |
| 177 | + |
| 178 | + # 3. Deposit to escrow for indexer |
| 179 | + echo " 🏦 Depositing to escrow..." |
| 180 | + docker exec chain cast send \ |
| 181 | + --rpc-url http://localhost:8545 \ |
| 182 | + --private-key $SENDER_KEY \ |
| 183 | + $TAP_ESCROW_V1 "deposit(address,uint256)" $RECEIVER_ADDRESS $AMOUNT |
| 184 | + |
| 185 | + echo " ✅ Completed setup for $SENDER_ADDR" |
| 186 | +done |
| 187 | + |
| 188 | +# 4. Update indexer config |
| 189 | +echo "📝 Updating indexer configuration..." |
| 190 | +docker exec -it indexer-service bash -c ' |
| 191 | + sed -i "s/trusted_senders = \[\]/trusted_senders = [\"0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266\", \"0x976EA74026E726554dB657fA54763abd0C3a0aa9\", \"0x14dC79964da2C08b23698B3D3cc7Ca32193d9955\"]/g" /opt/config.toml |
| 192 | +' |
| 193 | + |
| 194 | +# 5. Restart services |
| 195 | +echo "🔄 Restarting indexer services..." |
| 196 | +docker restart indexer-service tap-agent |
| 197 | + |
| 198 | +echo "✅ Multi-sender test setup complete!" |
| 199 | +``` |
| 200 | + |
| 201 | +## Integration Test Usage |
| 202 | + |
| 203 | +Add test sender constants to your `constants.rs`: |
| 204 | + |
| 205 | +```rust |
| 206 | +// constants.rs - Additional test senders for multi-sender testing |
| 207 | + |
| 208 | +// Trusted Sender 1 (independent random key) |
| 209 | +pub const TRUSTED_SENDER_1_KEY: &str = "7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6"; |
| 210 | +pub const TRUSTED_SENDER_1_ADDR: &str = "0x976EA74026E726554dB657fA54763abd0C3a0aa9"; |
| 211 | + |
| 212 | +// Trusted Sender 2 (independent random key) |
| 213 | +pub const TRUSTED_SENDER_2_KEY: &str = "8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba"; |
| 214 | +pub const TRUSTED_SENDER_2_ADDR: &str = "0x14dC79964da2C08b23698B3D3cc7Ca32193d9955"; |
| 215 | + |
| 216 | +// Untrusted Sender (independent random key - not in trusted_senders config) |
| 217 | +pub const UNTRUSTED_SENDER_KEY: &str = "92db14e403b83dfe3df233f83dfa3a0d7096f21ca9b0d6d6b8d88b2b4ec1564e"; |
| 218 | +pub const UNTRUSTED_SENDER_ADDR: &str = "0x23618e81E3f5cdF7f54C3d65f7FBc0aBf5B21E8f"; |
| 219 | + |
| 220 | +// EIP-712 Domain for TAP receipts (matches gateway configuration) |
| 221 | +pub const TAP_EIP712_DOMAIN_NAME: &str = "TAP"; |
| 222 | +pub const TAP_EIP712_DOMAIN_VERSION: &str = "2"; |
| 223 | +``` |
| 224 | + |
| 225 | +Example test: |
| 226 | + |
| 227 | +```rust |
| 228 | +#[tokio::test] |
| 229 | +async fn test_trusted_vs_untrusted_senders() -> Result<()> { |
| 230 | + // Create signers |
| 231 | + let trusted_signer = PrivateKeySigner::from_str(TRUSTED_SENDER_1_KEY)?; |
| 232 | + let untrusted_signer = PrivateKeySigner::from_str(UNTRUSTED_SENDER_KEY)?; |
| 233 | + |
| 234 | + let allocation_id = find_allocation().await?; |
| 235 | + let large_fee = MAX_RECEIPT_VALUE * 10; // Above normal limits |
| 236 | + |
| 237 | + // Test 1: Trusted sender can exceed escrow balance |
| 238 | + let trusted_receipt = create_signed_receipt(&trusted_signer, allocation_id, large_fee)?; |
| 239 | + let response = send_receipt_directly(INDEXER_URL, trusted_receipt).await?; |
| 240 | + assert!(response.status().is_success(), "Trusted sender should be accepted"); |
| 241 | + |
| 242 | + // Test 2: Untrusted sender is denied for large amounts |
| 243 | + let untrusted_receipt = create_signed_receipt(&untrusted_signer, allocation_id, large_fee)?; |
| 244 | + let response = send_receipt_directly(INDEXER_URL, untrusted_receipt).await?; |
| 245 | + assert_eq!(response.status(), 403, "Untrusted sender should be denied"); |
| 246 | + |
| 247 | + Ok(()) |
| 248 | +} |
| 249 | + |
| 250 | +async fn send_receipt_directly(indexer_url: &str, receipt: TapReceipt) -> Result<Response> { |
| 251 | + Client::new() |
| 252 | + .post(format!("{}/subgraphs/id/{}", indexer_url, SUBGRAPH_ID)) |
| 253 | + .header("tap-receipt", serde_json::to_string(&receipt)?) |
| 254 | + .header("content-type", "application/json") |
| 255 | + .json(&serde_json::json!({"query": "{ _meta { block { number } } }"})) |
| 256 | + .send() |
| 257 | + .await |
| 258 | + .map_err(Into::into) |
| 259 | +} |
| 260 | +``` |
| 261 | + |
| 262 | +## Test Scenarios You Can Cover |
| 263 | + |
| 264 | +With multiple senders, test: |
| 265 | + |
| 266 | +- ✅ **Trusted vs untrusted sender behavior** |
| 267 | +- ✅ **Balance limit enforcement** |
| 268 | +- ✅ **Deny list functionality** |
| 269 | +- ✅ **RAV generation with mixed senders** |
| 270 | +- ✅ **Receipt signature verification** |
| 271 | +- ✅ **Timestamp buffer edge cases** |
| 272 | +- ✅ **Sender timeout behavior** |
| 273 | + |
| 274 | +## Key Benefits |
| 275 | + |
| 276 | +1. **Fast execution** - No container restarts between tests |
| 277 | +2. **Precise control** - Test exact sender scenarios |
| 278 | +3. **Easy isolation** - Each sender can use different allocations |
| 279 | +4. **Comprehensive coverage** - Test all edge cases without gateway complexity |
| 280 | + |
| 281 | +Run `./setup_test_senders.sh` once, then execute all your multi-sender integration tests! 🚀 |
| 282 | + |
| 283 | +## Summary: What You Need to Know |
| 284 | + |
| 285 | +### ✅ **Signature Scheme** |
| 286 | +- **Standard Ethereum**: ECDSA secp256k1 + Keccak-256 addresses |
| 287 | +- **EIP-712 structured signing** for TAP receipts |
| 288 | +- **Use any Ethereum tooling** (cast, MetaMask, etc.) to generate keys |
| 289 | + |
| 290 | +### ✅ **On-Chain Requirements** |
| 291 | +- **GRT tokens** must be transferred to each test sender |
| 292 | +- **Escrow deposits** must be made for each sender → indexer pair |
| 293 | +- **Smart contract** maintains `escrowAccounts[sender][receiver]` balances |
| 294 | + |
| 295 | +### ✅ **Off-Chain Requirements** |
| 296 | +- **Indexer config** must include senders in `trusted_senders` array |
| 297 | +- **Aggregator endpoints** must be configured for each sender |
| 298 | +- **Services restart** required to pick up new configuration |
| 299 | + |
| 300 | +### ✅ **Testing Benefits** |
| 301 | +- **No gateway complexity** - sign receipts directly with test keys |
| 302 | +- **Precise control** over sender scenarios (trusted/untrusted/balance limits) |
| 303 | +- **Fast execution** - no container restarts between tests |
| 304 | +- **Complete coverage** of edge cases and corner scenarios |
| 305 | + |
| 306 | +### 🔑 **Key Insight** |
| 307 | +The system uses **standard Ethereum cryptography** throughout. Any Ethereum private key can be used as a TAP sender - you just need to fund the escrow and update the indexer configuration. |
| 308 | + |
| 309 | +**Direct indexer testing gives you full control over multi-sender scenarios without the overhead of multiple gateway instances!** 🎯 |
0 commit comments