Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 112 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# AGENTS Guidelines for This Repository

This repository contains Chainbench, a blockchain infrastructure benchmarking tool built on Locust. When working on the project interactively with an agent (e.g. the Codex CLI) please follow the guidelines below for efficient development and testing.

## 1. Use Headless Mode for Development Testing

* **Always use headless mode** with short test durations during development.
* **Start with minimal load** (few users, low spawn rate) to validate changes.
* **Do _not_ run long-duration tests** during agent development sessions.
* **Use `--autoquit`** flag to ensure tests terminate properly.

Example test command:
```bash
poetry run chainbench start --profile evm.light --users 5 --workers 1 --test-time 30s --target https://test-node --headless --autoquit
```

## 2. Keep Dependencies in Sync

If you add or update dependencies:

1. Use Poetry to manage dependencies: `poetry add <package>` or `poetry add --group dev <package>`.
2. Run `poetry lock --no-update` to update the lock file.
3. Install updated dependencies with `poetry install`.
4. Verify compatibility with Python 3.10+ as specified in the project.

## 3. Coding Conventions

* Follow Black formatting (120 character line length).
* Use isort for import sorting (Black-compatible profile).
* Follow Flake8 linting rules (E203 ignored for Black compatibility).
* Use type hints where appropriate.
* Keep MyPy checks passing.

## 4. Code Quality Checks

Before completing any task, run these quality checks:

| Command | Purpose |
| ------------------------ | ------------------------------------------ |
| `poetry run black .` | Format code to project standards |
| `poetry run isort .` | Sort imports |
| `poetry run flake8` | Run linting checks |
| `poetry run mypy .` | Run type checking |

Or use pre-commit hooks:
```bash
poetry run pre-commit run --all-files
```

## 5. Testing Guidelines

Test changes progressively:

1. **Unit testing**: Test individual user classes and methods
2. **Profile validation**: Verify profiles load correctly
```bash
poetry run chainbench list profiles
```
3. **Short headless tests**: Run brief tests with minimal load
4. **Method discovery**: Test endpoint compatibility
```bash
poetry run chainbench discover https://test-node --clients geth
```

Comment on lines +52 to +64
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Confirm discover/list subcommands and client flag names.

Please verify that list profiles|methods|shapes and discover (with --clients geth) are the current subcommands and option names.


🏁 Script executed:

#!/bin/bash
# Verify subcommands referenced by the doc
set -euo pipefail

echo "Look for 'list' subcommands:"
rg -nP -C2 --type=py '\blist\s+(profiles|methods|shapes)\b' || true

echo -e "\nLook for 'discover' command and client flag names:"
rg -nP -C2 --type=py '\bdiscover\b' || true
rg -nP -C2 --type=py '\b(--clients|--client|--node|--engine)s?\b' || true

Length of output: 1730


Verify CLI subcommands and flags

  • Confirmed that chainbench list profiles (with optional -d <profile_dir>) is supported.
  • Confirmed that chainbench list methods exists (used in the error message: “Use ‘chainbench list methods’ to list all available methods.”).
  • There is no chainbench list shapes subcommand in the code; please remove any references to list shapes from the documentation (or implement it in the CLI).
  • Confirmed that chainbench discover <endpoint> accepts a --clients <client1,client2…> flag for specifying one or more clients.
🤖 Prompt for AI Agents
In AGENTS.md around lines 52 to 64, the docs reference a non-existent
`chainbench list shapes` subcommand and include commands that should reflect
actual CLI behavior; remove any mention of `list shapes` (or implement the
subcommand if intended), and update the examples to match the real flags: note
that `chainbench list profiles` supports an optional `-d <profile_dir>` and that
`chainbench discover <endpoint>` accepts `--clients <client1,client2…>`; also
keep the `chainbench list methods` reference as-is since it exists.

## 6. Profile Development

When creating or modifying profiles:

* Place custom profiles in appropriate subdirectories.
* Follow existing profile structure and conventions.
* Include docstrings explaining profile purpose.
* Test with small data sizes first (`--size XS`).
* Validate against multiple node types when applicable.

## 7. Working with Test Data

* Start with smallest data size (`--size XS`) for development.
* Use `--use-latest-blocks` for nodes with limited history.
* Consider using `--ref-url` for test data generation from a reference node.
* Monitor memory usage with larger data sizes.

## 8. Development Workflow

1. Make changes to source code
2. Run formatting: `poetry run black . && poetry run isort .`
3. Run linting: `poetry run flake8`
4. Run type checking: `poetry run mypy .`
5. Test with minimal profile first
6. Gradually increase complexity and load

## 9. Useful Commands Recap

| Command | Purpose |
| ------------------------------------------------- | ------------------------------------- |
| `poetry install` | Install all dependencies |
| `poetry run chainbench --help` | Show all available commands |
| `poetry run chainbench list methods` | List supported RPC methods |
| `poetry run chainbench list profiles` | List available profiles |
| `poetry run chainbench list shapes` | List load pattern shapes |
| `poetry run chainbench discover <url>` | Discover available methods |

## 10. Safety Reminders

* **Test against test/dev nodes first** before production nodes.
* **Monitor target node health** during benchmarks.
* **Use appropriate rate limits** to avoid overwhelming nodes.
* **Start with light profiles** before heavy ones.
* **Keep test durations short** during development.

---

Following these practices ensures reliable development, prevents overwhelming blockchain nodes, and maintains code quality. Always prioritize controlled testing and gradual load increases when benchmarking infrastructure.