Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ async with Computer(
| [**Computer (Python)**](./libs/python/computer/README.md) | Python Interface for controlling virtual machines | `pip install "cua-computer[all]"` |
| [**Computer (Typescript)**](./libs/typescript/computer/README.md) | Typescript Interface for controlling virtual machines | `npm install @trycua/computer` |
| [**Agent**](./libs/python/agent/README.md) | AI agent framework for automating tasks | `pip install "cua-agent[all]"` |
| [**Snapshot Manager**](./libs/python/snapshot-manager/README.md) | Docker container snapshot management for agent state persistence | `pip install cua-snapshot-manager` |
| [**MCP Server**](./libs/python/mcp-server/README.md) | MCP server for using CUA with Claude Desktop | `pip install cua-mcp-server` |
| [**SOM**](./libs/python/som/README.md) | Self-of-Mark library for Agent | `pip install cua-som` |
| [**Computer Server**](./libs/python/computer-server/README.md) | Server component for Computer | `pip install cua-computer-server` |
Expand Down
47 changes: 47 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# CUA Snapshot Manager Integration TODO

## Current Status: Core MVP Features Complete
**Phase**: Named volume support completed, finalizing remaining MVP features
**Target**: Production-ready snapshot management for CUA Agent SDK

## Completed Tasks ✅
- [x] Draft PR created in CUA repo
- [x] ARCHITECTURE.md with concise technical overview
- [x] DockerSnapshotProvider wired to CUA callbacks
- [x] Integration tests with real CUA agent verified
- [x] CI compliance validated (Black, Ruff, pytest)
- [x] **Named Volume Support** - Tar-based backup/restore for named volumes
- [x] **Bind Mount Detection** - User warnings for non-portable bind mounts
- [x] **Volume Type Identification** - Full volume analysis and handling
- [x] **Critical MyPy Fixes** - All blocking type errors resolved

## Remaining MVP Requirements

### Enhanced Restore Modes
- [ ] Replace mode: stop/remove original, create replacement
- [ ] Minimal container healthcheck after restore
- [ ] Error handling for restore failures

### Deterministic Retention Policy
- [ ] Age-first cleanup (max_age then max_count)
- [ ] Deterministic ordering for snapshot removal
- [ ] Focused unit test for retention logic

### Final Polish
- [ ] CI validation in pipeline
- [ ] Documentation review and finalization

## Technical Debt (Non-blocking)
- [ ] MyPy type annotation cleanup (1 false positive remaining)
- [ ] CLI function return type annotations
- [ ] Test function type hints

## Test Coverage
- **19/19 tests passing** including comprehensive volume support tests
- Volume detection, backup, storage, restore, and error handling all verified
- CUA integration examples working end-to-end

## Next Steps
1. Add replace restore mode with healthcheck
2. Implement deterministic retention policy
3. Final CI validation before team review
170 changes: 170 additions & 0 deletions libs/python/snapshot-manager/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Snapshot Manager Architecture

## Overview
The CUA Snapshot Manager provides state capture and restoration for Docker containers during AI agent execution, enabling rollback, debugging, and reliable state recovery.

## System Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│ CUA Agent SDK │
├─────────────────────────────────────────────────────────────────┤
│ ComputerAgent │ Callbacks │ Computer │ Run Lifecycle │
└─────────────────┬───────────────────────────────────────────────┘
┌─────────────────────┐
│ SnapshotCallback │ ◄─── Automatic Triggers
│ (CUA Integration) │ (run_start, run_end, etc.)
└─────────┬───────────┘
┌─────────────────────┐
│ SnapshotManager │ ◄─── Manual API Calls
│ (Central Control) │ (CLI, Programmatic)
└─────────┬───────────┘
┌────────┴────────┐
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ SnapshotProvider│ │ SnapshotStorage │
│ (Abstract) │ │ (Abstract) │
└─────────┬───────┘ └─────────┬───────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ DockerSnapshot │ │ FileSystemSnap │
│ Provider │ │ shotStorage │
│ │ │ │
│ • docker commit │ │ • JSON metadata │
│ • volume backup │ │ • volume tar │
│ • container ops │ │ • indexing │
│ • validation │ │ • statistics │
└─────────┬───────┘ └─────────┬───────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Docker API │ │ File System │
│ │ │ │
│ • Images │ │ • ./snapshots/ │
│ • Containers │ │ ├─metadata/ │
│ • Volumes │ │ ├─volumes/ │
│ • Commit/Run │ │ └─index.json │
└─────────────────┘ └─────────────────┘
```

## What's Captured
**Container State:**
- Complete filesystem state via `docker commit` (creates new Docker image)
- Container configuration (environment variables, working directory, user, command)
- Mount point metadata (named volumes, bind mounts)
- Network settings and port mappings
- Labels and original image reference

**Named Volume Data:**
- Full volume contents backed up as tar archives
- Volume analysis with type identification (named vs bind mounts)
- Automatic volume restoration with data integrity verification
- Bind mount detection with portability warnings

**Metadata:**
- Timestamp and trigger context (manual, run_start, run_end, before/after actions)
- Human-readable descriptions and agent run IDs
- Storage statistics and restoration tracking
- Container relationship mapping
- Volume backup manifest and storage locations

## What's NOT Captured
**Excluded by Design:**
- Live running processes and their memory state
- Temporary filesystems (`/tmp`, `/proc`, `/sys`, `/dev`)
- Real-time network connections and sockets
- Host-mounted bind mount contents (warned about, not portable across systems)
- Anonymous volumes (Docker limitation, data lost on container removal)

**Technical Limitations:**
- Containers must be in valid state (running/paused/exited/created)
- Docker daemon must be accessible and responsive
- No cross-host container migration support

## Restore Behavior
**New Container Mode (Default):**
- Creates fresh container from snapshot image
- Generates unique name with timestamp suffix
- Preserves original configuration and mounts (if available)
- Original container remains untouched

**Replace Mode (Planned):**
- Stops and removes target container
- Creates replacement with same name and configuration
- Includes basic health check validation

**Volume Handling:**
- Named volumes: Full data backup as tar archives, automatic restoration with integrity verification
- Bind mounts: Configuration preserved, warnings displayed about portability concerns
- Anonymous volumes: Lost (Docker limitation, not backed up)

## Limits & Policies
**Storage Limits:**
- Per-container snapshot count (default: 10)
- Global snapshot count (default: 100)
- Total storage size (default: 10GB)

**Retention Policy (Deterministic):**
- Primary: Age-based cleanup (default: 7 days)
- Secondary: Count-based limit enforcement (oldest first)
- Execution order: max_age THEN max_count

**Operation Safety:**
- Concurrent operation locks per container
- Graceful degradation (snapshot failures don't break agent execution)
- Atomic metadata updates (index updated only after successful snapshot)
- Orphaned metadata cleanup on storage corruption

## Integration Points
**CUA Agent SDK:**
- Pluggable callback handler (enable/disable without code changes)
- Configurable trigger points (run lifecycle, action boundaries)
- Non-blocking operation design (async throughout)

**Storage Backend:**
- JSON-based metadata with efficient indexing
- Pluggable storage interface (filesystem, future: cloud/database)
- Human-readable format for debugging and inspection

**Docker Provider:**
- Uses Docker's native `docker commit` for efficiency
- Leverages Docker image layer sharing for space optimization
- Direct integration with Docker API (no CLI shelling)

## Volume Backup & Restore Logic

**Volume Detection:**
1. Analyze container mount configuration via Docker API
2. Categorize mounts by type: named volumes vs bind mounts
3. Generate volume analysis report with portability warnings

**Backup Process:**
1. For each named volume attached to container:
- Create temporary alpine container with volume mounted read-only
- Extract complete volume data using `docker get_archive` (tar format)
- Store tar archives in `./snapshots/volumes/{snapshot_id}_{volume_name}.tar`
2. Log warnings for bind mounts (not included in backup)
3. Record volume manifest in snapshot metadata

**Restore Process:**
1. Create or verify target volumes exist in Docker
2. For each backed-up volume:
- Create temporary alpine container with volume mounted read-write
- Clear existing volume contents (ensuring clean restore)
- Extract tar data using `docker put_archive`
- Verify extraction success
3. Attach restored volumes to new container with original mount paths

**Storage Structure:**
```
./snapshots/
├── metadata/{snapshot_id}.json # Container + volume metadata
├── volumes/{snapshot_id}_{vol}.tar # Volume data archives
└── index.json # Global snapshot index
```
134 changes: 134 additions & 0 deletions libs/python/snapshot-manager/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
<div align="center">
<h1>
<div class="image-wrapper" style="display: inline-block;">
<picture>
<source media="(prefers-color-scheme: dark)" alt="logo" height="150" srcset="img/logo_white.png" style="display: block; margin: auto;">
<source media="(prefers-color-scheme: light)" alt="logo" height="150" srcset="img/logo_black.png" style="display: block; margin: auto;">
<img alt="CUA Snapshot Manager">
</picture>
</div>

[![Python](https://img.shields.io/badge/Python-333333?logo=python&logoColor=white&labelColor=333333)](#)
[![Docker](https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=white)](#)
[![Discord](https://img.shields.io/badge/Discord-%235865F2.svg?&logo=discord&logoColor=white)](https://discord.com/invite/mVnXXpdE85)
[![PyPI](https://img.shields.io/pypi/v/cua-snapshot-manager?color=333333)](https://pypi.org/project/cua-snapshot-manager/)
</h1>
</div>

**CUA Snapshot Manager** is a Docker container snapshot management system for the CUA Agent SDK, enabling automated state capture, storage, and restoration during AI agent execution.

<div align="center">
<img src="img/infra_diagram.png" alt="Snapshot Manager Architecture">
</div>

```bash
pip install cua-snapshot-manager
```

## Features

- **🔄 Automatic Snapshots**: Integrate with CUA Agent callbacks for hands-free operation
- **🐳 Docker Native**: Uses `docker commit` for efficient container snapshots
- **📦 Volume Support**: Automatic backup/restore of named volumes with bind mount warnings
- **⚡ Configurable Triggers**: Manual, run start/end, before/after actions
- **🧹 Smart Retention**: Automatic cleanup with configurable policies
- **📊 Rich Metadata**: Timestamps, descriptions, agent context tracking
- **⚙️ CLI Interface**: Command-line tools for snapshot management
- **🔌 Pluggable Design**: Easy integration without modifying existing workflows

## Quick Start

### Basic Usage

```python
import asyncio
from snapshot_manager import SnapshotManager, SnapshotTrigger, SnapshotConfig

async def main():
config = SnapshotConfig(storage_path="./snapshots")
manager = SnapshotManager(config=config)

# Create snapshot
metadata = await manager.create_snapshot(
container_id="my-container",
trigger=SnapshotTrigger.MANUAL,
description="Before risky operation"
)
print(f"Created snapshot: {metadata.snapshot_id}")

asyncio.run(main())
```

### CUA Agent Integration

```python
from agent import ComputerAgent
from computer import Computer
from snapshot_manager import SnapshotCallback, SnapshotConfig, SnapshotTrigger

async def agent_with_snapshots():
# Configure automatic snapshots
config = SnapshotConfig(
triggers=[SnapshotTrigger.RUN_START, SnapshotTrigger.RUN_END]
)

snapshot_callback = SnapshotCallback(config=config)

async with Computer(os_type="linux", provider_type="docker") as computer:
agent = ComputerAgent(
model="anthropic/claude-3-5-sonnet-20241022",
tools=[computer],
callbacks=[snapshot_callback] # Enable automatic snapshots
)

async for result in agent.run("Take a screenshot"):
print(result)
```

### CLI Usage

```bash
# Create snapshot
cua-snapshot create my-container --description "Initial state"

# List snapshots
cua-snapshot list

# Restore snapshot
cua-snapshot restore <snapshot-id> --container-name restored-container

# Cleanup old snapshots
cua-snapshot cleanup --max-age-days 7
```

## Architecture

The system consists of pluggable components for maximum flexibility:

- **SnapshotManager**: Central orchestrator coordinating all operations
- **DockerSnapshotProvider**: Docker-specific implementation using `docker commit`
- **FileSystemSnapshotStorage**: JSON-based metadata storage with indexing
- **SnapshotCallback**: CUA Agent SDK integration for automatic snapshots

See [ARCHITECTURE.md](./ARCHITECTURE.md) for detailed design decisions and technical overview.

## Examples & Testing

**Examples** in [examples/](./examples/):
- [`cua_volume_example.py`](./examples/cua_volume_example.py) - CUA Agent integration with volume support
- [`cua_callback_integration.py`](./examples/cua_callback_integration.py) - Automatic callback triggers demonstration

**Tests** in [tests/](./tests/):
- Comprehensive test suite covering snapshot manager core functionality
- Volume support tests for detection, backup, storage, and restore
- All 19 tests passing with full coverage of critical features

## Contributing

We welcome and greatly appreciate contributions to CUA Snapshot Manager! Whether you're improving documentation, adding new features, fixing bugs, or enhancing integrations, your efforts help make the system better for everyone.

Join our [Discord community](https://discord.com/invite/mVnXXpdE85) to discuss ideas or get assistance.

## License

CUA Snapshot Manager is open-sourced under the MIT License - see the [LICENSE](LICENSE) file for details.
Loading