Skip to content

Conversation

tobidae-cb
Copy link
Contributor

@tobidae-cb tobidae-cb commented Aug 6, 2025

🚀 Core Features Added:

  1. AWS S3 Cloud Storage Integration

    • Added S3 service for uploading and downloading benchmark results
    • New export-to-cloud command to upload entire output directories to S3
    • S3 integration in the main benchmark runner with --enable-s3 flag
    • Automatic result upload during benchmark execution when S3 is enabled
  2. Import/Export System for Benchmark Results

    • New import-runs command to import benchmark results from local files or remote URLs
    • Support for merging metadata from different benchmark runs
  3. Enhanced Snapshot Management System

    • Optimized two-tier snapshot system for better performance
    • Initial snapshots downloaded once and reused across tests
    • Per-test snapshot copies using optimized rsync strategies
    • Remote snapshot source support for cloud-based initial snapshots
  4. Machine Information Collection

    • Added machine metadata collection (type, provider, region, filesystem)
    • Machine info included in benchmark results for better context
  5. Report Backend API Improvements

    • New Go-based REST API backend for serving benchmark data from S3
  6. Frontend Enhancements

    • Updated React frontend to work with new backend API
    • Support for displaying machine information in reports
    • Integration with cloud-stored benchmark results

🔧 Technical Improvements:

  • Optimized rsync strategies for faster snapshot copying
  • Better error handling and logging throughout the system
  • Structured configuration management for import/export operations
  • TTL-based caching for improved performance
  • Docker support for containerized deployment

@cb-heimdall
Copy link
Collaborator

cb-heimdall commented Aug 6, 2025

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

@tobidae-cb tobidae-cb changed the title feat: Add download/upload capabilities with cloud storage feat: Add cloud storage integration with S3 upload/download and snapshot management Sep 12, 2025
@tobidae-cb tobidae-cb marked this pull request as ready for review September 12, 2025 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants