EvoWorld

Official implementation of
EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory

EvoWorld is a generative world model that explicitly reconstructs and utilizes a 3D memory for egocentric video generation. Starting from a single panoramic image and a sequence of camera poses, EvoWorld synthesizes future frames with strong 3D consistency by projecting an evolving 3D point cloud onto future views as conditioning for a video diffusion model.

Table of Contents

• 🔧 Installation
• 📦 Pretrained Weights & Dataset
• ⚡ Quickstart
  • Single-clip generation
  • 3-clip iterative generation
• 🧪 Inference (full test set)
• 📊 Evaluation
• 🏋️ Training
• 🗺️ Spatial360 Dataset
• 📎 Acknowledgements

🔧 Installation

git clone [email protected]:JiahaoPlus/EvoWorld.git --recursive
cd EvoWorld
conda create -n evoworld python=3.11
conda activate evoworld
pip install -r requirements.txt
conda install -c conda-forge "cudnn>=9,<10"

📦 Pretrained Weights & Dataset

We have released:

• Model weights trained on Unity Curve Path (subset): Evoworld_Unity_Curve_Path (Hugging Face)
• Corresponding dataset: Google Drive link

How to use

1. Download model weights and dataset

• Download model weights from Hugging Face and place them in the MODELS/ directory.
• Download the dataset from Google Drive.
• After downloading the dataset, extract it (adjust paths as needed):

  tar -xvf /path/to/dataset.tar -C data/

• Update the model path in run_single_segment.sh and run_unified_pipeline.sh to point to your downloaded weights.

2. Run model on provided example

• For single clip generation, run:

  bash run_single_segment.sh

• For 3-clip iterative generation, run:

  bash run_unified_pipeline.sh

• Make sure the model path in these scripts matches your downloaded weights in MODELS/.

3. Run model across the whole test set

• Change your model path in inference_unity_curve.sh to the downloaded weights.
• Execute:

  bash inference_unity_curve.sh

4. Get evaluation metrics

• Run:

  bash calculate_metrics.sh

⚡ Quickstart

Single-clip generation (one clip)

bash run_single_segment.sh

3-clip iterative generation (memory carried across 3 clips)

bash run_unified_pipeline.sh

Notes:

• run_single_segment.sh runs a single-clip example.
• run_unified_pipeline.sh runs three clips iteratively, reusing the evolving 3D memory.
• Ensure the model path inside these scripts points to your downloaded weights in MODELS/.

🗺️ Spatial360 Dataset

We have released the Unity Curve Path subset — remaining subsets are coming soon!

🏋️ Training

To reproduce our performance, run the train script:

# Train EvoWorld
bash train.sh

🧪 Inference (full test set)

Update your model path in inference_unity_curve.sh and run:

bash inference_unity_curve.sh

📊 Evaluation

Compute evaluation metrics on the outputs:

bash calculate_metrics.sh

📎 Acknowledgements

This codebase builds upon:

• VGGT
• 360-1M
• SVD_Xtend
• GenEx