RL4Ising (Reinforcement Learning for Ising Models): Datasets and Benchmark

Reinforcement Learning for Ising Models is an open-source dataset and benchmark suite for Ising models. Our goal is to curate a public dataset of Ising models, provide a comprehensive benchmark of state-of-the-art reinforcement learning algorithms alongside an industry-standard solver baseline, and offer detailed tutorials for each reinforcement learning algorithm.

Explore our comprehensive benchmark suite and in-depth tutorials at rl4ising-docs.readthedocs.io.
Our curated datasets of various Ising models is at huggingface.co/datasets/SecureFinAI-Lab/Ising_Model_Instances.

Ising Model Datasets

We curate a diverse collection of over 170,000 Ising models and construct two distinct datasets:

• Dataset 1: Classification based on spin interactions
• Dataset 2: Classification based on model dimensionality

Dataset 1

Types of Ising Models	Description	Instances	Spin Range	Coupling Range
Spin Glass	...	14,045	0 - 0	0 - 0
Ferromagnetic	...	90,000	0 - 0	0 - 0
Anti-Ferromagnetic	...	90,007	0 - 0	0 - 0

Dataset 2

Types of Ising Models	Description	Instances	Spin Range	Coupling Range
1D ising Models	...		0 - 0	0 - 0
2D ising Models	...		0 - 0	0 - 0
3D ising Models	...		0 - 0	0 - 0
4D ising Models	...		0 - 0	0 - 0

Benchmark Suite

Our benchmark suite consists of two integral components, a solver baseline and SOTA RL benchmark, allowing for the direct comparison of the classical optimization methods vs modern RL-based approaches.

• Solver Baseline: Offers transparent and provable optimal or near-optimial solutions while leveraging highly optimized, memory-efficient algorithms. Most importantly, solvers provide an upper bound enabling the measurement of solution quality for approximate methods such as RL-based algorithms.
• SOTA RL Benchmark: Showcases key milestones of state-of-the-art RL algorithms over the years in terms of performance and scalability. Most importantly, we highlight the current top-of-the-line RL algorithms providing high quality solutions with a scaliability and time advantage against industry solvers.

Algorithm/Solver	Description	Reference
Gurobi	-----------	1
IBM CPLEX	-----------	2
COPT	-----------	3
SCIP	-----------	4
ECOLE	-----------	5
MOSEK	-----------	6
L2A	-----------	7
VCA	-----------	8
MCPG	-----------	9
ECO-DQN	-----------	10
S2V-DQN	-----------	11

Solver Baseline

Solvers	Spin Glass	Ferromagnetic	Anti-Ferromagnetic	1D Ising	2D Ising	3D Ising	4D Ising
Gurobi	✅	❌	❌	✅	✅	❌	❌
IBM CPLEX	✅	❌	❌	❌	❌	❌	❌
COPT	✅	❌	❌	✅	✅	❌	❌
SCIP	❌	❌	❌	❌	❌	❌	❌
ECOLE	❌	❌	❌	❌	❌	❌	❌
MOSEK	❌	❌	❌	❌	❌	❌	❌

Benchmarks

Becnhmark	Spin Glass	Ferromagnetic	Anti-Ferromagnetic	1D Ising	2D Ising	3D Ising	4D Ising
L2A	✅	❌	❌	✅	✅	❌	❌
VCA	✅	❌	❌	❌	❌	❌	❌
MCPG	✅	❌	❌	✅	✅	❌	❌
ECO-DQN	❌	❌	❌	❌	❌	❌	❌
S2V-DQN	❌	❌	❌	❌	❌	❌	❌

Tutorials

File Structure

    ├── docs            : ReadTheDocs website containing algorithms, tutorials, and dataset overview. 
    |
    └── src
        ├── algorithms
        |   ├── vca                 : Variational Classical Annealing algorithm.
        |   ├── mcpg                : Monte Carlo Policy Gradient algorithm modified for Ising models.
        |   ├── eco_dqn             : Exploratory Combinatorial Optimization with Reinforcement Learning modified for Ising models.
        |   └── vta                 : Variational Transformer Annealing algorithm, transformer based VCA.
        |       
        ├── baseline
        |   ├── gurobi.py           : Gurobi MIP Solver.
        |   ├── ilog_cplex.py       : IBM ILOG CPLEX MIP Solver.
        |   └── copt.py             : COPT Cardinal MIP Solver.
        |
        └── tutorials
            ├── vca.ipynb           : VCA tutorial for EA 100 node instance.
            ├── mcpg.ipynb          : MCPG tutorial for EA 100 node instance.
            └── eco_dqn.ipynb       : ECO-DQN tutorial for EA 100 node instance.

Motivation