A Declarative Benchmark Configuration Framework for architecture exploration with gem5.
The gem5-dbc framework allows for a simplified gem5 simulation workflow.
After installing, the g5dbc
command line tool can be used to
- Configure required binary artifacts including full system simulation images.
- Generate a set simulation scripts from a benchmark definition and initial architecture configuration.
- Run generated gem5 simulation scripts.
- Parse gem5 stats.txt output to flat json files.
- Evaluate parsed statistics to collect data of interest to a CSV file.
Currently, benchmark configuration and simulation is only supported for the ARM ISA. Support for general ISAs will be implemented soon.
Install gem5-dbc directly from the git repo
# Install gem5-dbc package locally
pip install --user git+https://github.com/FZJ-JSC/gem5-dbc.git
In the following, we describe the simulation workflow in more detail.
To use gem5-dbc, you need a working installation of the gem5 computer architecture simulator.
You also need to provide all binary artifacts required for configuring and running a full system simulation.
# Clone gem5 repository
git clone https://github.com/gem5/gem5.git
# Compile gem5 binary for ARM ISA
scons -C gem5 --ignore-style -j $(nproc) gem5/build/ARM/gem5.fast
Add gem5 binary to the local user artifact registry with the command g5dbc --resource-add <PATH> --resource-type GEM5
# Add gem5 binary to user artifact registry
g5dbc --resource-add gem5/build/ARM/gem5.fast --resource-type GEM5
When adding a gem5 binary for the first time, an artifact registry
is created by default in your platform’s local user configuration directory,
for example: $HOME/.config/gem5-dbc/artifacts.yaml
.
Example user artifact registry
arm64:
- bintype: GEM5
md5hash: 06fc82f21f6e82e03e538252a05131e6
metadata: Apr 3 2025 10:07:41
name: gem5.fast
path: /home/user/sources/gem5/build/ARM/gem5.fast
version: 24.1.0.2
An artifact registry is a collection of artifact lists, each indexed by an architecture name (e.g. arm64
).
Each list contains entries describing individual artifacts using a set of key-value pairs, as detailed below:
Key | Description |
---|---|
name | Artifact name identifier. |
path | Relative or absolute path to the binary or image file. |
bintype | The type of artifact (DISK, BOOT, KERNEL,GEM5). |
version | The version label or tag of the artifact. |
md5hash | The MD5 checksum used for integrity verification. |
metadata | Additional context, such as the disk image root partition (e.g. /dev/vda2 ) or kernel boot parameters. |
If you already have binary artifacts available for use with gem5 full-system simulation, you can add them via the command line, either to your user artifact registry or to a separate registry file.
Using separate registry files is recommended when working with multiple sets of binary images, as each set can be described by its own independent registry.
Example commands are listed below.
# Add Linux Image
# For DISK artifacts, gem5-dbc uses the metadata field to speficy the correct root partition.
g5dbc -a arm64/disks/disk.img -A arm64 -T DISK -N disk.img -V debian -M /dev/vda2 -i index.yaml
# Add Linux Kernel
# For KERNEL artifacts, gem5-dbc uses the metadata field to speficy kernel boot parameters
g5dbc -a arm64/binaries/vmlinux-5.15.68 -A arm64 -T KERNEL -N vmlinux -V 5.15.68 -M "earlyprintk=pl011,0x1c090000 console=ttyAMA0 lpj=19988480 norandmaps rw loglevel=8" -i index.yaml
# Add ARM64 Bootloader
# For BOOT artifacts, gem5-dbc uses the version field to select correct bootloader for selected platform
g5dbc -a arm64/binaries/boot.arm64 -A arm64 -T BOOT -N boot.arm64 -V V1 -i index.yaml
# Add ARM64 Bootloader
# For BOOT artifacts, gem5-dbc uses the version field to select correct bootloader for selected platform
g5dbc -a arm64/binaries/boot_v2.arm64 -A arm64 -T BOOT -N boot.arm64 -V V2 -i index.yaml
For DISK
artifacts, gem5-dbc uses the metadata
field to specify the correct root partition.
For KERNEL
artifacts, gem5-dbc uses the metadata
field to specify kernel boot parameters.
For BOOT
artifacts, gem5-dbc uses the version
field to select the appropriate bootloader for the selected platform.
This repository contains a set of Packer templates for building disk images to use for Full System Simulation with gem5.
Currently, artifact generation is supported only for the ARM64 architecture.
# Directory where Linux Binary Artifacts for Full System Simulation will be generated
export ARTIFACTS=$HOME/artifacts
# Init packer plugins
packer init gem5-dbc/artifacts/packer
# Generate binary artifacts
packer build -var artifacts_dir=$ARTIFACTS gem5-dbc/artifacts/packer
After succesful building of all binary images,
an index file $ARTIFACTS/index.yaml
is created containing
metadata and checksum information for all generated binaries.
The index file will be referenced later in order to automatically generate simulation scripts.
For more details, see artifacts/packer/README.md.
Linux Binary Artifact Registry
After the ARM64 image build is finished, the directory $ARTIFACTS
will contain the generated binary artifacts.
File | Description |
---|---|
$ARTIFACTS/index.yaml | Index of binary artifacts for use with g5dbc |
$ARTIFACTS/arm64/disks | Disk Images |
$ARTIFACTS/arm64/keys | SSH Keys for generated image |
$ARTIFACTS/arm64/binaries | ARM64 bootloaders and Linux kernels |
Example generated index.yaml
file after successful build of an ARM64 debian-testing image with Packer.
arm64:
- bintype: "DISK"
name: disk.img
path: arm64/disks/disk.img-debian-testing
md5hash: d37bfb08faa9ba10c861b649b6fbac8a
version: debian-testing
metadata: /dev/vda2
- bintype: "BOOT"
name: boot.arm64
path: arm64/binaries/boot.arm64-V1
md5hash: 64e7904dccf73a73f9cb8040f7a27a0c
version: V1
metadata: V1
- bintype: "BOOT"
name: boot.arm64
path: arm64/binaries/boot.arm64-V2
md5hash: c81d1a6d4ee9621a9b08284cfb00190e
version: V2
metadata: V2
- bintype: "KERNEL"
name: Image
path: arm64/binaries/Image-5.15.68
md5hash: 66d18afe5d0cb4c4ec9cb0641982ff38
version: 5.15.68
metadata: qemu
- bintype: "KERNEL"
name: vmlinux
path: arm64/binaries/vmlinux-5.15.68
md5hash: a98b65636bbac2a1fe4ab4eb5948d66b
version: 5.15.68
metadata: earlyprintk=pl011,0x1c090000 console=ttyAMA0 lpj=19988480 norandmaps rw loglevel=8
To let gem5-dbc automatically generate gem5 simulation scripts for you, you need to define
- A set of initial model parameters, the initial architecture configuration,
- A gem5 benchmark specification, by implementing a subclass of AbstractBenchmark.
The initial configuration is described using human-readable YAML format. Example configurations are located under share/gem5-dbc/configs.
A schema definition documenting currently supported configuration values is located at share/gem5-dbc/schema.json.
Initial configuration example
The following lists the example garnet configuration using Garnet and DDR4 memory. A schema definition documenting currently supported configuration values is located at share/gem5-dbc/schema.json.
Note that gem5-dbc uses a YAML loader with support for !include constructor.
An initial configuration can be defined as a single YAML file, or a directory containing an index file index.yaml
.
# Example configuration with Garnet interconnect and DDR4 memory model
simulation:
gem5_version: 24.1.0.2
full_system: True
system:
architecture: arm64
num_cpus: 2
num_slcs: 2
clock: 2.6GHz
sve_vl: 256
cache_line_size: 64
cpus:
simple:
model: AtomicSimple
clock: 2.6GHz
o3cpu: !include cpu.yaml
caches: !include caches.yaml
memory:
regions:
- model: DRAM
channels: 4
size: 2GiB
dram_settings: !include ../memory/DDR4_2400_16x4.yaml
interconnect:
model: garnet
network: !include network.yaml
A variety of Ruby network topologies can be defined as well.
Currently, gem5-dbc only supports the Simple2D
network topology model.
The Simple2D
model constructs simple 2D meshes based on a list of internal links.
An example for a simple 2D mesh topology is given below.
# 1x Link for all VNETs
mesh_vnet_support: [[0,1,2,3]]
node_vnet_support: [[0,1,2,3]]
# 2x3 2D mesh topology
#
# [3] - [4] - [5]
# | | |
# [0] - [1] - [2]
#
topology:
model: Simple2D
parameters:
num_mesh_routers: 6
router_numa_ids: [0,0,0,0,0,0]
cpu_routers: [[1,4]]
slc_routers: [[1,4]]
mem_routers: [[0,3]]
rom_routers: [2]
dma_routers: [2]
# Tuples describe internal links between routers A,B in the format:
# [Router A, Router B, Weight]
internal_links:
- [0, 1, 1]
- [1, 2, 1]
- [3, 4, 1]
- [4, 5, 1]
- [0, 3, 2]
- [1, 4, 2]
- [2, 5, 2]
A gem5 benchmark is defined by implementing a subclass of AbstractBenchmark
Example benchmark implementations are located under share/gem5-dbc/benchmarks.
Benchmark specification
Specify a gem5 benchmark by implementing following abstract methods:
class benchmark(AbstractBenchmark[T]):
@abstractmethod
def get_command(self, params: T, config: Config) -> str:
"""Return command to execute"""
@abstractmethod
def get_env(self, params: T, config: Config) -> dict:
"""Return dictionary of environment variables to set before running benchmark"""
@abstractmethod
def get_varparams(self) -> dict[str, list]:
"""Return the parameter space definition as a map
parameter name => list of parameter values"""
@abstractmethod
def filter_varparams(self, params: T, config: Config) -> bool:
"""Return True if given initial configuration and parameter combination is valid"""
@abstractmethod
def update_config(self, params: T, config: Config) -> Config:
"""Update configuration from given parameters"""
@abstractmethod
def get_data_rows(self, params: T, stats: dict) -> dict | list[dict]:
"""Return a single data row or multiple data rows to be written to a CSV file"""
Once the initial configuration and AbstractBenchmark implementation is defined,
g5dbc
may be invoked to generate simulations scripts.
For this example, we generate simulation scripts for the mini_triad
benchmark using garnet
as initial configuration.
The mini_triad
benchmark will modify
the initial garnet
configuration by attaching
m5.SimpleMemory
memory controllers with varying bandwidth parameters.
We also need to specify the artifacts directory, --artifacts-dir $ARTIFACTS
.
Generate the simulation scripts using 4 threads, --nprocs 4
.
# Generate simulation scripts for stream/mini_triad benchmark and example garnet configuration
# Include $ARTIFACTS directory containing artifacts.yaml index
g5dbc --generate stream/mini_triad --init-config garnet --artifact-index $ARTIFACTS --nprocs 4
The command will generate simulation scripts under mini_triad/work
.
Each subdirectory is labeled by an integer $benchId
, and contains the following files
File | Description |
---|---|
$benchName/work/$benchId/srun.sh |
Shell script to start gem5 simulation for $benchId configuration |
$benchName/work/$benchId/work.sh |
Shell script to execute under Full System Simulation |
$benchName/work/$benchId/config.yaml |
Architecture configuration file for $benchId |
The srun.sh
script is generated from a template
which can be modified to run simulations in a cluster environment, for example.
To run the first 10 benchmarks on the local machine, you may use the command line
# Run the first 10 benchmarks
for i in {00..09} ; do ./mini_triad/work/$i/srun.sh & done
After successful simulation run, a stats.txt
file will be written for each $benchId
.
We can use g5dbc
to parse the results to flat JSON,
# Parse the generated benchmark statistics to flat JSON
g5dbc --parse mini_triad --nprocs 4
Resulting JSON files will be written to $benchName/parsed/$benchId.$roiId.json
From the parsed JSON files we can generate a CSV file with relevant data,
as implemented in get_data_rows
:
# Evaluate the parsed results and write relevant data to a CSV file.
g5dbc --eval mini_triad --nprocs 4
The command will generate a file $benchName/parsed/data.csv