Introduction

Blink (Benchmark for Large-Scale Interconnection Networks) is designed to run multiple applications and/or benchmarks together and collect data on the interconnection network performance.

Requirements

• Python >= 3.6.8
• numpy
• pandas
• seaborn (Only for the plotting part)

Preliminary information

All the commands described below MUST be run from the root directory of the framework. If the system is not already supported by the framework, you need to add the support for it. See the section "How to add the support for a new system" for more details.

How to compile

Before compiling, change the BLINK_SYSTEM variable in the conf.sh file to the name system you want to compile for. Then, run the following command: $ ./compile.sh

How to run benchmarks

The purpose of Blink is to run multiple benchmarks together and collect data from them. To do so, you need to create configuration files (apps_mix) describing which applications must be executed together (e.g. a pingpong together with an alltoall). You can find some examples under the apps_mix directory, and more details about the file format in apps_mix/README.md file. You can then run multiple of those configurations sequentally by creating a test suite (basically, a list of apps_mix to be executed). You can find examples of test suites in the test_suites directory.

Once you have created the test suite, you can run it by running the following command: $ ./run.sh <path_to_test_suite> <path_to_node_files>

• <path_to_test_suite> is the path to the test suite file
• <path_to_node_files> is the path to the file listing the available nodes on the system. Default format is one hostname per line. If a custom manual allocation of the applications on nodes is used, then the file must specify one column per application/benchmark (csv style). If the system uses Slurm, you can specify auto as <path_to_node_files>, and the benchmark will use all the allocated nodes (it assumes that the nodes have already been allocated using salloc or similar).

You can get a full list of the available options by running the following command: $ ./run.sh -h

How to integrate new applications/benchmarks

To integrate new applications, you need to create a new wrapper for the application. The wrapper is a Python script that launches the application and collects the data from it. You can find examples of wrappers in the wrappers directory, and more details on how to create a new wrapper in the wrappers/README.md file. After creating a new wrapper, it can be used in applications mix files and test suites.

Repository structure

This repository contains the following files and directories:

• apps_mix: Contains the files describing the mix of applications and benchmarks that can be concurrently run by the framework.
• compile.sh: Script to compile the applications and benchmarks.
• conf: Contains the configuration files for the different systems.
• conf.sh: Configuration file for the framework.
• data: Contains the data collected by the framework.
• generate_script.py: Generate the bash script to run a specific test suite.
• plots: Contains the plots generated by the framework.
• plotter.py: The script to generate the plots from the data collected by the framework.
• run.sh: Generates the bash script to run a specific test suite and runs it.
• runner.py: The main script of the framework. It runs the applications and benchmarks and collects the data.
• specific_plotter: Contains additional visualization scripts.
• src: Contains the microbenchmarks code.
• test_suites: Contains the test suites to be run by the framework.
• wrappers: Contains the Python code to launch the applications and benchmarks supported by the framework and to parse their output.

How to add the support for a new system

Let's assume that the system name is newsystem'.

1. Add a newsystem.sh file in the conf/ directory. Copy the content from one of the existing files and adjust the parameters to match the new system.
2. Set the SYSTEM variable to newsystem in the conf.sh script.

Acknowledgement

Loic Holbein developed a preliminary version of Blink as part of his bachelor thesis at ETH Zurich.