darchr · erin-le · Aug 8, 2023 · Aug 15, 2023 · Aug 23, 2023 · Aug 17, 2023
diff --git a/example/README.md b/example/README.md
@@ -0,0 +1,40 @@
+# Running tests on the RISCV Matched Board with a Plackett-Burman Design
+
+This document describes the scripts for testing and the process followed to perform tests.
+
+## Part 1: Methodology
+
+For this investigation, we used the methodology outlined in the paper "A Statistically Rigorous Approach for Improving Simulation Methodology" by Yi et al. The methodology uses a Plackett Burman design, which is a matrix containing only 1s and -1s, and whose dimensions are divisible by 4.
+
+This matrix can also be multiplied by a -1 to form a "foldover matrix", thus doubling the number of rows/potential configurations.
+
+Each row of the matrix represents a configuration of the system, and each column represents one of the parameters being varied. A -1 represents a value of a parameter that is slightly below the range of standard values, and a +1 represents a value that is slightly higher.
+
+The number of columns in the matrix must be greater than the number of parameters. For the RISCV Matched Board, we used a matrix size of 16, as we had a total of 13 parameters. These are listed in `nopipe_params.json`. The first value (index 0) was the value from the original RISCV Matched Board code, while the second value (index 1) was an arbitrarily chosen larger value.
+
+## Part 2: Tools
+### Creating a Configured JSON
+
+The first step was to make a script that would take as input:
+1. a JSON file with a list of high and low values for each parameter, and
+2. a row of the PB design,
+
+and output a JSON with only one value for each parameter. We used `nopipe_params.json` as the input JSON file, and initially used a 16 by 16 Hadamard matrix generated in MATLAB to get rows of the PB design. We were not sure if all Hadamard matrices could be used as PB designs, so we later used a size 16 PB design from the original paper on PB designs.
+
+We used the script `json_maker.py` to generate the JSONs.
+
+### Using the Configured JSON to Parameterize the RISCV Matched Board
+
+The RISCV Matched Board code was modified to be able to take in a JSON, convert it to a Python dictionary, then set the values of its parameters to the values from the JSON.
+
+The files are found in src/python/gem5/prebuild/riscvmatched.
+
+### Running the benchmarks
+
+The benchmarks were run by using the bash scripts `pb_matched_run.sh` and `microbench_pb_run.sh`. `pb_matched_run.sh` was used for riscv-print-this, riscv-bubblesort, riscv-floatmm, and riscv-hello, while `microbench_pb_run.sh` was used to run a number of microbenchmarks.
+
+`pb_matched_run.sh` loops over all combinations of row (0-15) and foldover (0 or 1) for a given benchmark, while `microbench_pb_run.sh` loops over all microbenchmarks for a given row and foldover.
+
+The output files are put into either matched_board_tests_MATLAB or matched_board_tests_paper. Within these directories, the files are sorted by benchmark, then by the row, foldover, and matrix size. The name format for these subdirectories is foldover_row_matrixsize. For example, the directory 0_6_16 corresponds to the config json that was generated with row index 6 of the original (not the foldover) PB design.
+
+Some of the configurations under matched_board_tests_MATLAB have multiple runs, while all of the runs under matched_board_tests_paper have only one run. When running the simulations the first time, we wanted to see if there were any variations in simulation time between different runs of the same system. We did not observe any variation, so we only did one run for each system the second time.
diff --git a/example/benchmark_stats/dp1d.txt b/example/benchmark_stats/dp1d.txt
diff --git a/example/benchmark_stats/riscv-bw-ldst.txt b/example/benchmark_stats/riscv-bw-ldst.txt
diff --git a/example/benchmark_stats/riscv-cca.txt b/example/benchmark_stats/riscv-cca.txt
diff --git a/example/benchmark_stats/riscv-cce.txt b/example/benchmark_stats/riscv-cce.txt
diff --git a/example/benchmark_stats/riscv-cch-st.txt b/example/benchmark_stats/riscv-cch-st.txt
diff --git a/example/benchmark_stats/riscv-cch.txt b/example/benchmark_stats/riscv-cch.txt
diff --git a/example/benchmark_stats/riscv-cci.txt b/example/benchmark_stats/riscv-cci.txt
diff --git a/example/benchmark_stats/riscv-ccm.txt b/example/benchmark_stats/riscv-ccm.txt
diff --git a/example/benchmark_stats/riscv-crd.txt b/example/benchmark_stats/riscv-crd.txt
diff --git a/example/benchmark_stats/riscv-crf.txt b/example/benchmark_stats/riscv-crf.txt
diff --git a/example/benchmark_stats/riscv-cs1.txt b/example/benchmark_stats/riscv-cs1.txt
diff --git a/example/benchmark_stats/riscv-cs3.txt b/example/benchmark_stats/riscv-cs3.txt
diff --git a/example/benchmark_stats/riscv-dp1d.txt b/example/benchmark_stats/riscv-dp1d.txt
diff --git a/example/benchmark_stats/riscv-dp1f.txt b/example/benchmark_stats/riscv-dp1f.txt
diff --git a/example/benchmark_stats/riscv-dpcvt.txt b/example/benchmark_stats/riscv-dpcvt.txt
diff --git a/example/benchmark_stats/riscv-dpt.txt b/example/benchmark_stats/riscv-dpt.txt
diff --git a/example/benchmark_stats/riscv-dptd.txt b/example/benchmark_stats/riscv-dptd.txt
diff --git a/example/benchmark_stats/riscv-ed1.txt b/example/benchmark_stats/riscv-ed1.txt
diff --git a/example/benchmark_stats/riscv-ef.txt b/example/benchmark_stats/riscv-ef.txt
diff --git a/example/benchmark_stats/riscv-ei.txt b/example/benchmark_stats/riscv-ei.txt
diff --git a/example/benchmark_stats/riscv-em1.txt b/example/benchmark_stats/riscv-em1.txt
diff --git a/example/benchmark_stats/riscv-em5.txt b/example/benchmark_stats/riscv-em5.txt
diff --git a/example/benchmark_stats/riscv-m-dyn.txt b/example/benchmark_stats/riscv-m-dyn.txt
diff --git a/example/benchmark_stats/riscv-mc.txt b/example/benchmark_stats/riscv-mc.txt
diff --git a/example/benchmark_stats/riscv-mcs.txt b/example/benchmark_stats/riscv-mcs.txt
diff --git a/example/benchmark_stats/riscv-md.txt b/example/benchmark_stats/riscv-md.txt
diff --git a/example/benchmark_stats/riscv-mi.txt b/example/benchmark_stats/riscv-mi.txt
diff --git a/example/benchmark_stats/riscv-mim.txt b/example/benchmark_stats/riscv-mim.txt
diff --git a/example/benchmark_stats/riscv-mim2.txt b/example/benchmark_stats/riscv-mim2.txt
diff --git a/example/benchmark_stats/riscv-mip.txt b/example/benchmark_stats/riscv-mip.txt
diff --git a/example/benchmark_stats/riscv-ml2-bw-ld.txt b/example/benchmark_stats/riscv-ml2-bw-ld.txt
diff --git a/example/benchmark_stats/riscv-ml2-bw-ldst.txt b/example/benchmark_stats/riscv-ml2-bw-ldst.txt
diff --git a/example/benchmark_stats/riscv-ml2-bw-st.txt b/example/benchmark_stats/riscv-ml2-bw-st.txt
diff --git a/example/benchmark_stats/riscv-stc.txt b/example/benchmark_stats/riscv-stc.txt
diff --git a/example/benchmark_stats/riscv-stl2.txt b/example/benchmark_stats/riscv-stl2.txt
diff --git a/example/configured_jsons/pb/0_0_16.json b/example/configured_jsons/pb/0_0_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_10_16.json b/example/configured_jsons/pb/0_10_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/0_11_16.json b/example/configured_jsons/pb/0_11_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/0_12_16.json b/example/configured_jsons/pb/0_12_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_13_16.json b/example/configured_jsons/pb/0_13_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_14_16.json b/example/configured_jsons/pb/0_14_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_15_16.json b/example/configured_jsons/pb/0_15_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/0_1_16.json b/example/configured_jsons/pb/0_1_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/0_2_16.json b/example/configured_jsons/pb/0_2_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_3_16.json b/example/configured_jsons/pb/0_3_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/0_4_16.json b/example/configured_jsons/pb/0_4_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_5_16.json b/example/configured_jsons/pb/0_5_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_6_16.json b/example/configured_jsons/pb/0_6_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/0_7_16.json b/example/configured_jsons/pb/0_7_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/0_8_16.json b/example/configured_jsons/pb/0_8_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/0_9_16.json b/example/configured_jsons/pb/0_9_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_0_16.json b/example/configured_jsons/pb/1_0_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_10_16.json b/example/configured_jsons/pb/1_10_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/1_11_16.json b/example/configured_jsons/pb/1_11_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/1_12_16.json b/example/configured_jsons/pb/1_12_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_13_16.json b/example/configured_jsons/pb/1_13_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_14_16.json b/example/configured_jsons/pb/1_14_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_15_16.json b/example/configured_jsons/pb/1_15_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/1_1_16.json b/example/configured_jsons/pb/1_1_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/1_2_16.json b/example/configured_jsons/pb/1_2_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_3_16.json b/example/configured_jsons/pb/1_3_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/1_4_16.json b/example/configured_jsons/pb/1_4_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_5_16.json b/example/configured_jsons/pb/1_5_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 2}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_6_16.json b/example/configured_jsons/pb/1_6_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 5, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/1_7_16.json b/example/configured_jsons/pb/1_7_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Tournament"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/configured_jsons/pb/1_8_16.json b/example/configured_jsons/pb/1_8_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "4KiB"}}
diff --git a/example/configured_jsons/pb/1_9_16.json b/example/configured_jsons/pb/1_9_16.json
@@ -0,0 +1 @@
+{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "2MiB", "l2_assoc": 16, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "16KiB", "dptw_caches-size": "16KiB"}}
diff --git a/example/full_factorial_matrix_maker.py b/example/full_factorial_matrix_maker.py
@@ -0,0 +1,24 @@
+import argparse
+import math
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--factors",
+        type=int,
+        required=True,
+        help="Number of parameters or factors you want to test. The output file will have 2^factors rows.",
+    )
+    args = parser.parse_args()
+
+    with open(f"{args.factors}_full_factorial_matrix.txt", "w") as out:
+        for i in range(0, 2**args.factors):
+            temp = str(bin(i))[2:]
+            temp = f"{temp:0>{args.factors}}"
+            temp = " ".join(temp)
+            out.write(temp)
+            out.write("\n")
+
+
+main()
diff --git a/example/get_all.py b/example/get_all.py
@@ -0,0 +1,117 @@
+import os
+import sys
+
+
+def create_dir(microbench):
+    dir_name = "microbench_results" if microbench else "bench_results"
+    err_file = ".get_all_mb_error" if microbench else ".get_all_error"
+    os.system(f"touch {err_file}")
+    ret_code = os.system(f"mkdir {dir_name} 2> {err_file}")
+
+    if ret_code != 0:
+        with open(err_file, "r") as e_file:
+            line = e_file.readlines()[0]
+            if (
+                line
+                != f"mkdir: cannot create directory ‘{dir_name}’: File exists\n"
+            ):
+                print("Could not create output directory")
+                os.system(f"rm -rf {err_file}")
+                exit
+    os.system(f"rm -rf {err_file}")
+    return f"{dir_name}/"
+
+
+def run_command(
+    directory, pb_directory, pb_file, benchmark, matrix_size, resultsdir
+):
+    os.system(
+        f"python read_stats.py {directory} {pb_directory} {pb_file} {benchmark} {str(matrix_size)} > {resultsdir}{benchmark}.md"
+    )
+    return f"{resultsdir}{benchmark}.md"
+
+
+def parse_results_file(file):
+    results = []
+    with open(file, "r") as h_file:
+        lines = h_file.readlines()
+        for line in range(1, len(lines)):
+            parameter = lines[line].split(",")[0]
+            eff = float(lines[line].split(",")[1])
+            abs_v = float(lines[line].split(",")[2])
+            norm = float(lines[line].split(",")[3])
+            rank = line
+            results.append([parameter, eff, abs_v, norm, rank])
+    return sorted(results, key=lambda results: results[0])
+
+
+directory = (
+    "/home/lredivo/gem5-pb-test-board/example/matched_board_tests_paper/"
+)
+pb_directory = "./pb-designs"
+pb_file = "PB_16.txt"
+matrix_size = 16
+skip = 0
+
+bench_list = os.listdir(directory)
+bench_list_len = len(bench_list)
+micro_dir = directory + "microbench/"
+micro_bench_list = os.listdir(micro_dir)
+micro_bench_list_len = len(micro_bench_list)
+
+param_num = 13
+
+# [] * bench_list_len for i in range(0, bench_list_len)
+benchmark_results = [["parameter"]]
+micro_results = [["parameter"]]
+for parameters in range(0, param_num):
+    benchmark_results.append([])
+    micro_results.append([])
+
+
+for benchmark in range(0, bench_list_len):
+    if bench_list[benchmark] == "microbench" and not skip:
+        results_dir = create_dir(1)
+        for micro_bench in range(0, micro_bench_list_len):
+            file = run_command(
+                directory,
+                pb_directory,
+                pb_file,
+                micro_bench_list[micro_bench],
+                matrix_size,
+                results_dir,
+            )
+            results = parse_results_file(file)
+            micro_results[0].append(micro_bench_list[micro_bench])
+            for parameter in range(0, param_num):
+                if len(micro_results[1]) == 0:
+                    micro_results[parameter + 1].append(results[parameter][0])
+                micro_results[parameter + 1].append(results[parameter][2])
+    elif bench_list[benchmark] == "microbench" and skip:
+        continue
+    else:
+        results_dir = create_dir(0)
+        file = run_command(
+            directory,
+            pb_directory,
+            pb_file,
+            bench_list[benchmark],
+            matrix_size,
+            results_dir,
+        )
+        results = parse_results_file(file)
+        benchmark_results[0].append(bench_list[benchmark])
+        for parameter in range(0, param_num):
+            if len(benchmark_results[1]) == 0:
+                benchmark_results[parameter + 1].append(results[parameter][0])
+            benchmark_results[parameter + 1].append(results[parameter][2])
+
+print(
+    "\n".join([",".join([str(cell) for cell in row]) for row in micro_results])
+)
+print("-----------------------")
+print(
+    "\n".join(
+        [",".join([str(cell) for cell in row]) for row in benchmark_results]
+    )
+)
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		{"core": {"threadPolicy": "RoundRobin", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 1, "U74IntMulFU": 3, "U74IntDivFU": 18, "U74MemReadFU": 1, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 1, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "16KiB"}}
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		{"core": {"threadPolicy": "SingleThreaded", "BP": {"BPChoice": "Local"}, "FUPool": {"U74IntFU": 3, "U74IntMulFU": 9, "U74IntDivFU": 6, "U74MemReadFU": 2, "U74MemWriteFU": 1}}, "cache": {"l2_size": "16MiB", "l2_assoc": 64, "l1dcache-response_latency": 4, "l2cache-data_latency": 20, "iptw_caches-size": "4KiB", "dptw_caches-size": "4KiB"}}