Quick Start

Interpreter-guided Differential JIT Compiler Unit Testing

Getting Started Guide

Our solution is implemented as a tool called Ranger, available in github (https://github.com/Alamvic/ranger/) that runs on top of Pharo 10. Ranger tests the Pharo VM on it's simulation environment: a single machine can test different architectures. Thus, evaluating ranger requires downloading/installing the following artifacts:

• Ranger itself
• the Pharo VM source code and the Pharo IDE
• the LLVM disassembler dynamic library with support for at least x86 and ARM32
• the Unicorn machine simulator dynamic library
• the minizinc constraint solver (https://www.minizinc.org/)

Below we provide installation instructions for installing these components on a clean directory. We provide instructions for Linux and OSX X86-64, for which we provide pre-compiled binaries of the required dynamic libraries.

Clone the source code

git clone -b feat/ranger2 [email protected]:pharo-project/pharo-vm.git
git clone [email protected]:Alamvic/ranger.git

Download the external libraries

Download and unzip libllvm-full.zip and libunicorn.zip from:

• Linux x86-64: https://files.pharo.org/vm/pharo-spur64/Linux-x86_64/third-party/
• OSX x86-64: https://files.pharo.org/vm/pharo-spur64/Darwin-x86_64/third-party/
• OSX ARM64:
  • brew installation of llvm has everything needed brew install llvm
  • unicorn can be built from https://github.com/pharo-project/unicorn + make install

Minizinc Installation

Install MiniZinc following it's instructions (https://www.minizinc.org/) You will need

• MiniZinc (minizinc.org)
• Make sure that it's in the PATH (see https://www.minizinc.org/doc-2.2.3/en/installation.html)
• The or-tools solver. You can install the solver using the instructions found in this post.
  • Download from github releases (We have tested for example version 9.5)
  • Move it close to the rest of the files mv ~/Downloads/or-tools_arm64_macOS-13.0.1_cpp_v9.5.2237 .
  • configure the MiniZinc IDE to find it

Get Pharo, load the simulation environment and ranger

mkdir bin
cd bin
wget -O- https://get.pharo.org/110+vm | bash
cd ..
./bin/pharo-ui ./bin/Pharo.image

Once in the IDE, open a playground from the menu (Browse>Playground). Copy the following code snipped, click on the Play button and wait until it finishes

Metacello new
  baseline: 'VMMaker';
  repository: 'gitlocal://./pharo-vm';
  load;
  lock.
Metacello new
  baseline: 'Ranger';
  repository: 'gitlocal://./ranger';
  load;
  lock.

Verify everything is ok

1. Check minizinc is in the path (e.g., minizinc --version from the terminal should returna version)
2. Check that Pharo finds the libraries 2.1 Unicorn: Execute from a playground UnicornLibrary uniqueInstance libraryName with print it. It should return the library path 2.1 LLVM: Execute from a playground LibLLVMDisassembler uniqueInstance ffiLibraryName with print it. It should return the library path

3. Run LLVM disassembler and Minizinc tests in the test runner (see screenshot in https://github.com/Alamvic/ranger/wiki/Quick-Start)

Running Ranger

Open the test runner, filter packages by ranger-explorer, select any of:

• RARegisterAllocatingCogitTest
• RASimpleStackBasedCogitTest
• RAStackToRegisterMappingCogitTest
• RAPrimitiveAutoTest

And click on Run Selected

Step-by-Step Instructions

To reproduce the results of the article, the instructions above should be run against the feat/autotesting branch. Moreover, since the experiment was done using Unicorn 1 and an x86-64 OSX version, those should be used too.

Then, each test class should be run separately:

• RASimpleStackBasedCogitTest => corresponds to the Simple Stack BC Compiler in Table 2
• RAStackToRegisterMappingCogitTest => corresponds to the Stack-to-Register BC Compiler in Table 2
• RARegisterAllocatingCogitTest => corresponds to the Linear-Scan Allocator BC Compiler in Table 2
• RAPrimitiveAutoTest => corresponds to the Native Methods (primitives) in Table 2

Claims supported by the artifact (RASimpleStackBasedCogitTest)

• 1308 ran
• 20 errors

Claims supported by the artifact (RAStackToRegisterMappingCogitTest)

• 1308 ran
• 12 differences

Claims supported by the artifact (RARegisterAllocatingCogitTest)

• 1308 ran
• 12 differences

Claims supported by the artifact (RAPrimitiveAutoTest)

• 2024 ran
• 492 differences (488 errors + 4 failures)