Proposal: create common distribution formats

[mefellows]

It would be great to be able to install Pact CLI tool(s) using common packaging and distribution formats, e.g.

• gem install pact
• For MacOSXX brew install pact
• For Linux / Other curl -sSL https://get.pact.io | bash -s style install (e.g. for RVM curl -sSL https://get.rvm.io | bash -s stable --ruby)
• For Windows a Nuget package installed like chocolatey install pact or Install-package https://raw.githubusercontent.com/pact-standalone/releases/pact-1.0.0.config
• Potentially others (RPM, Deb etc.)

I would also propose that we only have 3 CLI tools: verifier, mock-service and the pact tool (containing broker commands, stubs etc.). The first two are for use by library implementations, where the latter could be installed by any user of Pact. I see this as being like a travis command, where it is used for other useful stuff - e.g. bootstrapping projects.

Thoughts?

[bethesque]

I like it. I did consider creating a brew installer a while ago, but I looked at the criteria for inclusion, and it had something about not being able to be installed via other package managers, and not being able to be submitted by the author of the tool itself. Hopefully we can work around this.

[mefellows]

Yeah, they can be a bit picky. There are more rigorous rules for the main brew repo, but you can tap other repos (or even just create one in our repo).

[bethesque]

I think we still want the single standalone package with everything together for embedding in the native libraries, because of the size of the Ruby runtime.

[bethesque]

Ah yes, I only just learned about making your own brew repos. It's a good backup plan.

[YOU54F]

Initial thoughts

Looking at the size of the pact-plugin-cli executable (11mb) I am inclined to suggest we seperate out and publish two sets of packages

• one containing the pure ruby executables
• one containing all our pact executables

├── pact
├── pactflow
├── pact-broker
├── pact-message
├── pact-mock-service
├── pact-stub-service
├── pact-provider-verifier
├── pact_mock_server_cli
├── pact-stub-server
├── pact_verifier_cli
└── pact-plugin-cli

If that is produced in one place, its easy for HomeBrew / Scoop and our scripted installed to provide a ruby-standalone or full-fat cli experience.

Ideally we could use Rust going forward as a way to provide a directory to install all the cli tooling, the pact-plugin-cli provides a nice template for that.

Could a pact-cli command in Rust, abstract away the rubyisms behind, so a pact-cli broker command feels natural and the fact it is ruby or rust in the backend is a moot point.

For tools there are duplicated in rust/ruby, we could provide an abstraction through a single command (rather than having two) and use a switch --legacy or PACT_USE_LEGACY=1 env var to switch between either the rust or the ruby impl.

├── pact-mock-service
├── pact-stub-service
├── pact-provider-verifier
├── pact_mock_server_cli
├── pact-stub-server
├── pact_verifier_cli

Brain dump

The pact-ruby-standalone project recently added the pact-plugin-cli to the bundle

This weighs in at 12Mb and hugely inflates our bundle.

This proposal suggests a couple of different packages

• pact - ruby only pact packages
• pact-rust - ruby + rust packages
• pact-ffi - ruby + pact_ffi lib and pact-ffi gem
• pact-cli - ruby + rust + pact_ffi lib and pact-ffi gem

for any of the combinations above, they can have

• pact--slim

augmentations, which seek to trim out as much as possible from the ruby runtime.

Note:- the slim versions may not work and need throughly testing, cursory testing has been
done so far.

The overall aim is provide users with a single bundled distribution called pact-cli

which contains every single pact executable so the users can get all the pact goodness at their
fingertips.

This will have benefits for packaging with HomeBrew/Scoop/Choco and our own install script.

In the future it would be nice to extend pact-plugin-cli which can discover and install plugins

to discover all of the pact tooling (official and community provided) and provide a mechanism to install

to a shared location such as PACT_HOME_DIR which would default to $HOME/.pact

This pattern is already in use in the pact-plugin-cli

Could we further this, so users only need the standalone in a single location, and the ffi libs in a single location, and our various client libraries to point to the relevant home folder, and not require the packages to contain the host libraries themselves. This may not be practical for some libs which need to build from source, but if the source was already on their machine, via our installer, maybe that makes it less of an issue.

The tree of all tools would currently look like this

├── pact
├── pactflow
├── pact-broker
├── pact-message
├── pact-mock-service
├── pact-stub-service
├── pact-provider-verifier
├── pact_mock_server_cli
├── pact-stub-server
├── pact_verifier_cli
└── pact-plugin-cli

Users would be able to interact with the FFI by

`${PATH_TO_RUBY}ruby -rpact/ffi -e "puts PactFfi.pactffi_version"`

Package sizes

Full Ruby Bundle - with pact-plugin-cli

As per todays release.

49M    pkg/pact

16M    pkg/pact-2.0.2-linux-arm64.tar.gz
16M    pkg/pact-2.0.2-linux-x86_64.tar.gz
14M    pkg/pact-2.0.2-osx-arm64.tar.gz
15M    pkg/pact-2.0.2-osx-x86_64.tar.gz
11M    pkg/pact-2.0.2-windows-x86.zip
16M    pkg/pact-2.0.2-windows-x86_64.zip

Full Ruby (only) Bundle

37M    pkg/pact

12M    pkg/pact-2.0.2-windows-x86_64.zip
12M    pkg/pact-2.0.2-windows-x86.zip
10M    pkg/pact-2.0.2-osx-x86_64.tar.gz
10M    pkg/pact-2.0.2-linux-x86_64.tar.gz
9.0M   pkg/pact-2.0.2-osx-arm64.tar.gz
9.0M   pkg/pact-2.0.2-linux-arm64.tar.gz

Trimmed Ruby (only) Bundle

TRIM_PACKAGE_FULL=true bundle exec rake package

28M    pkg/pact

10M    pact-2.0.2-windows-x86.zip
9.7M   pact-2.0.2-windows-x86_64.zip
7.0M   pact-2.0.2-linux-arm64.tar.gz
7.0M   pact-2.0.2-linux-x86_64.tar.gz
7.0M   pact-2.0.2-osx-arm64.tar.gz
7.0M   pact-2.0.2-osx-x86_64.tar.gz

Full Ruby Bundle + Pact Rust Tools

PACKAGE_PACT_RUST_TOOLS=true bundle exec rake package

98M    pkg/pact

41M    pkg/pact-2.0.2-linux-x86_64.tar.gz
39M    pkg/pact-2.0.2-linux-arm64.tar.gz
34M    pkg/pact-2.0.2-osx-x86_64.tar.gz
33M    pkg/pact-2.0.2-osx-arm64.tar.gz
33M    pkg/pact-2.0.2-windows-x86_64.zip
11M    pkg/pact-2.0.2-windows-x86.zip

du -sh pkg/pact/bin/* | sort -nr

18M    pkg/pact/bin/pact_verifier_cli
17M    pkg/pact/bin/pact_mock_server_cli
14M    pkg/pact/bin/pact-stub-server
12M    pkg/pact/bin/pact-plugin-cli

Ruby (only) + Pact-Ffi Gem (and pact_ffi libs) Bundle

PACKAGE_PACT_FFI=true bundle exec rake package

64M    pkg/pact

21M    pkg/pact-2.0.2-windows-x86_64.zip
21M    pkg/pact-2.0.2-windows-x86.zip
19M    pkg/pact-2.0.2-osx-x86_64.tar.gz
19M    pkg/pact-2.0.2-osx-arm64.tar.gz
19M    pkg/pact-2.0.2-linux-x86_64.tar.gz
19M    pkg/pact-2.0.2-linux-arm64.tar.gz

24M    pkg/pact/lib/vendor/ruby/3.2.0/gems/pact-ffi-0.0.2-arm64-darwin/ffi/macos-arm64/libpact_ffi.dylib

Full Ruby Bundle + Pact Rust Tools + Pact-Ffi Gem (and pact_ffi libs)

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true bundle exec rake package

123M    pkg/pact-cli
52M    pkg/pact-cli-2.0.2-linux-x86_64.tar.gz
51M    pkg/pact-cli-2.0.2-linux-arm64.tar.gz
43M    pkg/pact-cli-2.0.2-osx-x86_64.tar.gz
41M    pkg/pact-cli-2.0.2-osx-arm64.tar.gz
33M    pkg/pact-cli-2.0.2-windows-x86_64.zip
12M    pkg/pact-cli-2.0.2-windows-x86.zip
24M    pkg/pact/lib/vendor/ruby/3.2.0/gems/pact-ffi-0.0.2-arm64-darwin/ffi/macos-arm64/libpact_ffi.dylib

Building & Testing.

You can easily run through the testing by passing the same env vars.

The following scenario will

• package the pact-ruby-standalone for osx:arm64
• include the rust tools
• include the pact ffi
• unpack the built package that would be distributed to users
• runs a smoke test against the package

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true bundle exec rake package:osx:arm64

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true PACKAGE_NAME=pact-cli ./script/unpack-and-test.sh

You can run the smoke tests anyway, your standalone directory lives.

• Run the script /script/test.sh
• Set your env vars
  • PATH_TO_BIN=pkg/pact-cli/bin/ default is $PACKAGE_NAME/bin/
    • ensure it has a trailing /
    • if windows, reverse the slashes \
  • PACKAGE_NAME default is pact

Example scenario

• runs a smoke test against the package
  • includes tests for the rust tools
  • includes test for the pact ffi

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true PACKAGE_NAME=pact-cli PATH_TO_BIN=pkg/pact-cli/bin/ ./script/test.sh

[YOU54F]

I've been testing it out against the main branch on my fork

https://github.com/you54f/pact-ruby-standalone

the build workflow takes workflow inputs that should be set to true to trigger the required new behaviour

[YOU54F]

Regarding our native script installer.

I've made some modifications to it so it will

• support windows users (who are using bash/sh)
• set the path in github or cirrus ci, we can extend this for other ci providers

Also updated our actions to provide access via the bin folder to all the bundles tooling, so it saves maintaining multiple actions.

https://github.com/pactflow/actions

it accepts env vars being set so the user can pin to particular versions if required, otherwise it will pick up latest.

[YOU54F]

Regarding other package managers

• Scoop https://github.com/YOU54F/scoop
• HomeBrew https://github.com/YOU54F/homebrew-pact#installation

[YOU54F]

Thinking of a nice list of sources. Will incorporate and add yours

Package Managers

Windows

• Manual Download from GitHub Release
• Standalone script
• Scoop
• Choco
• Powershell Installer (like scoop gets installed)
• RubyGems (For Ruby based tools)
• Cargo (For Rust based tools)

MacOS

• Manual Download from GitHub Release
• Standalone script
• Homebrew
• RubyGems (For Ruby based tools)
• Cargo (For Rust based tools)

Linux

• Manual Download from GitHub Release
• Standalone script
• Homebrew (Linuxbrew)
• Snap (Ubuntu)
• FlatPak (Universal)
• .rpm (Fedora)
• .deb (Debian)
• .apk (Alpine)
• RubyGems (For Ruby based tools)
• Cargo (For Rust based tools)

CI Systems

• GitHub Actions
• CircleCI Orbs

Other OS's

• NetBSD
• OpenBSD
• FreeBSD
• Linux Android

[YOU54F]

CircleCi Orb created by the community

https://github.com/trustedshops-public/circleci-orb-pact

[YOU54F]

arch linux pact-cli (which is the pact-ruby-standalone)

https://aur.archlinux.org/packages/pact-cli-bin

now updated to include aarch64 for linux 🚀 however arch linux doesn't officially support aarch64

community maintained, I have emailed the reported author to reach out

Dockerfile

FROM --platform=linux/amd64 greyltc/archlinux-aur:yay
RUN sudo -u ab -D~ bash -c 'yay -Syu --removemake --needed --noprogressbar --noconfirm pact-cli-bin'  
ENTRYPOINT ["/bin/bash", "-c"]
CMD ["pact-mock-service --help"]

Instructions

docker build . -t archlinux-aur-pact-cli:latest -f Dockerfile.ArchLinux
docker run --platform=linux/amd64 --rm -it archlinux-aur-pact-cli:latest

Output

>docker run --platform=linux/amd64 --rm -it archlinux-aur-pact-cli:latest 'pact-mock-service version'
3.11.2

[YOU54F]

Being worked on as part of pact-foundation/roadmap#127