Vend

vend is a dependency management tool for Common Lisp. The concept is simple:

Just vendor your dependencies!

vend’s focus is first and foremost on simplicity.

vend get to fetch dependencies directly into your project.
vend repl to open an isolated Lisp session.
vend test to run all your test suites.
vend only has 3 dependencies itself and compiles to a 400kb binary.
Neither Quicklisp (the tool) nor Quicklisp (the repository) are used.
Trivial integration with Sly / Slime / Lem / Slimv.

It’s time for some peace of mind.

Table of Contents

Why vendor dependencies?
- Why vend?
- Best Practices
Installation
- From Source
- Linux
  - Arch
Editor Integration
- Emacs
  - Doom Emacs
- Lem
- Vim
Usage
- vend get
- vend repl
- vend check
- vend eval
- vend graph
- vend init
- vend search
- vend test
  - Setup
  - Integrations
CI Integration
Coverage
Compiler Compatibility
FAQ

Why vendor dependencies?

Fast internet connections and centralised repositories have gotten us used to the idea that dependencies are free; things we can pluck off a shelf and employ with no risk. In languages like Javascript and Rust, it’s not uncommon to have projects with several hundred dependencies.

But are these really free? Have you inspected each one? Do you know and trust the authors, as well as the pipeline that feeds you updates? Is your project still guaranteed to compile in 5-10 years with no extra intervention? Can you write code on a plane? Can users reliably build your project after little more than a git clone?

The truth is that your dependencies are your code. And quite literally so - they might make up the majority of your final compiled artifact. By vendoring your dependencies directly into your project, you’re taking responsibility for its true form.

Why vend?

Dependency management in Common Lisp has traditionally centred around Quicklisp. A desire for access to more rapid package updates spawned Ultralisp. The need for version pinning and isolation birthed Qlot. The want for a better distribution system brought us ocicl.

But, could there be a simpler paradigm than just downloading the code and putting it right there?

With vend:

We need not perform bespoke installation scripts to get started.
We need not wait for Quicklisp to update.
We need not relegate all our systems to ~/common-lisp/.
We need not worry about where ASDF is looking for systems.
We need not fret over tools performing strange internal overrides.
We need not manage extra config files or lockfiles.

Plus, vend is actually an external tool with extra commands to help you inspect and manage your dependencies.

Best Practices

For library development, you are encouraged to:

vend get to fetch dependencies.
Add vendored/* to your .gitignore.

As this allows downstream users the most freedom when consuming your library.

For application development, you are encouraged to:

vend get to fetch dependencies.
rm -rf vendored/**/.git/
Actively commit vendored/ to git.

By committing these dependencies directly, it is never a mystery to your users how your software should be provisioned.

Installation

In all cases, vend requires ECL to build and run. However, it can be used to manage projects of any underlying compiler (see vend repl).

From Source

vend’s dependencies are all vendored, so it is enough to run:

make
make install

or manually:

ecl --load build.lisp
mv vend ~/.local/bin/

Linux

Arch

vend is available on the AUR and can be installed with tools like Aura:

aura -A vend

Editor Integration

If rough order of support / integration quality.

⚠ In all cases below, when starting a Lisp session, you must do so from the top-level of the repository. Doing this from the .asd file makes it easy.

Emacs

Sly and Slime have variables for setting how Lisp REPLs should be launched:

(setq sly-default-lisp 'sbcl
      sly-lisp-implementations '((sbcl  ("vend" "repl" "sbcl")  :coding-system utf-8-unix)
                                 (ecl   ("vend" "repl" "ecl")   :coding-system utf-8-unix)
                                 (abcl  ("vend" "repl" "abcl")  :coding-system utf-8-unix)
                                 (clasp ("vend" "repl" "clasp") :coding-system utf-8-unix)))

Adjust as necessary for Slime.

Note that adding "--dynamic-space-size" "4GB" to the sbcl list is useful for hungry projects like Trial.

Doom Emacs

As of 2025 January, you also need to manually disable sly-stepper and sly-quicklisp or they will interfere with the REPL starting:

(package! sly-stepper :disable t)
(package! sly-quicklisp :disable t)

Lem

Lem is built and configured in Common Lisp and so offers excellent support for it. To start a REPL with vend:

C-u M-x slime <RET> vend repl

And all your local systems will be available for loading.

Vim

Slimv is a port of Slime from Emacs that utilises Slime’s Swank backend server for a very similar experience to Emacs. However, unlike Emacs which supports multiple running Lisps, Slimv requires one standalone server that persists through Vim restarts.

If we want our dependencies in vendored/ to be visible to Slimv, we must start its server manually from our project directory:

> cd project/
> vend repl ecl --load /home/YOU/.vim/pack/common-lisp/start/slimv/slime/start-swank.lisp

Now, ,c (REPL Connect) within Vim will automatically find the running server, and you can load any system available in your project and in vendored/.

If you want to switch projects, you would need to quit the REPL server manually and restart it as above. You may also wish to set a shell alias or create a wrapper script for the long invocation shown above.

Usage

vend get

From the top-level directory of your project, simply vend get to fetch all dependencies. They will be stored in vendored/. From here, they are yours. You are free to inspect, edit, and remove them as you please.

> vend get
[vend] Downloading dependencies.
[vend] Fetching FN-MACRO
[vend] Fetching ARROW-MACROS
[vend] Fetching TRANSDUCERS
...
[vend] Done.

If during your usage of vend you discover a project that fails to resolve, please open an Issue.

vend repl

From the top-level directory of your project, vend repl opens a Lisp REPL while instructing ASDF to only look within this directory tree for .asd files.

> vend repl
This is SBCL 2.4.9, an implementation of ANSI Common Lisp.
> (asdf:load-system :transducers)
; Lots of compilation here.
T
>

By default, vend repl starts SBCL. You can easily override this:

> vend repl ecl
ECL (Embeddable Common-Lisp) 24.5.10
> (+ 1 1)

vend repl actually accepts any number of arguments, which is useful for adding additional settings for hungry projects like Trial:

> vend repl sbcl --dynamic-space-size 4GB

vend check

Since your dependencies are your code, you should care about what’s in there.

> vend check
DYNAMIC-CLASSES is deprecated.
  PGLOADER -> CL-MARKDOWN -> DYNAMIC-CLASSES
TRIVIAL-BACKTRACE is deprecated.
  PGLOADER -> TRIVIAL-BACKTRACE

Woops! And while Common Lisp has a culture of “done means done, not dead”, it’s still important to know what you’re getting yourself into.

vend eval

Run arbitrary Lisp code with all your dependencies available.

> vend eval "(asdf:load-system :transducers) (in-package :transducers) (princ (transduce #'pass #'first '(1)))"

You can pass as many s-expressions as you want within the argument string. Useful also for scripting; you can load a known system and run any function within it.

vend graph

After running vend get, you can inspect your full dependency graph via vend graph:

> vend graph

This produces a deps.dot file, which can be viewed directly with xdot:

> xdot deps.dot

Or you can render it into a static PNG to send around to your friends to brag about how few dependencies you’re using:

> cat deps.dot | dot -Tpng -o deps.png

In the case of vend, this produces an image like:

If the graph is too messy, you can “focus” it with an extra argument to vend graph:

vend graph lem

In the case of the large Lem project, this would display a graph of only the core application and not its test suites, etc.

vend init

If you don’t even have a project yet, vend init will create a simple skeleton for you.

> vend init foo

This generates:

foo
├── foo.asd
└── src
    └── package.lisp

vend search

Search the known systems via some term.

> vend search woo
woo    https://github.com/fukamachi/woo.git
wookie https://github.com/orthecreedence/wookie.git

vend test

Detect and run testable systems. Yields a proper error code to the terminal if failures are detected (good for CI!).

> vend test
[vend] Running tests.
...
;; Summary:
Passed:    68
Failed:     0
Skipped:    0

Pass an additional arg to switch compilers:

> vend test ecl

Setup

In order for the test suite to be detected properly, your systems must look something like this:

(defsystem "foo"
  :components ((:module "src" :components ((:file "package"))))
  :in-order-to ((test-op (test-op :foo/tests))))

(defsystem "foo/tests"
  :depends-on (:foo :parachute)
  :components ((:module "tests" :components ((:file "tests"))))
  :perform (test-op (op c) (symbol-call :parachute :test :foo/tests)))

Integrations

Library	Compatibility	Notes
Parachute	✅
Clunit2	✅
FiveAM	✅*	Test system must export `all-tests` function
Rove	❌	Usage of `package-inferred-system`

If no specific (or an unknown) testing library is used, vend will fall back to a naive (asdf:test-system :foo) call. However, this will not yield the correct error code to the terminal in the event of test failures.

If you desire integration with libraries not listed here, please open an Issue.

CI Integration

A Github Action is available that utilises vend. In the simplest case:

on:
  push:
    branches: [master]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    name: Unit Tests
    steps:
      - name: Clone the Project
        uses: actions/checkout@v4

      - name: Set up Common Lisp
        uses: fosskers/common-lisp@v1

      - name: Test
        run: |
          vend test

See its README for more information.

Coverage

vend does not cover all of what’s available on Quicklisp, but it does have significant enough coverage to resolve and compile a number of large, modern projects:

Resolves: Does vend get complete?
Compiles: Does (asdf:load-system :foo) within vend repl complete?
Tests: Does vend test or (asdf:test-system :foo) within vend repl pass?

Project	Resolves?	Compiles?	Tests?	Category	Notes
Alloy	✅	✅	✅	UI
April	✅	✅	-	Language
Clog	✅	✅	-	GUI
Coalton	✅	✅	✅	Language
GTK4	✅	✅	-	GUI
Kandria	✅	✅	-	Game
Lem	✅	❌	-	Editor	Usage of `package-inferred-system`
Lisp-stat	✅	✅	-	Math
McCLIM	✅	✅	✅	GUI
Mito	✅	✅	✅	Database
Nodgui	✅	✅	✅	GUI
Nyxt	✅	✅	✅	Browser
OCICL	✅	✅	-	Dev tool
Postmodern	✅	✅	✅	Database
Qlot	❌	-	-	Dev tool	Usage of `package-inferred-system`
Quicklisp	✅	❌	-	Dev tool	System is unloadable: (1) (2)
Radiance	✅	✅	✅	Web
Roswell	✅	❌	-	Dev tool	Requires `quicklisp` internally
Tinmop	✅	✅	-	Chat
Trial	✅	✅	✅	Gamedev	trial-assets manual setup for demos
Woo	✅	✅	❌	Web	Usage of `package-inferred-system`

If during your usage of vend you discover a project that fails to resolve, please open an Issue.

Compiler Compatibility

vend repl and eval work with the following compilers:

Compiler	Status	Notes
SBCL	✅
ECL	✅
ABCL	✅
Clasp	✅
CCL	✅
Clisp	✅	Doesn’t support PLN
Allegro	✅
LispWorks	❓

Historical implementations are not considered.

FAQ

How do I update dependencies?

The intent is that by vendoring, you’re taking responsibility for the “true shape” of your program. So, upgrading dependencies should always be a conscious choice, done for a specific reason. Therefore there is no “bulk update” button.

To update a single dependency, you can git pull it specifically. If you’ve already committed that dependency to your repo (as in the application case), you’re still able to:

rm -rf vendored/old-dep
vend get
rm -rf vendored/old-dep/.git/

But you’re discouraged from doing this habitually.

How can I build executables of my application?

See build.lisp for how vend is built, which uses ECL. For SBCL, consider adding:

(sb-ext:save-lisp-and-die #p"foobar"
                          :toplevel #'foobar:launch
                          :executable t
                          :compression t)

Consider also vend eval. For projects that have sufficient build commands already set in their .asd files, this may be enough:

vend eval "(asdf:load-system :waytemp) (asdf:make :waytemp)"

How do I refer to local dependencies?

vend get fetches dependencies it knows about via git, but sometimes you want to refer to a dependency that already exists somewhere else on your local machine. To trick vend, you can either:

Make a copy of the local project within vendored/.
Create a symlink inside vendored/ that refers to the local project.

Then, when running vend get it will see the folder you added and assume it had already fetched it via a previous call to vend get. Likewise, vend repl should “just work”.

Can I install new dependencies while `vend repl` is running?

Probably not. At least, vend assumes that Quicklisp doesn’t exist, and it tells ASDF to only look for systems in the current directory tree. It’s not clear what a call to (ql:quickload ...) would do in that case.

If you want new packages available to vend repl, you can:

Manually clone them into vendored/ (discouraged).
Add them to your .asd explicitly and call vend get.

Either way, it’s expected that you ensure that when a user freshly clones your repository, runs vend get, and then loads your system, everything should work.

Does this use git submodules?

No. Submodules need to be recloned by your users, which is a weaker long-term guarantee than true vendoring. Submodules are also a pain in the neck to manage. With vend, if you want to change and make new commits to vendored dependencies, you’re still free to do so.

Why ECL?

To avoid dependencies, vend uses a number of ECL-specific extensions. However, you’re free to use it to manage Lisp projects of any (modern) compiler. ECL typically produces very small binaries; in the case of vend it’s only a few hundred kilobytes, which eases distribution.

License

fosskers/vend

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 9

Uh oh!

Contributors 4

Uh oh!

Languages