lisp-modules: remove pkgs/development/lisp-modules-new-obsolete

imported.nix linguist-vendored

result

*.sqlite

*.fasl

## The API

This page documents the Nix API of nix-cl.

## Overview

The core API functions are `build-asdf-system` and

`lispWithPackagesInternal`.

They are considered more low-level that the rest of the API, which

builds on top of them to provide a more convenient interface with sane

defaults.

The higher-level API provides a lot of pre-configured packages,

including all of Quicklisp, and consists of the functions:

- `lispPackagesFor`

- `lispWithPackages`

Finally, there are functions that provide pre-defined Lisps, for

people who don't need to customize that:

- `abclPackages`, `eclPackages`, `cclPackages`, `claspPackages`, `sbclPackages`

- `abclWithPackages`, `eclWithPackages`, `cclWithPackages`, `claspWithPackages`, `sbclWithPackages`

The following is an attempt to document all of this.

## Packaging systems - `build-asdf-system`

Packages are declared using `build-asdf-system`. This function takes

the following arguments and returns a `derivation`.

#### Required arguments

##### `pname`

Name of the package/library

##### `version`

Version of the package/library

##### `src`

Source of the package/library (`fetchzip`, `fetchgit`, `fetchhg` etc.)

##### `lisp`

This command must load the provided file (`$buildScript`) then exit

immediately. For example, SBCL's --script flag does just that.

#### Optional arguments

##### `patches ? []`

Patches to apply to the source code before compiling it. This is a

list of files.

##### `nativeLibs ? []`

Native libraries, will be appended to the library

path. (`pkgs.openssl` etc.)

##### `javaLibs ? []`

Java libraries for ABCL, will be appended to the class path.

##### `lispLibs ? []`

Lisp dependencies These must themselves be packages built with

`build-asdf-system`

##### `systems ? [ pname ]`

Some libraries have multiple systems under one project, for example,

[cffi] has `cffi-grovel`, `cffi-toolchain` etc.  By default, only the

`pname` system is build.

`.asd's` not listed in `systems` are removed before saving the library

to the Nix store. This prevents ASDF from referring to uncompiled

systems on run time.

Also useful when the `pname` is different than the system name, such

as when using [reverse domain naming]. (see `jzon` ->

`com.inuoe.jzon`)

[cffi]: https://cffi.common-lisp.dev/

[reverse domain naming]: https://en.wikipedia.org/wiki/Reverse_domain_name_notation

##### `asds ? systems`

The .asd files that this package provides. By default, same as

`systems`.

#### Return value

A `derivation` that, when built, contains the sources and pre-compiled

FASL files (Lisp implementation dependent) alongside any other

artifacts generated during compilation.

#### Example

[bordeaux-threads.nix] contains a simple example of packaging

`alexandria` and `bordeaux-threads`.

[bordeaux-threads.nix]: /examples/bordeaux-threads.nix

## Building a Lisp with packages: `lispWithPackagesInternal`

Generators of Lisps configured to be able to `asdf:load-system`

pre-compiled libraries on run-time are built with

`lispWithPackagesInternal`.

#### Required Arguments

##### `clpkgs`

An attribute set of `derivation`s returned by `build-asdf-system`

#### Return value

`lispWithPackagesInternal` returns a function that takes one argument:

a function `(lambda (clpkgs) packages)`, that, given a set of

packages, returns a list of package `derivation`s to be included in

the closure.

#### Example

The [sbcl-with-bt.nix] example creates a runnable Lisp where the

`bordeaux-threads` defined in the previous section is precompiled and

loadable via `asdf:load-system`:

[sbcl-with-bt.nix]: /examples/sbcl-with-bt.nix

## Reusing pre-packaged Lisp libraries: `lispPackagesFor`

`lispPackagesFor` is a higher level version of

`lispPackagesForInternal`: it only takes one argument - a Lisp command

to use for compiling packages. It then provides a bunch of ready to

use packages.

#### Required Arguments

##### `lisp`

The Lisp command to use in calls to `build-asdf-system` while building

the library-provided Lisp package declarations.

#### Return value

A set of packages built with `build-asdf-system`.

#### Example

The [abcl-package-set.nix] example generates a set of thousands of packages for ABCL.

[abcl-package-set.nix]: /examples/abcl-package-set.nix

## Reusing pre-packaged Lisp libraries, part 2: `lispWithPackages`

This is simply a helper function to avoid having to call

`lispPackagesFor` if all you want is a Lisp-with-packages wrapper.

#### Required Arguments

##### `lisp`

The Lisp command to pass to `lispPackagesFor` in order for it to

generate a package set. That set is then passed to

`lispWithPackagesInternal`.

#### Return value

A Lisp-with-packages function (see sections above).

#### Example

The [abcl-with-packages.nix] example creates an `abclWithPackages` function.

[abcl-with-packages.nix]: /examples/abcl-with-packages.nix

## Using the default Lisp implementations

This is the easiest way to get going with `nix-cl` in general. Choose

the CL implementation of interest and a set of libraries, and get a

lisp-with-packages wrapper with those libraries pre-compiled.

#### `abclPackages`, `eclPackages`, `cclPackages`, `claspPackages`, `sbclPackages`

Ready to use package sets.

#### `abclWithPackages`, `eclWithPackages`, `cclWithPackages`, `claspWithPackages`, `sbclWithPackages`

Ready to use wrapper generators.

#### Example

For example, to open a shell with SBCL + hunchentoot + sqlite in PATH:

```

nix-shell -p 'with import ./. {}; sbclWithPackages (ps: [ ps.hunchentoot ps.sqlite ])'

```

## Use cases

This page lists some possible use cases for nix-cl.

## Pinning down the exact commits of libraries

Sometimes, a bug is fixed upstream but is not yet available in package

repositories such as Quicklisp or Ultralisp. The users have to wait

for the repository maintainer to update it, or download and compile

the patched sources themselves.

This is a manual and hard to reproduce process. By leveraging Nix,

users of `nix-cl` can essentially "run their own package repository",

written as Nix code, with all the benefits of that (shareability,

cacheability, reproducibility, version-controllable etc.)

## Modifying libraries with patches

Other times, a bug in a library is not fixed upstream, but you fixed

it yourself. Or, you would like a change to the internals that the

maintainers don't like.

Sure, you could fork the code or maintain patches manually, but that

becomes hard to manage with a lot of patches. It also doesn't have the

benefits mentioned in the previous section.

`nix-cl` provides a way of applying version-controlled patches to any

package.

## Using libraries not available in repositories

There are useful and working libraries out there, that are nonetheless

unavailable to users of package managers such as Quicklisp or

Ultralisp. Two real-world examples are [jzon] and [cl-tar].

`nix-cl` is not tied to any particular package source: instead,

packages are written as a Nix expression, which can be done manually

or generated/imported.

This frees the user to have any package they want, and not be

constrained by a central repository.

## Reproducible environments

The usual way to develop a project involves several steps, such as:

1. Installing a Lisp implementation

2. Installing a package manager

3. Installing the chosen libraries

This is not necessarily reproducible. It's unlikely to come back a

year later and develop the project using the exact same versions of

the dependencies.

Things can break between attempts at different points in time. The

repository could have updated versions in the meantime. The source

tarballs could become unreachable.

With `nix-cl` you can have your own binary cache for Lisp libraries

and not be affected by downtime of other central repositories.

## Testing across CL implementations

One can manually download different Lisp implementations and run tests

of a package. This works well in most cases, but it is limited in how

you can tweak the software. Some practical examples are:

- Statically compiling [zlib] into [SBCL]

- Building SBCL with the `--fancy` flag

- Compiling [ECL] as a static library

These are usually hard to do manually, unless you have the necessary

compilers already configured. These combinations are usually not

available from package managers as well.

With Nix it's easier, because it will set up the build environment

automatically. It could be useful to, for example:

- Test against all possible compiler flag combinations

- Libc versions (ECL)

- JDK versions ([ABCL])

[zlib]: https://zlib.net

[SBCL]: https://sbcl.org

[ECL]: https://ecl.common-lisp.dev/

[Ultralisp]: https://ultralisp.org/

[jzon]: https://github.com/Zulu-Inuoe/jzon

[cl-tar]: https://gitlab.common-lisp.net/cl-tar/cl-tar

[bootstrap tools]: https://github.com/NixOS/nixpkgs/tree/master/pkgs/stdenv/linux/bootstrap-files

[nixpkgs]: https://github.com/NixOS/nixpkgs

## Windows note

Note that all of this still only applies to Unix systems - primarily because Nix doesn't work on Windows.

If you have an idea how to port some of the functionality to Windows, get in touch.

## Importing package definitions from Quicklisp

This page documents how to import packages from Quicklisp.

## Nix dumper

Run:

```

$ nix-shell

$ sbcl --script ql-import.lisp

```

This command runs a program that dumps a `imported.nix` file

containing Nix expressions for all packages in Quicklisp. They will be

automatically picked up by the `lispPackagesFor` and

`lispWithPackages` API functions.

It also creates a 'packages.sqlite' file. It's used during the

generation of the 'imported.nix' file and can be safely removed. It

contains the full information of Quicklisp packages, so you can use it

to query the dependency graphs using SQL, if you're interested.

## Tarball hashes

The Nix dumper program will re-use hashes from "imported.nix" if it

detects that it's being run for the first time. This saves a lot of

bandwidth by not having to download each tarball again.

But when upgrading the Quicklisp release URL, this can take a while

because it needs to fetch the source code of each new system to

compute its SHA256 hash. This is because Quicklisp only provides a

SHA1 , and Nix's `builtins.fetchTarball` requires a SHA256.

Later on, the hashes are cached in `packages.sqlite`, and are reused

in subsequent invocations. Therefore you might want to keep the

'packages.sqlite' file around if you'd like to keep hashes of

historical Quicklisp tarballs, for example for archival purposes.

## Choosing a Quicklisp release

Quicklisp release url's are currently hard-coded and can be changed

directly in the source code. See the `import` directory.

## Native and Java libraries

At the moment, native and Java libraries need to be added manually to

imported systems in `ql.nix` on an as-needed basis.

## Dependencies from packages.nix

Also worth noting is that systems imported from Quicklisp will prefer

packages from `packages.nix` as dependencies, so that custom versions

can be provided or broken versions replaced.