156 lines
6.2 KiB
ReStructuredText
156 lines
6.2 KiB
ReStructuredText
Building dependencies in CI
|
|
===========================
|
|
|
|
**Status as of 2024/Sep/11:** implemented in :pr:`896`.
|
|
Non-Rust dependencies are built from git tags and installed in the CI
|
|
container image. See the ``ci/build-dependencies.sh`` script that
|
|
does this.
|
|
|
|
Preamble
|
|
--------
|
|
|
|
Until Sep/2023, librsvg's CI has worked by building a container image
|
|
out of a snapshot of openSUSE Tumbleweed, a rolling release, that has
|
|
all of librsvg's dependencies in it. What this means for runtime
|
|
dependencies like Cairo, Pango, etc. is that they come in a
|
|
"reasonably recent" version, but they are not pinned, and can change
|
|
any time that the rolling release decides to update them.
|
|
|
|
This is not entirely terrible: librsvg's run-time dependencies are all
|
|
stable, mature libraries that don't change very much. From librsvg's
|
|
viewpoint, the only trouble comes in situations like these:
|
|
|
|
* A library involved in text rendering changes something, and text
|
|
output changes slightly. The test suite breaks as a result, since
|
|
it assumes exact rendering based on reference images.
|
|
|
|
* Less often, a library involved in Bézier path rendering changes
|
|
something, and parts of the test suite break because antialiasing is
|
|
not the same as before.
|
|
|
|
* Someone runs librsvg's test suite with a different set of dependency
|
|
versions than the test suite assumes; a bunch of tests fail for
|
|
them, and they file bugs that are not very useful. In the best
|
|
case, they are using newer libraries and the bug reports let me (the
|
|
maintainer) know that I'll need to re-generate the test suite's
|
|
images soon. In the worst case, I just close those bugs since they
|
|
don't provide useful information.
|
|
|
|
And probably the biggest problem of all: I am never sure of exactly
|
|
what set of versions are okay for the test suite to work; I just
|
|
regenerate test reference files when they break after a CI update.
|
|
|
|
Pinning dependency versions
|
|
---------------------------
|
|
|
|
Librsvg's CI needs to be able to build the library's dependencies in a
|
|
custom fashion, without necessarily assuming that the dependencies
|
|
come from system libraries. If we have that ability, then we can do a
|
|
few interesting things:
|
|
|
|
* Pin the versions of dependencies to a particular set, for example,
|
|
one that corresponds to a certain GNOME release. This is important
|
|
to keep CI working for old branches, so security patches are easy to
|
|
build.
|
|
|
|
* Compile the dependencies with a particular set of compiler options.
|
|
For example, for fuzzing, dependencies should be built with
|
|
sanitizers like asan/ubsan. For deep debugging, it would be nice to
|
|
have all the dependencies built in debug mode. For performance
|
|
testing, compile all the dependencies in release mode, etc.
|
|
|
|
Which dependencies? These ones; this list is already sorted in the
|
|
correct build order:
|
|
|
|
* glib
|
|
* gobject-introspection
|
|
* freetype2
|
|
* fontconfig
|
|
* cairo
|
|
* harfbuzz
|
|
* pango
|
|
* libxml2
|
|
* gdk-pixbuf
|
|
|
|
How do we achieve that?
|
|
-----------------------
|
|
|
|
**Option 1:** There is a script ``ci/build-dependencies.sh`` that can
|
|
already build librsvg's dependencies pinned to particular git tags.
|
|
In theory one can pass environment variables to change compiler
|
|
options; the script may need some tweaks to change the meson or
|
|
autotools invocations. The script builds and installs the
|
|
dependencies to a given prefix, and assumes that
|
|
``PATH/LD_LIBRARY_PATH/PKG_CONFIG_PATH`` are tweaked to use things
|
|
from that prefix.
|
|
|
|
**Option 2:** Alternatively, we can outsource the problem and use
|
|
GNOME's BuildStream images. These correspond to specific releases of
|
|
the GNOME platform libraries, including librsvg's dependencies... and
|
|
librsvg itself, as it *is* a platform library. One must be a little
|
|
careful to keep the test suite from using the "system's" librsvg in
|
|
that case, but this is not a huge problem.
|
|
|
|
Implementation notes - building dependencies explicitly
|
|
-------------------------------------------------------
|
|
|
|
As of Sep/2023 the CI builds three main images:
|
|
|
|
* An image with the MSRV, just used to test the promise that the MSRV
|
|
can still build the library and its Rust dependencies.
|
|
|
|
* An image with a recent, stable Rust compiler - the main image used
|
|
for most jobs, and what I use for local development.
|
|
|
|
* An image with the nightly compiler. This is not updated frequently
|
|
since the whole CI doesn't regenerate its images nightly.
|
|
|
|
* (Other images for other architectures or distros, not in scope for
|
|
this discussion.)
|
|
|
|
All images have whatever RPM versions are available in openSUSE for
|
|
librsvg's runtime dependencies.
|
|
|
|
Proposed change:
|
|
|
|
* Keep a single image, pre-populated with ``rustup toolchain install
|
|
<version>`` for the MSRV, the stable compiler, and nightly. Select
|
|
the compiler version on a per-job basis; hopefully this will be fast
|
|
since ``toolchain`` should cache them.
|
|
|
|
* In that single image, keep pre-built sets of runtime dependencies
|
|
(e.g. libraries built from git tags) in at least two configurations:
|
|
the minimum supported versions, and the latest stable ones. These
|
|
configurations can live in different prefixes,
|
|
e.g. ``/usr/local/librsvg-minimum`` and
|
|
``/usr/local/librsvg-stable``.
|
|
|
|
The idea is to ensure that the minimum-supported everything (rustc and
|
|
dependencies) actually works, in addition to testing the recent-stable
|
|
stuff.
|
|
|
|
Keeping everything in a single container image (versions of the Rust
|
|
toolchain, and sets of dependencies installed to different prefixes)
|
|
is just an optimization to build and maintain a single image, instead
|
|
of the three we have right now. If we determine that selecting
|
|
rustc/deps makes jobs too slow, we can go back to producing multiple
|
|
images.
|
|
|
|
Implementation note - BuildStream to test nightly dependencies
|
|
--------------------------------------------------------------
|
|
|
|
FIXME: alatiera's work goes here.
|
|
|
|
Other architectures, other distros
|
|
----------------------------------
|
|
|
|
I'd like to keep an ``aarch64`` image; it has let us notice
|
|
peculiarities like the different signedness of ``libc::c_char``. It
|
|
doesn't need the minimum/stable/nightly distinction; we can keep it
|
|
stable-only as currently.
|
|
|
|
I think we can drop the Fedora image. It still runs Fedora 36, is
|
|
seldom updated, and I don't pay attention to it. I'd rather make it
|
|
possible to have explicit git versions of dependencies.
|
|
|