--- layout: report year: "2021" month: "09" title: "Reproducible Builds in September 2021" draft: false date: 2021-10-06 16:56:05 --- [![]({{ "/images/reports/2021-09/reproducible-builds.png#right" | relative_url }})](https://reproducible-builds.org/) The goal behind "reproducible builds" is to ensure that no deliberate flaws have been introduced during compilation processes via promising or mandating that identical results are always generated from a given source. This allowing multiple third-parties to come to an agreement on whether a build was compromised or not by a system of distributed consensus. In these reports we outline the most important things that have been happening in the world of reproducible builds in the past month:
[![]({{ "/images/reports/2021-09/sigstore.png#right" | relative_url }})](https://www.sigstore.dev/) First mentioned in our [March 2021 report]({{ "/reports/2021-03/" | relative_url }}), [Martin Heinz](https://martinheinz.dev/) published two blog posts on [**sigstore**](https://www.sigstore.dev/), a project that endeavours to offer software signing as a "public good, [the] software-signing equivalent to [Let's Encrypt](https://letsencrypt.org/)". The two posts, the first entitled [*Sigstore: A Solution to Software Supply Chain Security*](https://martinheinz.dev/blog/55) outlines more about the project and justifies its existence: > Software signing is not a new problem, so there must be some solution already, right? Yes, but signing software and maintaining keys is very difficult especially for non-security folks and UX of existing tools such as PGP leave much to be desired. That's why we need something like sigstore - an easy to use software/toolset for signing software artifacts. The second post (titled [*Signing Software The Easy Way with Sigstore and Cosign*](https://martinheinz.dev/blog/56)) goes into some technical details of getting started.
There was an interesting thread in the [/r/Signal](https://www.reddit.com/r/signal/) subreddit that started from the observation that [*Signal's apk doesn't match with the source code*](https://www.reddit.com/r/signal/comments/pyj0uv/signals_apk_doesnt_match_with_the_source_code/): > Some time ago I checked Signal's reproducibility and it failed. I asked others to test in case I did something wrong, but nobody made any reports. Since then I tried to test the Google Play Store version of the apk against one I [compiled](https://github.com/signalapp/Signal-Android/tree/master/reproducible-builds) myself, and that doesn't match either.
[![]({{ "/images/reports/2021-09/bitcoinbinary.png#right" | relative_url }})](https://bitcoinbinary.org) [**BitcoinBinary.org**](https://bitcoinbinary.org) was announced this month, which aims to be a "repository of Reproducible Build Proofs for Bitcoin Projects": > Most users are not capable of building from source code themselves, but we can at least get them able enough to check signatures and shasums. When reputable people who can tell everyone they were able to reproduce the project's build, others at least have a secondary source of validation.
### Distribution work Frédéric Pierret [announced a new testing service](https://lists.reproducible-builds.org/pipermail/rb-general/2021-September/002386.html) at [**beta.tests.reproducible-builds.org**](https://beta.tests.reproducible-builds.org/), showing actual rebuilds of binaries distributed by both the Debian and Qubes distributions. [![]({{ "/images/reports/2021-09/debian.png#right" | relative_url }})](https://debian.org/) In Debian specifically, however, 51 reviews of Debian packages were added, 31 were updated and 31 were removed this month to [our database of classified issues](https://tests.reproducible-builds.org/debian/index_issues.html). As part of this, Chris Lamb refreshed a number of notes, including the [build_path_in_record_file_generated_by_pybuild_flit_plugin](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/3f309266) issue. Elsewhere in Debian, Roland Clobus posted his [*Fourth status update about reproducible live-build ISO images in Jenkins*](https://lists.reproducible-builds.org/pipermail/rb-general/2021-September/002387.html) to our mailing list, which mentions (amongst other things) that: > * All major configurations are [still built regularly](https://jenkins.debian.net/view/live/) using live-build and bullseye. > * All major configurations are reproducible now; Jenkins is green. > * I've worked around the issue for the Cinnamon image. > * [The patch](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993444) was accepted and released within a few hours. > * My main focus for the last month was on the live-build tool itself. > * It will properly [use the proxy for all HTTP traffic](https://salsa.debian.org/live-team/live-build/-/merge_requests/252). Related to this, there was continuing discussion on how to [embed/encode the build metadata for the Debian "live" images](https://lists.reproducible-builds.org/pipermail/rb-general/2021-September/002359.html) which were being worked on by Roland Clobus.
[![]({{ "/images/reports/2021-09/alpine.png#right" | relative_url }})](https://ariadne.space/2021/06/04/a-slightly-delayed-monthly-status-update/) [Ariadne Conill](https://ariadne.space/) published [another detailed blog post](https://ariadne.space/2021/09/07/bits-relating-to-alpine-security-initiatives-in-august/) related to various security initiatives within the [Alpine](https://alpinelinux.org/) Linux distribution. After summarising some conventional security work being done (eg. with [sudo](https://www.sudo.ws/) and the release of [OpenSSH](https://www.openssh.com/) version 3.0), Ariadne included another section on reproducible builds: "The main blocker [was] determining what to do about storing the build metadata so that a build environment can be recreated precisely". Finally, Bernhard M. Wiedemann posted his [monthly reproducible builds status report](https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/message/JGVWCDTZJS2HXTXVADS3ZKO3CIHXZZIN/).
### Community news [![]({{ "/images/reports/2021-09/website.png#right" | relative_url }})](https://reproducible-builds.org/) On [our website](https://reproducible-builds.org/) this month, Bernhard M. Wiedemann fixed some broken links [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/104a68e)] and Holger Levsen made a number of changes to the [*Who is Involved?*]({{ "/who/" | relative_url }}) page [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/9f43eb7)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/6b2a913)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/32b3aba)]. On our mailing list, Magnus Ihse Bursie started a thread with the subject [*Reproducible builds on Java*](https://lists.reproducible-builds.org/pipermail/rb-general/2021-September/thread.html#2374), which begins as follows: > I'm working for Oracle in the Build Group for [OpenJDK](https://openjdk.java.net/groups/build/) which is primary responsible for creating a built artifact of the OpenJDK source code. [...] For the last few years, we have worked on a low-effort, background-style project to make the build of OpenJDK itself building reproducible. We've come far, but there are still issues I'd like to address. [[...](https://lists.reproducible-builds.org/pipermail/rb-general/2021-September/002374.html)]
### [*diffoscope*](https://diffoscope.org) [![]({{ "/images/reports/2021-09/diffoscope.svg#right" | relative_url }})](https://diffoscope.org) [*diffoscope*](https://diffoscope.org) is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, [Chris Lamb](https://chris-lamb.co.uk) prepared and uploaded versions `183`, `184` and `185` as well as performed significant triaging of merge requests and other issues in addition to making the following changes: * New features: * Support a newer format version of the [R language](https://www.r-project.org/)'s `.rds` files. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/ac1bcb6a)] * Update tests for [OCaml](https://ocaml.org/) 4.12. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/1fcc0d51)] * Add a missing `format_class` import. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/73f49430)] * Bug fixes: * Don't call `close_archive` when garbage collecting `Archive` instances, unless `open_archive` definitely returned successfully. This prevents, for example, an `AttributeError` where `PGPContainer`'s cleanup routines were rightfully assuming that its temporary directory had actually been created. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/9007e009)] * Fix (and test) the comparison of [R language](https://www.r-project.org/)'s `.rdb` files after refactoring temporary directory handling. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/622d1cac)] * Ensure that "RPM archives" exists in the Debian package description, regardless of whether `python3-rpm` is installed or not at build time. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/fa530947)] * Codebase improvements: * Use our `assert_diff` routine in `tests/comparators/test_rdata.py`. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/d12af4cc)] * Move `diffoscope.versions` to `diffoscope.tests.utils.versions`. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/2fae7e85)] * Reformat a number of modules with Black. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/a33cf81e)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/e6024b3b)] However, the following changes were also made: * Mattia Rizzolo: * Fix an autopkgtest caused by the `androguard` module not being in the (expected) `python3-androguard` Debian package. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/7ae076c7)] * Appease a `shellcheck` warning in `debian/tests/control.sh`. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/46c468cb)] * Ignore a warning from `h5py` in our tests that doesn't concern us. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/e998eb9d)] * Drop a trailing `.1` from the `Standards-Version` field as it's required. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/080813f3)] * Zbigniew Jędrzejewski-Szmek: * Stop using the deprecated `distutils.spawn.find_executable` utility. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/fef2375b)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/fce00cf6)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/4c3109cb)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/953a599c)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/beebe3fd)] * Adjust an LLVM-related test for LLVM version 13. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/258d09a4)] * Update invocations of `llvm-objdump`. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/b2b1f731)] * Adjust a test with a one-byte text file for `file` version 5.40. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/d5707121)] And, finally, Benjamin Peterson added a `--diff-context` option to control unified diff context size [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/1ee45568)] and Jean-Romain Garnier fixed the Macho comparator for architectures other than `x86-64` [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/4ea2175c)].
### Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: * Bernhard M. Wiedemann: * [`gtk4`](https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3929) (date-related issue) * [`build-compare`](https://github.com/openSUSE/build-compare/pull/42) (random tempfile problem) * [`itinerary`](https://bugs.kde.org/show_bug.cgi?id=442821) (time-related build failure) * Chris Lamb: * [#993950](https://bugs.debian.org/993950) filed against [`lcalc`](https://tracker.debian.org/pkg/lcalc). * [#993952](https://bugs.debian.org/993952) filed against [`htscodecs`](https://tracker.debian.org/pkg/htscodecs). * [#994123](https://bugs.debian.org/994123) filed against [`osdlyrics`](https://tracker.debian.org/pkg/osdlyrics). * [#994976](https://bugs.debian.org/994976) filed against [`xtermcontrol`](https://tracker.debian.org/pkg/xtermcontrol). * [#994978](https://bugs.debian.org/994978) filed against [`rust-insta`](https://tracker.debian.org/pkg/rust-insta). * [#994979](https://bugs.debian.org/994979) filed against [`python-tomli`](https://tracker.debian.org/pkg/python-tomli). * [#995258](https://bugs.debian.org/995258) filed against [`python-pairix`](https://tracker.debian.org/pkg/python-pairix). * [#995259](https://bugs.debian.org/995259) filed against [`python-pybedtools`](https://tracker.debian.org/pkg/python-pybedtools) ([forwarded upstream](https://github.com/daler/pybedtools/pull/349)). * Simon McVittie: * [#993673](https://bugs.debian.org/993673) filed against [`clazy`](https://tracker.debian.org/pkg/clazy). * [#993674](https://bugs.debian.org/993674) filed against [`ga`](https://tracker.debian.org/pkg/ga). * [#993675](https://bugs.debian.org/993675) filed against [`opensmtpd`](https://tracker.debian.org/pkg/opensmtpd). * [#993681](https://bugs.debian.org/993681) filed against [`sphinxcontrib-programoutput`](https://tracker.debian.org/pkg/sphinxcontrib-programoutput). * Vagrant Cascadian: * [#995092](https://bugs.debian.org/995092) filed against [`guile-3.0`](https://tracker.debian.org/pkg/guile-3.0). * [#995131](https://bugs.debian.org/995131) filed against [`nut`](https://tracker.debian.org/pkg/nut). * [#995143](https://bugs.debian.org/995143) filed against [`xdmf`](https://tracker.debian.org/pkg/xdmf). * [#995145](https://bugs.debian.org/995145) filed against [`cyrus-sasl2`](https://tracker.debian.org/pkg/cyrus-sasl2). * [#995211](https://bugs.debian.org/995211) filed against [`libbrahe`](https://tracker.debian.org/pkg/libbrahe). * [#995217](https://bugs.debian.org/995217) filed against [`maildirsync`](https://tracker.debian.org/pkg/maildirsync). * [#995401](https://bugs.debian.org/995401) filed against [`canna`](https://tracker.debian.org/pkg/canna). * [#990246](https://bugs.debian.org/990246) filed against [`vlc`](https://tracker.debian.org/pkg/canna) (forwarded upstream [[...](https://code.videolan.org/videolan/vlc/-/merge_requests/698)][[...](https://code.videolan.org/videolan/vlc/-/issues/26035)])
#### Testing framework [![]({{ "/images/reports/2021-09/testframework.png#right" | relative_url }})](https://tests.reproducible-builds.org/) The Reproducible Builds project runs a testing framework at [tests.reproducible-builds.org](https://tests.reproducible-builds.org), to check packages and other artifacts for reproducibility. This month, the following changes were made: * Holger Levsen: * Drop my package rebuilder prototype as it's not useful anymore. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/3814aa00)] * Schedule old packages in Debian *bookworm*. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/38a659c3)] * Stop scheduling packages for Debian *buster*. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/99840169)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/dc2695c7)] * Don't include PostgreSQL debug output in package lists. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/8658f0dc)] * Detect Python library mismatches during build in the node health check. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ae3a3e30)] * Update a note on updating the FreeBSD system. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d8fe3916)] * Mattia Rizzolo: * Silence a warning from Git. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/60829d42)] * Update a setting to reflect that Debian *bookworm* is the new testing. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/846ec7aa)] * Upgrade the PostgreSQL database to version 13. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/e3658b54)] * Roland Clobus (Debian "live" image generation): * Workaround non-reproducible config files in the `libxml-sax-perl` package. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a9fdeb39)] * Use the new DNS for the 'snapshot' service. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/2afc5833)] * Vagrant Cascadian: * Also note that the `armhf` architecture also systematically varies by the kernel. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/2fcfdc98)]
#### Contributing If you are interested in contributing to the Reproducible Builds project, please visit our [*Contribute*](https://reproducible-builds.org/contribute/) page on our website. However, you can get in touch with us via: * IRC: `#reproducible-builds` on `irc.oftc.net`. * Twitter: [@ReproBuilds](https://twitter.com/ReproBuilds) * Mailing list: [`rb-general@lists.reproducible-builds.org`](https://lists.reproducible-builds.org/listinfo/rb-general)