--- layout: report year: "2023" month: "05" title: "Reproducible Builds in May 2023" draft: false date: 2023-06-05 17:35:40 --- **Welcome to the May 2023 report from the [Reproducible Builds](https://reproducible-builds.org) project** {: .lead} [![]({{ "/images/reports/2023-05/reproducible-builds.png#right" | relative_url }})](https://reproducible-builds.org/) In our reports, we outline the most important things that we have been up to over the past month. As always, if you are interested in contributing to the project, please visit our [*Contribute*]({{ "/contribute/" | relative_url }}) page on our website. --- [![]({{ "/images/reports/2023-05/hamburg.png#left" | relative_url }})](https://wiki.debian.org/DebianEvents/de/2023/DebianReunionHamburg) Holger Levsen gave a talk at the 2023 edition of the [Debian Reunion Hamburg](https://wiki.debian.org/DebianEvents/de/2023/DebianReunionHamburg), a semi-informal meetup of Debian-related people in northern Germany. The [slides](https://reproducible-builds.org/_lfs/presentations/2023-05-27-R-B-the-first-10-years/#/) are available online. --- [![]({{ "/images/reports/2023-05/foss-north.jpg#right" | relative_url }})](https://fsfe.org/news/2023/news-20230509-01.en.html) In April, Holger Levsen gave a talk at [foss-north 2023](https://foss-north.se/2023) titled [*Reproducible Builds, the first ten years*](https://foss-north.se/2023/speakers-and-talks.html#hlevsen). Last month, however, Holger's talk was covered in [a round-up of the conference on the Free Software Foundation Europe (FSFE) blog](https://fsfe.org/news/2023/news-20230509-01.en.html). --- [![]({{ "/images/reports/2023-05/ieeepaper.png#right" | relative_url }})](https://ieeexplore.ieee.org/document/9240695) Pronnoy Goswami, Saksham Gupta, Zhiyuan Li, Na Meng and Daphne Yao from [Virginia Tech](https://www.vt.edu/) published a paper investigating the [*Reproducibility of NPM Packages*](https://ieeexplore.ieee.org/document/9240695). The abstract includes: > When using open-source NPM packages, most developers download prebuilt packages on npmjs.com instead of building those packages from available source, and implicitly trust the downloaded packages. However, it is unknown whether the blindly trusted prebuilt NPM packages are reproducible (i.e., whether there is always a verifiable path from source code to any published NPM package). […] We downloaded versions/releases of 226 most popularly used NPM packages and then built each version with the available source on GitHub. Next, we applied a differencing tool to compare the versions we built against versions downloaded from NPM, and further inspected any reported difference. The paper reports that "among the 3,390 versions of the 226 packages, only 2,087 versions are reproducible," and furthermore that multiple factors contribute to the non-reproducibility including "flexible versioning information in package.json file and the divergent behaviors between distinct versions of tools used in the build process." The paper concludes with "insights for future verifiable build procedures." Unfortunately, a PDF is not available publically yet, but a [Digital Object Identifier](https://en.wikipedia.org/wiki/Digital_object_identifier) (DOI) is available on the [paper's IEEE page](https://ieeexplore.ieee.org/document/9240695). --- [![]({{ "/images/reports/2023-05/arXiv-2305.14157.png#left" | relative_url }})](https://arxiv.org/abs/2305.14157) Elsewhere in academia, Betul Gokkaya, Leonardo Aniello and Basel Halak of the [School of Electronics and Computer Science](https://www.southampton.ac.uk/about/faculties-schools-departments/school-of-electronics-and-computer-science) at the [University of Southampton](https://www.southampton.ac.uk/) published a new paper containing a broad overview of attacks and comprehensive risk assessment for software supply chain security. Their paper, titled [*Software supply chain: review of attacks, risk assessment strategies and security controls*](https://arxiv.org/abs/2305.14157), analyses the most common software supply-chain attacks by providing the latest trend of analyzed attack, and identifies the security risks for open-source and third-party software supply chains. Furthermore, their study "introduces unique security controls to mitigate analyzed cyber-attacks and risks by linking them with real-life security incidence and attacks". ([arXiv.org](https://arxiv.org/abs/2305.14157), [PDF](https://arxiv.org/pdf/2305.14157.pdf)) --- [![]({{ "/images/reports/2023-05/nixos.png#right" | relative_url }})](https://reproducible.nixos.org) [NixOS](https://nixos.org/) is now tracking two new reports at [*reproducible.nixos.org*](https://reproducible.nixos.org/). Aside from the collection of build-time dependencies of the minimal and Gnome installation ISOs, this page now also contains reports that are restricted to the artifacts that make it into the image. The minimal ISO is currently reproducible except for Python 3.10, which hopefully will be resolved with the coming update to Python version 3.11. --- On [our *rb-general* mailing list](https://lists.reproducible-builds.org/listinfo/rb-general/) this month: [![]({{ "/images/reports/2023-05/semantically.png#right" | relative_url }})](https://lists.reproducible-builds.org/pipermail/rb-general/2023-May/thread.html#2968) David A. Wheeler [started a thread](https://lists.reproducible-builds.org/pipermail/rb-general/2023-May/002968.html) noting that the [OSSGadget project's oss-reproducible tool](https://github.com/microsoft/OSSGadget/tree/main/src/oss-reproducible/README.md) was measuring something related to but not the same as reproducible builds. Initially they had adopted the term “semantically reproducible build” term for what it measured, which they defined as being "if its build results can be either recreated exactly (a bit for bit reproducible build), or if the differences between the release package and a rebuilt package are not expected to produce functional differences in normal cases." This [generated a significant number of replies](https://lists.reproducible-builds.org/pipermail/rb-general/2023-May/thread.html#2968), and several were concerned that people might confuse what they were measuring with "reproducible builds". After discussion, the OSSGadget developers decided to [switch to the term "semantically equivalent"](https://github.com/microsoft/OSSGadget/issues/426) for what they measured in order to reduce the risk of confusion. Vagrant Cascadian (*vagrantc*) posted an update about [GCC, binutils, and Debian's build-essential set](https://lists.reproducible-builds.org/pipermail/rb-general/2023-May/002961.html) with "some progress, some hope, and I daresay, some fears…". Lastly, *kpcyrd* asked a question about [building a reproducible Linux kernel package for Arch Linux](https://lists.reproducible-builds.org/pipermail/rb-general/2023-May/002991.html) ([answered](https://lists.reproducible-builds.org/pipermail/rb-general/2023-May/002992.html) by Arnout Engelen). In the same, thread David A. Wheeler pointed out that the [Linux Kernel documentation has a chapter about Reproducible kernel builds](https://docs.kernel.org/kbuild/reproducible-builds.html) now as well. --- [![]({{ "/images/reports/2023-05/debian.png#right" | relative_url }})](https://debian.org/) In Debian this month, nine reviews of Debian packages were added, 20 were updated and 6 were removed this month, all adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). In addition, Vagrant Cascadian added a link to the source code causing various `ecbuild` issues. [[...](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/a5e2eb23)] --- [![]({{ "/images/reports/2023-05/fdroid.png#right" | relative_url }})](https://f-droid.org/) The [F-Droid](https://f-droid.org/) project updated its Inclusion How-To with a [new section](https://f-droid.org/docs/Inclusion_How-To/#reproducible-builds) explaining why it considers reproducible builds to be best practice and hopes developers will support the team's efforts to make as many (new) apps reproducible as it reasonably can. --- [![]({{ "/images/reports/2023-05/diffoscope.png#right" | relative_url }})](https://diffoscope.org) In [*diffoscope*](https://diffoscope.org) development this month, version `242` was [uploaded to Debian unstable](https://tracker.debian.org/news/1430496/accepted-diffoscope-242-source-into-unstable/) by Chris Lamb who also made the following changes: * If `binwalk` is not available, ensure the user knows they may be missing more info. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/45ac81ba)] * Factor out generating a human-readable comment when missing a Python module. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/55742394)] In addition, Mattia Rizzolo documented how to (re)-produce a binary blob in the code [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/8d61a7b4)] and Vagrant Cascadian updated the version of *diffoscope* in [GNU Guix](https://guix.gnu.org/) to 242 [[...](https://issues.guix.gnu.org/63339)]. --- [*reprotest*](https://salsa.debian.org/reproducible-builds/reprotest) is our tool for building the same source code twice in different environments and then checking the binaries produced by each build for any differences. This month, Holger Levsen uploaded versions `0.7.24` and `0.7.25` to Debian *unstable* which added support for [Tox](https://tox.wiki/en/latest/) versions 3 and 4 with help from Vagrant Cascadian [[...](https://salsa.debian.org/reproducible-builds/reprotest/commit/3a95b6b)][[...](https://salsa.debian.org/reproducible-builds/reprotest/commit/4bec4e2)][[...](https://salsa.debian.org/reproducible-builds/reprotest/commit/8ee881b)] --- ## Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: * Alper Nebi Yasak: * [#1035375](https://bugs.debian.org/1035375) filed against [`mtools`](https://tracker.debian.org/pkg/mtools) ([forwarded upstream](https://lists.gnu.org/archive/html/info-mtools/2023-05/msg00000.html)). * Bernhard M. Wiedemann: * [`snobol4`](https://build.opensuse.org/request/show/1089591) (datetime-related issue) * [`python310`](https://bugzilla.opensuse.org/show_bug.cgi?id=1211765) (`.pyc` ordering issue) * [`xmvn`](https://github.com/fedora-java/xmvn/issues/289) (randomised issue) * Chris Lamb: * [#1036220](https://bugs.debian.org/1036220) filed against [`refnx`](https://tracker.debian.org/pkg/refnx). * [#1036221](https://bugs.debian.org/1036221) filed against [`mfem`](https://tracker.debian.org/pkg/mfem). * Vagrant Cascadian: * [#1035365](https://bugs.debian.org/1035365) filed against [`lombok`](https://tracker.debian.org/pkg/lombok). * [#1035394](https://bugs.debian.org/1035394) filed against [`lcov`](https://tracker.debian.org/pkg/lcov). * [#1035400](https://bugs.debian.org/1035400) filed against [`lucene8`](https://tracker.debian.org/pkg/lucene8). * [#1035405](https://bugs.debian.org/1035405) filed against [`bnd`](https://tracker.debian.org/pkg/bnd). * [#1035630](https://bugs.debian.org/1035630) filed against [`clc-intercal`](https://tracker.debian.org/pkg/clc-intercal). * [#1035704](https://bugs.debian.org/1035704) filed against [`proj`](https://tracker.debian.org/pkg/proj). * [#1036367](https://bugs.debian.org/1036367) filed against [`gcc-13`](https://tracker.debian.org/pkg/gcc-13). * [#1036521](https://bugs.debian.org/1036521) filed against [`pygopherd`](https://tracker.debian.org/pkg/pygopherd). * [#1036522](https://bugs.debian.org/1036522) filed against [`pytorch-audio`](https://tracker.debian.org/pkg/pytorch-audio). * [#1036571](https://bugs.debian.org/1036571) & [#1036572](https://bugs.debian.org/1036572) filed against [`vcmi`](https://tracker.debian.org/pkg/vcmi). * [#1036939](https://bugs.debian.org/1036939) filed against [`proj`](https://tracker.debian.org/pkg/proj). In addition, Jason A. Donenfeld filed a bug (now fixed in the latest alpha version) in the Android issue tracker to report that `generateLocaleConfig` in Android Gradle Plugin version 8.1.0 generates XML files using non-deterministic ordering, breaking reproducible builds. [[...](https://issuetracker.google.com/issues/281825213)] --- ## Testing framework [![]({{ "/images/reports/2023-05/testframework.png#right" | relative_url }})](https://tests.reproducible-builds.org/) The Reproducible Builds project operates a comprehensive testing framework (available at [tests.reproducible-builds.org](https://tests.reproducible-builds.org)) in order to check packages and other artifacts for reproducibility. In May, a number of changes were made by Holger Levsen: * Update the kernel configuration of `arm64` nodes only put required modules in the `initrd` to save space in the `/boot` partition. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/bb9db9556)] * A huge number of changes to a new tool to document/track Jenkins node maintenance, including adding `--fetch`, `--help`, `--no-future` and `--verbose` options [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/3c3dae18a)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/6d98bb36e)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/8e4f9bc44)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/beb89ee0e)] as well as adding a suite of new actions, such as `apt-upgrade`, `command`, `deploy-git`, `rmstamp`, etc. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/1b96205a5)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a96f60e37)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0ce2cc887)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f44dd74b9)] in addition a significant amount of refactoring [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/4d8b820c2)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/6e1b92aaf)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/99e11b3e9)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/e8436074f)]. * Issue warnings if `apt` has updates to install. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ce46c6e15)] * Allow Jenkins to run apt get update in maintenance job. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/747049364)] * Installed `bind9-dnsutils` on some Ubuntu 18.04 nodes. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/c1259e0d3)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/366293688)] * Fixed the Jenkins shell monitor to correctly deal with little-used directories. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/76526cee9)] * Updated the node health check to warn when `apt` upgrades are available. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/dde5a3c09)] * Performed some node maintenance. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/dae110d94)] In addition, Vagrant Cascadian added the `nocheck`, `nopgo` and `nolto` when building `gcc-*` and `binutils` packages [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/c51432de5)] as well as performed some node maintenance [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/82efa70ee)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/95c8f5d57)]. In addition, Roland Clobus updated the [openQA](http://open.qa/) configuration to specify longer timeouts and access to the developer mode [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ef0235102)] and updated the URL used for reproducible Debian Live images [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d5f600b34)].
--- If you are interested in contributing to the Reproducible Builds project, please visit our [*Contribute*](https://reproducible-builds.org/contribute/) page on our website. However, you can get in touch with us via: * IRC: `#reproducible-builds` on `irc.oftc.net`. * Mailing list: [`rb-general@lists.reproducible-builds.org`](https://lists.reproducible-builds.org/listinfo/rb-general) * Twitter: [@ReproBuilds](https://twitter.com/ReproBuilds)