--- layout: report year: "2023" month: "08" title: "Reproducible Builds in August 2023" draft: false date: 2023-09-08 20:56:50 --- **Welcome to the August 2023 report from the [Reproducible Builds](https://reproducible-builds.org) project!** {: .lead} [![]({{ "/images/reports/2023-08/reproducible-builds.png#right" | relative_url }})](https://reproducible-builds.org/) In these reports we outline the most important things that we have been up to over the past month. As a quick recap, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. If you are interested in contributing to the project, please visit our [*Contribute*]({{ "/contribute/" | relative_url }}) page on our website.
### Rust serialisation library moving to precompiled binaries [Bleeping Computer](https://www.bleepingcomputer.com/about/) reported that [Serde](https://serde.rs/), a popular Rust serialization framework, had [decided to ship its `serde_derive` macro as a precompiled binary](https://www.bleepingcomputer.com/news/security/rust-devs-push-back-as-serde-project-ships-precompiled-binaries/). As [Ax Sharma](https://www.bleepingcomputer.com/author/ax-sharma/) writes: > The move has generated a fair amount of push back among developers who worry about its future legal and technical implications, along with a **potential for supply chain attacks, should the maintainer account publishing these binaries be compromised**. After [intensive discussions](https://github.com/serde-rs/serde/issues/2538), use of the precompiled binary [was phased out](https://github.com/serde-rs/serde/pull/2590).
### "Reproducible builds, the first ten years" [![]({{ "/images/reports/2023-08/bornhack.jpg#right" | relative_url }})](https://bornhack.dk/bornhack-2023/) On August 4th, Holger Levsen gave a talk at [BornHack 2023](https://bornhack.dk/bornhack-2023/) on the Danish island of [Funen](https://en.wikipedia.org/wiki/Funen) titled [*Reproducible Builds, the first ten years*](https://bornhack.dk/bornhack-2023/program/reproducible-builds-the-first-ten-years/) which promised to contain: > […] an overview about reproducible builds, the past, the presence and the future. How it started with a small [meeting] at DebConf13 (and before), how it grew from being a Debian effort to something many projects work on together, until in 2021 it was mentioned in an executive order of the president of the United States. ([HTML slides](https://reproducible-builds.org/_lfs/presentations/2023-08-04-R-B-the-first-10-years)) Holger repeated the talk later in the month at [Chaos Communication Camp 2023](https://events.ccc.de/camp/2023/infos/) in [Zehdenick](https://en.wikipedia.org/wiki/Zehdenick), Germany:
A [video of the talk](https://media.ccc.de/v/camp2023-57236-reproducible_builds_the_first_ten_years) is available online, as are the [HTML slides](https://reproducible-builds.org/_lfs/presentations/2023-08-19-R-B-the-first-10-years).
### Reproducible Builds Summit [![]({{ "/images/reports/2023-08/summit.jpg#right" | relative_url }})]({{ "/events/hamburg2023/" | relative_url }}) Just another reminder that our [upcoming Reproducible Builds Summit]({{ "/events/hamburg2023/" | relative_url }}) is set to take place from **October 31st — November 2nd 2023 in Hamburg, Germany**. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. If you're interested in joining us this year, please make sure to read the [event page]({{ "/events/hamburg2023/" | relative_url }}), the [news item]({{ "/news/2023/07/05/reproducible-builds-hamburg-meeting/" | relative_url }}), or the [invitation email](https://lists.reproducible-builds.org/pipermail/rb-general/2023-August/003057.html) that Mattia Rizzolo sent out, which have more details about the event and location. We are also still looking for [sponsors]({{ "/images/hamburg2023/sponsor_invitation.pdf" | relative_url }}) to support the event, so do reach out to the [organizing team](mailto:2023-summit-team@lists.reproducible-builds.org) if you are able to help. (Also of note that [PackagingCon 2023](https://packaging-con.org/) is taking place in Berlin just before our summit, and [their schedule](https://cfp.packaging-con.org/2023/schedule/) has just been published.)
### Vagrant Cascadian on the Sustain podcast [![]({{ "/images/reports/2023-08/sustain.jpg#right" | relative_url }})](https://podcast.sustainoss.org/196) Vagrant Cascadian was interviewed on the [SustainOSS](https://podcast.sustainoss.org/) podcast on reproducible builds: > Vagrant walks us through his role in the project where the aim is to ensure identical results in software builds across various machines and times, enhancing software security and creating a seamless developer experience. Discover how this mission, supported by the Software Freedom Conservancy and a broad community, is changing the face of Linux distros, Arch Linux, openSUSE, and F-Droid. They also explore the challenges of managing random elements in software, and Vagrant’s vision to make reproducible builds a standard best practice that will ideally become automatic for users. Vagrant shares his work in progress and their commitment to the “last mile problem.” [The episode](https://podcast.sustainoss.org/196) is available to listen (or download) from the Sustain podcast website. As it happens, the episode was recorded at [FOSSY 2023](https://2023.fossy.us), and the video of Vagrant's talk from this conference ([*Breaking the Chains of Trusting Trust*](https://archive.org/details/fossy2023_Breaking_the_Chains_of_Trustin) is now available on [Archive.org](https://archive.org/): [![]({{ "/images/reports/2023-08/fossy2023.jpg#center" | relative_url }})](https://archive.org/details/fossy2023_Breaking_the_Chains_of_Trustin) It was also announced that Vagrant Cascadian will be presenting at the [Open Source Firmware Conference](https://osfc.io) in October on the topic of [*Reproducible Builds All The Way Down*](https://talks.osfc.io/open-source-firmware-conference-2023/talk/HYSNUF/).
### On our mailing list… Carles Pina i Estany wrote to [our mailing list](https://lists.reproducible-builds.org/listinfo/rb-general/) during August with an interesting question concerning the [practical steps to reproduce the `hello-traditional`](https://lists.reproducible-builds.org/pipermail/rb-general/2023-August/003045.html) package from Debian. The [entire thread](https://lists.reproducible-builds.org/pipermail/rb-general/2023-August/thread.html#3051) can be viewed from the archive page, as can [Vagrant Cascadian's reply](https://lists.reproducible-builds.org/pipermail/rb-general/2023-August/003051.html).
### Website updates Rahul Bajaj updated our website to add a series of environment variations related to reproducible builds [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/0990558a)], Russ Cox added the [Go](https://go.dev/) programming language to our [projects page]({{ "/who/projects/" | relative_url }}) [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/83656e0c)] and Vagrant Cascadian fixed a number of broken links and typos around the website [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/905a28d2)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/1f86eef8)][[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/02a8d70d)].
### Software development [![]({{ "/images/reports/2023-08/diffoscope.png#right" | relative_url }})](https://diffoscope.org) In [*diffoscope*](https://diffoscope.org) development this month, versions `247`, `248` and `249` were uploaded to Debian *unstable* by Chris Lamb, who also added documentation for the new `specialize_as` method and expanding the documentation of the existing `specialize` as well [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/1f8d9e17)]. In addition, Fay Stegerman added `specialize_as` and used it to optimise `.smali` comparisons when decompiling Android `.apk` files [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/fe513c02)], Felix Yan and Mattia Rizzolo corrected some typos in code comments [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/bf334e1d),[...](https://salsa.debian.org/reproducible-builds/diffoscope/-/commit/8ee4e4ef)], Greg Chabala merged the RUN commands into single layer in the package's `Dockerfile` [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/0798a2d1)] thus greatly reducing the final image size. Lastly, Roland Clobus updated tool descriptions to mark that the `xb-tool` has moved package within Debian [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/18f764f3)].
[*reprotest*](https://salsa.debian.org/reproducible-builds/reprotest) is our tool for building the same source code twice in different environments and then checking the binaries produced by each build for any differences. This month, Vagrant Cascadian updated the packaging to be compatible with [Tox](https://tox.wiki/en/latest/index.html) version 4. This was originally filed as Debian bug [#1042918](https://bugs.debian.org/1042918) and Holger Levsen uploaded this to change to Debian *unstable* as version 0.7.26 [[…](https://tracker.debian.org/news/1450119/accepted-reprotest-0726-source-into-unstable/)].
### Distribution work [![]({{ "/images/reports/2023-08/debian.png#right" | relative_url }})](https://debian.org/) In Debian, 28 reviews of Debian packages were added, 14 were updated and 13 were removed this month adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). A number of issue types were added, including Chris Lamb adding a [new `timestamp_in_documentation_using_sphinx_zzzeeksphinx_theme` toolchain issue](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/bb423533).
[![]({{ "/images/reports/2023-08/fdroid.png#right" | relative_url }})](https://f-droid.org/en/) In August, F-Droid added 25 new reproducible apps and saw 2 existing apps switch to reproducible builds, making 191 apps in total that are published with Reproducible Builds and using the upstream developer's signature. [[…](https://gitlab.com/obfusk/fdroid-misc-scripts/-/blob/master/reproducible/overview.md)]
[![]({{ "/images/reports/2023-08/opensuse.png#right" | relative_url }})](https://www.opensuse.org/) Bernhard M. Wiedemann published another [monthly report about reproducibility within openSUSE](https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/PZVK36KUEDRUSVSUP6KILC5NZPBKZ2VT/).
### Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: * Bernhard M. Wiedemann: * [`arimo`](https://github.com/notofonts/Arimo/pull/17) (modification time in build results) * [`apptainer`](https://github.com/apptainer/apptainer/issues/1623) (random Go build identifier) * [`arrow`](https://github.com/apache/arrow/issues/37276) (fails to build on single-CPU machines) * [`camlp`](https://github.com/ocaml/camlp-streams/issues/9) (parallelism-related issue) * [`developer`](https://github.com/linuxdeepin/developer-center/issues/5117) (Go ordering-related issue) * [`elementary-xfce-icon-theme`](https://build.opensuse.org/request/show/1106231) (font-related problem) * [`gegl`](https://gitlab.gnome.org/GNOME/gegl/-/issues/337) (parallelism issue) * [`grommunio`](https://github.com/grommunio/grommunio-web/pull/8) (filesystem ordering issue) * [`grpc`](https://build.opensuse.org/request/show/1102203) (drop nondetermistic log) * [`guile-parted`](https://build.opensuse.org/request/show/1102184) (parallelism-related issue) * [`icinga`](https://build.opensuse.org/request/show/1104874) (hostname-based issue) * [`liquid-dsp`](https://build.opensuse.org/request/show/1107870) (CPU-oriented problem) * [`memcached`](https://github.com/memcached/memcached/pull/1074) (package fails to build far in the future) * [`openmpi5/openpmix`](https://github.com/openpmix/openpmix/pull/3112) (date/copyright year issue) * [`openmpi5`](https://github.com/open-mpi/ompi/pull/11847) (date/copyright year issue) * [`orthanc-ohif+orthanc-volview`](https://orthanc.uclouvain.be/hg/orthanc-ohif/rev/154cb76a042f) (ordering related issue plus timestamp in a Gzip) * [`perl-Net-DNS`](https://rt.cpan.org/Ticket/Display.html?id=149456) (package fails to build far in the future) * [`postgis`](https://build.opensuse.org/request/show/1101964) (parallelism issue) * [`python-scipy`](https://github.com/scipy/scipy/issues/19101) (uses an arbitrary build path) * [`python-trustme`](https://github.com/python-trio/trustme/pull/620) (package fails to build far in the future) * [`qtbase/qmake/goldendict-ng`](https://codereview.qt-project.org/c/qt/qtbase/+/494174) (timestamp-related issue) * [`qtox`](https://build.opensuse.org/request/show/1103451) (date-related issue) * [`ring`](https://github.com/briansmith/ring/issues/1625) (filesytem ordering related issue) * `scipy` ([1](https://github.com/scipy/scipy/pull/19123) & [2](https://github.com/scipy/scipy/pull/19124)) (drop arbtirary build path and filesytem-ordering issue) * `snimpy` ([1](https://github.com/vincentbernat/snimpy/issues/105) & [3](https://github.com/vincentbernat/snimpy/issues/106)) (fails to build on single-CPU machines as well far in the future) * [`tango-icon-theme`](https://build.opensuse.org/request/show/1106230) (font-related issue) * Chris Lamb: * [#1042954](https://bugs.debian.org/1042954) filed against [`libcerf`](https://tracker.debian.org/pkg/libcerf). * [#1042955](https://bugs.debian.org/1042955) filed against [`zzzeeksphinx`](https://tracker.debian.org/pkg/zzzeeksphinx). * [#1043330](https://bugs.debian.org/1043330) filed against [`tox`](https://tracker.debian.org/pkg/tox). * [#1050357](https://bugs.debian.org/1050357) filed against [`pytds`](https://tracker.debian.org/pkg/pytds). * [#1050727](https://bugs.debian.org/1050727) filed against [`zlib`](https://tracker.debian.org/pkg/zlib). * [#1050944](https://bugs.debian.org/1050944) filed against [`jtreg6`](https://tracker.debian.org/pkg/jtreg6). * [#1050955](https://bugs.debian.org/1050955) filed against [`rpy2`](https://tracker.debian.org/pkg/rpy2). * Rebecca N. Palmer: * [#1050619](https://bugs.debian.org/1050619) filed against [`nbclient`](https://tracker.debian.org/pkg/nbclient).
### Testing framework [![]({{ "/images/reports/2023-08/testframework.png#right" | relative_url }})](https://tests.reproducible-builds.org/) The Reproducible Builds project operates a comprehensive testing framework (available at [tests.reproducible-builds.org](https://tests.reproducible-builds.org)) in order to check packages and other artifacts for reproducibility. In August, a number of changes were made by Holger Levsen: * [Debian](https://debian.org/)-related changes: * Disable Debian 'live' image creation jobs until an [OpenQA](https://open.qa/) credential problem has been fixed. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/85533aeab)] * Run our maintenance scripts every 3 hours instead of every 2. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/41ea0f463)] * Export data for *unstable* to the `reproducible-tracker.json` data file. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ad86258b8)] * Stop varying the build path, we want reproducible builds. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/8c2c7fb42)] * Temporarily stop updating the `pbuilder.tgz` for Debian *unstable* due to [#1050784](https://bugs.debian.org/1050784). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/17867dafe)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/05ce02f4d)] * Correctly document that we are not variying `usrmerge`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/746ca36e4)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0153b4ab0)] * Mark two `armhf` nodes (`wbq0` and `jtx1a`) as down; investigation is needed. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/5fe0e1f77)] * Misc: * Force reconfiguration of all Jenkins jobs, due to the recent rise of zombie processes. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/9b4cefb57)] * In the node health checks, also try to restart failed `ntpsec`, `postfix` and `vnstat` services. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/4082de753)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/9101176c3)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/bad24f439)] * System health checks: * Detect Debian 'live' build failures due to missing credentials. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/2dc8da1aa)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/b3d90bb8a)] * Ignore specific types of known zombie processes. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/8c8c82aa8)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/58a430a50)] In addition, Vagrant Cascadian updated the scripts to use a predictable build path that is consistent with the one used on `buildd.debian.org`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/624aa4fe2)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/9f078d223)]
--- If you are interested in contributing to the Reproducible Builds project, please visit our [*Contribute*](https://reproducible-builds.org/contribute/) page on our website. However, you can get in touch with us via: * IRC: `#reproducible-builds` on `irc.oftc.net`. * Mailing list: [`rb-general@lists.reproducible-builds.org`](https://lists.reproducible-builds.org/listinfo/rb-general) * Mastodon: [@reproducible_builds@fosstodon.org](https://fosstodon.org/@reproducible_builds) * Twitter: [@ReproBuilds](https://twitter.com/ReproBuilds)