---
layout: report
year: "2020"
month: "10"
title: "Reproducible Builds in October 2020"
draft: false
date: 2020-11-11 14:35:33
---

[![]({{ "/images/reports/2020-10/reproducible-builds.png#right" | relative_url }})](https://reproducible-builds.org/)

**Welcome to the October 2020 report from the [Reproducible Builds](https://reproducible-builds.org) project.**
{: .lead}

In our monthly reports, we outline the major things that we have been up to over the past month. As a brief reminder, the motivation behind the Reproducible Builds effort is to ensure flaws have not been introduced in the binaries we install on our systems. If you are interested in contributing to the project, [please visit our main website]({{ "/" | relative_url }}).

## General

[![]({{ "/images/reports/2020-10/archlinux.png#right" | relative_url }})](https://archlinux.org/)

On Saturday 10th October, Morten Linderud gave a talk at [Arch Conf Online 2020](https://conf.archlinux.org/) on [*The State of Reproducible Builds*](https://pretalx.com/arch-conf-online-2020/talk/39BGNS/) in [Arch](https://archlinux.org/). The video should be available later this month, but as a teaser:

> The previous year has seen great progress in Arch Linux to get reproducible builds in the hands of the users and developers. In this talk we will explore the current tooling that allows users to reproduce packages, the rebuilder software that has been written to check packages and the current issues in this space.

During the [Reproducible Builds summit in Marrakesh in 2019]({{ "/events/Marrakesh2019/" | relative_url }}), developers from the [GNU Guix](https://guix.gnu.org), [NixOS](https://nixos.org) and [Debian](https://debian.org) distributions were able to produce a bit-for-bit identical [GNU Mes](https://www.gnu.org/software/mes/) binary despite using three different versions of GCC. Since this summit, additional work resulted in a bit-for-bit identical Mes binary using `tcc`, and last month [a fuller update was posted to this effect]({{ "/news/2019/12/21/reproducible-bootstrap-of-mes-c-compiler/" | relative_url }}) by the individuals involved. This month, however, David Wheeler updated his extensive page on [*Fully Countering Trusting Trust through Diverse Double-Compiling*](https://dwheeler.com/trusting-trust/), remarking that:

> GNU Mes rebuild is definitely an application of [Diverse Double-Compiling]. [..] This is an awesome application of DDC, and I believe it's the first publicly acknowledged use of DDC on a binary

There was a [small, followup discussion](https://lists.reproducible-builds.org/pipermail/rb-general/2020-October/002056.html) on our mailing list.

In [openSUSE](https://www.opensuse.org/), Bernhard M. Wiedemann published his [monthly Reproducible Builds status update](https://lists.opensuse.org/opensuse-factory/2020-10/msg00328.html).

![]({{ "/images/reports/2020-10/irc.png#right" | relative_url }})

This month, the Reproducible Builds project restarted our IRC meetings, managing to convene twice: the first time on October 12th ([summary](https://lists.reproducible-builds.org/pipermail/rb-general/2020-October/002059.html) & [logs](http://meetbot.debian.net/reproducible-builds/2020/reproducible-builds.2020-10-12-17.58.html)), and later on the 26th ([logs](http://meetbot.debian.net/reproducible-builds/2020/reproducible-builds.2020-10-26-17.58.html)). As mentioned in previous reports, due to the unprecedented events throughout 2020, there will be [no in-person summit event this year](https://lists.reproducible-builds.org/pipermail/rb-general/2020-September/002045.html).

On our [mailing list this month](https://lists.reproducible-builds.org/pipermail/rb-general/) Elías Alejandro posted a request for [help with a local configuration](https://lists.reproducible-builds.org/pipermail/rb-general/2020-October/002060.html)

### Debian-related work

[![]({{ "/images/reports/2020-10/debian.png#right" | relative_url }})](https://debian.org/)

In August, [Lucas Nussbaum](https://members.loria.fr/LNussbaum/) performed an archive-wide rebuild of packages to test [enabling the `reproducible=+fixfilepath` Debian build flag](https://alioth-lists.debian.net/pipermail/reproducible-builds/Week-of-Mon-20200921/012586.html) by default. Enabling this `fixfilepath` feature will likely fix reproducibility issues in an estimated 500-700 packages. However, this month Vagrant Cascadian posted to the [debian-devel](https://lists.debian.org/debian-devel/) mailing list:

> It would be great to see the `reproducible=+fixfilepath` feature enabled by default in `dpkg-buildflags`, and we would like to proceed forward with this soon unless we hear any major concerns or other outstanding issues. [...] We would like to move forward with this change soon, so please raise any concerns or issues not covered already.

Debian Developer [Stuart Prescott](https://nanonanonano.net/) has been improving [`python-debian`](https://salsa.debian.org/python-debian-team/python-debian), a Python library that is used to parse Debian-specific files such as changelogs, `.dscs`, etc. In particular, Stuart is working on [adding support for `.buildinfo` files](https://bugs.debian.org/875306) used for recording reproducibility-related build metadata:

> This can mostly be a very thin layer around the existing `Deb822` types, using the existing `Changes` code for the file listings, the existing `PkgRelations` code for the package listing and `gpg_*` functions for signature handling.

A total of 159 Debian packages were categorised, 69 had their categorisation updated, and 33 had their classification removed this month, adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). As part of this, Chris Lamb identified and classified two new issues: [`build_path_captured_in_emacs_el_file`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/fc051391) and [`rollup_embeds_build_path`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/6baa174e).

## Software development

This month, we tried to fix a large number of currently-unreproducible packages, including:

* Bernhard M. Wiedemann:

    * [`go`](https://github.com/golang/go/issues/42159) (version 1.15.3 has improved reproducibility over 1.14)
    * [`goxel`](https://github.com/guillaumechereau/goxel/pull/215) (sort [SCons](https://scons.org/)-related filesystem ordering issue)
    * [`lal`](https://git.ligo.org/lscsoft/lalsuite/-/merge_requests/1452) (rework an old date-related patch)
    * [`lalmetaio`](https://build.opensuse.org/request/show/844707) (date)
    * [`libsemigroups`](https://github.com/libsemigroups/libsemigroups/issues/251) (build failure in single-CPU mode)
    * [`memcached`](https://github.com/memcached/memcached/pull/725) (build failure in 2025 due to expired SSL certificate)
    * [`octant`](https://build.opensuse.org/request/show/843657) (SUSE-specific date issue)
    * [`openmpi4`](https://github.com/open-mpi/ompi/pull/8136) (date-related problem, revive old patch)
    * [`sbcl`](https://github.com/sbcl/sbcl/pull/37) (datetime and hostname issue)
    * [`selinux-policy/policycoreutils`](https://github.com/SELinuxProject/selinux/pull/268) (date-related issue in timezone)

* Chris Lamb:

    * [#970383](https://bugs.debian.org/970383) filed against [`evince`](https://tracker.debian.org/pkg/evince) ([forwarded upstream](https://gitlab.gnome.org/GNOME/evince/-/merge_requests/286)).
    * [#971527](https://bugs.debian.org/971527) filed against [`libsass-python`](https://tracker.debian.org/pkg/libsass-python) ([forwarded upstream](https://github.com/sass/libsass-python/pull/319)).
    * [#972077](https://bugs.debian.org/972077) filed against [`pitivi`](https://tracker.debian.org/pkg/pitivi).
    * [#972078](https://bugs.debian.org/972078) filed against [`sound-juicer`](https://tracker.debian.org/pkg/sound-juicer).
    * [#972147](https://bugs.debian.org/972147) filed against [`pcbasic`](https://tracker.debian.org/pkg/pcbasic).
    * [#972336](https://bugs.debian.org/972336) filed against [`ora2pg`](https://tracker.debian.org/pkg/ora2pg) ([forwarded upstream](https://github.com/darold/ora2pg/pull/1009)).
    * [#972378](https://bugs.debian.org/972378) filed against [`fckit`](https://tracker.debian.org/pkg/fckit).
    * [#972493](https://bugs.debian.org/972493) filed against [`gita`](https://tracker.debian.org/pkg/gita).
    * [#972494](https://bugs.debian.org/972494) filed against [`libgrokj2k`](https://tracker.debian.org/pkg/libgrokj2k).
    * [#972496](https://bugs.debian.org/972496) filed against [`softether-vpn`](https://tracker.debian.org/pkg/softether-vpn).
    * [#972559](https://bugs.debian.org/972559) filed against [`perl`](https://tracker.debian.org/pkg/perl).
    * [#972561](https://bugs.debian.org/972561) filed against [`ruby-appraiser`](https://tracker.debian.org/pkg/ruby-appraiser).
    * [#972562](https://bugs.debian.org/972562) filed against [`gmerlin-avdecoder`](https://tracker.debian.org/pkg/gmerlin-avdecoder).
    * [#972631](https://bugs.debian.org/972631) filed against [`node-proxy`](https://tracker.debian.org/pkg/node-proxy).
    * [#972668](https://bugs.debian.org/972668) filed against [`yard`](https://tracker.debian.org/pkg/yard).
    * [#972861](https://bugs.debian.org/972861) filed against [`emacs`](https://tracker.debian.org/pkg/emacs).
    * [#972930](https://bugs.debian.org/972930) filed against [`netcdf-parallel`](https://tracker.debian.org/pkg/netcdf-parallel).
    * [#965255](https://bugs.debian.org/965255) re-opened with new patch [`dh-fortran-mod`](https://tracker.debian.org/pkg/netcdf-parallel).

Bernhard M. Wiedemann also reported three issues against [`bison`](https://bugzilla.opensuse.org/show_bug.cgi?id=1040589#c43), [`ibus`](https://github.com/ibus/ibus/issues/2272) and [`postgresql12`](https://www.postgresql.org/message-id/16689-57701daa23b377bf%40postgresql.org).

### Tools

[![]({{ "/images/reports/2020-09/diffoscope.svg#right" | relative_url }})](https://diffoscope.org)

[*diffoscope*](https://diffoscope.org) is our in-depth and content-aware diff utility. Not only could you locate and diagnose reproducibility issues, it provides human-readable diffs of all kinds too. This month, Chris Lamb [uploaded version `161` to Debian](https://tracker.debian.org/news/1184819/accepted-diffoscope-161-source-into-unstable/) (later [backported by Mattia Rizzolo](https://tracker.debian.org/news/1187659/accepted-diffoscope-161bpo101-source-into-buster-backports/)), as well as made the following changes:

* Move `test_ocaml` to the `assert_diff` helper. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/45b44dc)]
* Update tests to support [OCaml](https://ocaml.org/) version 4.11.1. Thanks to Sebastian Ramacher for the report. ([#972518](https://bugs.debian.org/972518))
* Bump minimum version of the [Black](https://black.readthedocs.io/en/stable/) source code formatter to `20.8b1`. ([#972518](https://bugs.debian.org/972518))

In addition, Jean-Romain Garnier temporarily updated the dependency on [`radare2`](https://rada.re/n/) to ensure our test pipelines continue to work [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/6ab7b31)], and for the [GNU Guix](https://guix.gnu.org/) distribution Vagrant Cascadian *diffoscope* to version 161 [[...](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=1e62c8114c233f96c2c569d27e02ccda10001efc)].

In related development, [*trydiffoscope*](https://try.diffoscope.org) is the web-based version of *diffoscope*. This month, Chris Lamb made the following changes:

* Mark a `--help`-only test as being a 'superficial' test. ([#971506](https://bugs.debian.org/971506))
* Add a real, albeit flaky, test that interacts with the `try.diffoscope.org` service. [[...](https://salsa.debian.org/reproducible-builds/trydiffoscope/commit/cc076b2)]
* Bump `debhelper` compatibility level to 13 [[...](https://salsa.debian.org/reproducible-builds/trydiffoscope/commit/0dfa946)] and bump `Standards-Version` to 4.5.0 [[...](https://salsa.debian.org/reproducible-builds/trydiffoscope/commit/9d30857)].

Lastly, *disorderfs* version `0.5.10-2` was [uploaded to Debian unstable](https://tracker.debian.org/news/1187208/accepted-disorderfs-0510-2-source-into-unstable/) by Holger Levsen, which enabled [security hardening](https://wiki.debian.org/Hardening) via `DEB_BUILD_MAINT_OPTIONS` [[...](https://salsa.debian.org/reproducible-builds/disorderfs/commit/65da8f2)] and dropped `debian/disorderfs.lintian-overrides` [[...](https://salsa.debian.org/reproducible-builds/disorderfs/commit/05b21c0)].

### [Website and documentation](https://reproducible-builds.org/)

[![]({{ "/images/reports/2020-09/website.png#right" | relative_url }})]({{ "/" | relative_url }})

This month, a number of updates to the [main Reproducible Builds website]({{ "/" | relative_url }}) and related documentation were made by Chris Lamb:

* Add a citation link to the academic article regarding [`dettrace`](https://github.com/dettrace/dettrace) [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/13d04c1)], and added yet another supply-chain security attack publication [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/158faf8)].
* Reformatted the [Jekyll](https://jekyllrb.com/)'s [Liquid templating language](https://jekyllrb.com/docs/liquid/) and CSS formatting to be consistent [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/80aa18c)] as well as expand a number of tab characters [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/3a99e54)].
* Used `relative_url` to fix missing translation icon on various pages. [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/f56ce13)]
* Published two announcement blog posts regarding the restarting of our IRC meetings. [[...](https://reproducible-builds.org/news/2020/10/12/reproducible-irc-meeting/)][[...](https://reproducible-builds.org/news/2020/10/26/reproducible-irc-meeting-2/)]
* Added an explicit note regarding the lack of an in-person summit in 2020 to [our events page](https://reproducible-builds.org/events/). [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/2d56895)]

## Testing framework

[![]({{ "/images/reports/2020-09/testframework.png#right" | relative_url }})](https://tests.reproducible-builds.org/)

The Reproducible Builds project operates a [Jenkins](https://jenkins.io/)-based testing framework that powers [`tests.reproducible-builds.org`](https://tests.reproducible-builds.org). This month, Holger Levsen made the following changes:

* [Debian](https://debian.org/)-related changes:

    * Refactor and improve the Debian dashboard. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/dddaa8c3)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d8adebff)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/30f6bfec)]
    * Track bugs which are [usertagged](https://wiki.debian.org/bugs.debian.org/usertags) as 'filesystem', 'fixfilepath', etc.. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ae766572)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/70601ea0)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d3b7bb2e)]
    * Make a number of changes to package index pages. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/aaf15b46)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/12d6122c)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/3920a755)]

* System health checks:

    * Relax disk space warning levels. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/14d05397)]
    * Specifically detect build failures reported by `dpkg-buildpackage`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/06566c61)]
    * Fix a regular expression to detect outdated package sets. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/88a24574)]
    * Detect [Lintian](https://lintian.debian.org/) issues in [*diffoscope*](https://diffoscope.org/). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/c039beac)]

[![]({{ "/images/reports/2020-10/ionos.png#right" | relative_url }})](https://www.ionos.co.uk)

* Misc:

    * Make a number of updates to reflect that our sponsor Profitbricks has renamed itself to [IONOS](https://www.ionos.co.uk/). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/91544ae8)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/31f3eb16)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/3e017bf8)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/2315057b)]
    * Run a [F-Droid](https://f-droid.org/) maintenance routine twice a month to utilise its cleanup features. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/66e9a017)]
    * Fix the target name in [OpenWrt](https://openwrt.org/) builds to `ath79` from `ath97`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/94e1bea9)]
    * Add a missing [Postfix](http://www.postfix.org/) configuration for a node. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f326c39c)]
    * Temporarily disable [Arch Linux](https://www.archlinux.org/) builds until a core node is back. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/5a29570b)]
    * Make a number of changes to our "thanks" page. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/46b50793)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/148045c8)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/be38ef3e)]

[![]({{ "/images/reports/2020-06/fdroid.png#right" | relative_url }})](https://f-droid.org/)

Build node maintenance was performed by both Holger Levsen [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/47dff8c2)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/5612c58a)] and Vagrant Cascadian [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/de8a610f)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f5211795)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0ac2c6fa)], Vagrant Cascadian also updated the [page listing the variations made when testing](https://tests.reproducible-builds.org/debian/index_variations.html) to reflect changes for in build paths [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/eec606ea)] and Hans-Christoph Steiner made a number of changes for [F-Droid](https://f-droid.org/), the free software app repository for Android devices, including:

* Do not fail reproducibility jobs when their cleanup tasks fail. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/fba1d0cd)]
* Skip [libvirt](https://libvirt.org/)-related [`sudo`](https://www.sudo.ws/) command if we are not actually running `libvirt`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/3d3ad562)]
* Use direct URLs in order to eliminate a useless HTTP redirect. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/cef63c34)]

---

If you are interested in contributing to the Reproducible Builds project, please visit the [*Contribute*](https://reproducible-builds.org/contribute/) page on our website. However, you can also get in touch with us via:

 * IRC: `#reproducible-builds` on `irc.oftc.net`.

 * Mailing list: [`rb-general@lists.reproducible-builds.org`](https://lists.reproducible-builds.org/listinfo/rb-general)

 * Social media: [@ReproBuilds](https://twitter.com/ReproBuilds), [@reproducible_builds@fosstodon.org](https://fosstodon.org/@reproducible_builds) & [Reddit](https://reddit.com/r/reproduciblebuilds)