--- layout: report year: "2020" month: "06" title: "Reproducible Builds in June 2020" draft: false date: 2020-07-06 08:11:05 --- *Welcome to the June 2020 report from the [Reproducible Builds]({{ "/" | relative_url }}) project.* In these reports we outline the most important things that we and the rest of the community have been up to over the past month. ### What are reproducible builds? [![]({{ "/images/reports/2020-06/reproducible-builds.png#right" | relative_url }})]({{ "/" | relative_url }}) One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. But whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes. ## News [![]({{ "/images/reports/2020-06/octopus.png#right" | relative_url }})](https://securitylab.github.com/research/octopus-scanner-malware-open-source-supply-chain) The [GitHub Security Lab](https://securitylab.github.com) published a long article on the discovery of a piece of malware designed to backdoor open source projects that used the build process and its resulting artifacts to spread itself. In the course of their analysis and investigation, the GitHub team uncovered 26 open source projects that were backdoored by this malware and were actively serving malicious code. ([Full article](https://securitylab.github.com/research/octopus-scanner-malware-open-source-supply-chain)) Carl Dong from Chaincode Labs uploaded a presentation on [*Bitcoin Build System Security*](https://www.youtube.com/watch?v=I2iShmUTEl8) and reproducible builds to YouTube: [![]({{ "/images/reports/2020-06/I2iShmUTEl8.jpg#center" | relative_url }})](https://www.youtube.com/watch?v=I2iShmUTEl8) The app intended to [trace infection chains of Covid-19 in Switzerland](https://github.com/DP-3T/dp3t-app-android-ch) published information on [how to perform a reproducible build](https://github.com/DP-3T/dp3t-app-android-ch/blob/master/REPRODUCIBLE_BUILDS.md). [![]({{ "/images/reports/2020-06/opentechfund.png#right" | relative_url }})](https://www.opentech.fund/) The Reproducible Builds project has received funding in the past from the [Open Technology Fund (OTF)](https://www.opentech.fund/) to reach specific technical goals, as well as to enable the project to meet in-person at our summits. The OTF has actually also assisted countless other organisations that promote transparent, civil society as well as those that provide tools to circumvent censorship and repressive surveillance. However, the OTF has now [been threatened with closure](https://saveinternetfreedom.tech). ([More info](https://lists.reproducible-builds.org/pipermail/rb-general/2020-June/001968.html)) It was noticed that Reproducible Builds was mentioned in the book [*End-user Computer Security*](https://en.wikibooks.org/wiki/End-user_Computer_Security) by Mark Fernandes (published by [WikiBooks](https://wikibooks.org/)) in the section titled [*Detection of malware in software*](https://en.wikibooks.org/wiki/End-user_Computer_Security/Main_content/Software_based#Detection_of_malware_in_software). Lastly, reproducible builds and other ideas around software supply chain were mentioned in a recent episode of the [Ubuntu Podcast](https://ubuntupodcast.org/) in a [wider discussion about the Snap and application stores](https://ubuntupodcast.org/2020/06/25/s13e14-ace-of-spades/) (at approx 16:00).
## Distribution work [![]({{ "/images/reports/2020-06/archlinux.png#right" | relative_url }})](https://www.archlinux.org/) In the [ArchLinux](https://www.archlinux.org/) distribution, a goal to [remove `.doctrees` from installed files](https://www.archlinux.org/todo/remove-doctrees-from-installed-files-for-reproducible-builds/) was created via Arch's '[TODO list](https://www.archlinux.org/todo/)' mechanism. These `.doctree` files are caches generated by the [Sphinx documentation generator](https://www.sphinx-doc.org/) when developing documentation so that Sphinx does not have to reparse all input files across runs. They should not be packaged, especially as they lead to the package being unreproducible as their [pickled](https://docs.python.org/3/library/pickle.html) format contains unreproducible data. Jelle van der Waa and Eli Schwartz submitted various upstream patches to fix projects that install these by default. Dimitry Andric was able to determine why the reproducibility status of [FreeBSD](https://www.freebsd.org/)'s `base.txz` depended on the number of CPU cores, attributing it to an optimisation made to the [Clang](https://clang.llvm.org/) C compiler [[...](https://github.com/llvm/llvm-project/commit/b4a99a061f517e60985667e39519f60186cbb469)]. After [further detailed discussion on the FreeBSD bug](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246630#c18) it was possible to get the binaries reproducible again [[...](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246630#c34)]. For the [GNU Guix](https://guix.gnu.org/) operating system, Vagrant Cascadian started a thread about [collecting reproducibility metrics](https://lists.reproducible-builds.org/pipermail/rb-general/2020-June/001952.html) and Jan "*janneke*" Nieuwenhuizen posted that they had [further reduced their "bootstrap seed" to 25%](https://guix.gnu.org/blog/2020/guix-further-reduces-bootstrap-seed-to-25/) which is intended to [reduce the amount of code to be audited](https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/) to avoid potential compiler backdoors. [![]({{ "/images/reports/2020-06/opensuse.png#right" | relative_url }})](https://www.opensuse.org/) In [openSUSE](https://www.opensuse.org/), Bernhard M. Wiedemann published his [monthly Reproducible Builds status update](https://lists.opensuse.org/opensuse-factory/2020-06/msg00348.html) as well as made the following changes within the distribution itself: * [`autogen`](https://build.opensuse.org/request/show/813252) (Date issue) * [`carla`](https://build.opensuse.org/request/show/817043) (Timestamp in Windows [Portable Executable](https://en.wikipedia.org/wiki/Portable_Executable) executables) * [`fonttosfnt/xorg-x11-fonts`](https://bugzilla.opensuse.org/show_bug.cgi?id=1173396) ([Address space layout randomization](https://en.wikipedia.org/wiki/Address_space_layout_randomization) issue) * [`fossil`](https://build.opensuse.org/request/show/813802) (Date issue) * [`gcc10 C++`](https://bugzilla.opensuse.org/show_bug.cgi?id=1172846) ([Link-time optimisation](https://en.wikipedia.org/wiki/Interprocedural_optimization#WPO_and_LTO) issue) * [`grep`](https://build.opensuse.org/request/show/817958) ([Profile-guided optimisation](https://en.wikipedia.org/wiki/Profile-guided_optimization) issue) * [`kubernetes1.18`](https://build.opensuse.org/request/show/813143) (Remove [Go](https://golang.org/) build identifier) * [`libjcat`](https://build.opensuse.org/request/show/816984) (Remove certificate) * [`lifelines`](https://build.opensuse.org/request/show/812914) (Date issue) * [`miredo`](https://build.opensuse.org/request/show/812923) (Drop hostname) * [`stressapptest`](https://build.opensuse.org/request/show/812947) (Override date, user & host) ### Debian [![]({{ "/images/reports/2020-06/debian.png#right" | relative_url }})](https://debian.org/) Holger Levsen filed three bugs ([#961857](https://bugs.debian.org/961857), [#961858](https://bugs.debian.org/961858) & [#961859](https://bugs.debian.org/961859)) against the [`reproducible-check`](https://salsa.debian.org/debian/devscripts/-/blob/master/scripts/reproducible-check) tool that reports on the reproducible status of installed packages on a running Debian system. They were subsequently all fixed by Chris Lamb [[...](https://salsa.debian.org/debian/devscripts/-/commit/bcb0b72cf3ff5c36af00197e96cd1c8b7fb7ac5c)][[...](https://salsa.debian.org/debian/devscripts/-/commit/fe56e8a8f7d50aac4dac2e83880116ff4d27b7a9)][[...](https://salsa.debian.org/debian/devscripts/-/commit/9b70f871dbbb7336038cf856a22fd8b7236857c9)]. Timo Röhling filed a wishlist bug against the `debhelper` build tool impacting the reproducibility status of 100s of packages that use the [CMake](https://cmake.org/) build system which led to a number of tests and next steps. [[...](https://bugs.debian.org/962474)] Chris Lamb contributed to a conversation regarding the nondeterministic execution of order of Debian [maintainer scripts](https://wiki.debian.org/MaintainerScripts) that results in the arbitrary allocation of [UNIX group IDs](https://en.wikipedia.org/wiki/Group_identifier), referencing the [Tails](https://tails.boum.org/) operating system's approach this [[...](https://bugs.debian.org/963788#10)]. Vagrant Cascadian also added to a discussion regarding [verification formats for reproducible builds](https://lists.reproducible-builds.org/pipermail/rb-general/2020-June/001954.html). 47 reviews of Debian packages were added, 37 were updated and 69 were removed this month adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). Chris Lamb identified and classified a new `uids_gids_in_tarballs_generated_by_cmake_kde_package_app_templates` issue [[...](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/e5c87f7c)] and updated the `paths_vary_due_to_usrmerge as deterministic` issue, and Vagrant Cascadian updated the `cmake_rpath_contains_build_path` and `gcc_captures_build_path` issues. [[...](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/e6346788)][[...](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/8b2b2034)][[...](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/643f8f14)]. Lastly, Debian Developer Bill Allombert started a mailing list thread regarding [setting the `-fdebug-prefix-map` command-line argument via an environment variable](https://lists.reproducible-builds.org/pipermail/rb-general/2020-June/thread.html#1969) and Holger Levsen also filed three bugs against the [`debrebuild`](https://salsa.debian.org/debian/devscripts/-/blob/master/scripts/debrebuild.pl) Debian package rebuilder tool ([#961861](https://bugs.debian.org/961861), [#961862](https://bugs.debian.org/961862) & [#961864](https://bugs.debian.org/961864)). ## Development [![]({{ "/images/reports/2020-06/website.png#right" | relative_url }})](https://reproducible-builds.org/) On our website this month, Arnout Engelen added a link to [our Mastodon account](https://fosstodon.org/@reproducible_builds) [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/fd782e2)] and moved the [`SOURCE_DATE_EPOCH`]({{ "/docs/source-date-epoch/" | relative_url }}) `git log` example to another section [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/0cd75fe)]. Chris Lamb also limited the number of [news posts]({{ "/news/" | relative_url }}) to avoid showing items from (for example) 2017 [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/4abb82d)]. [*strip-nondeterminism*](https://tracker.debian.org/pkg/strip-nondeterminism) is our tool to remove specific non-deterministic results from a completed build. It is used automatically in most Debian package builds. This month, Mattia Rizzolo bumped the `debhelper` compatibility level to 13 [[...](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/1517402)] and adjusted a related dependency to avoid potential circular dependency [[...](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/a203a13)]. ### Upstream work The Reproducible Builds project attempts to fix unreproducible packages and we try to to send all of our patches upstream. This month, we wrote a large number of such patches including: * Andreas Schleifer: * [`conky`](https://github.com/brndnmtthws/conky/pull/973) * [`kismet`](https://github.com/kismetwireless/kismet/pull/269) * [`nitrokey-app`](https://github.com/Nitrokey/nitrokey-app/pull/447) * Bernhard M. Wiedemann: * [`autogen`](https://sourceforge.net/p/autogen/code/merge-requests/1/) (race condition) * [`cockpit`](https://github.com/cockpit-project/cockpit/pull/14276) (date) * [`fossil`](https://fossil-scm.org/home/tktview/99038b83b489dddb5d6e68c56f48730aad6d72bb) (date) * [`libnvidia-container`](https://github.com/NVIDIA/libnvidia-container/pull/94) (date) * [`libv3270`](https://github.com/PerryWerneck/libv3270/pull/8) ( date) * Chris Lamb: * [#962401](https://bugs.debian.org/962401) filed against [`netcdf-fortran`](https://tracker.debian.org/pkg/netcdf-fortran). * [#962589](https://bugs.debian.org/962589) filed against [`seqtools`](https://tracker.debian.org/pkg/seqtools). * [#962702](https://bugs.debian.org/962702) filed against [`python-pauvre`](https://tracker.debian.org/pkg/python-pauvre). * [#963119](https://bugs.debian.org/963119) filed against [`petitboot`](https://tracker.debian.org/pkg/petitboot). * [#963120](https://bugs.debian.org/963120) filed against [`fonts-anonymous-pro`](https://tracker.debian.org/pkg/fonts-anonymous-pro). * [#963124](https://bugs.debian.org/963124) filed against [`python-pyqtgraph`](https://tracker.debian.org/pkg/python-pyqtgraph) ([forwarded upstream](https://github.com/pyqtgraph/pyqtgraph/pull/1265)) * [#963485](https://bugs.debian.org/963485) filed against [`libqmi`](https://tracker.debian.org/pkg/libqmi). * [#963486](https://bugs.debian.org/963486) filed against [`tkabber-plugins`](https://tracker.debian.org/pkg/tkabber-plugins). * [#963533](https://bugs.debian.org/963533) filed against [`python-stem`](https://tracker.debian.org/pkg/python-stem). * [#963537](https://bugs.debian.org/963537) filed against [`golang-v2ray-core`](https://tracker.debian.org/pkg/golang-v2ray-core). * [#963600](https://bugs.debian.org/963600) filed against [`critcl`](https://tracker.debian.org/pkg/critcl). * [#963602](https://bugs.debian.org/963602) filed against [`gftl`](https://tracker.debian.org/pkg/gftl). * [#963603](https://bugs.debian.org/963603) filed against [`libmbim`](https://tracker.debian.org/pkg/libmbim). * [#963688](https://bugs.debian.org/963688) filed against [`neovim-qt`](https://tracker.debian.org/pkg/neovim-qt). * [#963740](https://bugs.debian.org/963740) filed against [`golang-github-viant-toolbox`](https://tracker.debian.org/pkg/golang-github-viant-toolbox). * Hendrik Meyer: * [`gitality`](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2253) * [`lsof`](https://github.com/lsof-org/lsof/pull/94) * Nick Wellnhofer: * [`libxml2`](https://gitlab.gnome.org/GNOME/libxml2/-/commit/9f42f6baaa91b6461ddfce345c174ea5f1ee73c3) (random data corruption) * Jelle van der Waa: * [`gnome-builder`](https://gitlab.gnome.org/GNOME/gnome-builder/-/merge_requests/281) * [`PHP`](https://github.com/php/php-src/pull/5671) * Eli Schwartz: * [`ghc`](https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3573) * [`vigra`](https://github.com/ukoethe/vigra/pull/477) * Vagrant Cascadian: * [#961954](https://bugs.debian.org/961954) filed against [`lirc`](https://tracker.debian.org/pkg/lirc). * [#962021](https://bugs.debian.org/962021) filed against [`graphviz`](https://tracker.debian.org/pkg/graphviz). * [#962305](https://bugs.debian.org/962305) filed against [`libtommath`](https://tracker.debian.org/pkg/libtommath). * [#963466](https://bugs.debian.org/963466), [#963467](https://bugs.debian.org/963467), [#963470](https://bugs.debian.org/963470) and [#963472](https://bugs.debian.org/963472) filed against [`qemu`](https://tracker.debian.org/pkg/qemu). * [#963518](https://bugs.debian.org/963518) filed against [`source-highlight`](https://tracker.debian.org/pkg/source-highlight). Bernhard M. Wiedemann also filed reports for [`frr`](https://github.com/FRRouting/frr/issues/6576) (build fails on single-processor machines), [`ghc-yesod-static/git-annex`](https://github.com/yesodweb/yesod/issues/1684) (a filesystem ordering issue) and [`ooRexx`](https://sourceforge.net/p/oorexx/bugs/1712/) ([ASLR](https://en.wikipedia.org/wiki/Address_space_layout_randomization)-related issue). ### diffoscope [![]({{ "/images/reports/2020-06/diffoscope.svg#right" | relative_url }})](https://diffoscope.org) [diffoscope](https://diffoscope.org/) is our in-depth 'diff-on-steroids' utility which helps us diagnose reproducibility issues in packages. It does not define reproducibility, but rather provides a helpful and human-readable guidance for packages that are not reproducible, rather than relying essentially-useless binary diffs. This month, Chris Lamb uploaded versions `147`, `148` and `149` to Debian and made the following changes: * New features: * Add output from `strings(1)` to ELF binaries. ([#148](https://salsa.debian.org/reproducible-builds/diffoscope/issues/148)) * Dump `PE32+` executables (such as EFI applications) using `objdump(1)`. ([#181](https://salsa.debian.org/reproducible-builds/diffoscope/issues/181)) * Add support for [Zsh shell](http://zsh.sourceforge.net/) completion. ([#158](https://salsa.debian.org/reproducible-builds/diffoscope/issues/158)) * Bug fixes: * Prevent a traceback when comparing PDF documents that did not contain metadata (ie. a PDF `/Info` stanza). ([#150](https://salsa.debian.org/reproducible-builds/diffoscope/issues/150)) * Fix compatibility with [`jsondiff`](https://github.com/fzumstein/jsondiff) version 1.2.0. ([#159](https://salsa.debian.org/reproducible-builds/diffoscope/issues/159)) * Fix an issue in [GnuPG keybox](https://www.gnupg.org/documentation/manuals/gnupg/kbxutil.html) file handling that left filenames in the diff. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/9ce287d)] * Correct detection of JSON files due to missing call to `File.recognizes` that checks candidates against [`file(1)`](https://darwinsys.com/file/). [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/d284b58)] * Output improvements: * Use the CSS `word-break` property over manually adding `U+200B` zero-width spaces as these were making copy-pasting cumbersome. ([!53](https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/53)) * Downgrade the `tlsh` warning message to an 'info' level warning. ([#29](https://salsa.debian.org/reproducible-builds/diffoscope/issues/29)) * Logging improvements: * Log calls to [`subprocess.check_output`](https://docs.python.org/3/library/subprocess.html#subprocess.check_output) by using a wrapper. ([#151](https://salsa.debian.org/reproducible-builds/diffoscope/issues/151)) * Clarify that we are generating *presenter* formats in a debug-level message. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/7b814f8)] * Log the version of [jsondiff](https://github.com/fzumstein/jsondiff) used. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/6c43bfc)] * Testsuite improvements: * Update tests for [`file(1)`](https://darwinsys.com/file/) version 5.39. ([#179](https://salsa.debian.org/reproducible-builds/issues/179)) * Drop accidentally-duplicated copy of the `--diff-mask` tests. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/3cfc880)] * Don't mask an existing test. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/7b05056)] * Codebase improvements: * Replace obscure references to *WF* with "[Wagner-Fischer](https://en.wikipedia.org/wiki/Wagner%E2%80%93Fischer_algorithm)" for clarity. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/8329774)] * Use a semantic `AbstractMissingType` type instead of remembering to check for both types of 'missing' files. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/b2f9aa3)] * Add a comment regarding potential security issue in the `.changes`, `.dsc` and `.buildinfo` comparators. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/3b1a9ba)] * Drop a large number of unused imports. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/4fcd8b5)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/acea1e8)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/f14a0c6)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/73671b9)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/a065bc9)] * Make many code sections more Pythonic. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c22b9b0)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c5a803c)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/5591884)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/19476dc)] * Prevent some variable aliasing issues. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/8165a00)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/b628a17)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/61b22c3)] * Use some tactical `f-strings` to tidy up code [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/9e82293)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/4ddea94)] and remove explicit `u"unicode"` strings [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/3352546)]. * Refactor a large number of routines for clarity. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/d33572f)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/fdad13c)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/98c4666)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/14dd8d1)] [*trydiffoscope*](https://try.diffoscope.org) is the web-based version of diffoscope. This month, Chris Lamb also corrected the location for the [celerybeat](https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html) scheduler to ensure that the clean/tidy tasks are actually called which had caused an accidental resource exhaustion. ([#12](https://salsa.debian.org/reproducible-builds/try.diffoscope.org/issues/12)) In addition Jean-Romain Garnier made the following changes: * Fix the `--new-file` option when comparing directories by merging `DirectoryContainer.compare` and `Container.compare`. ([#180](https://salsa.debian.org/reproducible-builds/issues/180)) * Allow user to mask/filter diff output via `--diff-mask=REGEX`. ([!51](https://salsa.debian.org/reproducible-builds/diffoscope/merge_requests/51)) * Make child pages open in new window in the `--html-dir` presenter format. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/6b40118)] * Improve the diffs in the `--html-dir` format. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/db15a42)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/7d350df)] Lastly, Daniel Fullmer fixed the [Coreboot](https://doc.coreboot.org/) filesystem comparator [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c4c1a46)] and Mattia Rizzolo prevented warnings from the [`tlsh`](https://github.com/trendmicro/tlsh) fuzzy-matching library during tests [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/cbf8a2c)] and tweaked the build system to remove an unwanted `.build` directory [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/0abfbdf)]. For the [GNU Guix](https://guix.gnu.org/) distribution Vagrant Cascadian updated the version of *diffoscope* to version 147 [[...](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=526a0066ac243f9b740cd2df9e6bb56bcd51e378)] and later 148 [[...](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=d72009bf0841b55e460bcc049e3723c3bd4f6603)]. #### Testing framework [![]({{ "/images/reports/2020-06/testframework.png#right" | relative_url }})](https://tests.reproducible-builds.org/) We operate a large and many-featured [Jenkins](https://jenkins.io/)-based testing framework that powers [`tests.reproducible-builds.org`](https://tests.reproducible-builds.org). Amongst many other tasks, this tracks the status of our reproducibility efforts across many distributions as well as identifies any regressions that have been introduced. This month, Holger Levsen made the following changes: * [Debian](https://www.debian.org/)-related changes: * Prevent bogus failure emails from `rsync2buildinfos.debian.net` every night. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/6b922410)] * Merge a fix from David Bremner's [database of `.buildinfo` files](https://salsa.debian.org/bremner/builtin-pho) to include a fix regarding comparing source vs. binary package versions. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/e095c249)] * Only run the Debian package rebuilder job twice per day. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0b183ea2)] * Increase *bullseye* scheduling. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/384e2422)] * System health status page: * Add a note displaying whether a node needs to be rebooted for a kernel upgrade. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/2f4915e2)] * Fix sorting order of failed jobs. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/b5f23cdc)] * Expand footer to link to the related Jenkins job. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ca068566)] * Add `archlinux_html_pages`, `openwrt_rebuilder_today` and `openwrt_rebuilder_future` to 'known broken' jobs. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f0859704)] * Add HTML `` header to refresh the page every 5 minutes. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/7145cac0)] * Count the number of ignored jobs [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/6e109932)], ignore permanently 'known broken' jobs [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/04267ffd)] and jobs on 'known offline' nodes [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d282064c)]. * Only consider the 'known offline' status from Git. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ce4ed45c)] * Various output improvements. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/37f94ad7)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ff558c0c)] * Tools: * Switch URLs for the [Grml Live Linux](https://grml.org/) and [PureOS](https://pureos.net/) package sets. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/48eaef65)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/dc812c28)] * Don't try to build a [*disorderfs*](https://tracker.debian.org/pkg/disorderfs) Debian source package. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a2f2933d)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/12962144)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/24e8e746)] * Stop building [*diffoscope*](https://diffoscope.org/) as we are moving this to [Salsa](https://salsa.debian.org/reproducible-builds). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f69bcf4e)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0d14e667)] * Merge several "is diffoscope up-to-date on every platform?" test jobs into one [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0d14e667)] and fail less noisily if the version in Debian cannot be determined [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/1e64a889)]. [![]({{ "/images/reports/2020-06/fdroid.png#right" | relative_url }})](https://f-droid.org/) In addition: Marcus Hoffmann was added as a maintainer of the [F-Droid](https://f-droid.org/) reproducible checking components [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d3925c3f)], Jelle van der Waa updated the "is diffoscope up-to-date in every platform" check for [Arch Linux](https://www.archlinux.org/) and [diffoscope](https://diffoscope.org/) [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0762e782)], Mattia Rizzolo backed up a copy of a "remove script" run on the [Codethink](https://www.codethink.co.uk/)-hosted '[jump server](https://en.wikipedia.org/wiki/Jump_server)' [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/82da22af)] and Vagrant Cascadian [temporarily disabled the `fixfilepath` on *bullseye*](https://salsa.debian.org/qa/jenkins.debian.net/-/commit/f2a447ea), to get better data about the [`ftbfs_due_to_f-file-prefix-map`](https://tests.reproducible-builds.org/debian/issues/unstable/ftbfs_due_to_f-file-prefix-map_issue.html) categorised issue. Lastly, the usual build node maintenance was performed by Holger Levsen [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/91a99356)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a4aec7ef)], Mattia Rizzolo [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d84583a9)] and Vagrant Cascadian [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/b40c4a9b)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ddf89320)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/1d7653ed)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a01e745c)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f238c7ed)].
--- If you are interested in contributing to the Reproducible Builds project, please visit our [*Contribute*]({{ "/contribute/" | relative_url }}) page on our website. However, you can get in touch with us via: * IRC: `#reproducible-builds` on `irc.oftc.net`. * Twitter: [@ReproBuilds](https://twitter.com/ReproBuilds) * Mastodon: [@reproducible_builds@fosstodon.org](https://fosstodon.org/@reproducible_builds) * Reddit: [/r/ReproducibleBuilds](https://reddit.com/r/reproduciblebuilds) * Mailing list: [`rb-general@lists.reproducible-builds.org`](https://lists.reproducible-builds.org/listinfo/rb-general) --- This month's report was written by Bernhard M. Wiedemann, Chris Lamb, Eli Schwartz, Holger Levsen, Jelle van der Waa and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list. {: .small}