---
layout: blog
week: 185
published: 2018-11-13 13:56:58
---

Here's what happened in the [Reproducible Builds](https://reproducible-builds.org) effort between Sunday November 4 and Saturday November 10 2018:

* **We are excited to announce that the Reproducible Builds project [has joined the Software Freedom Conservancy](https://reproducible-builds.org/news/2018/11/08/reproducible-builds-joins-software-freedom-concervancy/)**!

   [![]({{ "/images/logos/rb_joins_sfc.png#center" | relative_url }})](https://reproducible-builds.org/news/2018/11/08/reproducible-builds-joins-software-freedom-concervancy/)

   [Conservancy](https://sfconservancy.org/about/) is a not-for-profit organisation that helps promote, develop and defend free software projects. We can now can take directed donations and the Conservancy can also provide projects us with basic legal services. The Reproducible Builds project is delighted and honoured to be associated with Conservancy's outreach work and other work of the project and look forward to a long and mutually beneficial relationship.

* The month-long session of students from the [Application Security](http://bulletin.engineering.nyu.edu/preview_course_nopop.php?catoid=9&coid=23997) course at [New York University](https://www.nyu.edu/), cataloguing, submitting and merging reproduciblity bugs concluded this week. This year, students made 55 tags and issues for Debian and [Arch Linux](https://www.archlinux.org/) packages and sent 18 pull requests upstream of which 4 have been merged.

* Richard Parkins [posted a detailed message to our mailing list](https://lists.reproducible-builds.org/pipermail/rb-general/2018-November/001251.html) on the topic of algorithms used for comparing binary files in a way that makes the result easily consumable by humans. Most binary file comparators just compare bytes and thus do not semantically detect deletions or insertions. This is relevant to our work on [diffoscope](https://diffoscope.org/). He linked to some [example code on GitHub](https://github.com/rparkins999/bindiff).

* There was further discussion on Debian bug [#869184](https://bugs.debian.org/869184) which relates to `dpkg` generating source uploads that include architecture in the name of the `.buildinfo` file (eg. `_amd64.buildinfo`). This week, Salvatore Bonaccorso reported that the [Debian Security Team](https://wiki.debian.org/Teams/Security) were [hit by this issue again](https://bugs.debian.org/869184#60).

* On Tuesday 6th November, Chris Lamb [hosted a seminar and a lengthy Q&A session](http://talks.cam.ac.uk/talk/index/114232) at the William Gates Building at the University of Cambridge on reproducible builds as part of the [Computer Laboratory NetOS Group](https://www.cl.cam.ac.uk/research/srg/netos/).

* [Simon McVittie](http://smcv.pseudorandom.co.uk/) kindly [provided a patch](https://bugs.debian.org/901473#33) to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org/) to vary whether we apply the "merged `/usr`" directory scheme between builds. This is where the `/{bin,sbin,lib}/` directories are symbolic links to `/usr/{bin,sbin,lib}/`. It was subsequently merged by Holger Levsen and resulted in some variations in (at least) [quilt](https://tests.reproducible-builds.org/debian/rb-pkg/buster/amd64/diffoscope-results/quilt.html) and [systemd](https://tests.reproducible-builds.org/debian/rb-pkg/buster/amd64/diffoscope-results/systemd.html).

* Chris Lamb updated `strip-nondeterminism` (our tool to post-process files to remove known non-deterministic output) to [catch invalid ZIP "local" field lengths](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/e5f5008) — we were previously blindly trusting the value supplied in the ZIP file ([#803503](https://bugs.debian.org/803503)). In addition, he applied a patch from Emmanuel Bourg to [update the Javadoc handler to handle OpenJDK 11](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/f745484) ([#913132](https://bugs.debian.org/913132)). He then subsequently uploaded version `0.044-1` [to Debian unstable](https://tracker.debian.org/news/1001570/accepted-strip-nondeterminism-0044-1-source-all-into-unstable/).

* Agustin Henze announced in a mail to the [`debian-devel` mailing list](https://lists.debian.org/debian-devel/) that [the new Debian CI pipeline](https://lists.debian.org/debian-devel/2018/11/msg00183.html) includes support testing for reproducibility using `reprotest`. These tests are currently available on-demand and need to be set up individually.

* 33 Debian package reviews were added, 14 were updated and 33 were removed in this week, adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). Chris Lamb also updated the [`dc_created_timestamp_in_javadoc`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/1b314210) issue and added a new [`cflags_recorded_in_in_ada_ali_files`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/f3c2f1be) toolchain issue.

* We have received more than 45 registrations for the upcoming [Reproducible Builds summit in Paris](https://reproducible-builds.org/events/paris2018/) between 11th—13th December 2018 and thus are in the process of closing registrations. If you are interested in attending and are contributing to a project not yet represented, please do get in touch as registrations will close shortly.

* Our [report from last week](https://reproducible-builds.org/blog/posts/184/) was quoted in [LWN](https://lwn.net/)'s ["Distribution quotes of the week"](https://lwn.net/Articles/770530/).

Packages reviewed and fixed, and bugs filed
-------------------------------------------

* Bernhard M. Wiedemann:
    * [geany](https://github.com/geany/geany/pull/1989) (Don't use [inode numbers](https://en.wikipedia.org/wiki/Inode) in useless ways)
    * [python-qscintilla-qt5](https://build.opensuse.org/request/show/647086) (sort `readdir(2)`, [submitted upstream](https://www.riverbankcomputing.com/pipermail/qscintilla/2018-November/001349.html))
    * [pesign-obs-integration](https://bugzilla.opensuse.org/show_bug.cgi?id=1114605) (bug)
    * [grep](https://build.opensuse.org/request/show/647618) ([PGO](https://github.com/bmwiedemann/theunreproduciblepackage/tree/master/pgo)/parallelism)

* Chris Lamb:
    * [#912957](https://bugs.debian.org/912957) filed against [python-multipletau](https://tracker.debian.org/pkg/python-multipletau) ([https://github.com/FCS-analysis/multipletau/pull/16](merged upstream))
    * Chris's previously-authored patches for [GNU mtools](https://www.gnu.org/software/mtools/) to ensure the [Debian Installer](https://www.debian.org/devel/debian-installer/) images could become reproducible which were sent upstream last week ([1](http://lists.gnu.org/archive/html/info-mtools/2018-10/msg00003.html) & [2](http://lists.gnu.org/archive/html/info-mtools/2018-10/msg00004.html)) were merged and should appear in the [upcoming mtools 4.0.20 release](http://lists.gnu.org/archive/html/info-mtools/2018-11/msg00000.html).

* Oskar Wirga:
    * [python-octaviaclient](https://salsa.debian.org/openstack-team/clients/python-octaviaclient/merge_requests/1) (Set `PYTHONHASHSEED`)
    * [felix-osgi-obr](https://salsa.debian.org/java-team/felix-osgi-obr/merge_requests/1) (Timestamp in documentation)
    * [easyconf](https://salsa.debian.org/java-team/easyconf/merge_requests/1) (Ended up being a bug in [strip-nondeterminism](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913132))
    * [commons-daemon](https://salsa.debian.org/java-team/commons-daemon/merge_requests/1) (Ended up being a bug in [strip-nondeterminism](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913132))

* Snahil Singh:
    * [#913195](https://bugs.debian.org/913195): Please make netmrg reproducible.

diffoscope development
----------------------

[diffoscope](https://diffoscope.org/) is our in-depth "diff-on-steroids" utility which helps us diagnose reproducibility issues in packages. This week, version `105` was [uploaded to Debian unstable](https://tracker.debian.org/news/1001952/accepted-diffoscope-105-source-into-unstable/) by Mattia Rizzolo. It [included contributions already covered in previous weeks](https://salsa.debian.org/reproducible-builds/diffoscope/commits/105) as well as new ones from:

* Chris Lamb:
    * [Don't assume all files called ".a" are ELF binaries because we specified a `FILE_EXTENSION_SUFFIX`](https://salsa.debian.org/reproducible-builds/diffoscope/commit/cd4c642). This prevents a potential "Unrecognized archive format" traceback. ([#903446](https://bugs.debian.org/903446))
    * [Prevent errors when obtaining PDF metadata from files with multiple PDF metadata dictionary definition entries](https://salsa.debian.org/reproducible-builds/diffoscope/commit/9624319). ([#913315](https://bugs.debian.org/913315))
    * [Display the reason when cannot extract metadata from PDF files](https://salsa.debian.org/reproducible-builds/diffoscope/commit/cf3bc34).

* Daniel Shahaf:
    * [Fix test failures with upcoming `file(1)` 5.35](https://salsa.debian.org/reproducible-builds/diffoscope/commit/0dfb818). Thanks to Christoph Biedl for the heads-up in advance. ([#912756](https://bugs.debian.org/912756))

* Mattia Rizzolo:
    * [Update GPG key](https://salsa.debian.org/reproducible-builds/diffoscope/commit/44e2c29).
    * [Fix a number of `flake8` and other deprecation warnings](https://salsa.debian.org/reproducible-builds/diffoscope/commit/becf992).

* Will Thompson:
    * [Add a `--list-missing-tools` option](https://salsa.debian.org/reproducible-builds/diffoscope/commit/339a431).


Website updates
---------------

There were a large number of changes to [our website](https://reproducibile-builds) this week:

* Bernhard M. Wiedemann:
    * [Update the "CMake" info](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/11828b3) to our page about the [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/docs/source-date-epoch/) environment variable.

* Chris Lamb:
    * [Use our "contribute to Salsa" page in the footer](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/f944a44).
    * [Add Cambridge University seminar](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/cb1f822).

* Holger Levsen:
    * Add Huawei, Alpine Linux and Bazel to [Paris summit event page](https://reproducible-builds.org/events/paris2018/). ([1](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/9559dc5), [2](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ad07633), [3](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ddd1aea))

* Mattia Rizzolo:
    * [Prepend `site.baseurl` (not `site.url`) in our logo](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/cc6f579).
    * [Add a donation page](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/77b419d).


In addition to that we had contributions from Deb Nicholson, Chris Lamb, Georg Faerber, Holger Levsen and Mattia Rizzolo *et al.* on the [press release regarding joining the Software Freedom Conservancy](https://reproducible-builds.org/news/2018/11/08/reproducible-builds-joins-software-freedom-concervancy/):


Test framework development
--------------------------

There were a large number of updates to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org/) by Holger Levsen this week (see below). The most important work was done behind the scenes outside of Git which was a long debugging session to find out why the Jenkins Java processes were suddenly consuming all of the system resources whilst the machine had a load of 60-200. This involved temporarily removing all 1,300 jobs, disabling plugins and other changes. In the end, it turned out that the underlying SSH/HDD performance was configured poorly and, after this was fixed, Jenkins returned to normal.

* [Debian](https://www.debian.org/)-specific changes:
    * Merge patch by Simon McVittie to apply the "merged `/usr`" directory scheme between builds ([#901473](https://bugs.debian.org/901473)). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d04769a7)]
    * Document that we vary by installing the `usr-merge` package [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/7902f640)] and add a link to the [corresponding Debian Wiki page](https://wiki.debian.org/UsrMerge]) [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/fb44311e)].
    * [Use `pbuilder` from the "backports" repositories everywhere](https://salsa.debian.org/qa/jenkins.debian.net/commit/2081b3a4), to achieve that also [force installation of `pbuilder` from backports on Ubuntu 16.04](https://salsa.debian.org/qa/jenkins.debian.net/commit/d28a62fb)
    * Deal with flaky `armhf` boards. ([1](https://salsa.debian.org/qa/jenkins.debian.net/commit/6121cd22), [2](https://salsa.debian.org/qa/jenkins.debian.net/commit/099a8de5), [3](https://salsa.debian.org/qa/jenkins.debian.net/commit/2bc5747f), [4](https://salsa.debian.org/qa/jenkins.debian.net/commit/410d530b))
    * Remove java and depends from all 49 build nodes manually. Also clean up cruft from the jessie2stretch upgrades on armhf nodes.

* Misc/generic changes:

    * [Increase heap size further and drop all other Java arguments](https://salsa.debian.org/qa/jenkins.debian.net/commit/005aab43).
    * [Do not recover `schroot` sessions](https://salsa.debian.org/qa/jenkins.debian.net/commit/69cfa0c1). Thanks to Helmut Grohne.
    * [Run our health check less often](https://salsa.debian.org/qa/jenkins.debian.net/commit/246b3c25).
    * [Make jenkins `schroot` configuration the common/default](https://salsa.debian.org/qa/jenkins.debian.net/commit/aba431d3).
    * [Drop code related to volume groups](https://salsa.debian.org/qa/jenkins.debian.net/commit/8f0c9c43).

In addition, Mattia Rizzolo fixed an issue in the web-based package rescheduling tool by [encoding a string before passing to `subprocess.run`](https://salsa.debian.org/qa/jenkins.debian.net/commit/5b1832b4) and to [fix the parsing of the "issue" selector option](https://salsa.debian.org/qa/jenkins.debian.net/commit/641cfb29).

---

This week's edition was written by Arnout Engelen, Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Oskar Wirga, Santiago Torres, Snahil Singh & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.