Language Selection

English French German Italian Portuguese Spanish

Debian

Syndicate content
Planet Debian - https://planet.debian.org/
Updated: 3 hours 20 sec ago

Simon Josefsson: Passive Icinga Checks: icinga-pusher

Monday 16th of December 2019 07:19:56 PM

I use Icinga to monitor the availability of my Debian/OpenWRT/etc machines. I have relied on server-side checks on the Icinga system that monitor the externally visible operations of the services that I care about. In theory, monitoring externally visible properties should be good enough. Recently I had one strange incident that was due to an out of disk space on one system. This prompted me to revisit my thinking, and to start monitor internal factors as well. This would allow me to detect problems before they happen, such as an out of disk space condition.

Another reason that I only had server-side checks was that I didn’t like the complexity of the Icinga agent nor wanted to open up for incoming SSH connections from the Icinga server on my other servers. Complexity and machine-based authorization tend to lead to security problems so I prefer to avoid them. The manual mentions agents that use the REST API which was that start of my journey into something better.

What I would prefer is for the hosts to push their self-test results to the central Icinga server. Fortunately, there is a Icinga REST API in modern versions of Icinga (including version 2.10 that I use). The process-check-result API can be used to submit passive check results. Getting this up and running required a bit more research and creativity than I would have hoped for, so I thought it was material enough for a blog post. My main requirement was to keep complexity down, hence I ended up with a simple shell script that is run from cron. None of the existing API clients mentioned in the manual appealed to me.

Prepare the Icinga server with some configuration changes to support the push clients (replace blahonga with a fresh long random password).

icinga# cat > /etc/icinga2/conf.d/api-users.conf object ApiUser "pusher" { password = "blahonga" permissions = [ "actions/process-check-result" ] } ^D icinga# icinga2 feature enable api && systemctl reload icinga2

Then add some Service definitions and assign it to some hosts, to /etc/icinga2/conf.d/services.conf:

apply Service "passive-disk" { import "generic-service" check_command = "passive" check_interval = 2h assign where host.vars.os == "Debian" } apply Service "passive-apt" { import "generic-service" check_command = "passive" check_interval = 2h assign where host.vars.os == "Debian" }

I’m using a relaxed check interval of 2 hours because I will submit results from a cron job that is run every hour. The next step is to setup the machines to submit the results. Create a /etc/cron.d/icinga-pusher with the content below. Note that % characters needs to be escaped in crontab files. I’m running this as the munin user which is a non-privileged account that exists on all of my machines, but you may want to modify this. The check_disk command comes from the monitoring-plugins-basic Debian package, which includes other useful plugins like check_apt that I recommend.

30 * * * * munin /usr/local/bin/icinga-pusher `hostname -f` passive-apt /usr/lib/nagios/plugins/check_apt 40 * * * * munin /usr/local/bin/icinga-pusher `hostname -f` passive-disk "/usr/lib/nagios/plugins/check_disk -w 20\% -c 5\% -X tmpfs -X devtmpfs"

My icinga-pusher script requires a configuration file with some information about the Icinga setup. Put the following content in /etc/default/icinga-pusher (again replacing blahonga with your password):

ICINGA_PUSHER_CREDS="-u pusher:blahonga" ICINGA_PUSHER_URL="https://icinga.yoursite.com:5665" ICINGA_PUSHER_CA="-k"

The parameters above are used by the icinga-pusher script. The ICINGA_PUSHER_CREDS contain the api user credentials, either a simple "-u user:password" combination or it could be "--cert /etc/ssl/yourclient.crt --key /etc/ssl/yourclient.key". The ICINGA_PUSHER_URL is the base URL of your Icinga setup, for the API port which is usually 5665. The ICINGA_PUSHER_CA is "--cacert /etc/ssl/icingaca.crt" or "-k" to not use any CA verification (not recommended!).

Below is the script icinga-pusher itself. Some error handling has been removed for brevity — I have put the script in a separate “icinga-pusher” git repository which will be where I make any updates to this project in the future.

#!/bin/sh # Copyright (C) 2019 Simon Josefsson. # Released under the GPLv3+ license. . /etc/default/icinga-pusher HOST="$1" SERVICE="$2" CMD="$3" OUT=$($CMD) RC=$? oIFS="$IFS" IFS='|' set -- $OUT IFS="$oIFS" OUTPUT="$1" PERFORMANCE="$2" data='{ "type": "Service", "filter": "host.name==\"'$HOST'\" && service.name==\"'$SERVICE'\"", "exit_status": '$RC', "plugin_output": "'$OUTPUT'", "performance_data": "'$PERFORMANCE'" }' curl $ICINGA_PUSHER_CA $ICINGA_PUSHER_CREDS \ -s -H 'Accept: application/json' -X POST \ "$ICINGA_PUSHER_URL/v1/actions/process-check-result" \ -d "$data" exit 0

What do you think? Is there a simpler way of achieving what I want? Thanks for reading.

Thomas Lange: FAI.me service now support kernel cmdline options

Monday 16th of December 2019 04:34:36 PM

The FAI.me service for creating customized installation and cloud images now supports additional kernel cmdline parameters. After toggling to the advanced settings, you can add your options. These will replace the default grub "quiet" option.

This feature is currently only available for the installation images, but not for the cloud images.

The URL of the FAI.me service is

https://fai-project.org/FAIme/

FAI.me

Dirk Eddelbuettel: BH 1.72.0-1 on CRAN

Monday 16th of December 2019 01:03:00 PM

The BH package provides a sizeable portion of the Boost C++ libraries as a set of template headers for use by R. It is quite popular, and frequently used together with Rcpp. The BH CRAN page shows e.g. that it is used by rstan, dplyr as well as a few other packages. The current count of reverse dependencies is at 193.

Boost releases every four months. The last release we packaged was 1.69 from last December, prepared just before CRAN’s winter break. As it needed corresponding changes in three packages using it, it arrived on CRAN early January of this year. The process was much smoother this time. Yesterday I updated the package to the Boost 1.72 release made last Wednesday, and we are on CRAN now as there are no apparent issues. Of course, this BH release was also preceded by a complete reverse-depends check on my end, as well as on CRAN.

As you may know, CRAN tightened policies some more. Pragmas suppressing compiler warnings are verboten so I had to disable a few (see this patch file). Expect compilations of packages using Boost, and BH, to be potentially very noisy. Consider adding flags to your local ~/.R/Makeconf and we should add them to the src/Makevars as much as we can. Collecting a few of these on a BH wiki page may not be a bad idea. Contributions welcome!

One change we now made is to actually start fresh, rather than from the previous release. That way we reflect upstream removals better than before. So even though the upstream source release grew, our release tarball is a little smaller than before. Yay. That is likely a one-off, though, and the file is still Yuge.

As far as regularly scheduled changes go, we responded to three issue tickets and added two more (small) libraries, and also attempted to clean up one (which does not fully disappear due to interdependencies).

A detailed list of our local changes from the NEWS file follows. Two diffs to upstream Boost (for diagnostics, plus another small one for path length and other minor issues) are in the repo as well.

Changes in version 1.72.0-1 (2019-12-15)
  • Upgraded to Boost 1.72.0 (plus the few local tweaks) (#65)

  • Applied the standard minimal patch with required changes, as well as the newer changeset for diagnostics pragma suppression.

  • No longer install filesystem _explicitly_ though some files are carried in (#55)

  • Added mp11 (as requested in #62)

  • Added polygon (as requested in #63)

Via CRANberries, there is a diffstat report relative to the previous release.

Comments and suggestions about BH are welcome via the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Enrico Zini: Environment links

Sunday 15th of December 2019 11:00:00 PM
Earth Temperature Timeline climate chart archive.org 2019-12-16 Methane (or ‘Natural Gas’) – the Other Major Greenhouse Gas — Information is Beautiful climate chart archive.org 2019-12-16 Where does methane or 'natural gas' come from? Is it a greenhouse gas? How much methane do cows produce? All these questions answered, visually. Kessler syndrome - Wikipedia science archive.org 2019-12-16 The Kessler syndrome (also called the Kessler effect,[1][2] collisional cascading, or ablation cascade), proposed by the NASA scientist Donald J. Kessler in 1978, is a scenario in which the density of objects in low Earth orbit (LEO) is high enough that collisions between objects could cause a cascade in which each collision generates space debris that increases the likelihood of further collisions.[3] One implication is that the distribution of debris in orbit could render space activities and the use of satellites in specific orbital ranges difficult for many generations.[3] Stuff in Space science archive.org 2019-12-16 Stuff in Space is a realtime 3D map of objects in Earth orbit, visualized using WebGL.

Giovanni Mascellani: Debian init systems GR

Sunday 15th of December 2019 02:30:00 PM

This is my vote:

-=-=-=-=-=- Don't Delete Anything Between These Lines =-=-=-=-=-=-=-=- 7b77e0f2-4ff9-4adb-85e4-af249191f27a [ 1 ] Choice 5: H: Support portability, without blocking progress [ 2 ] Choice 4: D: Support non-systemd systems, without blocking progress [ 3 ] Choice 7: G: Support portability and multiple implementations [ 4 ] Choice 3: A: Support for multiple init systems is Important [ 5 ] Choice 2: B: Systemd but we support exploring alternatives [ 6 ] Choice 1: F: Focus on systemd [ 7 ] Choice 8: Further Discussion [ 8 ] Choice 6: E: Support for multiple init systems is Required -=-=-=-=-=- Don't Delete Anything Between These Lines =-=-=-=-=-=-=-=---

I don't think that nowadays the choice of the init system can neutral: like the choice of a kernel, a libc and some other core components, you cannot just pretend that everything can be transparently swapped with anything else, therefore Debian rightly has to choose a default thing (Linux, glibc, the GNU userland, and now systemd) which is the standard proposal to the casual user. Systemd has clearly become the mainstream thing for a lot of good reasons, and it is the choice the most Debian users and developers should adopt (this is not to say that systemd, its development mode or its goals are perfect; but we have to choose between alternatives that exist, not between ideal ones). This is the reason why imposing that any init system should be supported at the same level is silly to me, and therefore why option E is the last one, and the only one below Further Discussion; in much the same way as it would be silly to impose that kFreeBSD, Mach and musl should be supported at the same level of their default counterparts (although I would be pretty excited to see that happening!). We cannot expect any Debian contributor to have the resources to contribute scripts/units for any init system randomly appearing.

At the same time, I like Debian to be Universal in the sense of potentially being home for any reasonable piece of software. Diversity, user choice and portability are still values, although not absolute ones, and nobody in Debian should make it impossible to pursue them. I also share the "glue" vision proposed by the principles of choices G and H. Therefore work extending support for non-default init systems (and, again, kernels, libc's, architectures, any core component) should be loyally accepted by all involved contributors, even if they do not see the need for such components.

For these reasons I consider option H the one that, despite being possibly a bit too verbose, spells out my thoughts. All the other options are essentially ordered in how close I percieve them to option H.

I'd like to thanks Ian Jackson for having proposed option H and for having collected a few important criteria to find a way in the jungle of all the available options.

Jonathan McDowell: This Week I Voted

Friday 13th of December 2019 07:04:01 PM

I voted this week. Twice, as it happens (vote early, vote often). This post is about my vote for Debian’s General Resolution: Init systems and systemd. You probably want to skip it, but I thought I’d write it up anyway.

[ 1 ] Choice 5: H: Support portability, without blocking progress [ 2 ] Choice 4: D: Support non-systemd systems, without blocking progress [ 3 ] Choice 7: G: Support portability and multiple implementations [ 4 ] Choice 3: A: Support for multiple init systems is Important [ 5 ] Choice 2: B: Systemd but we support exploring alternatives [ 6 ] Choice 1: F: Focus on systemd [ 7 ] Choice 6: E: Support for multiple init systems is Required [ 8 ] Choice 8: Further Discussion

Firstly, I’ve re-ordered the ballot in the order I ranked things in. I find the mix of numbers and letters that don’t match up confusing, and I think the ordering on the ballot indicates the bias of whoever did the ordering. I don’t think that’s intended to be anything other than helpful, but I’d have kept the numbers and letters matching in the expected order.

I made use of Ian Jackson’s voting guide (and should disclose that he and I have had conversations about this matter where he kindly took time to explain to me his position and rationale). However I’m more pro-systemd than he is, and also lazier, so hopefully this post is useful in some fashion rather than a simple rehash of anyone else’s logic.

I ranked Further Discussion last. I want this to go away. I feel it’s still sucking too much of the project’s time.

E was easy to rank as second last. While I want to support people who want to run non-systemd setups I don’t want to force us as a project to have to shoehorn that support in where it’s not easily done.

I put F third last. While I welcome the improvements brought by systemd I’m wary of buying into any ecosystem completely, and it has a lot of tentacles which will make any future move much more difficult if we buy in wholesale (and make life unnecessarily difficult for people who want to avoid systemd, and I’ve no desire to do that).

On the flip side I think those who want to avoid systemd should be able to do so within Debian. I don’t buy the argument that you can just fork and drop systemd there, it’s an invasive change that makes it much, much harder to produce a derivative system. So it’s one of those things we should care about as a project. (If you hate systemd so much you don’t want even its libraries on your system I can’t help you.)

I debated over my ordering for H and D. I am in favour of portability principles, and I’m happy to make a statement that if someone is prepared to do the work of sorting out non-systemd support for a package then as a project we should take that. I read that as it’s not my responsibility as a maintainer to do these things (though obviously if I can easily do so I will), but that I shouldn’t get in the way of someone else doing so. As someone who has built things on top of Debian I subscribe to the idea that it should be suitable glue for such things (as well as something I can run directly on my own machines), so I favoured H.

That leaves A and G. I deferred to Ian here; I’d rather systemd wasn’t the de facto result despite best intentions, which results in placing G first of the two.

For balance you might want to read the posts by Bernd Zeimetz, Jonathan Dowland, Gunnar Wolf, Lucas Nussbaum, Philipp Kern, Sam Hartman and Wouter Verhelst.

Molly de Blanc: Autonomy

Friday 13th of December 2019 09:52:20 AM

I’ve been stuck on the question: Why is autonomy an ethical imperative? or, worded another way Why does autonomy matter? I think if we’re going to argue that free software matters (or if I am anyway), there needs to be a point where we have to be able to answer why autonomy matters.

I’ve been thinking about this in the framing of technology and consent since the summer of 2018, when Karen Sandler and I spoke at HOPE and DebConf 18 on user and software freedom. Sitting with Karen before HOPE, I had a bit of a crisis of faith and lost track of why software freedom matters after I moved to the point that consent is necessary to our continued autonomy. But why does autonomy matter?

Autonomy matters because autonomy matters. It is the postulate on which not only have I built my arguments, but my entire world view. It is an idea that is instilled in us very deeply that all arguments about what should be a legal right are framed. We have the idea of autonomy so fundamental as part of our society, that we have been trained to have negative, sometimes physical, reactions to the loss of autonomy. Pro-choice and anti-choice arguments both boil down to the question of respecting autonomy — but whose autonomy?  Arguments against euthanasia come down to autonomy — questions of whether someone really have the agency to decide to die versus concerns about autonomy being ignored and death being forced on a person. Even climate change is a question of autonomy — how can we be autonomous if we can’t even be?

Person autonomy means we can consent, user freedom is a tool for consent, software freedom is a tool for user freedom, free software is a tool for software freedom. We can also think about this in reverse:

Free software is the reality of software freedom. Software freedom protects user freedom. User freedom enables consent. Consent is necessary to autonomy. Autonomy is essential. Autonomy is essential because autonomy is essential. And that’s enough.

Olivier Berger: Antidote and NRELabs presentation at Paris Open Source Summit 2019

Friday 13th of December 2019 09:22:32 AM

I’ve just presented Antidote and the NRELabs platform at Paris Open Source Summit 2019. Here are the slides of the presentation : Antidote: virtualized learning labs running over kubernetes

I made a demo and the speech in front of the few people left, unfortunately, as the conference attendance suffered from the ongoing strikes in France (opposing the pensions system reform).

In any case, I hope it triggered some interest for the platform and the project.

Many thanks to the organizers for the very nice athmosphere, despite the rather low attendance.

Also, thanks to the HubLinked project who allowed me to work on the project and participate to the summit.

Anisa Kuci: Outreachy post 1

Friday 13th of December 2019 12:00:00 AM

Couple of months ago I decided to apply for the winter 2019-2020 round of Outreachy.

Outreachy, for those who don’t know, provides internships in Free and Open Source Software (FOSS) with the aim to support underrepresented groups of people. Outreachy internships are open to applicants around the world, and interns are able to work remotely.

There were quite a few very interesting projects to choose among this round, but since I have been a Debian user and contributor for a while and it is a project I really like, I decided to work towards it. I have been doing small Debian related events or gatherings in the community I was part of. The Debian project I applied for is “Create fundraising material for DebConf20+, document the fundraising processes and support a cycle”.

The initial tasks were very interesting and applicable to my skill-set, so I was really enjoying working on them. Also the mentor of the project was very responsive and helpful when I would have questions or feel in doubt and quite supportive, which was motivating me to keep contributing in such a great project.

While doing research for the initial application tasks I was surprised to also learn much more about the Debian community and the Debian project itself. I learned information about how past DebConfs have been organized, tools and methods that are used to make DebConfs happen around the world, and go deeper into understanding the different structures that make the whole Debian project so functional.

I would encourage people to find the courage within themselves and apply for the projects they like on Outreachy. And if they don’t get selected one round that shouldn’t mean giving up but trying again next round. I find it a very welcoming environment!

I am very happy I got selected to be one of the Outreachy interns for this round and will do my best to have great results and help on supporting Debian!

Reproducible Builds: Reproducible Builds in November 2019

Thursday 12th of December 2019 04:28:15 PM

Welcome to the November 2019 report from the Reproducible Builds project.

As a summary of our project, whilst anyone can inspect the source code of free software for malicious flaws almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is therefore to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

In this month’s report, we cover:

  • Media coverage and eventsEnter the Reproducibility Challenge, etc.
  • Upstream newsOCaml, Mes, Maven, etc.
  • Distribution workThe latest reports from Arch, Debian and openSUSE, etc.
  • Software developmentHoliday bonanza of patches, work on diffoscope, etc.
  • ContributingHow to get in touch…

If you are interested in contributing to our project, please visit our Contribute page on our website.

Media coverage and events

We held our fifth annual Reproducible Builds summit between the 1st and 8th December in Marrakesh, Morocco. A full, in-depth report will be posted next month…

On November 16th, Vagrant Cascadian presented There and Back Again, Reproducibly at the SeaGL in Seattle, Washington.

Chris Lamb was featured on The Manifest package management podcast in an episode called Reproducible Builds project and Debian package management.

ReScience C is an open-access journal that targets computational research and encourages the explicit replication of already published research. This month they announced their Ten Years Reproducibility Challenge which promotes the idea that old code — in this instance, a “scientific article [published] before January 1st 2010” — should also run on modern hardware and software in order to check one can obtain the same scientific results in the future.

Upstream news

Mike Hommey pushed a change to Mozilla build system to add and print error messages when differences are found between builds as requested in bug #1597903.

There was fresh activity on an old pull request for the OCaml programming language regarding the usage and adoption of the BUILD_PATH_PREFIX_MAP environment variable that is used to ensure that software packages do not embed build-time paths into generated files. On the pull request in question Gabriel Scherer was kind enough to provide many helpful examples on how to use the rewrite rules.

Jan Nieuwenhuizen announced the release of GNU Mes 0.21 and Jeremiah Orians announced the release of mescc-tools-seed version 1.1:

Capable of bootstrapping from a simple hex assembler all the way to a cross-platform C compiler Work is still ongoing [to] result in a full bootstrap from a 357 byte bootstrap binary all the way to GCC.

Hervé Boutemy announced the release of three base Apache Maven plugins (maven-source-plugin, maven-jar-plugin and maven-assembly-plugin 3.2.0) to get Reproducible Builds as a “direct output” from this build system. For more information, please see the “Configuring for Reproducible Builds” section of their documentation.

Eli Schwartz reported a bug against the GNU groff typesetting system for incomplete SOURCE_DATE_EPOCH environment variable support; the output files appeared to be embedding the build timezone.

Distribution work Arch Linux

A slight but temporary decline in the Arch Linux reproducibility status was determined to be due to a bug in the continuous integration framework where one build was building with --nocheck whilst the other did not, resulting in the test dependencies being installed on one build. This led to differences in the BUILDINFO file which records the build dependencies.

Morten Linderud (Foxboron) wrote a blog post on the progress of reproducible builds for Arch packages, including how to reproduce packages and a roadmap of future of work.

The standard Arch development tools package (devtools) now contains a new tool called makerepropkg which can reproduce a package from the Arch repositories given a seed PKGBUILD file.

A lot of work has been put into getting the “[core]” system more reproducible; every package has been rebuilt with a new version of pacman which resolved a previous issue with storing the package size. Build failures and download issues have also been resolved which have lead to an increase of reproducible packages in this distributions continuous integration setup.

openSUSE

Bernhard M. Wiedemann posted a summary of openSUSE updates for 2019 including rpm, a high level openSUSE status and fixing problems with .pyc files which is also relevant to Arch Linux.

The report also summarises the current reproducibility status as follows:

In addition to this, Bernhard also published his monthly Reproducible Builds status update.

Debian

Thorsten Glaser filed a bug against the debhelper packaging library to request that it sets and exports a umask of 022 for all operations as a possible “harmonisation potential”. A varying umask can result in unreproducible packages as the file permissions on the build system can be embedded into archives generated by the build system.

Chris Lamb categorised a large number of packages and issues in the Reproducible Builds “notes” repository, including adding a new ocaml_dune_captures_build_path toolchain issue [].

Vagrant Cascadian filed a bug against the Lintian Debian static analyser for Debian packages to request that it checks for missing and/or unsigned .buildinfo files. He also uploaded the latest version of GNU Mes to the unstable distribution.

Other

Natanael Copa (@n_copa) posted on Twitter that he was finally able to make a fully reproducible package) for Alpine Linux.

The NixOS distribution announced that they plan to run a Christmas Hackathon hosted by Smarkets in London, England on 9th December.

Software development Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. It is run countless times a day on our testing infrastructure and is essential for identifying fixes and causes of non-deterministic behaviour.

diffoscope versions 131, 132 and 133 were uploaded to Debian unstable by Chris Lamb. He also made the following changes:

  • New features / improvements:
    • Allow all possible .zip file variations to return from external tools with non-zero exit codes, not just known types we can identify (e.g. Java .jmod and .jar files). (#78)
    • Limit .dsc and .buildinfo file matching to files in ASCII or UTF-8 format. (#77)
    • Bump the previous max_page_size limit from 400 kB to 4 MB. []
    • Clarify in the HTML and text outputs that the limits are per-format, not global. (#944882)
    • Don’t use line-based buffering when communicating with subprocesses in “binary” mode. (#75)
  • Regression fixes:
    • Correct the substitution/filtering of paths in ELF output to avoid unnecessary differences depending on the path name provided and commandline. (#945572)
    • Silence/correct a Python SyntaxWarning message due to incorrectly comparing an integer by identity vs. equality. (#945531)
  • Testsuite improvements:
    • Refresh the OCaml test fixtures to support versions greater than 4.08.1. []
    • Update an Android manifest test to reflect that parsed XML attributes are returned in a new/sorted manner under Python 3.8. []
    • Dramatically Truncate the tcpdump expected diff to 8KB from ~600KB to reduce the size of the release tarball. []
    • Add a self-test to encourage that new test data files are generated dynamically or at least no new ones are added without an explicit override. []
    • Add a comment that the text_ascii1 and text_ascii2 fixture files are used in multiple tests so is not trivial to remove/replace them. []
    • Drop two more test fixture files for the directory tests. []
    • Don’t run our self-test against the output of the Black source code reformatter with versions earlier than “ours” as it will generate different results. []
    • Update an XML test for Python 3.8. []
    • Drop unused an unused BASE_DIR global. []
  • Code improvements:
    • Rework a long string of or statements into a loop with a break. []
    • Update code to reflect the latest version of the Black source code reformatter. []
    • Correct a reference to the .rdx extension suffix in a comment. []

Other contributions were also made from:

  • Jelle van der Waa:
    • Add support for comparing .zst files created by Zstandard compression algorithm. (#34)
  • Mattia Rizzolo:
    • Install python3-all whilst running the autopkgtests as we want to run the tests against all supported Python versions. []
    • Use apt-get instead of apt in our Dockerfile. []
    • Add zstd to our test dependencies after the resolution of #34. []
strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month, Chris Lamb added file as a dependency for libfile-stripnondeterminism-perl (#945212) and moved away from deprecated $ADTTMP variable [] and made two uploads in total (1.6.2-1 & 1.6.3-1).

Project website

There was yet more effort put into our our website this month, including:

Test framework

We operate a comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, the following changes were made:

  • Alexander Couzens (OpenWrt): Fix a typo in the kirkwood architecture. []

  • Holger Levsen:

    • Debian:
      • Display newer suites first on pages showing the oldest build results. []
      • Use the fully qualified-domain name (FQDN) when specifying hostnames in our list of offline nodes. []
      • Reflect that coccia.debian.org has changed IP address. []
      • Ignore the Maximum transmission Unit (MTU) on eth0 when checking for host health. []
      • Perform the “/usr merge” variation in the unstable, experimental and bullseye distributions but not on buster. []
    • FreeBSD: Upgrade the test VM to FreeBSD 12.1. []

    • Arch Linux:
      • Don’t fail build jobs if the call to diffoscope --version fails; be a bit more verbose in the job output instead. [][]
      • Attempt to be less error prone when ending schroot sessions. []
    • OpenWrt:
      • Additionally build the brcm47xx, kirkwood, lantiq, mediatek, omap, sunxi and tegra targets. [][]
      • Make build job outputs easier to read and thus understand. []
      • Include the build target and subtarget in summary paragraphs at the top of report pages. []
      • Add a reminder to fix the job URL later. []
    • Misc:
      • Attempt to fix the PureOS package set. []
      • Shorten a “HOWTO” header a tiny bit. []
      • Drop hack to fix the clock. []
      • Improve a script header; patches are even more welcome than bugs! []
      • Disable the use of the OpenSSH ControlMaster feature to prevent Jenkins killing connections. []
      • Make a number of improvements to our boilerplate texts/scripts. [][][]
  • Jelle van der Waa: Skip running the Arch Linux tests for continuous builds and rebuilds. [][]

  • Mattia Rizzolo:
    • Set the maximum size for HTML pages generated by diffoscope to 1MB (current default is 400 KB). [][]
    • Update and improve the backup routines for the email relay system managing reproducible-builds.org. [][]
  • Vagrant Cascadian:
    • Ensure OpenSSH authorized_keys files are processed in the correct directory regardless of where they are run from. []
    • Reduce the level of parallelism on armhf systems with a lot of cores to reduce swapping on highly parallel builds, additionally ensuring level of parallelism are odd and even numbers on the first and second builds respectfully. []

The usual node maintenance was performed by Holger Levsen. [][][][]

Contributing

If you are interested in contributing the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:


This month’s report was written by Arnout Engelen, Chris Lamb, Holger Levsen, Jelle van der Waa, Bernhard M. Wiedemann and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

Enrico Zini: Init systems documentation

Thursday 12th of December 2019 11:20:00 AM

Inspired by this post about documentation1 I started a systemd/documentation page in the Debian wiki.

Systemd has an excellent reference documentation in its manpages, but it does a lot of things, and a reference documentation isn't the best starting point for getting introduced to them.

I would like to see a bit more documentation of the kind that sits between a systemctl start|stop|status and the reference manpages. Things like simple HOWTO posts on how to get a simple job done, or high-level explanations of how some specific feature works.

I put some of what I know and used (or wrote) into systemd/documentation, I'll try to add to it when I find more, and I encourage you to do the same.

If you remember an article that has been useful to you and it is missing from the page, link it.

If you found a systemd feature useful to get a task done, write a little HOWTO to the wiki or your blog, and link it.

If you don't use systemd, go ahead and start a similar index for the init system that you use, and link it, too.

I'd love it if each init system we have in Debian had an excellent documentation index in the wiki!

  1. Thanks anarcat for pointing me to it. 

Enrico Zini: On crisis

Thursday 12th of December 2019 09:40:57 AM

There is a quote that goes around allegedly attributed to Albert Einstein:

Let’s not pretend that things will change if we keep doing the same things. A crisis can be a real blessing to any person, to any nation. For all crises bring progress.

Creativity is born from anguish, just like the day is born form the dark night. It’s in crisis that inventiveness is born, as well as discoveries made and big strategies. He who overcomes crisis, overcomes himself, without getting overcome. He who blames his failure to a crisis neglects his own talent and is more interested in problems than in solutions. Incompetence is the true crisis. The greatest inconvenience of people and nations is the laziness with which they attempt to find the solutions to their problems.

There’s no challenge without a crisis. Without challenges, life becomes a routine, a slow agony.

There’s no merit without crisis. It’s in the crisis where we can show the very best in us. Without a crisis, any wind becomes a tender touch. To speak about a crisis is to promote it. Not to speak about it is to exalt conformism. Let us work hard instead. Let us stop, once and for all, the menacing crisis that represents the tragedy of not being willing to overcome it.

I like the idea of considering more valuable how a group overcomes a crisis, rather than how a group avoids it.

Russ Allbery: Astronomy!

Thursday 12th of December 2019 03:58:00 AM

Starting in January of 2020, I will be joining the Data Management team (specifically the Science Quality and Reliability Engineering team) for the Large Synoptic Survey Telescope.

There's a much longer description at the above Wikipedia link and also at LSST's own site, but the short version is that the mission of LSST is to survey the entire southern night sky about twice a week for ten years. This in turn will provide vast amounts of data that will be used to do wide-ranging research in astronomy. All of that data requires indexing and processing so that scientists can use it. The team I'm joining is applying current software engineering techniques (containers, Jupyter notebooks, continuous integration, and so on) to that problem.

For me, this is an opportunity to return to the academic, non-profit world that's always been my first love. It's also an opportunity to learn a bunch of new things (astronomy, for the most obvious, and scientific research computing more generally, but also some areas of technology that I've never had enough time to explore). Even better, everything my new team does is free software and is developed on GitHub, which means I'll be returning to a job where free software is at the center of the work instead of an optional side project for which there's rarely time.

Dropbox has been a great employer and I wasn't looking to leave, but this was too good of an opportunity to pass up.

I'm going to be staying in the Bay Area and working remotely, which also means that I'm going to get about six hours a week back from the commute, and have an opportunity to wander around the area and find interesting places from which to work, something that I'm looking forward to.

When I went to Dropbox, I thought that would mean a bit more time for Debian, and was sadly completely incorrect. No promises this time, but I have some reasons to hope that I'll at least be able to get back to the levels of involvement I had at Stanford. Even if the new job, since it's scientific computing, is using CentOS....

Ian Jackson: Debian GR on init systems - Ballot paper format

Wednesday 11th of December 2019 03:35:43 PM
This can get a bit confusing. The ballot options have letters (eg, "E"). They also have numbers, which show up on the vote page as "Choice 6" or whatever. Separately, there are the ranks you have to assign when voting, where 1 is your first preference, etc.

On the ballot paper, the choices are numbered from 1 to 8. The letters appear too along with the Secretary's summaries. Your preferences also have to be numbered. It is important not to get confused.

Reorder the ballot paper! You are allowed to reorder the choices on your ballot paper, and this is effective.

That is, you can take the ballot paper in the CFV and edit the lines in it into your preferred order with cut and paste. You can look at the letters, or the Secretary's summary lines, when you do that.

It's important to use a proper text editor and not linewrap things while you do this.

After, that you can simply write numbers 1 to 8 into the boxes down the left hand side.

Rank all the options. That way when you get your vote ack back, any parse failure will show up as a blank space in the ack.

Worked example

If after reading my voting guide you answer "maintainers MUST support non-systemd" to Q1, and "Accepting contributions is more important" to Q2, and "I like the vision" to Q3, you fall into the top left corner of my voting table. Then you would vote like this:

> - - -=-=-=-=-=- Don't Delete Anything Between These Lines =-=-=-=-=-=-=-=- > 7b77e0f2-4ff9-4adb-85e4-af249191f27a > [ 1 ] Choice 6: E: Support for multiple init systems is Required > [ 2 ] Choice 5: H: Support portability, without blocking progress > [ 3 ] Choice 4: D: Support non-systemd systems, without blocking progress > [ 4 ] Choice 7: G: Support portability and multiple implementations > [ 5 ] Choice 3: A: Support for multiple init systems is Important > [ 6 ] Choice 8: Further Discussion > [ 7 ] Choice 2: B: Systemd but we support exploring alternatives > [ 8 ] Choice 1: F: Focus on systemd > - - -=-=-=-=-=- Don't Delete Anything Between These Lines =-=-=-=-=-=-=-=- When you get your ack back from the vote system, it lists, from left to right, the preferences numbers for each vote.

In the case above, that's

Your vote has been recorded as follows -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= V: 87532146 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Understanding the devotee ack The V: list in the ack is listed in the order of the "Choice" numbers on the ballot, not in your preference order: Your vote has been recorded as follows -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= V: 87532146 [ 6 ] ||||||| `-- Choice 8: Further Discussion [ 4 ] |||||| `--- Choice 7: G: Support portability [ 1 ] ||||| `---- Choice 6: E: Support for multipl [ 2 ] |||| `----- Choice 5: H: Support portability [ 3 ] ||| `------ Choice 4: D: Support non-systemd [ 5 ] || `------- Choice 3: A: Support for multipl [ 7 ] | `-------- Choice 2: B: Systemd but we supp [ 8 ] `--------- Choice 1: F: Focus on systemd’ Edited 2019-12-11 15:35 to add the "Understanding the devotee ack" section. Thanks to my correspondent for the confusion-report and review.



comments

More in Tux Machines

HACKERS and HOSPITALS: How you can help

Free software activists, as well as many scientists and medical professionals, have long since realized that proprietary medical software and devices are neither ethical nor adequate to our needs. The COVID-19 pandemic has illuminated some of these shortcomings to a broader audience -- and also given our community a unique opportunity to offer real, material help at a difficult time. We're putting together a plan to pitch in, and we hope you'll join us: keep reading to find out what you can do! You may already be aware that software and hardware restrictions are actively hampering the ability of hospitals to repair desperately needed ventilators all over the world, and how some Italian volunteers ran into problems when they 3D printed ventilator valves. (As you can see from the link, the stories vary about exactly what their interaction with the manufacturer was, but it's clear that the company refused to release proprietary design files, forcing the volunteers to reverse-engineer the parts.) Read more In LWN: HACKERS and HOSPITALS<

OCRFeeder - Where images go to text

Recently, finding really cool, new, unique Linux software has become a difficult task. A chore. And by recently, I actually meant these past four or five years, even since the slow decline of enthusiasm and innovation in the desktop space started. After all, there's a limit to how much good stuff can exist in a finite volume of intellect, but let's not forget the wrong shift of focus to mobile and the shattering of the year-of-the-Linux dream. This makes my test of a four-year-old piece of software named OCRFeeder valid, I think. For two reasons. If it's good, it's good. Second, I've always been interested in the progress of optical character recognition, and whether our tools (read AI) can do a reasonable job here. I wrote about this in detail a while back, and then reviewed YAGF in 2015. Now, let's have a look at OCRFeeder and what it can do. After me, brave Linux warriors. Read more

LibreOffice Online Guide translated into Czech and Some LibreOffice 7.0 Previews

  • LibreOffice Online Guide translated into Czech

    LibreOffice Online Guide was created as part of the Google Season of Docs programme, and released in December 2019. Today we’re announcing that the Czech LibreOffice community has finished translating the guide, and it can be downloaded here. (See this page for English documentation.) It was a team effort, and participants were Petr Kuběj, Zuzana Pitříková, Zdeněk Crhonek, Roman Toman, Tereza Portešová, Petr Valach and Stanislav Horáček. Thanks to all volunteers! The Czech team continues with the translation of the Getting Started Guide, and is always open for new volunteers, translators and correctors. Give them a hand!

  • Fontwork update

    Jun Nogata help the LibreOffice community with new Fontwork. And now it’s ready to be in use.

  • Bullet images update

    LibreOffice 7.0 will get new bullet imges. Hope you like them. In general you can use whatever image you like, want or find from the internet, so in the Bullet image dialog there are the following examples...

Audiocasts/Shows: LINUX Unplugged, Late Night Linux, Linux Headlines and More

  • Arm is Here | LINUX Unplugged 347

    We discover a few simple Raspberry Pi tricks that unlock incredible performance and make us re-think the capabilities of Arm systems. Plus we celebrate Wireguard finally landing in Linux, catch up on feedback, and check out the new Manjaro laptop.

  • User Error: What Will Change Post-virus? | Jupiter Extras 67

    Joe, Alan, and Dan speculate about what the world will be like after the situation with Coronavirus is under control and life returns to something resembling normality.

  • Late Night Linux – Episode 86

    The impacts of Coronovirus on Linux and open source, KDE Korner, and whether we are seeing the second big split in the FOSS world.

  • All Backup Solutions for the Home | Rsync, Synology, and FreeNAS
  • 2020-03-31 | Linux Headlines

    The MANRS initiative gains several new members, GitLab wants customers to help migrate premier features to its free tier, Eclipse Theia reaches 1.0, Lutris lands Humble Bundle game store integration, and Steam scales back automatic updates.

  • An Open Source Toolchain For Natural Language Processing From Explosion AI

    The state of the art in natural language processing is a constantly moving target. With the rise of deep learning, previously cutting edge techniques have given way to robust language models. Through it all the team at Explosion AI have built a strong presence with the trifecta of SpaCy, Thinc, and Prodigy to support fast and flexible data labeling to feed deep learning models and performant and scalable text processing. In this episode founder and open source author Matthew Honnibal shares his experience growing a business around cutting edge open source libraries for the machine learning developent process.