Language Selection

English French German Italian Portuguese Spanish


Syndicate content
Planet Debian -
Updated: 4 hours 3 min ago

Norbert Preining: Updating Dovecot for Debian

Sunday 10th of May 2020 12:54:01 PM

A tweet of a friend pointed me at the removal of dovecot from Debian/testing, which surprised me a bit. Investigating the situation it seems that Dovecot in Debian is lagging a bit behind in releases, and hasn’t seen responses to some RC bugs. This sounds critical to me as dovecot is a core part of many mail setups, so I prepared updated packages.

Based on the latest released version of Dovecot, 2.3.10, I have made a package starting from the current Debian packaging and adjusted to the newer upstream. The package builds on Debian Buster (10), Testing, and Unstable on i386 and x64 archs. The packages are available on OBS, as usual:

For Unstable:

deb ./

For Testing:

deb ./

For Debian 10 Buster:

deb ./

To make these repositories work, don’t forget that you need to import my OBS gpg key: obs-npreining.asc, best to download it and put the file into /etc/apt/trusted.gpg.d/obs-npreining.asc.

These packages are provided without any warranty. Enjoy.

NOKUBI Takatsugu: Virtual Background using webcam

Sunday 10th of May 2020 10:50:30 AM

I made a webpage to produce virtual background with webcam.


Some online meeting software (Zoom, Microsoft Teams) supports virtual background, but I want to use other software like Jitsi (or google meet), so I made it.

To make this, I referred the article “Open Source Virtual Background”. 
The following figure is the diagram.

It depends on docker, GPU, v4l2loopback (only works on Linux), so I want to make more generic solution. To make as a webpage, and using OBS
Studio with plugins (obs-v4l2sink, OBS-VirtualCam or OBS (macOS) Virtual Camera) you can use the solution on more platforms.

To make as a single webpage, I can reduce overhead using inter-process commuication using http via docker.

This is an example animation:

Using jisti snapshot:

Unfortunately, BodyPix releases only pretraind models, no training data.

I need more improvements:

  • Accept any background images
  • Suppot choose camera device
  • Useful UI

Russ Allbery: Review: Golden Gates

Sunday 10th of May 2020 04:24:00 AM

Review: Golden Gates, by Conor Dougherty

Publisher: Penguin Copyright: 2020 ISBN: 0-525-56022-X Format: Kindle Pages: 249

This review, for reasons that will hopefully become clear later, starts with a personal digression.

I have been interested in political theory my entire life. That sounds like something admirable, or at least neutral. It's not. "Interested" means that I have opinions that are generally stronger than my depth of knowledge warrants. "Interested" means that I like thinking about and casting judgment on how politics should be done without doing the work of politics myself. And "political theory" is different than politics in important ways, not the least of which is that political actions have rarely been a direct danger to me or my family. I have the luxury of arguing about politics as a theory.

In short, I'm at high risk of being one of those people who has an opinion about everything and shares it on Twitter.

I'm still in the process (to be honest, near the beginning of the process) of making something useful out of that interest. I've had some success when I become enough a part of a community that I can do some of the political work, understand the arguments at a level deeper than theory, and have to deal with the consequences of my own opinions. But those communities have been on-line and relatively low stakes. For the big political problems, the ones that involve governments and taxes and laws, those that decide who gets medical treatment and income support and who doesn't, to ever improve, more people like me need to learn enough about the practical details that we can do the real work of fixing them, rather than only making our native (and generally privileged) communities better for ourselves.

I haven't found my path helping with that work yet. But I do have a concrete, challenging, local political question that makes me coldly furious: housing policy. Hence this book.

Golden Gates is about housing policy in the notoriously underbuilt and therefore incredibly expensive San Francisco Bay Area, where I live. I wanted to deepen that emotional reaction to the failures of housing policy with facts and analysis. Golden Gates does provide some of that. But this also turns out to be a book about the translation of political theory into practice, about the messiness and conflict that results, and about the difficult process of measuring success. It's also a book about how substantial agreement on the basics of necessary political change can still founder on the shoals of prioritization, tribalism, and people who are interested in political theory.

In short, it's a book about the difficulty of changing the world instead of arguing about how to change it.

This is not a direct analysis of housing policy, although Dougherty provides the basics as background. Rather, it's the story of the political fight over housing told primarily through two lenses: Sonja Trauss, founder of BARF (the Bay Area Renters' Federation); and a Redwood City apartment complex, the people who fought its rent increases, and the nun who eventually purchased it. Around that framework, Dougherty writes about the Howard Jarvis Taxpayers Association and the history of California's Proposition 13, a fight over a development in Lafayette, the logistics challenge of constructing sufficient housing even when approved, and the political career of Scott Wiener, the hated opponent of every city fighting for the continued ability to arbitrarily veto any new housing.

One of the things Golden Gates helped clarify for me is that there are three core interest groups that have to be part of any discussion of Bay Area housing: homeowners who want to limit or eliminate local change, renters who are vulnerable to gentrification and redevelopment, and the people who want to live in that area and can't (which includes people who want to move there, but more sympathetically includes all the people who work there but can't afford to live locally, such as teachers, day care workers, food service workers, and, well, just about anyone who doesn't work in tech). (As with any political classification, statements about collectives may not apply to individuals; there are numerous people who appear to fall into one group but who vote in alignment with another.) Dougherty makes it clear that housing policy is intractable in part because the policies that most clearly help one of those three groups hurt the other two.

As advertised by the subtitle, Dougherty's focus is on the fight for more housing. Those who already own homes whose values have been inflated by artificial scarcity, or who want to preserve such stratified living conditions as low-density, large-lot single-family dwellings within short mass-transit commute of one of the densest cities in the United States, don't get a lot of sympathy or focus here except as opponents. I understand this choice; I also don't have much sympathy. But I do wish that Dougherty had spent more time discussing the unsustainable promise that California has implicitly made to homeowners: housing may be impossibly expensive, but if you can manage to reach that pinnacle of financial success, the ongoing value of your home is guaranteed. He does mention this in passing, but I don't think he puts enough emphasis on the impact that a single huge, illiquid investment that is heavily encouraged by government policy has on people's attitude towards anything that jeopardizes that investment.

The bulk of this book focuses on the two factions trying to make housing cheaper: Sonja Trauss and others who are pushing for construction of more housing, and tenant groups trying to manage the price of existing housing for those who have to rent. The tragedy of Bay Area housing is that even the faintest connection of housing to the economic principle of supply and demand implies that the long-term goals of those two groups align. Building more housing will decrease the cost of housing, at least if you build enough of it over a long enough period of time. But in the short term, particularly given the amount of Bay Area land pre-emptively excluded from housing by environmental protection and the actions of the existing homeowners, building more housing usually means tearing down cheap lower-density housing and replacing it with expensive higher-density housing. And that destroys people's lives.

I'll admit my natural sympathy is with Trauss on pure economic grounds. There simply aren't enough places to live in the Bay Area, and the number of people in the area will not decrease. To the marginal extent that growth even slows, that's another tale of misery involving "super commutes" of over 90 minutes each way. But the most affecting part of this book was the detailed look at what redevelopment looks like for the people who thought they had housing, and how it disrupts and destroys existing communities. It's impossible to read those stories and not be moved. But it's equally impossible to not be moved by the stories of people who live in their cars during the week, going home only on weekends because they have to live too far away from their jobs to commute.

This is exactly the kind of politics that I lose when I take a superficial interest in political theory. Even when I feel confident in a guiding principle, the hard part of real-world politics is bringing real people with you in the implementation and mitigating the damage that any choice of implementation will cause. There are a lot of details, and those details matter. Without the right balance between addressing a long-term deficit and providing short-term protection and relief, an attempt to alleviate unsustainable long-term misery creates more short-term misery for those least able to afford it. And while I personally may have less sympathy for the relatively well-off who have clawed their way into their own mortgage, being cavalier with their goals and their financial needs is both poor ethics and poor politics. Mobilizing political opponents who have resources and vote locally isn't a winning strategy.

Dougherty is a reporter, not a housing or public policy expert, so Golden Gates poses problems and tells stories rather than describes solutions. This book didn't lead me to a brilliant plan for fixing the Bay Area housing crunch, or hand me a roadmap for how to get effectively involved in local politics. What it did do is tell stories about what political approaches have worked, how they've worked, what change they've created, and the limitations of that change. Solving political problems is work. That work requires understanding people and balancing concerns, which in turn requires a lot of empathy, a lot of communication, and sometimes finding a way to make unlikely allies.

I'm not sure how broad the appeal of this book will be outside of those who live in the region. Some aspects of the fight for housing generalize, but the Bay Area (and I suspect every region) has properties specific to it or to the state of California. It has also reached an extreme of housing shortage that is rivaled in the United States only by New York City, which changes the nature of the solutions. But if you want to seriously engage with Bay Area housing policy, knowing the background explained here is nearly mandatory. There are some flaws — I wish Dougherty would have talked more about traffic and transit policy, although I realize that could be another book — but this is an important story told well.

If this somewhat narrow topic is within your interests, highly recommended.

Rating: 8 out of 10

Andrew Cater: CD / DVD testing for Buster release 4 - 202005092130 - Slowing down a bit - but still going.

Saturday 9th of May 2020 09:39:06 PM
Last few architectures are being built in the background. Schweer has just confirmed successful testing of all the Debian Edu images - thanks to him, as ever, and to all involved. We're slowing up a bit - it's been a long, hot day and it's not quite over yet. The images release looks to be well on course. As ever, the point release incorporates security fixes and some packages have been removed. The release announcement at gives the details.

Andrew Cater: CD image testing for Buster release 4 - 202005091950 - Most install images checking out well

Saturday 9th of May 2020 07:56:31 PM
Lots of hard work going on. schweer has just validated all of the Debian Edu images.  Most of the normal install images have gone through tests with only a few minor hitches. Now moving on to the Live images. These take longer to download and test but we're working through them gradually.

As ever: a point release doesn't mean that the Debian you have is now obsolete - an apt-get / aptitude update will bring you up to the latest release very quickly. If you are updating regularly, you will have most of these files anyway. One small thing: the tools may report that the release version has changed. This is quite normal - base files have changed to reflect the new point release and this causes the notification. The notification is a small warning so that you are not taken by complete surprise but it is quite normal in the circumstances of a Debian point release.

Thanks to the other folk doing the hard work: 10+ hours and continuing.

Michael Stapelberg: Hermetic packages (in distri)

Saturday 9th of May 2020 04:55:43 PM

In distri, packages (e.g. emacs) are hermetic. By hermetic, I mean that the dependencies a package uses (e.g. libusb) don’t change, even when newer versions are installed.

For example, if package libusb-amd64-1.0.22-7 is available at build time, the package will always use that same version, even after the newer libusb-amd64-1.0.23-8 will be installed into the package store.

Another way of saying the same thing is: packages in distri are always co-installable.

This makes the package store more robust: additions to it will not break the system. On a technical level, the package store is implemented as a directory containing distri SquashFS images and metadata files, into which packages are installed in an atomic way.

Out of scope: plugins are not hermetic by design

One exception where hermeticity is not desired are plugin mechanisms: optionally loading out-of-tree code at runtime obviously is not hermetic.

As an example, consider glibc’s Name Service Switch (NSS) mechanism. Page 29.4.1 Adding another Service to NSS describes how glibc searches $prefix/lib for shared libraries at runtime.

Debian ships about a dozen NSS libraries for a variety of purposes, and enterprise setups might add their own into the mix.

systemd (as of v245) accounts for 4 NSS libraries, e.g. nss-systemd for user/group name resolution for users allocated through systemd’s DynamicUser= option.

Having packages be as hermetic as possible remains a worthwhile goal despite any exceptions: I will gladly use a 99% hermetic system over a 0% hermetic system any day.

Side note: Xorg’s driver model (which can be characterized as a plugin mechanism) does not fall under this category because of its tight API/ABI coupling! For this case, where drivers are only guaranteed to work with precisely the Xorg version for which they were compiled, distri uses per-package exchange directories.

Implementation of hermetic packages in distri

On a technical level, the requirement is: all paths used by the program must always result in the same contents. This is implemented in distri via the read-only package store mounted at /ro, e.g. files underneath /ro/emacs-amd64-26.3-15 never change.

To change all paths used by a program, in practice, three strategies cover most paths:

ELF interpreter and dynamic libraries

Programs on Linux use the ELF file format, which contains two kinds of references:

First, the ELF interpreter (PT_INTERP segment), which is used to start the program. For dynamically linked programs on 64-bit systems, this is typically

Many distributions use system-global paths such as /lib64/, but distri compiles programs with -Wl,--dynamic-linker=/ro/glibc-amd64-2.31-4/out/lib/ so that the full path ends up in the binary.

The ELF interpreter is shown by file(1), but you can also use readelf -a $BINARY | grep 'program interpreter' to display it.

And secondly, the rpath, a run-time search path for dynamic libraries. Instead of storing full references to all dynamic libraries, we set the rpath so that will find the correct dynamic libraries.

Originally, we used to just set a long rpath, containing one entry for each dynamic library dependency. However, we have since switched to using a single lib subdirectory per package as its rpath, and placing symlinks with full path references into that lib directory, e.g. using -Wl,-rpath=/ro/grep-amd64-3.4-4/lib. This is better for performance, as uses a per-directory cache.

Note that program load times are significantly influenced by how quickly you can locate the dynamic libraries. distri uses a FUSE file system to load programs from, so getting proper -ENOENT caching into place drastically sped up program load times.

Instead of compiling software with the -Wl,--dynamic-linker and -Wl,-rpath flags, one can also modify these fields after the fact using patchelf(1). For closed-source programs, this is the only possibility.

The rpath can be inspected by using e.g. readelf -a $BINARY | grep RPATH.

Environment variable setup wrapper programs

Many programs are influenced by environment variables: to start another program, said program is often found by checking each directory in the PATH environment variable.

Such search paths are prevalent in scripting languages, too, to find modules. Python has PYTHONPATH, Perl has PERL5LIB, and so on.

To set up these search path environment variables at run time, distri employs an indirection. Instead of e.g. teensy-loader-cli, you run a small wrapper program that calls precisely one execve system call with the desired environment variables.

Initially, I used shell scripts as wrapper programs because they are easily inspectable. This turned out to be too slow, so I switched to compiled programs. I’m linking them statically for fast startup, and I’m linking them against musl libc for significantly smaller file sizes than glibc (per-executable overhead adds up quickly in a distribution!).

Note that the wrapper programs prepend to the PATH environment variable, they don’t replace it in its entirely. This is important so that users have a way to extend the PATH (and other variables) if they so choose. This doesn’t hurt hermeticity because it is only relevant for programs that were not present at build time, i.e. plugin mechanisms which, by design, cannot be hermetic.

Shebang interpreter patching

The Shebang of scripts contains a path, too, and hence needs to be changed.

We don’t do this in distri yet (the number of packaged scripts is small), but we should.

Performance requirements

The performance improvements in the previous sections are not just good to have, but practically required when many processes are involved: without them, you’ll encounter second-long delays in magit which spawns many git processes under the covers, or in dracut, which spawns one cp(1) process per file.

Downside: rebuild of packages required to pick up changes

Linux distributions such as Debian consider it an advantage to roll out security fixes to the entire system by updating a single shared library package (e.g. openssl).

The flip side of that coin is that changes to a single critical package can break the entire system.

With hermetic packages, all reverse dependencies must be rebuilt when a library’s changes should be picked up by the whole system. E.g., when openssl changes, curl must be rebuilt to pick up the new version of openssl.

This approach trades off using more bandwidth and more disk space (temporarily) against reducing the blast radius of any individual package update.

Downside: long env variables are cumbersome to deal with

This can be partially mitigated by removing empty directories at build time, which will result in shorter variables.

In general, there is no getting around this. One little trick is to use tr : '\n', e.g.:

distri0# echo $PATH /usr/bin:/bin:/usr/sbin:/sbin:/ro/openssh-amd64-8.2p1-11/out/bin distri0# echo $PATH | tr : '\n' /usr/bin /bin /usr/sbin /sbin /ro/openssh-amd64-8.2p1-11/out/bin Edge cases

The implementation outlined above works well in hundreds of packages, and only a small handful exhibited problems of any kind. Here are some issues I encountered:

Issue: accidental ABI breakage in plugin mechanisms

NSS libraries built against glibc 2.28 and newer cannot be loaded by glibc 2.27. In all likelihood, such changes do not happen too often, but it does illustrate that glibc’s published interface spec is not sufficient for forwards and backwards compatibility.

In distri, we could likely use a per-package exchange directory for glibc’s NSS mechanism to prevent the above problem from happening in the future.

Issue: wrapper bypass when a program re-executes itself

Some programs try to arrange for themselves to be re-executed outside of their current process tree. For example, consider building a program with the meson build system:

  1. When meson first configures the build, it generates ninja files (think Makefiles) which contain command lines that run the meson --internal helper.

  2. Once meson returns, ninja is called as a separate process, so it will not have the environment which the meson wrapper sets up. ninja then runs the previously persisted meson command line. Since the command line uses the full path to meson (not to its wrapper), it bypasses the wrapper.

Luckily, not many programs try to arrange for other process trees to run them. Here is a table summarizing how affected programs might try to arrange for re-execution, whether the technique results in a wrapper bypass, and what we do about it in distri:

technique to execute itself uses wrapper mitigation run-time: find own basename in PATH yes wrapper program compile-time: embed expected path no; bypass! configure or patch run-time: argv[0] or /proc/self/exe no; bypass! patch

One might think that setting argv[0] to the wrapper location seems like a way to side-step this problem. We tried doing this in distri, but had to revert and go the other way.

Misc smaller issues Appendix: Could other distributions adopt hermetic packages?

At a very high level, adopting hermetic packages will require two steps:

  1. Using fully qualified paths whose contents don’t change (e.g. /ro/emacs-amd64-26.3-15) generally requires rebuilding programs, e.g. with --prefix set.

  2. Once you use fully qualified paths you need to make the packages able to exchange data. distri solves this with exchange directories, implemented in the /ro file system which is backed by a FUSE daemon.

The first step is pretty simple, whereas the second step is where I expect controversy around any suggested mechanism.

Appendix: demo (in distri)

This appendix contains commands and their outputs, run on upcoming distri version supersilverhaze, but verified to work on older versions, too.

Large outputs have been collapsed and can be expanded by clicking on the output.

The /bin directory contains symlinks for the union of all package’s bin subdirectories:

distri0# readlink -f /bin/teensy_loader_cli/ro/teensy-loader-cli-amd64-2.1+g20180927-7/bin/teensy_loader_cli

The wrapper program in the bin subdirectory is small:

distri0# ls -lh $(readlink -f /bin/teensy_loader_cli)-rwxr-xr-x 1 root root 46K Apr 21 21:56 /ro/teensy-loader-cli-amd64-2.1+g20180927-7/bin/teensy_loader_cli

Wrapper programs execute quickly:

distri0# strace -fvy /bin/teensy_loader_cli |& head | cat -n 1 execve("/bin/teensy_loader_cli", ["/bin/teensy_loader_cli"], ["USER=root", "LOGNAME=root", "HOME=/root", "PATH=/ro/bash-amd64-5.0-4/bin:/r"..., "SHELL=/bin/zsh", "TERM=screen.xterm-256color", "XDG_SESSION_ID=c1", "XDG_RUNTIME_DIR=/run/user/0", "DBUS_SESSION_BUS_ADDRESS=unix:pa"..., "XDG_SESSION_TYPE=tty", "XDG_SESSION_CLASS=user", "SSH_CLIENT= 42556 22", "SSH_CONNECTION= 42556 10"..., "SSHTTY=/dev/pts/0", "SHLVL=1", "PWD=/root", "OLDPWD=/root", "=/usr/bin/strace", "LD_LIBRARY_PATH=/ro/bash-amd64-5"..., "PERL5LIB=/ro/bash-amd64-5.0-4/ou"..., "PYTHONPATH=/ro/bash-amd64-5.b0-4/"...]) = 0 2 arch_prctl(ARCH_SET_FS, 0x40c878) = 0 3 set_tid_address(0x40ca9c) = 715 4 brk(NULL) = 0x15b9000 5 brk(0x15ba000) = 0x15ba000 6 brk(0x15bb000) = 0x15bb000 7 brk(0x15bd000) = 0x15bd000 8 brk(0x15bf000) = 0x15bf000 9 brk(0x15c1000) = 0x15c1000 10 execve("/ro/teensy-loader-cli-amd64-2.1+g20180927-7/out/bin/teensy_loader_cli", ["/ro/teensy-loader-cli-amd64-2.1+"...], ["USER=root", "LOGNAME=root", "HOME=/root", "PATH=/ro/bash-amd64-5.0-4/bin:/r"..., "SHELL=/bin/zsh", "TERM=screen.xterm-256color", "XDG_SESSION_ID=c1", "XDG_RUNTIME_DIR=/run/user/0", "DBUS_SESSION_BUS_ADDRESS=unix:pa"..., "XDG_SESSION_TYPE=tty", "XDG_SESSION_CLASS=user", "SSH_CLIENT= 42556 22", "SSH_CONNECTION= 42556 10"..., "SSHTTY=/dev/pts/0", "SHLVL=1", "PWD=/root", "OLDPWD=/root", "=/usr/bin/strace", "LD_LIBRARY_PATH=/ro/bash-amd64-5"..., "PERL5LIB=/ro/bash-amd64-5.0-4/ou"..., "PYTHONPATH=/ro/bash-amd64-5.0-4/"...]) = 0

Confirm which ELF interpreter is set for a binary using readelf(1):

distri0# readelf -a /ro/teensy-loader-cli-amd64-2.1+g20180927-7/out/bin/teensy_loader_cli | grep 'program interpreter'[Requesting program interpreter: /ro/glibc-amd64-2.31-4/out/lib/]

Confirm the rpath is set to the package’s lib subdirectory using readelf(1):

distri0# readelf -a /ro/teensy-loader-cli-amd64-2.1+g20180927-7/out/bin/teensy_loader_cli | grep RPATH 0x000000000000000f (RPATH) Library rpath: [/ro/teensy-loader-cli-amd64-2.1+g20180927-7/lib]

…and verify the lib subdirectory has the expected symlinks and target versions:

distri0# find /ro/teensy-loader-cli-amd64-*/lib -type f -printf '%P -> %l\n' -> /ro/glibc-amd64-2.31-4/out/lib/ -> /ro/glibc-amd64-2.31-4/out/lib/ -> /ro/glibc-amd64-2.31-4/out/lib/ -> /ro/libudev-amd64-245-11/out/lib/ -> /ro/libusb-compat-amd64-0.1.5-7/out/lib/ -> /ro/libusb-amd64-1.0.23-8/out/lib/

To verify the correct libraries are actually loaded, you can set the LD_DEBUG environment variable for

distri0# LD_DEBUG=libs teensy_loader_cli[…] 678: find [0]; searching 678: search path=/ro/teensy-loader-cli-amd64-2.1+g20180927-7/lib (RPATH from file /ro/teensy-loader-cli-amd64-2.1+g20180927-7/out/bin/teensy_loader_cli) 678: trying file=/ro/teensy-loader-cli-amd64-2.1+g20180927-7/lib/ 678: […]

NSS libraries that distri ships:

find /lib/ -name "libnss_*.so.2" -type f -printf '%P -> %l\n' -> ../systemd-amd64-245-11/out/lib/ -> ../systemd-amd64-245-11/out/lib/ -> ../systemd-amd64-245-11/out/lib/ -> ../systemd-amd64-245-11/out/lib/ -> ../glibc-amd64-2.31-4/out/lib/ -> ../glibc-amd64-2.31-4/out/lib/ -> ../glibc-amd64-2.31-4/out/lib/ -> ../glibc-amd64-2.31-4/out/lib/ -> ../glibc-amd64-2.31-4/out/lib/

Andrew Cater: CD image testing for Buster release 4 - 202005091538 - Coming along nicely

Saturday 9th of May 2020 04:13:03 PM
We're most of the way through testing the various installs, with good success. Waiting on a few things to be built. It's a good way to pass an afternoon / evening.

Andrew Cater: CD / DVD image testing happening now - 202005091330

Saturday 9th of May 2020 01:33:12 PM
The usual suspects very much involved - Isy, myself, RattusRattus, schweer and Sledge. Image testing going on to good effect. Update pulses are starting to hit mirrors: just done a dist-upgrade on one of the machines here. All good

Dirk Eddelbuettel: ttdo 0.0.5: Reflect tinytest update

Saturday 9th of May 2020 11:36:00 AM

A maintenance release of our (still small) ttdo package just arrived on CRAN. As introduced last fall, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam to give us test results with visual diffs:

tinytest has an extension mechanism we use, and as tinytest was just upgraded to version 1.2.0 changing, among other nice extensions, one interface by allowing for a new error class argument, we had to rebuild as well in order to document the new argument.

The release was actually prepared three days ago when tinytest itself was updated, but we waited for the binaries at CRAN to be updated and rebuilt to take advantage of the fully automated submission and test process at CRAN.

The NEWS entry follow.

Changes in ttdo version 0.0.5 (2020-05-06)
  • Rebuilt under tinytest 1.2.0 to add support for class argument in error-code test predicates

CRANberries provides the usual summary of changes to the previous version. Please use the GitHub repo and its issues for any questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Andrew Cater: And again - CD release for Buster release 4 is going to happen over the weekend ...

Saturday 9th of May 2020 10:30:23 AM
And for our regular readers - it's happening again. I'm sitting here with two laptops, a desktop, a connection to IRC - and friends in Cambridge and elsewhere involved in this too.

All good fun, as ever :)

Thorsten Alteholz: My Debian Activities in April 2020

Saturday 9th of May 2020 10:23:31 AM

FTP master

This month I accepted 384 packages and rejected 47. The overall number of packages that got accepted was 457.

Debian LTS

This was my seventieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

This month my all in all workload has been 28.75h. During that time I did LTS uploads of:

  • [DLA 2183-1] libgsf security update for one CVE
  • [DLA 2184-1] jsch security update for one CVE
  • [DLA 2185-1] eog security update for one CVE
  • [DLA 2187-1] radicale security update for one CVE
  • [DLA 2186-1] ncmpc security update for one CVE
  • [DLA 2188-1] php5 security update for three CVEs
  • [DLA 2189-1] rzip security update for one CVE
  • [DLA 2195-1] w3m security update for two CVEs
  • [DLA 2194-1] yodl security update for one CVE
  • [DLA 2197-1] miniupnpc security update for one CVE
  • [DLA 2196-1] pound security update for one CVE

As there have been lots of no-dsa-CVEs I continued my work on wireshark.

Last but not least I did some days of frontdesk duties.

Debian ELTS

This month was the twenty second ELTS month.

During my small allocated time I only uploaded:

  • ELA-227-1 for php5 fixing four CVEs

I also did some days of frontdesk duties.

Other stuff

Unfortunately this month again strange things happened outside Debian and I only got some stuff done.

I improved packaging of …

I sponsored uploads of …

  • … ulfius

I uploaded the new package …

  • … puppet-module-cirrax-gitolite

On my Go challenge I uploaded:
golang-github-facebookgo-subset, golang-github-facebookgo-ensure, golang-github-shurcool-gopherjslib, golang-github-grafana-grafana-plugin-model, golang-github-crewjam-httperr, golang-github-hashicorp-terraform-svchost, golang-github-neelance-sourcemap, golang-github-neelance-astrewrite, golang-github-kisielk-gotool, golang-github-gopherjs-gopherjs, golang-github-yvasiyarov-newrelic-platform-go, golang-github-rhnvrm-simples3, golang-github-robfig-go-cache, golang-github-xorcare-pointer, golang-github-goburrow-serial

Sandro Tosi: It's a waiting game... but just how long we gotta wait?

Saturday 9th of May 2020 03:08:26 AM
While waiting for my priority date to become current, and with enough "quarantine time" on my hand, i just come up with a very simple Python tool to parse the USCIS Visa Bulletin to gather some data from that.

You can find code and images in this GitHub repo.

For now it only contains a single plot for the EB3 final action date; it answers a simple question: how many months ago your priority date should be if you want to file your AOS on that month. We started from FY2016, to cover the final full year of the Obama administration.

If you're interested in more classes/visas, let me know and the tool could be easily extended to cover that too. PRs are always welcome.

Petter Reinholdtsen: Jami as a Zoom client, a trick for password protected rooms...

Friday 8th of May 2020 11:30:00 AM

Half a year ago, I wrote about the Jami communication client, capable of peer-to-peer encrypted communication. It handle both messages, audio and video. It uses distributed hash tables instead of central infrastructure to connect its users to each other, which in my book is a plus. I mentioned briefly that it could also work as a SIP client, which came in handy when the higher educational sector in Norway started to promote Zoom as its video conferencing solution. I am reluctant to use the official Zoom client software, due to their copyright license clauses prohibiting users to reverse engineer (for example to check the security) and benchmark it, and thus prefer to connect to Zoom meetings with free software clients.

Jami worked OK as a SIP client to Zoom as long as there was no password set on the room. The Jami daemon leak memory like crazy (approximately 1 GiB a minute) when I am connected to the video conference, so I had to restart the client every 7-10 minutes, which is not a great. I tried to get other SIP Linux clients to work without success, so I decided I would have to live with this wart until someone managed to fix the leak in the dring code base. But another problem showed up once the rooms were password protected. I could not get my dial tone signaling through from Jami to Zoom, and dial tone signaling is used to enter the password when connecting to Zoom. I tried a lot of different permutations with my Jami and Asterisk setup to try to figure out why the signaling did not get through, only to finally discover that the fundamental problem seem to be that Zoom is simply not able to receive dial tone signaling when connecting via SIP. There seem to be nothing wrong with the Jami and Asterisk end, it is simply broken in the Zoom end. I got help from a very skilled VoIP engineer figuring out this last part. And being a very skilled engineer, he was also able to locate a solution for me. Or to be exact, a workaround that solve my initial problem of connecting to password protected Zoom rooms using Jami.

So, how do you do this, I am sure you are wondering by now. The trick is already documented from Zoom, and it is to modify the SIP address to include the room password. What is most surprising about this is that the automatically generated email from Zoom with instructions on how to connect via SIP do not mention this. The SIP address to use normally consist of the room ID (a number), an @ character and the IP address of the Zoom SIP gateway. But Zoom understand a lot more than just the room ID in front of the at sign. The format is "[Meeting ID].[Password].[Layout].[Host Key]", and you can hear see how you can both enter password, control the layout (full screen, active presence and gallery) and specify the host key to start the meeting. The full SIP address entered into Jami to provide the password will then look like this (all using made up numbers):


Now if only jami would reduce its memory usage, I could even recommend this setup to others. :)

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Dirk Eddelbuettel: Rcpp Virtual Talk on June 5

Friday 8th of May 2020 12:48:00 AM

We had to cancel R/Finance 2020 due to what is happening all around us. But I plan to present the one-hour workshop I often give in the tutorial session preceding the first day—but this time online!

To keep it simple, we will stick with the same day, and possibly the same time: Friday morning at 8:00am! So that makes Friday, June 5, at 08:00h Central time.

This YouTube! link should then provide the stream, I reckon there may also be a recording afterwards.

The talk / demo / presentation will be about an hour long, and material should be similar to the previous ones (of the same length) still available at the talks page (which also has longer talks all the way to the two-day workshops).

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Norbert Preining: KDE/Plasma 5.18.5 for Debian

Thursday 7th of May 2020 07:52:42 AM

After the KDE Apps update 20.04, now the recently released Plasma 5.18.5 is ready for Debian.

Furthermore, since the most recent version of the KDE frameworks have been uploaded to Debian/experimental, I have adapted the packages to make upgrades to the versions in experimental – and hopefully soon in unstable – smooth. I am also working with the Debian KDE Qt Team to update KDE Apps and Plasma in Debian proper. Stay tuned.

For now, here is what you need: Use the following APT sources in /etc/apt/sources.list.d/obs-npreining-kde.list:

For Unstable:

deb ./ deb ./ deb ./ deb ./ deb ./

For Testing:

deb ./ deb ./ deb ./ deb ./ deb ./

And don’t forget that you need to import my OBS gpg key: obs-npreining.asc, best to download it and put the file into /etc/apt/trusted.gpg.d/obs-npreining.asc.


Reproducible Builds: Reproducible Builds in April 2020

Wednesday 6th of May 2020 03:11:15 PM

Welcome to the April 2020 report from the Reproducible Builds project. In our regular reports we outline the most important things that we and the rest of the community have been up to over the past month.

What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. But whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes.


It was discovered that more than 725 malicious packages were downloaded thousands of times from RubyGems, the official channel for distributing code for the Ruby programming language. Attackers used a variation of “typosquatting” and replaced hyphens and underscores (for example, uploading a malevolent atlas-client in place of atlas_client) that executed a script that intercepted Bitcoin payments. (Ars Technica report)

Bernhard M. Wiedemann launched, a service that takes a package name as input and displays whether the package is reproducible in a number of distributions. For example, it can quickly show the status of Perl as being reproducible on openSUSE but not in Debian. Bernhard also improved the documentation of his “unreproducible package” to add some example patches for hash issues. [].

There was a post on Chaos Computer Club’s website listing Ten requirements for the evaluation of “Contact Tracing” apps in relation to the SARS-CoV-2 epidemic. In particular:

4. Transparency and verifiability: The complete source code for the app and infrastructure must be freely available without access restrictions to allow audits by all interested parties. Reproducible build techniques must be used to ensure that users can verify that the app they download has been built from the audited source code.

Elsewhere, Nicolas Boulenguez wrote a patch for the Ada programming language component of the GCC compiler to skip -f.*-prefix-map options when writing Ada Library Information files. Amongst other properties, these .ali files embed the compiler flags used at the time of the build which results in the absolute build path being recorded via -ffile-prefix-map, -fdebug-prefix-map, etc.

In the Arch Linux project, kpcyrd reported that they held their first “rebuilder workshop”. The session was held on IRC and participants were provided a document with instructions on how to install and use Arch’s repro tool. The meeting resulted in multiple people with no prior experience of Reproducible Builds validate their first package. Later in the month he also announced that it was now possible to run independent rebuilders under Arch in a “hands-off, everything just works™” solution to distributed package verification.

Mathias Lang submitted a pull request against dmd, the canonical compiler for the ‘D’ programming languageto add support for our SOURCE_DATE_EPOCH environment variable as well the other C preprocessor tokens such __DATE__, __TIME__ and __TIMESTAMP__ which was subsequently merged. SOURCE_DATE_EPOCH defines a distribution-agnostic standard for build toolchains to consume and emit timestamps in situations where they are deemed to be necessary. []

The Telegram instant-messaging platform announced that they had updated to version 5.1.1 continuing their claim that they are reproducible according to their full instructions and therefore verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully.

Lastly, Hervé Boutemy reported that 97% of the current development versions of various Maven packages appear to have a reproducible build. []

Distribution work

In Debian this month, 89 reviews of Debian packages were added, 21 were updated and 33 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including:

In addition, Holger Levsen filed a feature request against debrebuild, a tool for rebuilding a Debian package given a .buildinfo file, proposing to add --standalone or --one-shot-mode functionality.

In openSUSE, Bernhard M. Wiedemann made the following changes:

In Arch Linux, a rebuilder instance has been setup at that is rebuilding Arch’s [core] repository directly. The first rebuild has led to approximately 90% packages reproducible contrasting with 94% on the Reproducible Build’s project own ArchLinux status page on that continiously builds packages and does not verify Arch Linux packages. More information may be found on the corresponding wiki page and the underlying decisions were explained on our mailing list.

Software development diffoscope

Chris Lamb made the following changes to diffoscope, the Reproducible Builds project’s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues (including preparing and uploading versions 139, 140, 141, 142 and 143 to Debian which were subsequently uploaded to the backports repository):

  • Comparison improvements:

    • Dalvik .dex files can also serve as APK containers so restrict the narrower identification of .dex files to files ending with this extension and widen the identification of APK files to when file(1) discovers a Dalvik file. (#28)
    • Add support for Hierarchical Data Format (HD5) files. (#95)
    • Add support for .p7c and .p7b certificates. (#94)
    • Strip paths from the output of zipinfo(1) warnings. (#97)
    • Don’t uselessly include the JSON “similarity” percentage if it is “0.0%”. []
    • Render multi-line difference comments in a way to show indentation. (#101)
  • Testsuite improvements:

    • Add pdftotext as a requirement to run the PDF test_metadata text. (#99)
    • apktool 2.5.0 changed the handling of output of XML schemas so update and restrict the corresponding test to match. (#96)
    • Explicitly list python3-h5py in debian/tests/ to ensure that we have this module installed during a test run to generate the fixtures in these tests. []
    • Correct parsing of ./ test --pytest-args arguments. []
  • Misc:

    • Capitalise “Ordering differences only” in text comparison comments. []
    • Improve documentation of FILE_TYPE_HEADER_PREFIX and FALLBACK_FILE_TYPE_HEADER_PREFIX to highlight that only the first 16 bytes are used. []

Michael Osipov created a well-researched merge request to return diffoscope to using zipinfo directly instead of piping input via /dev/stdin in order to ensure portability to the BSD operating system []. In addition, Ben Hutchings documented how --exclude arguments are matched against filenames [] and Jelle van der Waa updated the LLVM test fixture difference for LLVM version 10 [] as well as adding a reference to the name of the h5dump tool in Arch Linux [].

Lastly, Mattia Rizzolo also fixed in incorrect build dependency [] and Vagrant Cascadian enabled diffoscope to locate the openssl and h5dump packages on GNU Guix [][], and updated diffoscope in GNU Guix to version 141 [] and 143 [].


strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In April, Chris Lamb made the following changes:

  • Add deprecation plans to all handlers documenting how — or if — they could be disabled and eventually removed, etc. (#3)
  • Normalise *.sym files as Java archives. (#15)
  • Add support for custom .zip filename filtering and exclude two patterns of files generated by Maven projects in “fork” mode. (#13)

disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues.

This month, Chris Lamb fixed a long-standing issue by not drop UNIX groups in FUSE multi-user mode when we are not root (#1) and uploaded version 0.5.9-1 to Debian unstable. Vagrant Cascadian subsequently refreshed disorderfs in GNU Guix to version 0.5.9 [].

Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

In addition, Bernhard informed the following projects that their packages are not reproducible:

  • acoular (report unknown non-determinism)
  • cri-o (report a date issue)
  • gnutls (report certtool being unable to extend certificates beyond 2049)
  • gnutls (report copyright year variation)
  • libxslt (report a bug about non-deterministic output from data corruption)
  • python-astropy (report a future build failure in 2021)
Project documentation

This month, Chris Lamb made a large number of changes to our website and documentation in the following categories:

  • Community engagement improvements:

    • Update instructions to register for Salsa on our Contribute page now that the signup process has been overhauled. []
    • Make it clearer that joining the rb-general mailing list is probably a first step for contributors to take. []
    • Make our full contact information easier to find in the footer (#19) and improve text layout using bullets to separate sections [].
  • Accessibility:

    • To improve accessibility, make all links underlined. (#12)
    • Use an enhanced foreground/background contrast ratio of 7.04:1. (#11)
  • General improvements:

  • Internals:

    • Move to using jekyll-redirect-from over manual redirect pages [][] and add a redirect from /docs/buildinfo/ to /docs/recording/. (#23)
    • Limit the website self-check to not scan generated files [] and remove the “old layout” checker now that I have migrated all them [].
    • Move the news archive under the /news/ namespace [] and improve formatting of archived news links [].
    • Various improvements to the draft template generation. [][][][]

In addition, Holger Levsen clarified exactly which month we ceased to do weekly reports [] and Mattia Rizzolo adjusted the title style of an event page [].

Marcus Hoffman also started a discussion on our website’s issue tracker asking for clarification on embedded signatures and Chris Lamb subsequently replied and asked Marcus to go ahead and propose a concrete change.

Testing framework

We operate a large and many-featured Jenkins-based testing framework that powers that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced.

  • Chris Lamb:

    • Print the build environment prior to executing a build. []
    • Drop a misleading disorderfs-debug prefix in log output when we change non-disorderfs things in the file and, as it happens, do not run disorderfs at all. []
    • The CSS for the package report pages added a margin to all <a> HTML elements under <li> ones, which was causing a comma/bullet spacing issue. []
    • Tidy the copy in the project links sidebar. []
  • Holger Levsen:

    • General:
    • Debian:

      • Reduce scheduling frequency of the buster distribution on the arm64 architecture, etc.. [][]
      • Show builds per day on a per-architecture basis for the last year on the Debian dashboard. []
      • Drop the Subgraph OS package set as development halted in 2017 or 2018. []
      • Update debrebuild to version from the latest version of devscripts. [][]
      • Add or improve various parts of the documentation. [][][]
    • Work on a Debian rebuilder:

      • Integrate sbuild. [][][][][]
      • Select a random .buildinfo file and attempt to build and compare the result. [][][][]
      • Improve output and related output formatting. [][][][][]
      • Outline next steps for the development of the tool. [][][]
      • Various refactoring and code improvements. [][][]

Lastly, Mattia Rizzolo fixed some log parsing code regarding potentially-harmless warnings from package installation [][] and the usual build node maintenance was performed by Holger Levsen [][][] and Mattia Rizzolo [][][].

Misc news

On our mailing list this month, Santiago Torres asked whether we were still publishing releases of our tools to our website and Chris Lamb replied that this was not the case and fixed the issue. Later in the month Santiago also reported that the signature for the disorderfs package did not pass its GPG verification which was also fixed by Chris Lamb.

Hans-Christoph Steiner of the Guardian Project asked whether there would be interest in making our website translatable which resulted in a WIP merge request being filed against the website and a discussion on how to track translation updates.

If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

This month’s report was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa, kpcyrd, Mattia Rizzolo and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

Russell Coker: About Reopening Businesses

Wednesday 6th of May 2020 07:12:28 AM

Currently there is political debate about when businesses should be reopened after the Covid19 quarantine.

Small Businesses

One argument for reopening things is for the benefit of small businesses. The first thing to note is that the protests in the US say “I need a haircut” not “I need to cut people’s hair”. Small businesses won’t benefit from reopening sooner.

For every business there is a certain minimum number of customers needed to be profitable. There are many comments from small business owners that want it to remain shutdown. When the government has declared a shutdown and paused rent payments and provided social security to employees who aren’t working the small business can avoid bankruptcy. If they suddenly have to pay salaries or make redundancy payouts and have to pay rent while they can’t make a profit due to customers staying home they will go bankrupt.

Many restaurants and cafes make little or no profit at most times of the week (I used to be 1/3 owner of an Internet cafe and know this well). For such a company to be viable you have to be open most of the time so customers can expect you to be open. Generally you don’t keep a cafe open at 3PM to make money at 3PM, you keep it open so people can rely on there being a cafe open there, someone who buys a can of soda at 3PM one day might come back for lunch at 1:30PM the next day because they know you are open. A large portion of the opening hours of a most retail companies can be considered as either advertising for trade at the profitable hours or as loss making times that you can’t close because you can’t send an employee home for an hour.

If you have seating for 28 people (as my cafe did) then for about half the opening hours you will probably have 2 or fewer customers in there at any time, for about a quarter the opening hours you probably won’t cover the salary of the one person on duty. The weekend is when you make the real money, especially Friday and Saturday nights when you sometimes get all the seats full and people coming in for takeaway coffee and snacks. On Friday and Saturday nights the 60 seat restaurant next door to my cafe used to tell customers that my cafe made better coffee. It wasn’t economical for them to have a table full for an hour while they sell a few cups of coffee, they wanted customers to leave after dessert and free the table for someone who wants a meal with wine (alcohol is the real profit for many restaurants).

The plans of reopening with social distancing means that a 28 seat cafe can only have 14 chairs or less (some plans have 25% capacity which would mean 7 people maximum). That means decreasing the revenue of the most profitable times by 50% to 75% while also not decreasing the operating costs much. A small cafe has 2-3 staff when it’s crowded so there’s no possibility of reducing staff by 75% when reducing the revenue by 75%.

My Internet cafe would have closed immediately if forced to operate in the proposed social distancing model. It would have been 1/4 of the trade and about 1/8 of the profit at the most profitable times, even if enough customers are prepared to visit – and social distancing would kill the atmosphere. Most small businesses are barely profitable anyway, most small businesses don’t last 4 years in normal economic circumstances.

This reopen movement is about cutting unemployment benefits not about helping small business owners. Destroying small businesses is also good for big corporations, kill the small cafes and restaurants and McDonald’s and Starbucks will win. I think this is part of the motivation behind the astroturf campaign for reopening businesses.

Forbes has an article about this [1].

Psychological Issues

Some people claim that we should reopen businesses to help people who have psychological problems from isolation, to help victims of domestic violence who are trapped at home, to stop older people being unemployed for the rest of their lives, etc.

Here is one article with advice for policy makers from domestic violence experts [2]. One thing it mentions is that the primary US federal government program to deal with family violence had a budget of $130M in 2013. The main thing that should be done about family violence is to make it a priority at all times (not just when it can be a reason for avoiding other issues) and allocate some serious budget to it. An agency that deals with problems that affect families and only has a budget of $1 per family per year isn’t going to be able to do much.

There are ongoing issues of people stuck at home for various reasons. We could work on better public transport to help people who can’t drive. We could work on better healthcare to help some of the people who can’t leave home due to health problems. We could have more budget for carers to help people who can’t leave home without assistance. Wanting to reopen restaurants because some people feel isolated is ignoring the fact that social isolation is a long term ongoing issue for many people, and that many of the people who are affected can’t even afford to eat at a restaurant!

Employment discrimination against people in the 50+ age range is an ongoing thing, many people in that age range know that if they lose their job and can’t immediately find another they will be unemployed for the rest of their lives. Reopening small businesses won’t help that, businesses running at low capacity will have to lay people off and it will probably be the older people. Also the unemployment system doesn’t deal well with part time work. The Australian system (which I think is similar to most systems in this regard) reduces the unemployment benefits by $0.50 for every dollar that is earned in part time work, that effectively puts people who are doing part time work because they can’t get a full-time job in the highest tax bracket! If someone is going to pay for transport to get to work, work a few hours, then get half the money they earned deducted from unemployment benefits it hardly makes it worthwhile to work. While the exact health impacts of Covid19 aren’t well known at this stage it seems very clear that older people are disproportionately affected, so forcing older people to go back to work before there is a vaccine isn’t going to help them.

When it comes to these discussions I think we should be very suspicious of people who raise issues they haven’t previously shown interest in. If the discussion of reopening businesses seems to be someone’s first interest in the issues of mental health, social security, etc then they probably aren’t that concerned about such issues.

I believe that we should have a Universal Basic Income [3]. I believe that we need to provide better mental health care and challenge the gender ideas that hurt men and cause men to hurt women [4]. I believe that we have significant ongoing problems with inequality not small short term issues [5]. I don’t think that any of these issues require specific changes to our approach to preventing the transmission of disease. I also think that we can address multiple issues at the same time, so it is possible for the government to devote more resources to addressing unemployment, family violence, etc while also dealing with a pandemic.

Related posts:

  1. McDonalds – Wifi Without Power I am writing this post at a small cafe while...
  2. Nando’s Voucher Interpretation Every year my parents buy a book of vouchers for...
  3. economics of a computer store (why they don’t stock what you want) In some mailing list discussions recently some people demonstrated a...

Jonathan Dowland: Introducing Red Hat UBI OpenJDK runtime images

Tuesday 5th of May 2020 01:27:39 PM

At Red Hat we've just shipped Universal Base Image (UBI) build images for OpenJDK. Download them with your favourite container manager, e.g.:

podman pull podman pull

UBI, announced a year ago, is an initiative where you can obtain, share and build upon official Red Hat container images without needing a Red Hat subscription. Unlike something like CentOS, they aren't modified in any way (e.g. to remove branding), they're exactly the same base images that Red Hat products are built upon, composed entirely of Open Source software. Your precise rights are covered in the EULA.

I work on the Red Hat OpenJDK container images, which are designed primarily for use with OpenShift. We've been based upon the RHEL base images since inception. Although our containers are open source (of course), we haven't been able to distribute the binary images more widely than to Red Hat customers, until now.

These are based upon the "minimal" flavour base image and add an OpenJDK runtime and development packages (either JDK8 or JDK11, as per the image name) as well as Maven and OpenShift integration scripts that allow the image to be used with Source To Image.

If you give these a try, please let us know what you think! Comment here, on twitter, or file Issues or Pull Requests in the GitHub project.

Jonathan Dowland: Amiga floppy recovery project: what next?

Monday 4th of May 2020 07:08:32 PM

This is the ninth part in a series of blog posts. The previous post was Amiga floppy recovery project scope. The whole series is available here: Amiga.

It's been a while since I've reported on my Amiga floppy recovery project. With my bulky Philips CRT attached I slowly ground through the process of importing all my floppy disks, which is now done. The majority of disks were imported without errors. Commercial disks were the most likely to fail to import. Possibly I'd have more success with them if I used a different copying technique than X-COPY's default, but my focus was not on the commercial disks.

I have not yet restored the use of my LCD TV with the Amiga. I was waiting to hear back from Amiga Kit about an agreed return, but despite their web store claiming they're still open for business, I haven't been able to get any response to my emails to them since mid-February. I've given up, written off that order and bought an RGB/SCART adaptor elsewhere instead. Meanwhile my bulky CRT has returned to the loft.

The next step is pretty boring: time to check over my spreadsheet of disks, the imported copies of them, make sure everything is accounted for and decide whether to make further attempts for any that failed or write them off. For commercial disks it's almost certainly less effort to track down someone else's rip than to try mine again.

After that I can return to moving the Gotek floppy adaptor inside the Amiga case, as described in the last entry, and start playing.

Dima Kogan: Redirection and /dev/stderr

Monday 4th of May 2020 02:30:00 PM

I just hit some unexpected unix-ism (or maybe Linux-ism?) that I'd like to mention here. I found a workaround, but if anybody knows what's actually happening and can enlighten me, please send me an email, and I'll update the post.

OK, so I have a program that does stuff, and outputs its progress to stderr. And I'm invoking this from a shell script that also outputs stuff to stderr. A minimized sample:

echo "Outer print 1111" > /dev/stderr perl -E 'say STDERR ("1111 1111 1111 1111 1111 1111 1111 1111\n")' echo "Outer print 2222" > /dev/stderr perl -E 'say STDERR ("2222 2222 2222 2222 2222 2222 2222 2222\n")'

Let's call this This works just fine. Now let's say I want to log the output to a file. Naturally I do this:

./ 2> /tmp/log

Surprisingly, this does not work:

$ bash 2> /tmp/log && cat /tmp/log Outer print 2222 2222 2222 2222 2222 2222 2222 2222 2222 $ bash 2> /tmp/log && xxd /tmp/log 0000000: 4f75 7465 7220 7072 696e 7420 3232 3232 Outer print 2222 0000010: 0a00 0000 0000 0000 0000 0000 0000 0000 ................ 0000020: 0000 0000 0000 0000 0032 3232 3220 3232 .........2222 22 0000030: 3232 2032 3232 3220 3232 3232 2032 3232 22 2222 2222 222 0000040: 3220 3232 3232 2032 3232 3220 3232 3232 2 2222 2222 2222 0000050: 0a0a ..

The 1111 prints are gone, and we now have a block of binary 0 bytes in there for some reason. I had been assuming that writes to stderr block until they're finished. So I can, without thinking about it, mix writing to fd 2 and opening, and then writing to /dev/stderr. Apparently the right way to do this is to write to fd 2 in the script directly, instead of opening /dev/stderr:

echo "Outer print 1111" >&2 perl -E 'say STDERR ("1111 1111 1111 1111 1111 1111 1111 1111\n")' echo "Outer print 2222" >&2 perl -E 'say STDERR ("2222 2222 2222 2222 2222 2222 2222 2222\n")'



So I poked at it for a bit more, and it turns out that the obvious scapegoat (buffering) is actually not the issue. The problem is this:

$ ls -l /dev/stderr(:A) -rw-r--r-- 1 dima dima 36 May 4 17:02 /tmp/log

:A is the zsh syntax for recursive link-following. /dev/stderr is thus ultimately a link to /tmp/log. So the file on disk /tmp/log is being opened for writing and written-to twice at the same time

  • once in the outer 2>/tmp/log redirection
  • again in the inner >/dev/stderr redirection

Now we know. Thanks to Larry Doolittle for pointers.

More in Tux Machines

Software: Bashtop, Cointop and Auto-cpufreq

  • Bashtop – A Resource Monitoring Tool for Linux

    Bashtop is a terminal-based resource monitoring utility in Linux. It’s a nifty command-line tool that intuitively displays statistics for your CPU, memory, running processes, and bandwidth to mention just a few. It ships with a game-inspired and responsive terminal UI with a customizable menu. Monitoring various system metrics is made easy by the neat arrangement of various display sections. With Bashtop, you can also sort processes, as well as easily switch between the various sorting options. Additionally, you can send SIGKILL, SIGTERM, and SIGINT to the processes that you want.

  • Tracking Your CRYPTO INVESTMENTS Is Dead Simple With Cointop
  • Automatic CPU Speed & Power Optimizer Auto-cpufreq 1.2 Released

    Auto-cpufreq, automatic CPU speed & power optimizer for Linux to improve battery life, released version 1.2 with AMD support. Different to cpufreq indicator and / or TLP, Auto-cpufreq automatically make “cpufreq” related changes based on active monitoring of laptop’s battery state, CPU usage and system load. Ultimately allowing you to improve battery life without making any compromises.

today's howtos

Security, Fear, Uncertainty, Doubt

SUSE/OpenSUSE: Tumbleweed, YaST and Corporate Stuff

  • Skopeo, xxHash, GCC 10.2 are Among Updates in Tumbleweed

    openSUSE Tumbleweed had continuous daily snapshots with a handful of software package updates this week. Many minor-version updates and one major-version update became available to Tumbleweed users and the newest snapshot, 20200804, updated the iso-codes package, which lists country, language and currency names; the new 4.5.0 version updated translations and the subdivision names for Belarus. The Greybird Geeko theme was updated to improve contrast of gtk2 selection background color. The desktop calculator qalculate was updated to version 3.12.0 and improved exact simplification of roots. The fast hash algorithm xxhash 0.8.0 stablized the XXH3. Both libyui-ncurses and ncurses had minor updates. The snapshot is trending stable with a rating of 97, according to the Tumbleweed snapshot reviewer.

  • Digest of YaST Development Sprint 105

    Although a significant part of the YaST Team is enjoying their well deserved summer vacations, the development wheel keeps turning. During the latest two weeks we have fixed quite some bugs in several parts of (Auto)YaST. But listing fixed bugs it’s quite boring, so let’s focus on more interesting stuff we have also achieved.

  • Open Source for the Edge at IoT World

    As technologies converge to drive new innovation at the edge, organizations are working together more than ever to pave the road forward by combining the likes of 5G, AI/ML, Embedded Systems, High Performance Computing, Kubernetes, private/public environments and more. Companies are bringing specific domain expertise to the table, and SUSE is uniquely positioned with 28 years of Linux and open source expertise to serve as the foundation for developing, distributing and managing edge systems and the critical workloads they will support.

  • SUSE Partner Summit – Coming to a digital platform in mid-September!