Language Selection

English French German Italian Portuguese Spanish

Coverage From 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM)

Filed under
Linux
  • Is it time to remove ZONE_DMA?

    The DMA zone (ZONE_DMA) is a memory-management holdover from the distant past. Once upon a time, many devices (those on the ISA bus in particular) could only use 24 bits for DMA addresses, and were thus limited to the bottom 16MB of memory. Such devices are hard to find on contemporary computers. Luis Rodriguez scheduled the last memory-management-track session of the 2018 Linux Storage, Filesystem, and Memory-Management Summit to discuss whether the time has come to remove ZONE_DMA altogether.

  • Zone-lock and mmap_sem scalability

    The memory-management subsystem is a central point that handles all of the system's memory, so it is naturally subject to scalability problems as systems grow larger. Two sessions during the memory-management track of the 2018 Linux Storage, Filesystem, and Memory-Management Summit looked at specific contention points: the zone locks and the mmap_sem semaphore.

  • Hotplugging and poisoning

    Memory hotplugging is one of the least-loved areas of the memory-management subsystem; there are many use cases for it, but nobody has taken ownership of it. A similar situation exists for hardware page poisoning, a somewhat neglected mechanism for dealing with memory errors. At the 2018 Linux Storage, Filesystem, and Memory-Management summit, Michal Hocko and Mike Kravetz dedicated a pair of brief memory-management track sessions to problems that have been encountered in these subsystems, one of which seems more likely to get the attention it needs than the other.

  • Reworking page-table traversal

    A system's page tables are organized into a tree that is as many as five levels deep. In many ways those levels are all similar, but the kernel treats them all as being different, with the result that page-table manipulations include a fair amount of repetitive code. During the memory-management track of the 2018 Linux Storage, Filesystem, and Memory-Management Summit, Kirill Shutemov proposed reworking how page tables are maintained. The idea was popular, but the implementation is likely to be tricky.

  • get_user_pages() continued

    At a plenary session held relatively early during the 2018 Linux Storage, Filesystem, and Memory-Management Summit, the developers discussed a number of problems with the kernel's get_user_pages() interface. During the waning hours of LSFMM, a tired (but dedicated) set of developers convened again in the memory-management track to continue the discussion and try to push it toward a real solution.

    Jan Kara and Dan Williams scheduled the session to try to settle on a way to deal with the issues associated with get_user_pages() — in particular, the fact that code that has pinned pages in this way can modify those pages in ways that will surprise other users, such as filesystems. During the first session, Jérôme Glisse had suggested using the MMU notifier mechanism as a way to solve these problems. Rather than pin pages with get_user_pages(), kernel code could leave the pages unpinned and respond to notifications when the status of those pages changes. Kara said he had thought about the idea, and it seemed to make some sense.

  • XFS parent pointers

    At the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Allison Henderson led a session to discuss an XFS feature she has been working on: parent pointers. These would be pointers stored in extended attributes (xattrs) that would allow various tools to reconstruct the path for a file from its inode. In XFS repair scenarios, that path will help with reconstruction as well as provide users with better information about where the problems lie.

  • Shared memory mappings for devices

    In a short filesystem-only discussion at the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Jérôme Glisse wanted to talk about some (more) changes to support GPUs, FPGAs, and RDMA devices. In other talks at LSFMM, he discussed changes to struct page in support of these kinds of devices, but here he was looking to discuss other changes to support mapping a device's memory into multiple processes. It should be noted that I had a hard time following the discussion in this session, so there may be significant gaps in what follows.

  • A new API for mounting filesystems

    The mount() system call suffers from a number of different shortcomings that has led some to consider a different API. At last year's Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), that someone was Miklos Szeredi, who led a session to discuss his ideas for a new filesystem mounting API. Since then, David Howells has been working with Szeredi and VFS maintainer Al Viro on this API; at the 2018 LSFMM, he presented that work.

    He began by noting some of the downsides of the current mounting API. For one thing, you can pass a data page to the mount() call, but it is limited to a single page; if too many options are needed, or simply options with too many long parameters, they won't fit. The error messages and information on what went wrong could be better. There are also filesystems that have a bug where an invalid option will fail the mount() call but leave the superblock in an inconsistent state due to earlier options having been applied. Several in the audience were quick to note that both ext4 and XFS had fixed the latter bug along the way, though there may still be filesystems that have that behavior.

  • Controlling block-I/O latency

    Chris Mason and Josef Bacik led a brief discussion on the block-I/O controller for control groups (cgroups) in the filesystem track at the 2018 Linux Storage, Filesystem, and Memory-Management Summit. Mostly they were just aiming to get feedback on the approach they have taken. They are trying to address the needs of their employer, Facebook, with regard to the latency of I/O operations.

    Mason said that the goal is to strictly control the latency of block I/O operations, but that the filesystems themselves have priority inversions that make that difficult. For Btrfs and XFS, they have patches to tag the I/O requests, which mostly deals with the problem. They have changes for ext4 as well, but those are not quite working yet.

  • A mapping layer for filesystems

    In a plenary session on the second day of the Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Dave Chinner described his ideas for a virtual block address-space layer. It would allow "space accounting to be shared and managed at various layers in the storage stack". One of the targets for this work is for filesystems on thin-provisioned devices, where the filesystem is larger than the storage devices holding it (and administrators are expected to add storage as needed); in current systems, running out of space causes huge problems for filesystems and users because the filesystem cannot communicate that error in a usable fashion.

    His talk is not about block devices, he said; it is about a layer that provides a managed logical-block address (LBA) space. It will allow user space to make fallocate() calls that truly reserve the space requested. Currently, a filesystem will tell a caller that the space was reserved even though the underlying block device may not actually have that space (or won't when user space goes to use it), as in a thin-provisioned scenario. He also said that he would not be talking about his ideas for a snapshottable subvolume for XFS that was the subject of his talk at linux.conf.au 2018.

More in Tux Machines

Android Leftovers

today's leftovers

Games: Atari VCS Console, Humble Store and TUNG (The Ultimate Nerd Game)

  • Atari VCS Console Runs a Custom Linux Distro Called “AtariOS”
    Following criticism of its mediocre internal makeup the Ataris VCS console will now ship with 8GB RAM by default, up from the 4GB proposed during the funding push. It’s a decent increase in memory that should help the system cope better with more intensive indie games (don’t expect AAA titles to play nicely on the machine with the middling AMD Bristol Ridge APU).
  • Humble Store is doing a 'Pixel Perfect Platformers Sale' and it has some top Linux games for cheap
    For those of you who love your platformers, regardless of them being 2D, 3D, puzzle or action adventures there's bound to be something for the bored Linux gamer in the Humble Store Pixel Perfect Platformers Sale.
  • TUNG (The Ultimate Nerd Game) made me realise how stupid I really am
    The Ultimate Nerd Game or TUNG for short, is a first-person sandbox game about building intricate machines and it made me feel so very dumb. If you loved Minecraft's Redstone circuits or anything remotely similar, this is probably a free game you're going to love. For me, it was an exercise in frying my brain like it's in a microwave.

OSS Leftovers

  • Pharmaceutical industry gets first open source platform for Level 4 serialization
    Pharmaceutical companies today for the first time have an open source alternative for level 4 serialization with the launch of QU4RTET, a platform that provides them with new flexibility, transparency and affordability as they comply with global drug anti-counterfeiting laws.
  • Kontron Uses Open Source to Move Beyond Bare Metal
    Kontron, a company known for its embedded computing technology, is leveraging virtualization and open source to become a direct supplier to large service providers, promising to integrate hardware and operating system software with best-of-breed virtual network functions. That new sales strategy has evolved to support containers, particularly as they fit at the edge of the network, which for Kontron AG is the cell tower. In May, Kontron announced that its integrated SYMKLOUD open source platform now supports the latest versions of OpenStack for virtual machines and bare metal, as well as Kubernetes v1.10 for Docker and containers, via its distribution partnership with Canonical.
  • Open Source Expands In Finance With The FINOS Platform
  • Global Open Source Services Market Forecast to 2025 Published by Marketresearchnest
  • Synopsys ARC HS4x Processors Now Supported By GCC
    The GCC 8 compiler brought the Synopsys ARC CPU target while for the GCC 9 release is going to be support for the company's HS4x processors. Merged today to mainline GCC is support for the HS4x CPUs within the ARC target. Adding this newer generation of ARC processors to the GNU Compiler Collection code-base was just a few hundred lines of code with building off the existing target code.
  • GPL Cooperation Commitment gets more support for open source licensing
    Red Hat has announced its open source license enforcement initiative is making new strides. As part of the GPL Cooperation Commitment, 14 new companies have joined the effort to promote greater predictability for GPLv2 and LGPLv2.x licenses. “Through this initiative, we hope ultimately to increase participation in the use and development of open source software by helping to ensure that enforcement, when it takes place, is fair and predictable,” according to the commitment’s website.
  • The Global IP Exchange: Human ingenuity and open source technology
    He said: “Customers do increasingly care about open source, and if you don’t comply you are at risk of upsetting authors, as well as litigation and injunctions.” “If you’re just distributing internally, then you’re fine, but as soon as it leaves your company, then you’ve triggered an obligation.” For those who don’t comply, he warned that either the licensor, or the Free Software Foundation will find out.
  • How to Setup Python Virtual Environment on Ubuntu 18.04
    Python is a versatile programming language that can be used for many different programming projects(Web - Mobile - Desktop). Easy to set up, and written in a relatively straightforward style with immediate feedback on errors, Python is a great choice for beginners and experienced developers alike. Python 3 is the most current version of the language and is considered to be the future of Python. This article will guide you through installing Python 3 on your local Linux machine and setting up a programming virtual environment via the command line. This article will explicitly cover the installation procedures for Ubuntu 18.04, but the general principles apply to any other distribution of Debian Linux.
  • How expensive is globbing for sources in large projects
    Since we have the measurement script, let's use it for something more interesting. Modules are an upcoming C++ feature to increase build times and a ton of other coolness depending on who you ask. The current specification works by having a kind of "module export declaration" at the beginning of source files. The idea is that you first compile those to generate a sort of a module declaration file and then you can start the actual compilation that uses said files. If you thought "waitaminute, that sounds exactly like how FORTRAN is compiled", you are correct. Because of this it has the same problem that you can't compile source files in an arbitrary order, but instead you must first somehow scan them to find out the interdependencies between source (not header) files. In practice what this means is that instead of single-phase compilation all files must be processed twice. All scan operations must be done before any compilation jobs can start because otherwise you might start to compile a file before its dependencies are fully processed. The scanning can be done in one of two ways. Either the build system scans the sources meaning it needs to understand the syntax of source files or the compiler can be invoked in a special preprocessing mode. Note that build systems such as Ninja do not do any such operations by themselves but instead always invoke external processes to do their work.
  • Security updates for Monday