Language Selection

English French German Italian Portuguese Spanish

Coverage From 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM)

Filed under
Linux
  • Is it time to remove ZONE_DMA?

    The DMA zone (ZONE_DMA) is a memory-management holdover from the distant past. Once upon a time, many devices (those on the ISA bus in particular) could only use 24 bits for DMA addresses, and were thus limited to the bottom 16MB of memory. Such devices are hard to find on contemporary computers. Luis Rodriguez scheduled the last memory-management-track session of the 2018 Linux Storage, Filesystem, and Memory-Management Summit to discuss whether the time has come to remove ZONE_DMA altogether.

  • Zone-lock and mmap_sem scalability

    The memory-management subsystem is a central point that handles all of the system's memory, so it is naturally subject to scalability problems as systems grow larger. Two sessions during the memory-management track of the 2018 Linux Storage, Filesystem, and Memory-Management Summit looked at specific contention points: the zone locks and the mmap_sem semaphore.

  • Hotplugging and poisoning

    Memory hotplugging is one of the least-loved areas of the memory-management subsystem; there are many use cases for it, but nobody has taken ownership of it. A similar situation exists for hardware page poisoning, a somewhat neglected mechanism for dealing with memory errors. At the 2018 Linux Storage, Filesystem, and Memory-Management summit, Michal Hocko and Mike Kravetz dedicated a pair of brief memory-management track sessions to problems that have been encountered in these subsystems, one of which seems more likely to get the attention it needs than the other.

  • Reworking page-table traversal

    A system's page tables are organized into a tree that is as many as five levels deep. In many ways those levels are all similar, but the kernel treats them all as being different, with the result that page-table manipulations include a fair amount of repetitive code. During the memory-management track of the 2018 Linux Storage, Filesystem, and Memory-Management Summit, Kirill Shutemov proposed reworking how page tables are maintained. The idea was popular, but the implementation is likely to be tricky.

  • get_user_pages() continued

    At a plenary session held relatively early during the 2018 Linux Storage, Filesystem, and Memory-Management Summit, the developers discussed a number of problems with the kernel's get_user_pages() interface. During the waning hours of LSFMM, a tired (but dedicated) set of developers convened again in the memory-management track to continue the discussion and try to push it toward a real solution.

    Jan Kara and Dan Williams scheduled the session to try to settle on a way to deal with the issues associated with get_user_pages() — in particular, the fact that code that has pinned pages in this way can modify those pages in ways that will surprise other users, such as filesystems. During the first session, Jérôme Glisse had suggested using the MMU notifier mechanism as a way to solve these problems. Rather than pin pages with get_user_pages(), kernel code could leave the pages unpinned and respond to notifications when the status of those pages changes. Kara said he had thought about the idea, and it seemed to make some sense.

  • XFS parent pointers

    At the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Allison Henderson led a session to discuss an XFS feature she has been working on: parent pointers. These would be pointers stored in extended attributes (xattrs) that would allow various tools to reconstruct the path for a file from its inode. In XFS repair scenarios, that path will help with reconstruction as well as provide users with better information about where the problems lie.

  • Shared memory mappings for devices

    In a short filesystem-only discussion at the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Jérôme Glisse wanted to talk about some (more) changes to support GPUs, FPGAs, and RDMA devices. In other talks at LSFMM, he discussed changes to struct page in support of these kinds of devices, but here he was looking to discuss other changes to support mapping a device's memory into multiple processes. It should be noted that I had a hard time following the discussion in this session, so there may be significant gaps in what follows.

  • A new API for mounting filesystems

    The mount() system call suffers from a number of different shortcomings that has led some to consider a different API. At last year's Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), that someone was Miklos Szeredi, who led a session to discuss his ideas for a new filesystem mounting API. Since then, David Howells has been working with Szeredi and VFS maintainer Al Viro on this API; at the 2018 LSFMM, he presented that work.

    He began by noting some of the downsides of the current mounting API. For one thing, you can pass a data page to the mount() call, but it is limited to a single page; if too many options are needed, or simply options with too many long parameters, they won't fit. The error messages and information on what went wrong could be better. There are also filesystems that have a bug where an invalid option will fail the mount() call but leave the superblock in an inconsistent state due to earlier options having been applied. Several in the audience were quick to note that both ext4 and XFS had fixed the latter bug along the way, though there may still be filesystems that have that behavior.

  • Controlling block-I/O latency

    Chris Mason and Josef Bacik led a brief discussion on the block-I/O controller for control groups (cgroups) in the filesystem track at the 2018 Linux Storage, Filesystem, and Memory-Management Summit. Mostly they were just aiming to get feedback on the approach they have taken. They are trying to address the needs of their employer, Facebook, with regard to the latency of I/O operations.

    Mason said that the goal is to strictly control the latency of block I/O operations, but that the filesystems themselves have priority inversions that make that difficult. For Btrfs and XFS, they have patches to tag the I/O requests, which mostly deals with the problem. They have changes for ext4 as well, but those are not quite working yet.

  • A mapping layer for filesystems

    In a plenary session on the second day of the Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Dave Chinner described his ideas for a virtual block address-space layer. It would allow "space accounting to be shared and managed at various layers in the storage stack". One of the targets for this work is for filesystems on thin-provisioned devices, where the filesystem is larger than the storage devices holding it (and administrators are expected to add storage as needed); in current systems, running out of space causes huge problems for filesystems and users because the filesystem cannot communicate that error in a usable fashion.

    His talk is not about block devices, he said; it is about a layer that provides a managed logical-block address (LBA) space. It will allow user space to make fallocate() calls that truly reserve the space requested. Currently, a filesystem will tell a caller that the space was reserved even though the underlying block device may not actually have that space (or won't when user space goes to use it), as in a thin-provisioned scenario. He also said that he would not be talking about his ideas for a snapshottable subvolume for XFS that was the subject of his talk at linux.conf.au 2018.

More in Tux Machines

today's howtos

Software: Relational Databases, Low Footprint Web Browsers and Top 5 Linux Apps For Writing

  • 10 Excellent Free Linux Relational Databases
    A relational database matches data using common characteristics found within the data set. The resulting groups of data are organized and are much easier for people to understand. In such a database the data and relations between them are organized in tables. A table is a collection of records and each record in a table contains the same fields. Certain fields may be designated as keys, which means that searches for specific values of that field will use indexing to speed them up. The term relational database was originally defined by Edgar Codd, a British computer scientist, whilst working at IBM Almaden Research Center. He recognized that the concept of a relational database could be derived from the principles of relational algebra and relational calculus. A Relational Database Management System (RDBMS) is a Database Management System (DBMS) that is based on the relational model. Most database systems today are based on this type of system. A database is a vital system for any organization that stores mission critical information. The continual failure of a company’s database system can only lead to the demise of the organization – companies cannot do business without a working database system. To provide an insight into the quality of software that is available, we have compiled a list of 10 open source RDBMS. Hopefully, there will be something of interest here for anyone who wants to store data in an efficient way.
  • 10 Best lightweight browsers for Linux or Ubuntu
    Web Browsers, the day when they started making our lives easier by allowing us to crawl the internet to today’s world; they have been gone through numerous technological advancements. Browsers are quite advance to handle high-end graphics, online videos, apps and more without the help of third-party software. But this also has made them heavy in terms of consuming hardware resources, means more RAM and storage space. Such kind of browsers works well on good system configuration machines, however, Linux operating systems those are running on old PC or laptops or low configuration systems require light browsers with a minimal approach to work fast. Mainstream browser or shall I say the dominated one: Google Chrome that Linux users refrain themselves from instaling it on their machines is rather resourced consuming browser. This is the main reason why most of the Linux OS like Ubuntu, Linux Mint, Centos and more come with Firefox Mozilla but somewhere it still not that much lightweight as we need it to be. So, I have done some research and gathered some lightweight Linux browsers.
  • TenFourFox FPR13 SPR1 available
    TenFourFox Feature Parity Release 13 Security Parity Release 1 ("FPR13.1") is now available and live (downloads, hashes, release notes). The Pwn2Own vulnerabilities do not work on TenFourFox in their present form (not only because we're PowerPC but also because of our hybrid-endian typed arrays and other differences), but I have determined that TenFourFox-specific variant attacks could be exploitable, so we are patched as well. This should also reduce the risk of crashes from attempts to exploit mainline x86 Firefox. Meanwhile, H.264 support for TenFourFox FPR14 appears to be sticking. Yes, folks: for the first time you can now play Vimeo and other H.264-only videos from within TenFourFox using sidecar ffmpeg libraries, and it actually works pretty well! Kudos to Olga for the integration code! That said, however, it comes with a couple significant caveats. The first is that while WebM video tends not to occur in large numbers on a given page, H.264 videos nowadays are studded everywhere (Vimeo's front page, Twitter threads, Imgur galleries, etc.) and sometimes try to autoplay simultaneously. In its first iteration this would cause the browser to run out of memory if a large number of higher resolution videos tried to play at once, and sometimes crash when an infallible memory allocation fallibled. Right now there is a lockout in the browser to immediately halt all H.264 decoding if any instance runs out of memory so that the browser can save itself, but this needs a lot more testing to make sure it's solid, and is clearly a suboptimal solution. Remember that we are under unusual memory constraints because of the large amount of stack required for our JIT.
  • Top 5 Linux Apps For Writing
    Top 5 Linux Apps For Writing. Keeping in mind that not all Linux apps for writing are created equal nor do they each provide the same functionality. Today I’ll provide you with my top five Linux app picks for writing.

End of LibrePlanet 2019

  • Questioning and Finding Purpose
    This is copied over from my spiritual blog. I'm nervous doing that, especially at a point when I'm more vulnerable than usual in the Debian community. Still, this is who I am, and I want to be proud of that rather than hide it. And Debian and the free software community are about far more than just the programs we write. So hear goes: The Libreplanet opening keynote had me in tears. It was a talk by Dr. Tarek Loubani. He described his work as an emergency physician in Gaza and how 3d printers and open hardware are helping save lives. They didn't have enough stethoscopes; that was one of the critical needs. So, they imported a 3d printer, used that to print another 3d printer, and then began iterative designs of 3d-printable stethoscopes. By the time they were done, they had a device that performed as well or better than than a commercially available model. What was amazing is that the residents of Gaza could print their own; this didn't introduce dependencies on some external organization. Instead, open/free hardware was used to help give people a sense of dignity, control of some part of their lives, and the ability to better save those who depended on them. Even more basic supplies were unavailable. The lack of tourniquets caused the death of some significant fraction of casualties in the 2014 war. The same solution—3d-printed tourniquets had an even more dramatic result. Dr. Loubani talked about how he felt powerless to change the world around him. He talked about how he felt like an insignificant ant.
  • LibrePlanet Day 2: Welcoming everyone to the world of free software
    One of the most important questions that free software is facing in the year 2019 is: how do we make the world of free software accessible to broader audiences? Vast numbers of people are using software every day -- how do we relate our message to something that is important to them, and then welcome them into our community? In order to achieve our mission, we need to invite people and get them to use, create, and proliferate ethical software, until it replaces until all technology is free. Many of the best talks at LibrePlanet 2019 echoed a message for the free software community to focus on building a culture that's respectful and encouraging for new people, respecting a wide variety of personalities and values. The first way to get people invested in the culture of free software is to make it fun, and that was the focus of the morning keynote, "Freedom is fun!", delivered by free software veteran Bdale Garbee. A prominent name in the free software world for decades, Bdale talked about how he has a habit of turning all of his hobbies into free software projects, starting with model rockets.

Python Programming: PyPy 7.1 and More

  • PyPy v7.1 released; now uses utf-8 internally for unicode strings
    The interpreters are based on much the same codebase, thus the double release. This release, coming fast on the heels of 7.0 in February, finally merges the internal refactoring of unicode representation as UTF-8. Removing the conversions from strings to unicode internally lead to a nice speed bump. We merged the utf-8 changes to the py3.5 branch (Python3.5.3) but will concentrate on 3.6 going forward. We also improved the ability to use the buffer protocol with ctype structures and arrays.
  • PyPy 7.1 As The Well Known Alternative Python Implementation
    Last month brought the release of PyPy 7.0 as the JIT-ed performance-optimized Python implementation. PyPy 7.0 brought alpha Python 3.6 support, an updated CFFI module, and other enhancements. Out now is PyPy 7.1 as its successor.
  • Python’s “else” clause for loops
  • EuroPython 2019: Presenting our conference logo for Basel
    The logo is inspired by graphical elements from the Basel Jean Tinguely Museum and Basel Rhine Swimming. It was again created by our designer Jessica Peña Moro from Simétriko, who had already helped us in previous years with the conference design.