Language Selection

English French German Italian Portuguese Spanish

Coverage From 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM)

Filed under
  • Is it time to remove ZONE_DMA?

    The DMA zone (ZONE_DMA) is a memory-management holdover from the distant past. Once upon a time, many devices (those on the ISA bus in particular) could only use 24 bits for DMA addresses, and were thus limited to the bottom 16MB of memory. Such devices are hard to find on contemporary computers. Luis Rodriguez scheduled the last memory-management-track session of the 2018 Linux Storage, Filesystem, and Memory-Management Summit to discuss whether the time has come to remove ZONE_DMA altogether.

  • Zone-lock and mmap_sem scalability

    The memory-management subsystem is a central point that handles all of the system's memory, so it is naturally subject to scalability problems as systems grow larger. Two sessions during the memory-management track of the 2018 Linux Storage, Filesystem, and Memory-Management Summit looked at specific contention points: the zone locks and the mmap_sem semaphore.

  • Hotplugging and poisoning

    Memory hotplugging is one of the least-loved areas of the memory-management subsystem; there are many use cases for it, but nobody has taken ownership of it. A similar situation exists for hardware page poisoning, a somewhat neglected mechanism for dealing with memory errors. At the 2018 Linux Storage, Filesystem, and Memory-Management summit, Michal Hocko and Mike Kravetz dedicated a pair of brief memory-management track sessions to problems that have been encountered in these subsystems, one of which seems more likely to get the attention it needs than the other.

  • Reworking page-table traversal

    A system's page tables are organized into a tree that is as many as five levels deep. In many ways those levels are all similar, but the kernel treats them all as being different, with the result that page-table manipulations include a fair amount of repetitive code. During the memory-management track of the 2018 Linux Storage, Filesystem, and Memory-Management Summit, Kirill Shutemov proposed reworking how page tables are maintained. The idea was popular, but the implementation is likely to be tricky.

  • get_user_pages() continued

    At a plenary session held relatively early during the 2018 Linux Storage, Filesystem, and Memory-Management Summit, the developers discussed a number of problems with the kernel's get_user_pages() interface. During the waning hours of LSFMM, a tired (but dedicated) set of developers convened again in the memory-management track to continue the discussion and try to push it toward a real solution.

    Jan Kara and Dan Williams scheduled the session to try to settle on a way to deal with the issues associated with get_user_pages() — in particular, the fact that code that has pinned pages in this way can modify those pages in ways that will surprise other users, such as filesystems. During the first session, Jérôme Glisse had suggested using the MMU notifier mechanism as a way to solve these problems. Rather than pin pages with get_user_pages(), kernel code could leave the pages unpinned and respond to notifications when the status of those pages changes. Kara said he had thought about the idea, and it seemed to make some sense.

  • XFS parent pointers

    At the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Allison Henderson led a session to discuss an XFS feature she has been working on: parent pointers. These would be pointers stored in extended attributes (xattrs) that would allow various tools to reconstruct the path for a file from its inode. In XFS repair scenarios, that path will help with reconstruction as well as provide users with better information about where the problems lie.

  • Shared memory mappings for devices

    In a short filesystem-only discussion at the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Jérôme Glisse wanted to talk about some (more) changes to support GPUs, FPGAs, and RDMA devices. In other talks at LSFMM, he discussed changes to struct page in support of these kinds of devices, but here he was looking to discuss other changes to support mapping a device's memory into multiple processes. It should be noted that I had a hard time following the discussion in this session, so there may be significant gaps in what follows.

  • A new API for mounting filesystems

    The mount() system call suffers from a number of different shortcomings that has led some to consider a different API. At last year's Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), that someone was Miklos Szeredi, who led a session to discuss his ideas for a new filesystem mounting API. Since then, David Howells has been working with Szeredi and VFS maintainer Al Viro on this API; at the 2018 LSFMM, he presented that work.

    He began by noting some of the downsides of the current mounting API. For one thing, you can pass a data page to the mount() call, but it is limited to a single page; if too many options are needed, or simply options with too many long parameters, they won't fit. The error messages and information on what went wrong could be better. There are also filesystems that have a bug where an invalid option will fail the mount() call but leave the superblock in an inconsistent state due to earlier options having been applied. Several in the audience were quick to note that both ext4 and XFS had fixed the latter bug along the way, though there may still be filesystems that have that behavior.

  • Controlling block-I/O latency

    Chris Mason and Josef Bacik led a brief discussion on the block-I/O controller for control groups (cgroups) in the filesystem track at the 2018 Linux Storage, Filesystem, and Memory-Management Summit. Mostly they were just aiming to get feedback on the approach they have taken. They are trying to address the needs of their employer, Facebook, with regard to the latency of I/O operations.

    Mason said that the goal is to strictly control the latency of block I/O operations, but that the filesystems themselves have priority inversions that make that difficult. For Btrfs and XFS, they have patches to tag the I/O requests, which mostly deals with the problem. They have changes for ext4 as well, but those are not quite working yet.

  • A mapping layer for filesystems

    In a plenary session on the second day of the Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Dave Chinner described his ideas for a virtual block address-space layer. It would allow "space accounting to be shared and managed at various layers in the storage stack". One of the targets for this work is for filesystems on thin-provisioned devices, where the filesystem is larger than the storage devices holding it (and administrators are expected to add storage as needed); in current systems, running out of space causes huge problems for filesystems and users because the filesystem cannot communicate that error in a usable fashion.

    His talk is not about block devices, he said; it is about a layer that provides a managed logical-block address (LBA) space. It will allow user space to make fallocate() calls that truly reserve the space requested. Currently, a filesystem will tell a caller that the space was reserved even though the underlying block device may not actually have that space (or won't when user space goes to use it), as in a thin-provisioned scenario. He also said that he would not be talking about his ideas for a snapshottable subvolume for XFS that was the subject of his talk at 2018.

More in Tux Machines

today's leftovers

  • GUADEC 2018 Reminiscences
    This year’s GUADEC in Almería, Spain, was over two months ago, and so here is a long overdue post about it. It was so long ago that I might as well call it a reminiscence! This will be a different kind of post than the ones I’ve done in past years, as plenty of other bloggers have already posted summaries about the talks.
  • Rugged, Linux-ready transportation PC has four SIM slots
    Nexcom’s Apollo Lake based “VTC 6220-BK” in-vehicle PC features triple displays, 2x SATA bays, 3x GbE with optional PoE, Ublox GPS, and 4x mini-PCIe or M.2 slots paired with SIM slots. Intel-based in-vehicle computers have been around for a while — here’s a Linux-friendly Kontron model from 2004 -– but over the last year or two the market has picked up considerably. Like many in-vehicle systems, Nexcom’s VTC 6220-BK is not an automotive IVI computer, but like Lanner’s Apollo Lake based V3G and V3S systems, is designed for buses. The rugged VTC 6220-BK straddles the IVI and telematics worlds, offering triple display support for passenger entertainment plus CAN and OBD connections.
  • FreeBSD Desktop – Part 16 – Configuration – Pause Any Application
    After using UNIX for so many years I knew that I could freeze (or pause) any process in the system with kill -17 (SIGSTOP) signal and then unfreeze it with with kill -19 (SIGCONT) signal as I described in the Process Management section of the Ghost in the Shell – Part 2 article. Doing it that way for the desktop applications is PITA to say the least. Can you imagine opening xterm(1) terminal and searching for all Chromium or Firefox processes and then freezing them one by one every time you need it? Me neither. Fortunately with introduction of so called X11 helper utilities – like xdotool(1) – it is now possible to implement it in more usable manner.
  • Custom Sustes Malware Infects Linux and IoT Servers Worldwide [Ed: This only impacts poorly-secured and already-cracked servers. The article overstates the risk.]
    The dangerous characteristic is the fact that an estimate of the infected computers cannot be made at this time. The only way to prevent the infiltrations is to strengthen the network security of the Linux and IoT servers exposed in public. It is very possible that further attacks will be carried out with other distribution tactics.
  • C Programming | Introduction | Features – For Beginners
    C is a general-purpose programming language developed by the ultimate god of the programming world, “Mr.Dennis Ritchie” (Creator of C programming ). The language is mainly used to create a wide range of applications for operating systems like windows and iOS. The popularity of the language can be clearly seen as this language has made to the list of top 10 programming languages in the world.

'We expect this is the bottom' in enterprise growth: Red Hat CEO

OSS Leftovers

  • AxonIQ Launches New Open Source Server
    AxonIQ, the company behind the open source Axon Framework, launches Axon 4.0 the open, integrated development and operations tool for Microservices and Event Sourcing on the JVM.
  • L10N Report: September Edition
  • Tidelift surpasses $1M to pay open source software maintainers
    Tidelift announced that it has surpassed one million dollars committed via its platform to pay open source software maintainers to provide professional assurances for their projects, as momentum behind this new approach to professional open source continues to build. Over 100 packages are already on the Tidelift platform, with maintainers getting paid to provide support for their packages through the Tidelift Subscription. Top packages featured include Vue, Material-UI, Babel, Gulp, Fabric, Active Admin, Doctrine, and StandardJS. With Tidelift, software development teams receive assurances around maintenance, security, and licensing from a single source. By bringing together maintainers with a global market of customers, Tidelift is helping make open source work better for everyone.
  • Artifex and First National Title Insurance Company Reach Settlement Over MuPDF Open Source Dispute
    Artifex Software, Inc. and First National Title Insurance Company announced today a confidential agreement to settle their legal dispute. Case No. 4: 18-cv-00503-SBA, filed by Artifex in the United States District Court for the Northern District of California, concerned the use of Artifex's open source software MuPDF under the GNU Affero General Public License and the GNU General Public License. While the parties had their differences in the interpretation of the open source licenses, the companies were able to reach an amicable resolution based on their mutual respect for and recognition of copyright protection and the open source philosophy. Terms of the settlement remain confidential.

EEE, Entryism and Openwashing

  • New Linux distro specifically designed for Windows comes to the Microsoft Store [Ed: WLinux or Whitewater Foundry not the first time people exploit Microsoft to put a price tag on FOSS such as LibreOffice. Microsoft is doing a fine job sabotaging the GNU/Linux 'ecosystem'.]
    WLinux is based on Debian, and the developer, Whitewater Foundry, claims their custom distro will also allow faster patching of security and compatibility issues that appear from time to time between upstream distros and WSL. [...] In return for saving developers time Whitewater Foundry is charging $19.99 (though the app is currently 50% off and the distribution can be downloaded from Github for free).
  • Open source dev gets Win32 apps running on Xbox One [Ed: Running blobs on two DRM platforms does not make you "Open source dev"]
  • Building Blocks of Secure Development: How to Make Open Source Work for You [Ed: Veracode self-promotion in "webinar" form, badmouthing FOSS to push their proprietary things. They work with Microsoft.]
  • SD Times open source project of the week: TonY [Ed: Openwashing of a surveillance operation at Microsoft]
    Unsatisfied with the available solutions for connecting the analytics-generating power of their TensorFlow machine learning implementations with the scalable data computation and storage capabilities of their Apache Hadoop clusters, developers at LinkedIn decided that they’d take matters into their own hands with the development of this week’s highlighted project, TonY.
  • Open Source: Automating Release Notes in Github [Ed: The New York Times is still propping up Microsoft hosting]
  • Opendesk launches augmented-reality shopping for its open-source furniture [Ed: Calling furniture "open"]
    Opendesk customers can now use augmented reality to see how the furniture brand's pieces look in their homes before ordering them from local makers. The augmented-reality (AR) experience launched with the arrival of Apple's iOS 12 operating system this week. It enables customers to use their smartphones to view some of Opendesk's furniture superimposed on the room in front of them.
  • Open Source Testing Startup Cypress Leaves Beta With Thousands of Users, Launches Paid Plans [Ed: This is not Open Source; they misuse the label and even put dashes ("open-source") because they know they're faking it.]‘s CEO Drew Lanham explains that the startup’s tool is software created by developers, for developers. The company was founded in 2014 by technologist Brian Mann, after observing that while computing and application development had changed drastically over the past decade, software testing had not. Large companies now release thousands of software updates a year, often on a daily basis across their organization. Technology teams aim to move rapidly, iterating on an agile basis and working in parallel so they can sync their code together even faster. But, as Lanham explains, the testing software out there was far outdated for these agile processes.
  • Kindred Introduces SenseAct, the First Reinforcement Learning Open-Source Toolkit for Physical Robots [Ed: Kindred or SenseAct not actually FOSS; but they sure try to make it seem that way, by focusing on a toolkit.]