Language Selection

English French German Italian Portuguese Spanish

Custom scripting gives users a safe-du

Filed under
HowTos

My company has a Linux cluster with a terabyte of attached storage. Over time we noticed the head node was becoming more overloaded. Inspection of the system showed that users were starting dozens of copies of the du utility to determine disk space usage. This was a natural thing for them to do, because they had a need to know how much disk space was available. A lack of disk space would cause their software builds and tests to fail. The problem was that it takes five to seven hours for a du of the entire shared filesystem. Thus, when the filesystem was nearly full (as it of course usually was), the number of du processes would increase almost exponentially.

To address this problem, we first set up automated nightly disk space reports, so that users could check the status without running du. This still did not solve the problem, as the amount of used space could fluctuate dramatically over the course of 24 hours. Users still wanted and needed to run their own du processes throughout the workday.

While adding more disk space would have solved the problem, we are using a large disk array that is already filled to maximum capacity. In general, users tend to fill up all available disk space anyway, no matter how much you give them.

We then developed a policy: users could run du on any directory they owned. In addition, user du processes would be allowed to run for a maximum of one hour of wall time. Users in the wheel group would be exempt from these restrictions.

I was given the task of developing a tool to implement this policy. Some sort of wrapper around the existing du seemed like an obvious choice: the script could validate the input, abort if an invalid path was given, and terminate the du process if it ran too long.

I wrote a basic bash script in perhaps an hour's time. Then I thought about how to run it, and that is where I ran into trouble. I had thought that I would make the script set user id (setuid) or set group id (setgid) root, i.e. when run by any user it would actually run in the root group. Then, I could change the permissions on the real du so that only root could run it. The result would be that normal users could only access the real du through the wrapper script.

Of course that would make a pretty boring article, and in reality it didn't turn out to be that simple:

Full Story.

More in Tux Machines

Openwashing Apple and Microsoft Proprietary Frameworks/Services

Viperr Linux Keeps Crunchbang Alive with a Fedora Flair

Do you remember Crunchbang Linux? Crunchbang (often referred to as #!) was a fan-favorite, Debian-based distribution that focused on using a bare minimum of resources. This was accomplished by discarding the standard desktop environment and using a modified version of the Openbox Window Manager. For some, Crunchbang was a lightweight Linux dream come true. It was lightning fast, easy to use, and hearkened back to the Linux of old. Read more

Openwashing Cars

  • Open source: sharing patents to speed up innovation
    Adjusting to climate change will require a lot of good ideas. The need to develop more sustainable forms of industry in the decades ahead demands vision and ingenuity. Elon Musk, chief executive of Tesla and SpaceX, believes he has found a way for companies to share their breakthroughs and speed up innovation. Fond of a bold gesture, the carmaker and space privateer announced back in 2014 that Tesla would make its patents on electric vehicle technology freely available, dropping the threat of lawsuits over its intellectual property (IP). Mr Musk argued the removal of pesky legal barriers would help “accelerate the advent of sustainable transport”. The stunning move has already had an impact. Toyota has followed Tesla by sharing more than 5,600 patents related to hydrogen fuel cell cars, making them available royalty free. Ford has also decided to allow competitors to use its own electric vehicle-related patents, provided they are willing to pay for licences. Could Telsa’s audacious strategy signal a more open approach to patents among leading innovators? And if more major companies should decide to adopt a carefree attitude to IP, what are the risks involved?
  • Autonomous car platform Apollo doesn't want you to reinvent the wheel
    Open source technologies are solving many of our most pressing problems, in part because the open source model of cooperation, collaboration, and almost endless iteration creates an environment where problems are more readily solved. As the adage goes, "given enough eyeballs, all bugs are shallow." However, self-driving vehicle technology is one rapidly growing area that hasn't been greatly influenced by open source. Most of today's autonomous vehicles, including those from Volkswagen, BMW, Volvo, Uber, and Google, ride on proprietary technology, as companies seek to be the first to deliver a successful solution. That changed recently with the launch of Baidu's Apollo.

today's leftovers

  • KDE Applications 18.04 Brings Dolphin Improvements, JuK Wayland Support
    The KDE community has announced the release today of KDE Applications 18.04 as the first major update to the open-source KDE application set for 2018.
  • Plasma Startup
    Startup is one of the rougher aspects of the Plasma experience and therefore something we’ve put some time into fixing [...] The most important part of any speed work is correctly analysing it. systemd-bootchart is nearly perfect for this job, but it’s filled with a lot of system noise.
  • Announcing Virtlyst – a web interface to manage virtual machines
    Virtlyst is a web tool that allows you to manage virtual machines. In essence it’s a clone of webvirtmgr, but using Cutelyst as the backend, the reasoning behind this was that my father in law needs a server for his ASP app on a Win2k server, the server has only 4 GiB of RAM and after a week running webvirtmgr it was eating 300 MiB close to 10% of all available RAM. To get a VNC or SPICE tunnel it spawns websockify which on each new instance around 20 MiB of RAM get’s used. I found this unacceptable, a tool that is only going to be used once in a while, like if the win2k freezes or goes BSOD, CPU usage while higher didn’t play a role on this.
  • OPNFV: driving the network towards open source "Tip to Top"
    Heather provides an update on the current status of OPNFV. How is its work continuing and how is it pursuing the overall mission? Heather says much of its work is really ‘devops’ and it's working on a continuous integration basis with the other open source bodies. That work continues as more bodies join forces with the Linux Foundation. Most recently OPNFV has signed a partnership agreement with the open compute project. Heather says the overall OPNFV objective is to work towards open source ‘Tip to top’ and all built by the community in ‘open source’. “When we started, OPNFV was very VM oriented (virtual machine), but now the open source movement is looking more to cloud native and containerisation as the way forward,” she says. The body has also launched a C-RAN project to ensure that NFV will be ready to underpin 5G networks as they emerge.
  • Ubuntu Podcast from the UK LoCo: S11E07 – Seven Years in Tibet - Ubuntu Podcast
  • Failure to automate: 3 ways it costs you
    When I ask IT leaders what they see as the biggest benefit to automation, “savings” is often the first word out of their mouths. They’re under pressure to make their departments run as efficiently as possible and see automation as a way to help them do so. Cost savings are certainly a benefit of automation, but I’d argue that IT leaders who pursue automation for cost-savings alone are missing the bigger picture of how it can help their businesses. The true value of automation doesn’t lie in bringing down expenses, but rather in enabling IT teams to scale their businesses.
  • Docker Enterprise Edition 2.0 Launches With Secured Kubernetes
    After months of development effort, Kubernetes is now fully supported in the stable release of the Docker Enterprise Edition. Docker Inc. officially announced Docker EE 2.0 on April 17, adding features that have been in development in the Docker Community Edition (CE) as well as enhanced enterprise grade capabilities. Docker first announced its intention to support Kubernetes in October 2017. With Docker EE 2.0, Docker is providing a secured configuration of Kubernetes for container orchestration. "Docker EE 2.0 brings the promise of choice," Docker Chief Operating Officer Scott Johnston told eWEEK. "We have been investing heavily in security in the last few years, and you'll see that in our Kubernetes integration as well."