Language Selection

English French German Italian Portuguese Spanish

Custom scripting gives users a safe-du

Filed under
HowTos

My company has a Linux cluster with a terabyte of attached storage. Over time we noticed the head node was becoming more overloaded. Inspection of the system showed that users were starting dozens of copies of the du utility to determine disk space usage. This was a natural thing for them to do, because they had a need to know how much disk space was available. A lack of disk space would cause their software builds and tests to fail. The problem was that it takes five to seven hours for a du of the entire shared filesystem. Thus, when the filesystem was nearly full (as it of course usually was), the number of du processes would increase almost exponentially.

To address this problem, we first set up automated nightly disk space reports, so that users could check the status without running du. This still did not solve the problem, as the amount of used space could fluctuate dramatically over the course of 24 hours. Users still wanted and needed to run their own du processes throughout the workday.

While adding more disk space would have solved the problem, we are using a large disk array that is already filled to maximum capacity. In general, users tend to fill up all available disk space anyway, no matter how much you give them.

We then developed a policy: users could run du on any directory they owned. In addition, user du processes would be allowed to run for a maximum of one hour of wall time. Users in the wheel group would be exempt from these restrictions.

I was given the task of developing a tool to implement this policy. Some sort of wrapper around the existing du seemed like an obvious choice: the script could validate the input, abort if an invalid path was given, and terminate the du process if it ran too long.

I wrote a basic bash script in perhaps an hour's time. Then I thought about how to run it, and that is where I ran into trouble. I had thought that I would make the script set user id (setuid) or set group id (setgid) root, i.e. when run by any user it would actually run in the root group. Then, I could change the permissions on the real du so that only root could run it. The result would be that normal users could only access the real du through the wrapper script.

Of course that would make a pretty boring article, and in reality it didn't turn out to be that simple:

Full Story.

More in Tux Machines

Intel Sandy Bridge Gets A Surprise Boost From Linux 3.17

Besides the recent work to support OpenGL Geometry Shaders for Sandy Bridge in Mesa, users of Intel "Sandy Bridge" HD Graphics can also be thankful for the forthcoming Linux 3.17 kernel. Early testing of Linux 3.17 has revealed that for at least some Intel Sandy Bridge hardware are OpenGL performance improvements with the newer kernel code. Read more

Open Source Okavango14: The Heartbeat of the Delta

We can hear this heartbeat by listening to what the environment tells us through sensors and testing. I proposed that we build low cost sensors using open source hardware and software. In recent years there has been quite a disruption in computing ability as a result of the prevalence of smartphones. Increasingly small and powerful components and processors have created an opportunities that we would have never thought possible. One of the results of that is the single-board Raspberry Pi computer. Originally, the Raspberry Pi was created to enable students to learn hardware and software development. For the Okavango Wilderness Project, we are using them to take environmental readings and send those to us for inclusion into the Into The Okavango website. Jer will cover this more in his expedition post. We are using them to measure water temperature, pH, conductivity, total dissolved solids, salinity, and specific gravity. Read more

Kochi innovator Arvind Sanjeev makes Google Glass clone for Rs 4,500

Instead of commercializing the product and with the intention of contributing to the community, Sanjeev posted a blog explaining how his 'Smart Cap' can be built by anyone using opensource hardware such as a Rasberry Pi computer, an Arduino board and Android software. Read more

Alfresco Raises A Fresh $45M To Fuel Open-Source Enterprise Content Management

Alfresco, an open source, enterprise content management startup, is today announcing a new round of funding of $45 million — a Series D round that is more than twice as big as all of its previous rounds put together. The UK-based company competes against legacy services like Documentum (which was co-founded by one of Alfresco’s co-founders, John Newton) and Sharepoint to help large organisations manage their disparate document storage both in the cloud and on-premises, and also offer versioning control and other compliance requirements across mobile, PC and other devices. Alfresco will use the new funding to step its business up a gear, with new sales and marketing efforts, and moves into more cloud-based services that could see it competing more directly also against the likes of Dropbox, Box and Huddle. Read more