Language Selection

English French German Italian Portuguese Spanish

Debian

Syndicate content
Planet Debian - https://planet.debian.org/
Updated: 4 hours 6 min ago

Matthew Garrett: Filesystem deduplication is a sidechannel

Monday 27th of July 2020 07:57:19 PM
First off - nothing I'm going to talk about in this post is novel or overly surprising, I just haven't found a clear writeup of it before. I'm not criticising any design decisions or claiming this is an important issue, just raising something that people might otherwise be unaware of.

With that out of the way: Automatic deduplication of data is a feature of modern filesystems like zfs and btrfs. It takes two forms - inline, where the filesystem detects that data being written to disk is identical to data that already exists on disk and simply references the existing copy rather than, and offline, where tooling retroactively identifies duplicated data and removes the duplicate copies (zfs supports inline deduplication, btrfs only currently supports offline). In a world where disks end up with multiple copies of cloud or container images, deduplication can free up significant amounts of disk space.

What's the security implication? The problem is that deduplication doesn't recognise ownership - if two users have copies of the same file, only one copy of the file will be stored[1]. So, if user a stores a file, the amount of free space will decrease. If user b stores another copy of the same file, the amount of free space will remain the same. If user b is able to check how much free space is available, user b can determine whether the file already exists.

This doesn't seem like a huge deal in most cases, but it is a violation of expected behaviour (if user b doesn't have permission to read user a's files, user b shouldn't be able to determine whether user a has a specific file). But we can come up with some convoluted cases where it becomes more relevant, such as law enforcement gaining unprivileged access to a system and then being able to demonstrate that a specific file already exists on that system. Perhaps more interestingly, it's been demonstrated that free space isn't the only sidechannel exposed by deduplication - deduplication has an impact on access timing, and can be used to infer the existence of data across virtual machine boundaries.

As I said, this is almost certainly not something that matters in most real world scenarios. But with so much discussion of CPU sidechannels over the past couple of years, it's interesting to think about what other features also end up leaking information in ways that may not be obvious.

(Edit to add: deduplication isn't enabled on zfs by default and is explicitly triggered on btrfs, so unless it's something you've enabled then this isn't something that affects you)

[1] Deduplication is usually done at the block level rather than the file level, but given zfs's support for variable sized blocks, identical files should be deduplicated even if they're smaller than the maximum record size

comments

Wouter Verhelst: giphy.gif

Monday 27th of July 2020 04:00:09 PM

Wouter Verhelst: On Statements, Facts, Hypotheses, Science, Religion, and Opinions

Monday 27th of July 2020 03:52:03 PM

The other day, we went to a designer's fashion shop whose owner was rather adamant that he was never ever going to wear a face mask, and that he didn't believe the COVID-19 thing was real. When I argued for the opposing position, he pretty much dismissed what I said out of hand, claiming that "the hospitals are empty dude" and "it's all a lie". When I told him that this really isn't true, he went like "well, that's just your opinion". Well, no -- certain things are facts, not opinions. Even if you don't believe that this disease kills people, the idea that this is a matter of opinion is missing the ball by so much that I was pretty much stunned by the level of ignorance.

His whole demeanor pissed me off rather quickly. While I disagree with the position that it should be your decision whether or not to wear a mask, it's certainly possible to have that opinion. However, whether or not people need to go to hospitals is not an opinion -- it's something else entirely.

After calming down, the encounter got me thinking, and made me focus on something I'd been thinking about before but hadn't fully forumlated: the fact that some people in this world seem to misunderstand the nature of what it is to do science, and end up, under the claim of being "sceptical", with various nonsense things -- see scientology, flat earth societies, conspiracy theories, and whathaveyou.

So, here's something that might (but probably won't) help some people figuring out stuff. Even if it doesn't, it's been bothering me and I want to write it down so it won't bother me again. If you know all this stuff, it might be boring and you might want to skip this post. Otherwise, take a deep breath and read on...

Statements are things people say. They can be true or false; "the sun is blue" is an example of a statement that is trivially false. "The sun produces light" is another one that is trivially true. "The sun produces light through a process that includes hydrogen fusion" is another statement, one that is a bit more difficult to prove true or false. Another example is "Wouter Verhelst does not have a favourite color". That happens to be a true statement, but it's fairly difficult for anyone that isn't me (or any one of the other Wouters Verhelst out there) to validate as true.

While statements can be true or false, combining statements without more context is not always possible. As an example, the statement "Wouter Verhelst is a Debian Developer" is a true statement, as is the statement "Wouter Verhelst is a professional Volleybal player"; but the statement "Wouter Verhelst is a professional Volleybal player and a Debian Developer" is not, because while I am a Debian Developer, I am not a professional Volleybal player -- I just happen to share a name with someone who is.

A statement is never a fact, but it can describe a fact. When a statement is a true statement, either because we trivially know what it states to be true or because we have performed an experiment that proved beyond any possible doubt that the statement is true, then what the statement describes is a fact. For example, "Red is a color" is a statement that describes a fact (because, yes, red is definitely a color, that is a fact). Such statements are called statements of fact. There are other possible statements. "Grass is purple" is a statement, but it is not a statement of fact; because as everyone knows, grass is (usually) green.

A statement can also describe an opinion. "The Porsche 911 is a nice car" is a statement of opinion. It is one I happen to agree with, but it is certainly valid for someone else to make a statement that conflicts with this position, and there is nothing wrong with that. As the saying goes, "opinions are like assholes: everyone has one". Statements describing opinions are known as statements of opinion.

The differentiating factor between facts and opinions is that facts are universally true, whereas opinions only hold for the people who state the opinion and anyone who agrees with them. Sometimes it's difficult or even impossible to determine whether a statement is true or not. The statement "The numbers that win the South African Powerball lottery on the 31st of July 2020 are 2, 3, 5, 19, 35, and powerball 14" is not a statement of fact, because at the time of writing, the 31st of July 2020 is in the future, which at this point gives it a 1 in 24,435,180 chance to be true). However, that does not make it a statement of opinion; it is not my opinion that the above numbers will win the South African powerball; instead, it is my guess that those numbers will be correct. Another word for "guess" is hypothesis: a hypothesis is a statement that may be universally true or universally false, but for which the truth -- or its lack thereof -- cannot currently be proven beyond doubt. On Saturday, August 1st, 2020 the above statement about the South African Powerball may become a statement of fact; most likely however, it will instead become a false statement.

An unproven hypothesis may be expressed as a matter of belief. The statement "There is a God who rules the heavens and the Earth" cannot currently (or ever) be proven beyond doubt to be either true or false, which by definition makes it a hypothesis; however, for matters of religion this is entirely unimportant, as for believers the belief that the statement is correct is all that matters, whereas for nonbelievers the truth of that statement is not at all relevant. A belief is not an opinion; an opinion is not a belief.

Scientists do not deal with unproven hypotheses, except insofar that they attempt to prove, through direct observation of nature (either out in the field or in a controlled laboratory setting) that the hypothesis is, in fact, a statement of fact. This makes unprovable hypotheses unscientific -- but that does not mean that they are false, or even that they are uninteresting statements. Unscientific statements are merely statements that science cannot either prove or disprove, and that therefore lie outside of the realm of what science deals with.

Given that background, I have always found the so-called "conflict" between science and religion to be a non-sequitur. Religion deals in one type of statements; science deals in another. The do not overlap, since a statement can either be proven or it cannot, and religious statements by their very nature focus on unprovable belief rather than universal truth. Sure, the range of things that science has figured out the facts about has grown over time, which implies that religious statements have sometimes been proven false; but is it heresy to say that "animals exist that can run 120 kph" if that is the truth, even if such animals don't exist in, say, Rome?

Something very similar can be said about conspiracy theories. Yes, it is possible to hypothesize that NASA did not send men to the moon, and that all the proof contrary to that statement was somehow fabricated. However, by its very nature such a hypothesis cannot be proven or disproven (because the statement states that all proof was fabricated), which therefore implies that it is an unscientific statement.

It is good to be sceptical about what is being said to you. People can have various ideas about how the world works, but only one of those ideas -- one of the possible hypotheses -- can be true. As long as a hypothesis remains unproven, scientists love to be sceptical themselves. In fact, if you can somehow prove beyond doubt that a scientific hypothesis is false, scientists will love you -- it means they now know something more about the world and that they'll have to come up with something else, which is a lot of fun.

When a scientific experiment or observation proves that a certain hypothesis is true, then this probably turns the hypothesis into a statement of fact. That is, it is of course possible that there's a flaw in the proof, or that the experiment failed (but that the failure was somehow missed), or that no observance of a particular event happened when a scientist tried to observe something, but that this was only because the scientist missed it. If you can show that any of those possibilities hold for a scientific proof, then you'll have turned a statement of fact back into a hypothesis, or even (depending on the exact nature of the flaw) into a false statement.

There's more. It's human nature to want to be rich and famous, sometimes no matter what the cost. As such, there have been scientists who have falsified experimental results, or who have claimed to have observed something when this was not the case. For that reason, a scientific paper that gets written after an experiment turned a hypothesis into fact describes not only the results of the experiment and the observed behavior, but also the methodology: the way in which the experiment was run, with enough details so that anyone can retry the experiment.

Sometimes that may mean spending a large amount of money just to be able to run the experiment (most people don't have an LHC in their backyard, say), and in some cases some of the required materials won't be available (the latter is expecially true for, e.g., certain chemical experiments that involve highly explosive things); but the information is always there, and if you spend enough time and money reading through the available papers, you will be able to independently prove the hypothesis yourself. Scientists tend to do just that; when the results of a new experiment are published, they will try to rerun the experiment, partially because they want to see things with their own eyes; but partially also because if they can find fault in the experiment or the observed behavior, they'll have reason to write a paper of their own, which will make them a bit more rich and famous.

I guess you could say that there's three types of people who deal with statements: scientists, who deal with provable hypotheses and statements of fact (but who have no use for unprovable hypotheses and statements of opinion); religious people and conspiracy theorists, who deal with unprovable hypotheses (where the religious people deal with these to serve a large cause, while conspiracy theorists only care about the unprovable hypotheses); and politicians, who should care about proven statements of fact and produce statements of opinion, but who usually attempt the reverse of those two these days

Anyway...

Steve Kemp: Growing food is fun.

Monday 27th of July 2020 12:00:00 PM

"I grew up on a farm" is something I sometimes what I tell people. It isn't true, but it is a useful shorthand. What is true is that my parents both come from a farming background, my father's family up in Scotland, my mother's down in Yorkshire.

Every summer my sisters and myself would have a traditional holiday at the seaside, which is what people do in the UK (Blackpool, Scarborough, Great Yarmouth, etc). Before, or after, that we'd spend the rest of the summer living on my grandmother's farm.

I loved spending time on the farm when I was a kid, and some of my earliest memories date from that time. For example I remember hand-feeding carrots to working dogs (alsatians) that were taller than I was. I remember trying to ride on the backs of those dogs, and how that didn't end well. In fact the one and only time I can recall my grandmother shouting at me, or raising her voice at all, was when my sisters and I spent an afternoon playing in the coal-shed. We were filthy and covered in coal-dust from head to toe. Awesome!

Anyway the only reason I bring this up is because I have a little bit of a farming background, largely irrelevant in my daily life, but also a source of pleasant memories. Despite it being an animal farm (pigs, sheep, cows) there was also a lot of home-grown food, which my uncle Albert would deliver/sell to people nearby out of the back of a van. That same van that would be used to ferry us to see the fireworks every November. Those evenings were very memorable too - they would almost always involve flasks of home-made vegetable soup.

Nowadays I live in Finland, and earlier in the year we received access to an allotment - a small piece of land (10m x 10m) for €50/year - upon which we can grow our own plants, etc.

My wife decided to plant flowers and make it look pretty. She did good.

I decided to plant "food". I might not have done this stuff from scratch before, but I was pretty familiar with the process from my youth, and also having the internet to hand to make the obvious searches such as "How do you know when you can harvest your garlic?"

Before I started I figured it couldn't be too hard, after all if you leave onions/potatoes in the refrigerator for long enough they start to grow! It isn't like you have to do too much to help them. In short it has been pretty easy and I'm definitely going to be doing more of it next year.

I've surprised myself by enjoying the process as much as I have. Every few days I go and rip up the weeds, and water the things we've planted. So far I've planted, and harvested, Radish, Garlic, Onions, and in a few more weeks I'll be digging up potatoes.

I have no particular point to this post, except to say that if you have a few hours spare a week, and a slab of land to hand upon which you can dig and plant I'd recommend it. Sure there were annoyances, and not a single one of the carrot-seeds I planted showed any sign of life, but the other stuff? The stuff that grew? Very tasty, om nom nom ..

(It has to be said that when we received the plot there was a jungle growing upon it. Once we tidied it all up we found raspberries, roses, and other things. The garlic I reaped was already growing so I felt like a cheat to harvest it. That said I did plant a couple of bulbs on my balcony so I could say "I grew this from scratch". Took a while, but I did indeed harvest my own garlic.)

Martin Michlmayr: ledger2beancount 2.4 released

Monday 27th of July 2020 08:07:57 AM

I released version 2.4 of ledger2beancount, a ledger to beancount converter.

There are two notable changes in this release:

  1. I fixed two regressions introduced in the last release. Sorry about the breakage!
  2. I improved support for hledger. I believe all syntax differences in hledger are supported now.

Here are the changes in 2.4:

  • Fix regressions introduced in version 2.3
    • Handle price directives with comments
    • Don't assume implicit conversion when price is on second posting
  • Improve support for hledger
    • Fix parsing of hledger tags
    • Support commas as decimal markers
    • Support digit group marks through commodity and D directives
    • Support end aliases directive
    • Support regex aliases
    • Recognise total balance assertions
    • Recognise sub-account balance assertions
  • Add support for define directive
  • Convert all uppercase metadata tags to all lowercase
  • Improve handling of ledger lots without cost
  • Allow transactions without postings
  • Fix parsing issue in commodity declarations
  • Support commodities that contain quotation marks
  • Add --version option to show version
  • Document problem of mixing apply and include

Thanks to Kirill Goncharov for pointing out one regressions, to Taylor R Campbell for for a patch, to Stefano Zacchiroli for some input, and finally to Simon Michael for input on hledger!

You can get ledger2beancount from GitHub

Russ Allbery: Summer haul

Monday 27th of July 2020 04:31:00 AM

I'm buying rather too many books at the moment and not reading enough of them (in part because I got back into Minecraft and in part because I got a bit stuck on a few difficult books). I think I've managed to get myself unstuck again, though, and have started catching up on reviews.

2020. It's kind of a lot. And I'm not even that heavily affected.

Katherine Addison — The Angel of the Crows (sff)
Marie Brennan — A Natural History of Dragons (sff)
Kacen Callender — Queen of the Conquered (sff)
Jo Clayton — Diadem from the Stars (sff)
Jo Clayton — Lamarchos (sff)
Jo Clayton — Irsud (sff)
Clifford D. Conner — The Tragedy of American Science (nonfiction)
Kate Elliott — Unconquerable Sun (sff)
Rory Fanning & Craig Hodges — Long Shot (nonfiction)
Michael Harrington — Socialism: Past & Future (nonfiction)
Nalo Hopkinson — Brown Girl in the Ring (sff)
Kameron Hurley — The Stars Are Legion (sff)
N.K. Jemisin — Emergency Skin (sff)
T. Kingfisher — A Wizard's Guide to Defensive Baking (sff)
T. Kingfisher — Nine Goblins (sff)
Michael Lewis — The Fifth Risk (nonfiction)
Paul McAuley — War of the Maps (sff)
Gretchen McCulloch — Because Internet (nonfiction)
Hayao Miyazaki — Nausicaä of the Valley of the Wind (graphic novel)
Annalee Newitz — The Future of Another Timeline (sff)
Nick Pettigrew — Anti-Social (nonfiction)
Rivers Solomon, et al. — The Deep (sff)
Jo Walton — Or What You Will (sff)
Erik Olin Wright — Stardust to Stardust (nonfiction)

Of these, I've already read and reviewed The Fifth Risk (an excellent book).

Russ Allbery: Review: Rise of the Warrior Cop

Monday 27th of July 2020 02:20:00 AM

Review: Rise of the Warrior Cop, by Radley Balko

Publisher: PublicAffairs Copyright: 2013 ISBN: 1-61039-212-4 Format: Kindle Pages: 336

As the United States tries, in fits and starts, to have a meaningful discussion about long-standing police racism, brutality, overreach, corruption, and murder, I've realized that my theoretical understanding of the history of and alternative frameworks for law enforcement is woefully lacking. Starting with a book by a conservative white guy is not the most ideal of approaches, but it's what I already had on hand, and it won't be the last book I read and review on this topic. (Most of my research so far has been in podcast form. I don't review those here, but I can recommend Ezra Klein's interviews with Ta-Nehisi Coates, Paul Butler, and, most strongly, sujatha baliga.)

Rise of the Warrior Cop is from 2013 and has had several moments of fame, no doubt helped by Balko's connections to the conservative and libertarian right. One of the frustrating facts of US politics is that critiques of the justice system from the right (and from white men) get more media attention than critiques from the left. That said, it's a generally well-respected book on the factual history of the topic, and police brutality and civil rights are among the points on which I have stopped-clock agreements with US libertarians.

This book is very, very libertarian.

In my callow youth, I was an ardent libertarian, so I've read a lot of US libertarian literature. It's a genre with its own conventions that become obvious when you read enough of it, and Rise of the Warrior Cop goes through them like a checklist. Use the Roman Republic (never the Roman Empire) as the starting point for any political discussion, check. Analyze the topic in the context of pre-revolutionary America, check. Spend considerable effort on discerning the opinions of the US founders on the topic since their opinions are always relevant to the modern world, check. Locate some point in the past (preferably before 1960) where the political issue was as good as it has ever been, check. Frame all changes since then as an erosion of rights through government overreach, check. Present your solution as a return to a previous era of respect for civil rights, check. Once you start recognizing the genre conventions, their prevalence in libertarian writing is almost comical.

The framing chapters therefore leave a bit to be desired, but the meat of the book is a useful resource. Starting with the 1970s and its use as a campaigning tool by Nixon, Balko traces a useful history of the war on drugs. And starting with the 1980s, the number of cites to primary sources and the evidence of Balko's own research increases considerably. If you want to know how US police turned into military cosplayers with body armor, heavy weapons, and armored vehicles, this book provides a lot of context and history.

One of the reasons why I view libertarians as allies of convenience on this specific issue is that drug legalization and disgust with the war on drugs have been libertarian issues for decades. Ideologically honest libertarians (and Balko appears to be one) are inherently skeptical of the police, so when the police overreach in an area of libertarian interest, they notice. Balko makes a solid argument, backed up with statistics, specific programs, legislation, and court cases, that the drug war and its accompanying lies about heavily-armed drug dealers and their supposed threat to police officers was the fuel for the growth of SWAT teams, no-knock search warrants, erosion of legal protections for criminal defendants, and de facto license for the police to ignore the scope and sometimes even the existence of warrants.

This book is useful support for the argument that fears for the safety of officers underlying the militarization of police forces are imaginary. One telling point that Balko makes repeatedly and backs with statistical and anecdotal evidence is that the police generally do not use raid tactics on dangerous criminals. On the contrary, aggressive raids are more likely to be used on the least dangerous criminals because they're faster, they're fun for the police (they provide an adrenaline high and let them play with toys), and they're essentially risk-free. If the police believe someone is truly dangerous, they're more likely to use careful surveillance and to conduct a quiet arrest at an unexpected moment. The middle-of-the-night armed break-ins with battering rams, tear gas, and flash-bangs are, tellingly, used against the less dangerous suspects.

This is part of Balko's overall argument that police equipment and tactics have become untethered from any realistic threat and have become cultural. He traces an acceleration of that trend to 9/11 and the resulting obsession with terrorism, which further opened the spigot of military hardware and "special forces" training. This became a point of competition between police departments, with small town forces that had never seen a terrorist and had almost no chance of a terrorist incident demanding their own armored vehicles. I've encountered this bizarre terrorism justification personally; one of the reasons my local police department gave in a public hearing for not having a policy against shooting at moving vehicles was "but what if terrorism?" I don't believe there has ever been a local terrorist attack.

SWAT in such places didn't involve the special training or dedicated personnel of large city forces; instead, it was a part-time duty for normal police officers, and frequently they were encouraged to practice SWAT tactics by using them at random for some otherwise normal arrest or search. Balko argues that those raids were more exciting than normal police work, leading to a flood of volunteers for that duty and a tendency to use them as much as possible. That in turn normalizes disconnecting police tactics from the underlying crime or situational risk.

So far, so good. But despite the information I was able to extract from it, I have mixed feelings about Rise of the Warrior Cop as a whole. At the least, it has substantial limitations.

First, I don't trust the historical survey of policing in this book. Libertarian writing makes for bad history. The constraints of the genre require overusing only a few points of reference, treating every opinion of the US founders as holy writ, and tying forward progress to a return to a previous era, all of which interfere with good analysis. Balko also didn't do the research for the historical survey, as is clear from the footnotes. The citations are all to other people's histories, not to primary sources. He's summarizing other people's histories, and you'll almost certainly get better history by finding well-respected historians who cover the same ground. (That said, if you're not familiar with Peel's policing principles, this is a good introduction.)

Second, and this too is unfortunately predictable in a libertarian treatment, race rarely appears in this book. If Balko published the same book today, I'm sure he would say more about race, but even in 2013 its absence is strange. I was struck while reading by how many examples of excessive police force were raids on west coast pot farms; yes, I'm sure that was traumatic, but it's not the demographic I would name as the most vulnerable to or affected by police brutality. West coast pot growers are, however, mostly white.

I have no idea why Balko made that choice. Perhaps he thought his target audience would be more persuaded by his argument if he focused on white victims. Perhaps he thought it was an easier and less complicated story to tell. Perhaps, like a lot of libertarians, he doesn't believe racism has a significant impact on society because it would be a market failure. Perhaps those were the people who more readily came to mind. But to talk about police militarization, denial of civil rights, and police brutality in the United States without putting race at the center of both the history and the societal effects leaves a gaping hole in the analysis.

Given that lack of engagement, I also am dubious of Balko's policy prescriptions. His reform suggestions aren't unreasonable, but they stay firmly in the centrist and incrementalist camp and would benefit white people more than black people. Transparency, accountability, and cultural changes are all fine and good, but the cultural change Balko is focused on is less aggressive arrest tactics, more use of mediation, and better physical fitness. I would not object to those things (well, maybe the last, which seemed odd), but we need to have a discussion about police white supremacist organizations, the prevalence of spousal abuse, and the police tendency to see themselves not as public servants but as embattled warriors who are misunderstood by the naive sheep they are defending.

And, of course, you won't find in Rise of the Warrior Cop any thoughtful wrestling with whether there are alternative approaches to community safety, whether punitive rather than restorative justice is effective, or whether crime is a symptom of deeper societal problems we could address but refuse to. The most radical suggestion Balko has is to legalize drugs, which is both the predictable libertarian position and, as we have seen from recent events in the United States, far from the only problem of overcriminalization.

I understand why this book is so frequently mentioned on-line, and its author's political views may make it more palatable to some people than a more race-centered or radical perspective. But I don't think this is the best or most useful book on police violence that one could read today. I hope to find a better one in upcoming reviews.

Rating: 6 out of 10

Enrico Zini: Consent links

Sunday 26th of July 2020 10:00:00 PM
Love potions do not equal consent consent archive.org 2020-07-27 Teaching Consent to Small Children consent education archive.org 2020-07-27 Teaching consent is ongoing, but it starts when children are very young. It involves both teaching children to pay attention to and respect others' consent (or lack thereof) and teaching children that they should expect their own bodies and their own space to be respected---even by their parents and other relatives. And if children of two or four can be expected to read the nonverbal cues and expressions of children not yet old enough to talk in order to assess whether there is consent, what excuse do full grown adults have? Darling, We Don't Play With Our Vulvas At The Table consent education sex archive.org 2020-07-27 Small children have no sense of shame or disgust or fear of their bodies. A body is what it is. It does what it does. This is creepy and eye opening consent education archive.org 2020-07-27 About commonly accepted violation of children boundaries Personal boundaries - Wikipedia consent empowerment selfcare archive.org 2020-07-27 Personal boundaries are guidelines, rules or limits that a person creates to identify reasonable, safe and permissible ways for other people to behave towards them and how they will respond when someone passes those limits.[1] They are built out of a mix of conclusions, beliefs, opinions, attitudes, past experiences and social learning.[2][3] This concept or life skill has been widely referenced in self-help books and used in the counseling profession since the mid-1980s.[4]

Holger Levsen: 20200726-lts-survey

Sunday 26th of July 2020 12:40:48 PM
Final call to participate in the LTS survey

After 6 years of existence, we, the Debian LTS contributors, wanted to run a survey to learn more about how Debian LTS is used and perceived. Please take a few minutes to participate in the survey at https://surveys.debian.net/...!

Also, please hurry up, the survey will close at the end of July 27th on Samoa, which is in roughly 48h from now.

Russ Allbery: Review: Paladin's Grace

Sunday 26th of July 2020 04:25:00 AM

Review: Paladin's Grace, by T. Kingfisher

Publisher: Red Wombat Studio Copyright: 2020 ASIN: B0848Q8JVW Format: Kindle Pages: 399

Stephen was a paladin. Then his god died.

He was a berserker, an unstoppable warrior in the service of his god. Now, well, he's still a berserker, but going berserk when you don't have a god to control the results is not a good idea. He and his brothers were taken in by the Temple of the Rat, where they serve as guards, watch out for each other, and try to get through each day with an emptiness in their souls where a god should be.

Stephen had just finished escorting a healer through some of the poorer parts of town when a woman runs up to him and asks him to hide her. Their awkward simulated tryst is sufficient to fool the two Motherhood priests who were after her for picking flowers from the graveyard. Stephen then walks her home and that would have been the end of it, except that neither could get the other out of their mind.

Despite first appearances, and despite being set in the same world and sharing a supporting character, this is not the promised sequel to Swordheart (which is apparently still coming). It's an entirely different paladin story. T. Kingfisher (Ursula Vernon's nom de plume when writing for adults) has a lot of things to say about paladins! And, apparently, paladin-involved romances.

On the romance front, Kingfisher clearly has a type. The general shape of the story will be familiar from Swordheart and The Wonder Engine: An independent and occasionally self-confident woman with various quirks, a hunky paladin who is often maddeningly dense, and a lot of worrying on both sides about whether the other person is truly interested in them and if their personal liabilities make a relationship a horrible idea. This is not my preferred romance formula (it provokes the occasional muttered "for the love of god just talk to each other"), but I liked this iteration of it better than the previous two, mostly because of Grace.

Grace is a perfumer, a trade she went into by being picked out of a lineup of orphans by a master perfumer for her sense of smell. One of Kingfisher's strengths as a writer is showing someone get lost in their routine day-to-day competence. When mixed with an inherently fascinating profession, this creates a great reading experience. Grace is also an abuse survivor, which made the communication difficulties with Stephen more interesting and subtle. Grace has created space and a life for herself, and her unwillingness to take risks on changes is a deep part of her sense of self and personal safety. As her past is slowly revealed, Kingfisher puts the reader in a position to share Stephen's anger and protectiveness, but then consistently puts Grace's own choices, coping mechanisms, and irritated refusal to be protected back into the center of the story. She has to accept some help as she gets entangled in the investigation of a highly political staged assassination attempt, but both that help and the relationship come on her own terms. It's very well-done.

The plot was enjoyable enough, although it involved a bit too much of constantly rising stakes and turns for the worst for my taste, and the ending had a touch of deus ex machina. Like Kingfisher's other books, though, the delight is in the unexpected details. Stephen knitting socks. Grace's frustrated obsession with why he smells like gingerbread. The beautifully practical and respectful relationship between the Temple of the Rat and Stephen's band of former paladins. (After only two books in which they play a major role, the Temple of the Rat is already one of my favorite fantasy religions.) Everything about Bishop Beartongue. Grace's friend Marguerite. And a truly satisfying ending.

The best part of this book, though, is the way Grace is shown as a complete character in a way that even most books with well-rounded characterization don't manage. Some things she does make the reader's heart ache because of the hints they provide about her past, but they're also wise and effective safety mechanisms for her. Kingfisher gives her space to be competent and prickly and absent-minded. She has a complete life: friends, work, goals, habits, and little rituals. Grace meets someone and falls in love, but one can readily imagine her not falling in love and going on with her life and that result wouldn't be tragic. In short, she feels like a grown adult who has made her own peace with where she came from and what she is doing. The book provides her an opportunity for more happiness and more closure without undermining her independence. I rarely see this in a novel, and even more rarely done this well.

If you haven't read any of Kingfisher's books and are in the mood for faux-medieval city romance involving a perfumer and a bit of political skulduggery, this is a great place to start. If you liked Swordheart, you'll probably like Paladin's Grace; like me, you may even like it a bit more. Recommended, particularly if you want something light and heart-warming.

Rating: 8 out of 10

Niels Thykier: Support for Debian packaging files in IDEA (IntelliJ/PyCharm)

Saturday 25th of July 2020 04:28:34 PM

I have been using the community editions of IntelliJ and PyCharm for a while now for Python or Perl projects. But it started to annoy me that for Debian packaging bits it would “revert” into a fancy version of notepad. Being fed up with it, I set down and spent the last week studying how to write a plugin to “fix” this.

After a few prototypes, I have now released IDEA-debpkg v0.0.3 (Link to JetBrain’s official plugin listing with screenshots). It provides a set of basic features for debian/control like syntax highlighting, various degree of content validation, folding of long fields, code completion and “CTRL + hover” documentation. For debian/changelog, it is mostly just syntax highlighting with a bit of fancy linking for now. I have not done anything for debian/rules as I noted there is a Makefile plugin, which will have to do for now.

The code is available from github and licensed under Apache-2.0. Contributors, issues/feature requests and pull requests are very welcome. Among things I could help with are:

  • Icons – both for the plugin and for the file types. Currently it is just colored text, which is as far as my artistic skills got with the space provided.
  • Color and text formatting for syntax highlighting.
  • Reports of papercut or features that would be very useful to prioritize.
  • Review of the “CTRL + hover” documentation. I am hoping for something that is help for new contributors but I am very unlikely to have gotten it right (among other because I wrote most of it to “get it done” rather than “getting it right”)

I hope you will take it for spin if you have been looking for a bit of Debian packaging support to your PyCharm or other IDEA IDE.

Andrew Cater: How to use the signed checksum files to verify Debian media images

Saturday 25th of July 2020 09:52:47 AM
Following on from the blog post the other day in some sense: someone has asked on the debian-user list: "I do not understand from the given page (https://www.debian.org/CD/verify)  how to use .sign files and gpg in order to check verify the authenticity of debian cds. I understand the part with using sha256sum or sha512sum or md5sum to check whether the files were downloaded correctly."

Distributed with the CD and other media images on Debian CD mirrors, there are files like MD5SUM, MD5SUM.sign, SHA256SUM, SHA256SUM.sign and so on.

SHA512SUM is a plain text list of the SHA512SUMs for each of the files in the directory. SHA512SUM.sign is the GPG-signed version of that file. This allws for non-repudiation - if the signature is valid, then the plain text file has been signed by the owner of that key. Nothing has tampered with the checksums file since it was signed.

After downloading the SHA1SUM, SHA256SUM and SHA512SUM files and the corresponding .sign files from, say, the prime Debian CD mirror at http://cdimage.debian.org

Assuming that you already have GPG installed: sha256sum and sha512sum are installed by the coreutils package, which Debian installs by default.
gpg --verify SHA512SUMS.sign SHA512SUMS will verify the .sign signature file against the signed file.

gpg --verify SHA512SUMS.sign SHA512SUMS
gpg: Signature made Sun 10 May 2020 00:16:52 UTC
gpg:                using RSA key DF9B9C49EAA9298432589D76DA87E80D6294BE9B


The signature is as given on the Debian CD verification page given above.

You can import that key from the Debian key servers if you wish.
gpg --keyserver keyring.debian.org --recv-keys DF9B9C49EAA9298432589D76DA87E80D6294BE9B

You can import the signature for checking from the SKS keyservers which are often more available:
gpg --keyserver pool.sks-keyservers.net --recv-keys DF9B9C49EAA9298432589D76DA87E80D6294BE9B 

and you then get:

gpg --verify SHA512SUMS.sign SHA512SUMS
gpg: Signature made Sun 10 May 2020 00:16:52 UTC
gpg:                using RSA key DF9B9C49EAA9298432589D76DA87E80D6294BE9B
gpg: Good signature from "Debian CD signing key " [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D 6294 BE9B


My own key isn't in the Debian CD signing key ring - but this does now show me that this is a good signature from the primary key fingerprint as given.

Repeating the exercise from the other day and producing a Debian amd64 netinst file using jigdo, I can now check the checksum on the local .iso file against the checksum file distributed by Debian. If they match, it's a good sign that the CD I've generated is bit for bit identical. For my locally generated file:

sha512sum debian-10.4.0-amd64-netinst.iso
ec69e4bfceca56222e6e81766bf235596171afe19d47c20120783c1644f72dc605d341714751341051518b0b322d6c84e9de997815e0c74f525c66f9d9eb4295  debian-10.4.0-amd64-netinst.iso


and for the file checksum as distributed by Debian:

less SHA512SUMS | grep *iso
ec69e4bfceca56222e6e81766bf235596171afe19d47c20120783c1644f72dc605d341714751341051518b0b322d6c84e9de997815e0c74f525c66f9d9eb4295  debian-10.4.0-amd64-netinst.iso


and they match! 

As ever, I hope this blog post will help somebody.
[Edit: Someone has kindly pointed out that grep *iso SHA512SUMS | sha512sum -c will check this more efficiently.]














 


 

Craig Small: 25 Years of Free Software

Saturday 25th of July 2020 01:04:03 AM

When did I start writing Free Software, now called Open Source? That’s a tricky question. Does the time start with the first file edited, the first time it compiles or perhaps even some proto-program you use to work out a concept for the real program formed later on.

So using the date you start writing, especially in a era before decent version control systems, is problematic. That is why I use the date of the first release of the first package as the start date. For me, that was Monday 24th July 1995.

axdigi and before

My first released Free Software program was axdigi which was a layer-2 packet repeater for hamradio. This was uploaded to some FTP server, probably UCSD in late July 1995. The README is dated 24th July 1995.

There were programs before this. I had written a closed-source (probably undistributable) driver for the Gracilis PackeTwin serial card and also some sort of primitive wireshark/tcpdump thing for capturing packet radio. Funny thing is that the capture program is the predecessor of both axdigi and a system that was used by a major Australian ISP for their internet billing system.

Choosing Free Software

So you have written something you think others might like, what software license will you use to distribute it? In 1995 it wasn’t that clear. This was the era of strange boutique licenses including ones where it was ok to run the program as a hamradio operator but not a CB radio operator (or at least they tried to work it that way).

A friend of mine and the author of the Linux HAM HOWTO amongst other documents, Terry Dawson, suggested I use GPL or another Free Software license. He explained what this Free Software thing was and said that if you want your program to be the most useful then something like GPL will do it. So I released axdigi under the GPL license and most of my programs since then have used the same license. Something like MIT or BSD licenses would have been fine too, I was just not going to use something closed or hand-crafted.

That was a while ago, I’ve written or maintained many programs since then. I also became a Debian maintainer (23 years so far) and adopted both procps and psmisc which I still maintain as both the Debian developer and upstream to this day.

What Next?

So it has been 25 years or a quarter of a century, what will happen next? Probably more of the same, though I’m not sure I will be maintaining Free Software by the end of the next 25 years (I’ll be over 70 then). Perhaps the singularity will arrive and writing software will be something people only do at Rennie Festivals.

Come to the Festival! There is someone making horseshoes! Other there is a steam engine. See this other guy writing computer programs on a thing called keyboard!

Dirk Eddelbuettel: anytime 0.3.8: Minor Maintenance

Friday 24th of July 2020 08:11:00 PM

A new minor release of the anytime package arrived on CRAN overnight. This is the nineteenth release, and it comes just over six months after the previous release giving further indicating that we appear to have reached a nice level of stability.

anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, … format to either POSIXct or Date objects – and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples.

This release mostly plays games with CRAN. Given the lack of specification for setups on their end, reproducing test failures remains, to put it mildly, “somewhat challenging”. So we eventually gave up—and weaponed up once more and now explicitly test for the one distribution where tests failed (when they clearly passed everywhere else). With that we now have three new logical predicates for various Linux distribution flavours, and if that dreaded one is seen in one test file the test is skipped. And with that we now score twelve out of twelve OKs. This being a game of cat and mouse, I am sure someone somewhere will soon invent a new test…

The full list of changes follows.

Changes in anytime version 0.3.8 (2020-07-23)
  • A small utility function was added to detect the Linux distribution used in order to fine-tune tests once more.

  • Travis now uses Ubuntu 'bionic' and R 4.0.*.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. The issue tracker tracker off the GitHub repo can be use for questions and comments.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Mike Gabriel: Ayatana Indicators / IDO - Menu Rendering Fixed with vanilla GTK-3+

Friday 24th of July 2020 01:56:05 PM

At DebConf 17 in Montreal, I gave a talk about Ayatana Indicators [1] and the project's goal to continue the — by then already dropped out of maintenance — Ubuntu Indicators in a separate upstream project, detached from Ubuntu and its Ubuntu'isms.

Stalling

The whole Ayatana Indicators project received a bit of a show stopper by the fact that the IDO (Indicator Display Object) rendering was not working in vanilla GTK-3 without a certain patch [2] that only Ubuntu has in their GTK-3 package. Addressing GTK developers upstream some years back (after GTK 3.22 had already gone into long term maintenance mode) and asking for a late patch acceptance did not work out (as already assumed). Ayatana Indicators stalled at a level of 90% actually working fine, but those nice and shiny special widgets, like the calendar widget, the audio volume slider widgets, switch widgets, etc. could not be rendered appropriately in GTK based desktop environments (e.g. via MATE Indicator Applet) on other distros than Ubuntu.

I never really had the guts to sit down without a defined ending and find a patch / solution to this nasty problem. Ayatana Indicators stalled as a whole. I kept it alive and defended its code base against various GLib and what-not deprecations and kept it in Debian, but the software was actually partially broken / dysfunctional.

Taking the Dog for a Walk and then It Became all Light+Love

Several days back, I received a mail from Robert Tari [3]. I was outside on a hike with our dog and thought, ah well, let's check emails... I couldn't believe what I read then, 15 seconds later. I could in fact, hardly breathe...

I have known Robert from earlier email exchanges. Robert maintains various "little" upstream projects, like e.g. Caja Rename, Odio, Unity Mail, etc. that I have looked into earlier regarding Debian packaging. Robert is also a Manjaro contributor and he has been working on bringing Ayatana Indicators to Manjaro MATE. In the early days, without knowing Robert, I even forked one of his projects (indicator-notification) and turned it into an Ayatana Indicator.

Robert and I also exchanged some emails about Ayatana Indicators already a couple of weeks ago. I got the sense of him maybe being up to something already then. Oh, yeah!!!

It turned out that Robert and I share the same "love" for the Ubuntu Indicators concept [4]. From his email, it became clear that Robert had spent the last 1-2 weeks drowned in the Ayatana IDO and libayatana-indicator code and worked him self through the bowels of it in order to understand the code concept of Indicators to its very depth.

When emerging back from his journey, he presented me (or rather: the world) a patch [5] against libayatana-indicator that makes it possible to render IDO objects even if a vanilla GTK-3 is installed on the system. This patch is a game changer for Indicator lovers.

When Robert sent me his mail pointing me to this patch, I think, over the past five years, I have never felt more excited (except from the exact moment of getting married to my wife two-to-three years ago) than during that moment when my brain tried to process his email. "Like a kid on Christmas Eve...", Robert wrote in one of his later mails to me. Indeed, like a "kid on Christmas Eve", Robert.

Try It Out

As a proof of all this to the Debian people, I have just done the first release of ayatana-indicator-datetime and uploaded it to Debian's NEW queue. Robert is doing the same for Manjaro. The Ayatana Indicator Sound will follow after my vacation.

For fancy widget rendering in Ayatana Indicator's system indicators, make sure you have libayatana-indicator 0.7.0 or newer installed on your system.

Credits

One of the biggest thanks ever I send herewith to Robert Tari! Robert is now co-maintainer of Ayatana Indicators. Welcome! Now, there is finally a team of active contributors. This is so delightful!!!

References P.S.

Expect more Ayatana Indicators to appear in your favourite distro soon...

Raphaël Hertzog: The Debian Handbook has been updated for Debian 10

Friday 24th of July 2020 10:39:30 AM

Better late than never as we say… thanks to the work of Daniel Leidert and Jorge Maldonado Ventura, we managed to complete the update of my book for Debian 10 Buster.

You can get the electronic version on debian-handbook.info or the paperback version on lulu.com. Or you can just read it online.

Translators are busy updating their translations, with German and Norvegian Bokmal leading the way…

One comment | Liked this article? Click here. | My blog is Flattr-enabled.

Evgeni Golov: Building documentation for Ansible Collections using antsibull

Friday 24th of July 2020 08:01:10 AM

In my recent post about building and publishing documentation for Ansible Collections, I've mentioned that the Ansible Community is currently in the process of making their build tools available as a separate project called antsibull instead of keeping them in the hacking directory of ansible.git.

I've also said that I couldn't get the documentation to build with antsibull-docs as it wouldn't support collections yet. Thankfully, Felix Fontein, one of the maintainers of antsibull, pointed out that I was wrong and later versions of antsibull actually have partial collections support. So I went ahead and tried it again.

And what should I say? Two bug reports by me and four patches by Felix Fontain later I can use antsibull-docs to generate the Foreman Ansible Modules documentation!

Let's see what's needed instead of the ugly hack in detail.

We obviously don't need to clone ansible.git anymore and install its requirements manually. Instead we can just install antsibull (0.17.0 contains all the above patches). We also need Ansible (or ansible-base) 2.10 or never, which currently only exists as a pre-release. 2.10 is the first version that has an ansible-doc that can list contents of a collection, which antsibull-docs requires to work properly.

The current implementation of collections documentation in antsibull-docs requires the collection to be installed as in "Ansible can find it". We had the same requirement before to find the documentation fragments and can just re-use the installation we do for various other build tasks in build/collection and point at it using the ANSIBLE_COLLECTIONS_PATHS environment variable or the collections_paths setting in ansible.cfg1. After that, it's only a matter of passing --use-current to make it pick up installed collections instead of trying to fetch and parse them itself.

Given the main goal of antisibull-docs collection is to build documentation for multiple collections at once, it defaults to place the generated files into <dest-dir>/collections/<namespace>/<collection>. However, we only build documentation for one collection and thus pass --squash-hierarchy to avoid this longish path and make it generate documentation directly in <dest-dir>. Thanks to Felix for implementing this feature for us!

And that's it! We can generate our documentation with a single line now!

antsibull-docs collection --use-current --squash-hierarchy --dest-dir ./build/plugin_docs theforeman.foreman

The PR to switch to antsibull is open for review and I hope to get merged in soon!

Oh and you know what's cool? The documentation is now also available as a preview on ansible.com!

  1. Yes, the paths version of that setting is deprecated in 2.10, but as we support older Ansible versions, we still use it. 

Martin Michlmayr: beancount2ledger 1.1 released

Friday 24th of July 2020 07:04:56 AM

Martin Blais recently announced that he'd like to re-organize the beancount code and split out some functionality into separate projects, including the beancount to ledger/hledger conversion code previously provided by bean-report.

I agreed to take on the maintenance of this code and I've now released beancount2ledger, a beancount to ledger/hledger converter.

You can install beancount2ledger with pip:

pip3 install beancount2ledger

Please report issues to the GitHub tracker.

There are a number of outstanding issues I'll fix soon, but please report any other issues you encounter.

Note that I'm not very familiar with hledger. I intend to sync up with hledger author Simon Michael soon, but please file an issue if you notice any problems with the hledger conversion.

Version 1.1 contains a number of fixes compared to the latest code in bean-report:

1.1 (2020-07-24)
  • Preserve metadata information (issue #3)
  • Preserve cost information (lot dates and lot labels/notes) (issue #5)
  • Avoid adding two prices in hledger (issue #2)
  • Avoid trailing whitespace in account open declarations (issue #6)
  • Fix indentation issue in postings (issue #8)
  • Fix indentation issue in price entries
  • Drop time information from price (P) entries
  • Add documentation
  • Relicense under GPL-2.0-or-later (issue #1)
1.0 (2020-07-22)
  • Split ledger and hledger conversion from bean-report into a standalone tool
  • Add man page for beancount2ledger(1)

Reproducible Builds (diffoscope): diffoscope 153 released

Friday 24th of July 2020 12:00:00 AM

The diffoscope maintainers are pleased to announce the release of diffoscope version 153. This version includes the following changes:

[ Chris Lamb ] * Drop some legacy argument styles; --exclude-directory-metadata and --no-exclude-directory-metadata have been replaced with --exclude-directory-metadata={yes,no}. * Code improvements: - Make it easier to navigate the main.py entry point. - Use a relative import for get_temporary_directory in diffoscope.diff. - Rename bail_if_non_existing to exit_if_paths_do_not_exist. - Rewrite exit_if_paths_do_not_exist to not check files multiple times. * Documentation improvements: - CONTRIBUTING.md: - Add a quick note about adding/suggesting new options. - Update and expand the release process documentation. - Add a reminder to regenerate debian/tests/control. - README.rst: - Correct URL to build job on Jenkins. - Clarify and correct contributing info to point to salsa.debian.org.

You find out more by visiting the project homepage.

Sean Whitton: keyboardingupdates

Thursday 23rd of July 2020 06:55:24 PM
Marks and mark rings in GNU Emacs

I recently attempted to answer the question of whether experienced Emacs users should consider partially or fully disabling Transient Mark mode, which is (and should be) the default in modern GNU Emacs.

That blog post was meant to be as information-dense as I could make it, but now I’d like to describe the experience I have been having after switching to my custom pseudo-Transient Mark mode, which is labelled “mitigation #2” in my older post.

In summary: I feel like I’ve uncovered a whole editing paradigm lying just beneath the surface of the editor I’ve already been using for years. That is cool and enjoyable in itself, but I think it’s also helped me understand other design decisions about the basics of the Emacs UI better than before – in particular, the ideas behind how Emacs chooses where to display buffers, which were very frustrating to me in the past. I am now regularly using relatively obscure commands like C-x 4 C-o. I see it! It all makes sense now!

I would encourage everyone who has never used Emacs without Transient Mark mode to try turning it off for a while, either fully or partially, just to see what you can learn. It’s fascinating how it can come to seem more convenient and natural to pop the mark just to go back to the end of the current line after fixing up something earlier in the line, even though doing so requires pressing two modified keys instead of just C-e.

Eshell

I was amused to learn some years ago that someone was trying to make Emacs work as an X11 window manager. I was amazed and impressed to learn, more recently, that the project is still going and a fair number of people are using it. Kudos! I suspect that the basic motivation for such projects is that Emacs is a virtual Lisp machine, and it has a certain way of managing visible windows, and people would like to be able to bring both of those to their X11 window management.

However, I am beginning to suspect that the intrinsic properties of Emacs buffers are tightly connected to the ways in which Emacs manages visible windows, and the intrinsic properties of Emacs buffers are at least as fundamental as its status as a virtual Lisp machine. Thus I am not convinced by the idea of trying to use Emacs’ ways of handling visible windows to handle windows which do not contain Emacs buffers. (but it’s certainly nice to learn it’s working out for others)

The more general point is this. Emacs buffers are as fundamental to Emacs as anything else is, so it seems unlikely to be particularly fruitful to move something typically done outside of Emacs into Emacs, unless that activity fits naturally into an Emacs buffer or buffers. Being suited to run on a virtual Lisp machine is not enough.

What could be more suited to an Emacs buffer, however, than a typical Unix command shell session? By this I mean things like running commands which produce text output, and piping this output between commands and into and out of files. Typically the commands one enters are sort of like tiny programs in themselves, even if there are no pipes involved, because you have to spend time determining just what options to pass to achieve what you want. It is great to have all your input and output available as ordinary buffer text, navigable just like all your other Emacs buffers.

Full screen text user interfaces, like top(1), are not the sort of thing I have in mind here. These are suited to terminal emulators, and an Emacs buffer makes a poor terminal emulator – what you end up with is a sort of terminal emulator emulator. Emacs buffers and terminal emulators are just different things.

These sorts of thoughts lead one to Eshell, the Emacs Shell. Quoting from its documentation:

The shell’s role is to make [system] functionality accessible to the user in an unformed state. Very roughly, it associates kernel functionality with textual commands, allowing the user to interact with the operating system via linguistic constructs. Process invocation is perhaps the most significant form this takes, using the kernel’s fork' andexec’ functions.

Emacs is … a user application, but it does make the functionality of the kernel accessible through an interpreted language – namely, Lisp. For that reason, there is little preventing Emacs from serving the same role as a modern shell. It too can manipulate the kernel in an unpredetermined way to cause system changes. All it’s missing is the shell-ish linguistic model.

Eshell has been working very well for me for the past month or so, for, at least, Debian packaging work, which is very command shell-oriented (think tools like dch(1)).

The other respects in which Eshell is tightly integrated with the rest of Emacs are icing on the cake. In particular, Eshell can transparently operate on remote hosts, using TRAMP. So when I need to execute commands on Debian’s ftp-master server to process package removal requests, I just cd /ssh:fasolo: in Eshell. Emacs takes care of disconnecting and connecting to the server when needed – there is no need to maintain a fragile SSH connection and a shell process (or anything else) running on the remote end.

Or I can cd /ssh:athena\|sudo:root@athena: to run commands as root on the webserver hosting this blog, and, again, the text of the session survives on my laptop, and may be continued at my leisure, no matter whether athena reboots, or I shut my laptop and open it up again the next morning. And of course you can easily edit files on the remote host.

More in Tux Machines

today's leftovers

  • Want Social Justice? The Free Software Movement Fights For Everyone!

    Everyone wants freedom but most people have no idea just how enslaved they have become to their computing devices and the proprietary software that controls those devices. The Free Software Movement aims to spread awareness of this issue and to advocate for the use of freedom-respecting software ("free software").

  • Participate in Hacktoberfest, Help Develop Contributions

    The month-long, virtual-festival event that celebrates open source contributions, Hacktoberfest, is coming soon and members of the openSUSE community can make a difference. The event that is in its seventh year and run by Digital Ocean and DEV encourages people to make their first contributions to open source projects. The event is for developers, designers who contribute artwork, people who can contribute to documentation,and more. As the event brings more awareness to open-source projects and encourages contributions that benefit communities, having developers and community members available to help people who want to contribute can be beneficial to the project.

  • Are universities spending enough on cybersecurity? [iophk: Windows TCO]

    Such attacks “will absolutely continue”, said Mark Ford, who leads higher education risk and financial advisory services for the audit firm Deloitte. As higher education becomes known as an “easy target”, this increasingly “attracts the bad guys”, he explained.

    The threat comes not just from criminals seeking money. Universities now house arguably the most valuable secrets on earth – plans for a coronavirus vaccine – putting them in the sights of state-backed [cr]ackers. In July, UK, US and Canadian intelligence services warned that Russian groups were attempting to target Covid-19 vaccine research and development.

    This raises the question: are universities doing enough to defend themselves against [cr]acking?

  • vScaler Integrates SLURM with GigaIO FabreX for Elastic HPC Cloud Device Scaling
  • vScaler Announces SLURM integration with GigaIO FabreX

    The additional integration of the SLURM workload manager, an open-source job scheduler for Linux and Unix-like kernels, means that vScaler Cloud users can request traditional resources like memory and compute cores to be available for jobs.

  • Profiling slow-running queries in Amazon DocumentDB (with MongoDB compatibility)

    Amazon DocumentDB (with MongoDB compatibility) is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. You can use the same MongoDB 3.6 application code, drivers, and tools to run, manage, and scale workloads on Amazon DocumentDB without having to worry about managing the underlying infrastructure. As a document database, Amazon DocumentDB makes it easy to store, query, and index JSON data. AWS built Amazon DocumentDB to uniquely solve your challenges around availability, performance, reliability, durability, scalability, backup, and more. In doing so, we built several tools, like the profiler, to help you run analyze your workload on Amazon DocumentDB. The profiler gives you the ability to log the time and details of slow-running operations on your cluster. In this post, we show you how to use the profiler in Amazon DocumentDB to analyze slow-running queries to identify bottlenecks and improve individual query performance and overall cluster performance.

Programming Leftovers

  • Self-publishing and the 2nd edition of Ansible for DevOps

    Five years, 834 commits, and 24 major revisions later, I've just published the 2nd edition of Ansible for DevOps, a book which has now sold over 60,000 copies and spawned a popular free Ansible 101 video series on YouTube.

  • Open Standards Are Simple

    If you want to create a truly open standard, you _need_ to make it simple.

    There are no exceptions to this rule. When a standard becomes harder to fully implement than what your average motivated programmer can do in two months (max!), it _shouldn't_ be considered "open" anymore.

    Why?

  • In Which COVID-19 Misinformation Leads To A Bunch of Graphs Made With Rust

    A funny — and by funny, I mean sad — thing has happened. Recently the Kansas Department of Health and Environment (KDHE) has been analyzing data from the patchwork implementation of mask requirements in Kansas. They came to a conclusion that shouldn’t be surprising to anyone: masks help. They published a chart showing this. A right-wing propaganda publication got ahold of this, and claimed the numbers were “doctored” because there were two-different Y-axes. I set about to analyze the data myself from public sources, and produced graphs of various kinds using a single Y-axis and supporting the idea that the graphs were not, in fact, doctored. Here’s one graph that’s showing that: In order to do that, I had imported COVID-19 data from various public sources. Many states in the US are large enough to have significant variation in COVID-19 conditions, and many of the source people look at don’t show county-level data over time. I wanted to do that.

  • Basics of Working with the SQLite Database in Python

    A database is one of the most useful and popular files for storing data; they can be used to store any kind of data, including text, numbers, images, binary data, files, etc. SQLite is a relational database management system based on the SQL language. It is a C library, and it provides an API to work with other programming languages, including Python. It does not require a separate server process to be run as needed in large database engines like MySQL and Postgresql. It is swift and lightweight, and the entire database is stored in a single disk file, which makes it portable like CSV or other data storage files. Many applications use SQLite for internal data storage, mainly in environments like mobile devices or small applications.

  • Perl 7 By Default

    Perl 7 has been announced as the next direction of Perl development. My previous blog post explored at a high level the risks and benefits of the announced direction, as well as those of a more incremental proposal. The primary and critical difference between these two approaches is the decision to change interpreter defaults in an incompatible manner. I would like to explore each of the arguments presented for this design choice.

  • CY's Recent Submission for PWC(068-073)

    Skipped blogging on Perl Weekly Challenge(PWC) for a few weeks!

  • SSH vs. kubectl exec

    There’s a lot of similarities between SSH and kubectl, and both have their strengths and weaknesses. While SSH is architecturally set in stone, higher-level software can learn a thing or two from Kubernetes about centralized configuration when managing a fleet of machines. See Teleport for an example of how this can be done. SSH could also borrow the credential management approach from kubeconfigs (i.e. “put all my client creds and server info into one file that I can copy around”).

    kubectl could improve on its non-shell features like port forwarding and file transfer. It’s raw data throughput is also lacking, which precludes it from becoming a transport-layer protocol like SSH. In practice, these tools are complementary and get used for different tasks, it’s not “one or the other”. I hope this post helped you learn something new about both!

  • Can we do better than our C compiler?

    Today, I wanted to become a C compiler. I added a hand-compiled assembly version of echo from our previous coding exercise and added a new make target, make asm, that will assemble it. Let's look at our hand-compiled assembly and compare it to our C compiler and ask whether or not it was worth it.

  • Benign Data Races Considered Harmful

    The series of posts about so called benign data races stirred a lot of controversy and led to numerous discussions at the startup I was working at called Corensic. Two bastions formed, one claiming that no data race was benign, and the other claiming that data races were essential for performance. Then it turned out that we couldn’t even agree on the definition of a data race. In particular, the C++11 definition seemed to deviate from the established notions.

  • Micronaut 2.0 Full-Stack Java Framework Released

    The Micronaut framework uses Java's annotation processors, which work with any JVM language that supports them, as well as an HTTP server and client built on the Netty non-blocking I/O client server framework. To provide a programming model similar to Spring and Grails, these annotation processors pre-compile the required metadata to perform DI, define AOP proxies, and configure applications to run in a low-memory environment, the company says. Many of the APIs in Micronaut were "heavily inspired" by Spring and Grails," which was by design and aids in bringing developers up to speed quickly," the company says.

  • Understanding computer vision and AI, part 1

    An active area in the field of computer vision is object detection, where the goal is to not only localize objects of interest within an image but also assign a label to each of these objects of interest. Considerable recent successes in the area of object detection stem from modern advances in deep learning, particularly leveraging deep convolutional neural networks. Much of the initial focus was on improving accuracy, leading to increasingly more complex object detection networks such as SSD, R-CNN, Mask R-CNN, and other extended variants of these networks. While such networks demonstrated state-of-the-art object detection performance, they were very challenging, if not impossible, to deploy on edge and mobile devices due to computational and memory constraints. This greatly limits the widespread adoption for a wide range of applications such as robotics, video surveillance, autonomous driving where local embedded processing is required. [...] Model Evaluation is an integral part of the model development process. It helps to find the best model that represents our data and how well the chosen model performs on unseen data. To improve the model we tune the hyper-parameters; parameter that determines the network structure (number of neurons in the network, network activation functions) or training parameter (gradient descent learning rate, adding parameters like momentum in the weight update rule). Tuning those parameters is an inevitable and important step to obtain better performance. Methods like GridSearch and RandomizedSearch can be used to navigate through the different parameters.

  • Qt Design Studio 1.6 Beta released

    We are happy to announce the beta release of Qt Design Studio 1.6 Qt Design Studio is a UI design and development tool that enables designers and developers to rapidly prototype and develop complex UIs. Both designers and developers use Qt Design Studio and this makes collaboration between the two a lot simpler and more streamlined. To get an impression, you should watch this video.

Raspberry Pi Projects and News

Today in Techrights