Language Selection

English French German Italian Portuguese Spanish

Programming/Development Leftovers

Filed under
Development
  • Training at Qt World Summit 2020

    Qt World Summit 2020 has been postponed until October 20-22. It will take place in Palm Springs, USA.

  • Select all Google Photos (or Videos)

    Few weeks ago my uncle told me that he was unable to receive any new emails. A quick investigation led us to the idea that he had no more space left in his Google account, and that was caused by the hundreds and hundreds of photos and videos he was taking, that were automatically being backed up to Google Photos service.

    So the solution was simple, all he had to do is deleting some of the old photos to clear some space. but simple as that sound at first, turned out it wasn’t that an easy task after all.

  • Marcin Kolny: HawkTracer - low-overhead instrumentation-based profiler

    A while ago, at Amazon we've open-sourced instrumentation-based profiler - HawkTracer - which introduces very low overhead so it can be used on low-end platforms, where development environment is somehow limited (e.g. no ssh access, very limited disk storage etc). We used it to fix performance issues of Prime Video app on living room devices (SmartTVs, Streaming sticks, Game consoles etc).

  • If you use GNU Grep on text files, use the -a (--text) option

    Ah. Yes. How helpful. While reading along in what it had up until then thought was a text file, GNU Grep encountered some funny characters (in a DKIM signature information line, as it happened) and decided that the file was actually binary and so it wouldn't report anything more for the rest of the file than that final line.

  • DIY Single Sign-On for SSH

    TL;DR In this post we're going to set up Google single sign-on for SSH. Behind the scenes, we'll use OpenID Connect (OIDC), short-lived SSH certificates, a couple of clever SSH configuration tweaks, and Smallstep's open-source step-ca and step packages. We will set up an SSH Certificate Authority, and use it to bootstrap a new host and a new user in our system. While this approach requires more up-front work than a typical SSH public/private key setup, it comes with a lot of benefits beyond single sign-on. It eliminates the need for gathering and shipping and managing authorized_keys files.

  • Evaluate 3 ways to run Kubernetes locally

    The main purpose of Kubernetes is to host applications across a cluster of servers with sophisticated load balancing and resource allocation features. This ensures applications run smoothly, even if some servers fail. In production deployments, the use of multiple servers for Kubernetes is essential.

    However, there are situations where an IT admin or developer might want to run Kubernetes locally on a PC or laptop. A local Kubernetes environment, for example, enables developers to test new application code quickly without having to upload it to a production cluster first. Local Kubernetes is also a great way for newcomers to play around with the container orchestration system without the complexity and cost of a full-scale, multiserver deployment.

    Below are three approaches to run Kubernetes locally, all of which will work on a PC or laptop with Windows, Linux or macOS.

  • 5 IDEs for sysadmins

    Many sysadmins don’t consider themselves coders. They acknowledge that they regularly write complex scripts to help themselves automate their job, but they don’t consider themselves developers. I think it’s for that reason that most sysadmins also don’t think they have any use for an IDE. After all, an IDE is an Integrated Development Environment, and a sysadmin isn’t a developer, right? If that’s been your thought process, then it’s time to reconsider because a good IDE (or robust text editor) offers many benefits over a basic text editor.

    A good IDE provides syntax validation and smart auto-completion, important for catching mistakes that are at best, bothersome, and, at worst, harmful. An IDE also offers integration with the rest of your system so you can test scripts as you write them. Should something fail, a good IDE has a debugger to help identify the problem and, in some cases, help you solve it. Additional features include Git integration, quick access to a shell, plugins, and much more.

    There are lots of IDEs out there, and most are very flexible, but some arguably are best left to people who do nothing but develop software all day. Sysadmins have different needs and expectations than dedicated programmers, so here are five IDEs that stand out from the rest for sysadmins.

  • PyCharm: Here's what Python programming language developers get in new IDE update

    JetBrains has released the latest stable version of its PyCharm integrated development environment (IDE) for the Python programming language.

    PyCharm is a popular IDE for Python developers on Windows, macOS, and Linux. According to JetBrains' 2019 Python survey, PyCharm is by far the most widely used IDE for Python programming, ahead of Microsoft's Visual Studio Code for Python.

  • DBLD: a syslog-ng developer tool not just for developers

    DBLD is a central tool when it comes to syslog-ng development, but even after multiple blogs about the tool, it is still not much used outside of the developers’ team. So, what is DBLD and how could it be used even by you? The abbreviation stands for Docker BuiLD. Using containers ensures both that you have an easily reproducible build environment, and also that you do not have to “pollute” your base system with development-related software packages. You can use DBLD to build the release tarball or ready-to-use packages for a number of Linux distributions. It can even be used as a development environment with all necessary tools installed.

    Still not convinced? Yes, the listed possibilities are mostly interesting for syslog-ng developers and 3rd party packagers. If you are lucky, you will never need DBLD. On the other hand, it can come handy if you reported a problem and the syslog-ng team fixed it. Even if you are not a developer, only a junior sysadmin, you can still easily build fixed syslog-ng packages for testing (and even for production use) until an official release with the fix arrives.

  • Writing Java with Quarkus in VS Code

    In the previous articles in this series about cloud-native Java applications, I shared 6 requirements of cloud-native software and 4 things cloud-native Java must provide. But now you might want to implement these advanced Java applications in your local machine without climbing a steep learning curve. In this article, I will walk through using the open source technologies Quarkus and Visual Studio Code (VS Code) to accelerate the development of both traditional cloud-native Java stacks and also serverless, reactive applications with easier and more familiar methods.

    Quarkus is a Kubernetes-native Java stack tailored for GraalVM and OpenJDK HotSpot. It's crafted from best-of-breed Java libraries and standards with live coding, unified configuration, superfast startup, small memory footprint, and unified imperative and reactive development. VS Code is an open source integrated development environment (IDE) for editing code.

More in Tux Machines

Raspberry Pi HAT offer NMEA 2000 link for marine applications

Copperhill’s $99 “PiCAN-M” HAT for the Raspberry Pi provides CAN-based NMEA 2000 and RS-422 driven NMEA 0183 ports for marine applications. The HAT includes a 3A SMPS supply and a Qwiic link. In 2019, Copperhill Technologies launched a PiCAN3 CAN-Bus HAT for the Raspberry Pi 4. The new PiCAN-M (for Marine), built for Copperhill by SK Pang, offers a marine-specific, NMEA 2000 compatible version of CAN. The $98.50 HAT also supplies an RS-422-based NMEA 0183 interface plus a Qwiic interface for adding SparkFun’s Qwiic add-ons. Read more

How a Linux migration led to the creation of Amazon Web Services

Dan Rose, chairman of Coatue Ventures and Coatue Growth, posted a thread on Twitter the other day, 280 characters or less at a time, in which he chronicled how it came about that AWS infrastructure is built on Linux. Rose was at Amazon from 1999 to 2006, where he managed retail divisions and helped incubate the Kindle reader before moving to Facebook. So he was at Amazon in 2000 when the internet bubble popped,and one high-flying dot-com after another was shriveling up and dying, having burned through ridiculous amounts of capital on luxurious offices while often having nothing by way of a product to show for it. Rose said Amazon’s biggest expense was the data center outfitted with expensive Sun servers. Amazon’s motto was “get big fast,” and site stability was critical. Every second of downtime meant lost sales, and Sun was the gold standard for internet servers back then. I can recall them having a significant software business led by a VP named Eric Schmidt. Read more

Kernel: Restricted DMA and AMD Work in Linux 5.11

  • Restricted DMA

    A key component of system hardening is restricting access to memory; this extends to preventing the kernel itself from accessing or modifying much of the memory in the system most of the time. Memory that cannot be accessed cannot be read or changed by an attacker. On many systems, though, these restrictions do not apply to peripheral devices, which can happily use direct memory access (DMA) on most or all of the available memory. The recently posted restricted DMA patch set aims to reduce exposure to buggy or malicious device activity by tightening up control over the memory that DMA operations are allowed to access. DMA allows devices to directly read from or write to memory in the system; it is needed to get reasonable I/O performance from anything but the slowest devices. Normally, the kernel is in charge of DMA operations; device drivers allocate buffers and instruct devices to perform I/O on those buffers, and everything works as expected. If the driver or the hardware contains bugs, though, the potential exists for DMA transfers to overwrite unrelated memory, leading to corrupted systems and unhappy users. Malicious (or compromised) hardware can use DMA to compromise the system the hardware is attached to, making users unhappier still; examples of this type of attack have been posted over the years. One way to address this problem is to place an I/O memory-management unit (IOMMU) between devices and memory. The kernel programs the IOMMU to allow access to a specific region of memory; the IOMMU then keeps devices from straying outside of that region. Not all systems are equipped with an IOMMU, though; they are mostly limited to the larger processors found in desktop machines, data centers, and the like. Mobile systems usually lack an IOMMU.

  •  
  • A Fix Has Been Proposed For The Slower AMD Performance On Linux 5.11

    With the in-development Linux 5.11 kernel there are many great features and improvements especially for AMD users with some new drivers and other pleasant enhancements. But as I outlined back on Christmas day: Linux 5.11 Is Regressing Hard For AMD Performance With Schedutil. Fortunately, a fix is now en route to the Linux 5.11 kernel for fixing that performance regression affecting AMD Zen 2/3 desktops and servers.  As outlined in that original article after bisecting the sizable performance regressions and in follow-up tests, AMD hardware performing slower on Linux 5.11 came down to the CPU frequency invariance support introduced this cycle and is utilized by the "Schedutil" CPU frequency scaling governor. With Schedutil often being the default for AMD systems on newer versions of the Linux kernel, this regression on Linux 5.11 compared to prior kernel releases has been unfortunate. 

  • Linux 5.11 Is Now Looking Great For AMD Zen 2 / Zen 3 Performance - Phoronix

    Not only is the AMD "CPU frequency invariance regression" from that new support with the in-development Linux 5.11 kernel on course to address the performance shortcomings I outlined last month, but with the patched kernel for a number of workloads the performance is now ahead of where it was at with Linux 5.10.

System76 Brings Back the Darter Pro Linux Laptop with Longer Battery Life, Tiger Lake CPUs

The Darter Pro is one of System76's most versatile all-around Linux laptops and the 2021 refresh is here with 11th Gen Intel Core i5-1135G7 and i7-1165G7 CPUs with 4 cores / 8 threads and integrated Iris Xe graphics, up to 64GB dual-channel DDR4 3200MHz RAM, and up to 4TB M.2 SSD storage. Best of all, the new Darter Pro comes with System76’s Coreboot-based Open Firmware and Open Source Embedded Controller Firmware to give customers full control over the hardware, and also make the Linux laptop faster and more secure. Read more