Language Selection

English French German Italian Portuguese Spanish

Storage Instantiation Daemon in Fedora, IBM/Spark and Talospace Project/POWER

Filed under
Red Hat
  • Fedora Looks To Introduce The Storage Instantiation Daemon

    As one of the last minute change proposals for Fedora 33 is to introduce the Red Hat backed Storage Instantiation Daemon "SID" though at least for this first release would be off by default. The Storage Instantiation Daemon is one of the latest storage efforts being worked on by Red Hat engineers.

    The Storage Instantiation Daemon is intended to help manage Linux storage device state tracking atop udev and reacts to changes via uevents. This daemon can offer an API for various device subsystems and provides insight into the Linux storage stack. More details on this newer open-source effort via sid-project.github.io.

  • Explore best practices for Spark performance optimization

    I am a senior software engineer working with IBM’s CODAIT team. We work on open source projects and advocacy activities. I have been working on open source Apache Spark, focused on Spark SQL. I have also been involved with helping customers and clients with optimizing their Spark applications. Apache Spark is a distributed open source computing framework that can be used for large-scale analytic computations. In this blog, I want to share some performance optimization guidelines when programming with Spark. The assumption is that you have some understanding of writing Spark applications. These are guidelines to be aware of when developing Spark applications.

    [...]

    Spark has a number of built-in user-defined functions (UDFs) available. For performance, check to see if you can use one of the built-in functions since they are good for performance. Custom UDFs in the Scala API are more performant than Python UDFs. If you have to use the Python API, use the newly introduced pandas UDF in Python that was released in Spark 2.3. The pandas UDF (vectorized UDFs) support in Spark has significant performance improvements as opposed to writing a custom Python UDF. Get more information about writing a pandas UDF.

  • The Talospace Project: Firefox 78 on POWER

    Firefox 78 is released and is running on this Talos II. This version in particular features an updated RegExp engine but is most notable (notorious) for disabling TLS 1.0/1.1 by default (only 1.2/1.3). Unfortunately, because of craziness at $DAYJOB and the lack of a build waterfall or some sort of continuous integration for ppc64le, a build failure slipped through into release but fortunately only in the (optional) tests. The fix is trivial, another compilation bug in the profiler that periodically plagues unsupported platforms, and I have pushed it upstream in bug 1649653. You can either apply that bug to your tree or add ac_add_options --disable-tests to your .mozconfig. Speaking of, as usual, the .mozconfigs we use for debug and optimized builds have been stable since Firefox 67.

More in Tux Machines

Android Leftovers

LibreOffice 7: Why a paid enterprise edition could be a positive change

The RC1 candidate of LibreOffice was recently released. Soon after, something odd was discovered in the About LibreOffice window. It seems the developers had tagged that release candidate as a Personal Edition. What did this mean? Of course speculation is running amok with rumors that LibreOffice is no longer going to be free. After all, the open source office suite has always been free and never before hinted at ever not being free. Yet, there was that oh-so-obvious change in the About dialog indicating something was amiss. After all, it's a pretty easy conclusion to draw when said dialog clearly states (Figure A): "The Personal Edition is supported by volunteers and is intended for individual use." Read more

today's leftovers

  • Fedora program update: 2020-28

    Here’s your report of what has happened in Fedora this week. The Nest With Fedora Call for Participation is now open. I have weekly office hours in #fedora-meeting-1. Drop by if you have any questions or comments about the schedule, Changes, elections, or anything else.

  • openSUSE Tumbleweed – Review of the week 2020/28

    This week I have been fighting a bit with the size of the DVD. Due to some mistake in the pattern definition, it was for a good chunk ignored. Serves me right for fixing the error and then getting an ISO file that would not ever fit on a DVD, eh? Well, long story short: I needed to save some space, so looked at what the ‘fix’ newly brought on the DVD and aligned to what the DVD looked like before.

  • Nour E-Din ElNhass: An Appointment Up the Hill

    In my last post I showed screenshots for contacts appearing in Evolution, and explained that the .source file was created manually and that the credentials were hard coded for retrieving a specific journal form a specific EteSync account. After finishing this, I extended so that I can also retrieve calenders and tasks in the same manner which was quite easy as I already understood what should be done. Then I created an etesync-backend file, which generally handles the user’s collection account in evolution (retrieving/ creating /deleting) journals which are address-book or calenders .source files. The next step was then to make a user enter his credentials, So it isn’t hard coded. In this stage I had faced some issues regarding the implementation, I asked for my mentors help. Some of the problems that I faced were I needed to create a new dialog that will appear ask the user for his credentials and retrieve the data from EteSync, this had some implementation problems for me at first. Other issues appeared while integrating had to change some pieces.

  • Kiwi TCMS 8.5

    We're happy to announce Kiwi TCMS version 8.5! [...] Our website has been nominated in the 2020 .eu Web Awards and we've promised to do everything in our power to greet future FOSDEM visitors with an open source billboard advertising at BRU airport.

  • What is Application Definition?

    Application definition is the process of creating a codified operational runbook. It formalizes the process of describing in code everything an application needs to be built, run, and managed.

  • Open source contributions face friction over company IP

    Now, the overwhelming majority -- more than 90% -- of proposed open source contributions are approved by the council, said Nithya Ruff, the head of the Comcast open source program office. Under the current advisory council process, once engineers are approved to contribute to existing projects, they can make further contributions without having to go through the process all over again, according to Ruff. The process typically takes a few days at most. And since 2016, Comcast has donated several entire projects to open source, such as its Traffic Control CDN and Web PA client-server interface.

  • Virtualization Is Key to the Future of IT. Therefore ... What?

    If you look at commercial servers around the world, including those that provide the cloud in all its many forms to consumers of cloud-based infrastructures, platforms, services, and applications, you’ll soon learn that the vast majority of them run some version of Linux on the hardware, and consume a great many more Linux-based VMs than Windows-based ones. Even Microsoft has had to become more catholic in its approach: Today, its Azure cloud environment spawns Linux and Windows VMs with equal facility. Windows 10 has also become ever more accommodating of Linux, thanks to the Windows Subsystem for Linux (WSL) support for Linux within the Windows OS, along with Hyper-V’s ability to accommodate both Linux and Windows VMs. Specific versions of Linux have been developed as “network operating systems” that run on switches, and provide fully virtualized complex, networking environments. Thus, for example, Nvidia subsidiary Cumulus Networks offers a free virtual appliance called Cumulus VX that runs on KVM (a Linux-based or bare-metal hypervisor), Virtualbox (from a provider or as a local hypervisor), and Vagrant (from a libvirt provider). Cumulus Networks’ offering is based around Cumulus Linux, which adds a Network Command Line Utility (NCLU) to the basic Linux environment. This supports configuration, provisioning, and virtualization of network switches to support fully virtualized network infrastructures and let people learn about complex routing protocols such as BGP, XBGP, OSPF, and so forth. Other network players also offer virtualized toolsets and learning environments for their networking products and services as well, so they’re worth looking into as well, if networking is your thing.

  • Docker partners with AWS to smooth container deployments

    Docker containers, of course, can be used anywhere. But while developers may use Docker Desktop and the Docker CLI for building applications on their desktop, they may also have used Docker Compose to define and run multi-container applications via a YAML file. So far, so good. The problem is that there hasn't been a simple seamless way to use Docker and deploy to AWS from their desktop. That's because essential Amazon ECS constructs were not part of the Docker Compose specification. For instance, to run even a simple Compose file and deploy to ECS, developers are required to leave Docker tools and configure an Amazon VPC, Amazon ECS Cluster, and Amazon ECS Task Definition.

today's howtos

  • Linux networking - A weird little problem with DNS, DHCP

    At this point, or rather, just before this point, I was inclined to blame systemd. But it turns out, there's nothing wrong with it. Now, there were/are bugs in it, which is why it comes up quite high if you search for any connectivity problems in Linux.

  • Top Linux Interview Questions
  • Pandoc Flavoured Markdown: Guide To Your First Document
  • How to play Mount & Blade II: Bannerlord on Linux
  • Light OpenStreetMapping with GPS

    Now that lockdown is lifting a bit in Scotland, I’ve been going a bit further for exercise. One location I’ve been to a few times is Tyrebagger Woods. In theory, I can walk here from my house via Brimmond Hill although I’m not yet fit enough to do that in one go. Instead of following the main path, I took a detour along some route that looked like it wanted to be a path but it hadn’t been maintained for a while. When I decided I’d had enough of this, I looked for a way back to the main path but OpenStreetMap didn’t seem to have the footpaths mapped out here yet. I’ve done some OpenStreetMap surveying before so I thought I’d take a look at improving this, and moving some of the tracks on the map closer to where they are in reality. In the past I’ve used OSMTracker which was great, but now I’m on iOS there doesn’t seem to be anything that matches up.