Programming: mmap. Python and More
-
Making the Most of your Memory with mmap
Sometimes it seems that we have nearly infinite memory resources, especially compared to the tiny 48K RAM of yesteryear’s 8-bit computers. But today’s complex applications can soak up megabytes before you know it. While it would be great if developers planned their memory management for all applications, thinking through a memory management strategy is crucial for applications with especially RAM intensive features like image/video processing, massive databases, and machine learning.
How do you plan a memory management strategy? It’s very dependent on your application and its requirements, but a good start is to work with your operating system instead of against it. That’s where memory mapping comes in. mmap can make your application’s performance better while also improving its memory profile by letting you leverage the same virtual memory paging machinery that the OS itself relies on. Smart use of the memory mapping API (Qt, UNIX, Windows) allows you to transparently handle massive data sets, automatically paging them out of memory as needed – and it’s much better than you’re likely to manage with a roll-your-own memory management scheme.
Here’s a real-life use case of how we used mmap to optimize RAM use in QiTissue, a medical image application. This application loads, merges, manipulates, and displays highly detailed microscope images that are up to gigabytes in size. It needs to be efficient or risks running out of memory even on desktops loaded with RAM.
-
Moving Kolla images to Python 3
Python… To use 2.7 or to go for 3.x? To “be compatible” or to “use fancy new features”. Next year Python 2 gets finally unsupported upstream.
-
PyCharm 2019.1 RC 2
New in PyCharm 2019.1: completely redesigned Jupyter Notebooks, improved HTML & CSS quick documentation, custom themes, and more. Get the release candidate from our website
-
13 Project Ideas for Intermediate Python Developers
Learning the basics of Python is a wonderful experience. But the euphoria of just learning can be replaced by the hunger for hands-on projects. It’s normal to want to build projects, hence the need for project ideas.
The problem though is that some projects are either too simple for an intermediate Python developer or too hard. This article will suggest projects you can work on as an intermediate Python developer. These project ideas will provide the appropriate level of challenge for you.
-
Speed: Default value vs checking for None
-
Announcing SLE 15 SP1 RC 1 and SES 6 Beta 11!
-
Coding in Python 19 - More fun with the OS Module
-
Coding in Python 20 - Subprocess
-
Coding in Python 21 - Handling Exceptions
-
Starting A Django Project
-
Microsoft buffs up its open-source halo to fine sheen with PostgreSQL GUI in Azure Data Studio [Ed: These are all proprietary software falsely marketed using "free bait" a.k.a. "open core" or 'open' plugins. It's also surveillance.]
- Login or register to post comments
- Printer-friendly version
- 1606 reads
- PDF version
More in Tux Machines
- Highlights
- Front Page
- Latest Headlines
- Archive
- Recent comments
- All-Time Popular Stories
- Hot Topics
- New Members
digiKam 7.7.0 is releasedAfter three months of active maintenance and another bug triage, the digiKam team is proud to present version 7.7.0 of its open source digital photo manager. See below the list of most important features coming with this release. |
Dilution and Misuse of the "Linux" Brand
|
Samsung, Red Hat to Work on Linux Drivers for Future TechThe metaverse is expected to uproot system design as we know it, and Samsung is one of many hardware vendors re-imagining data center infrastructure in preparation for a parallel 3D world. Samsung is working on new memory technologies that provide faster bandwidth inside hardware for data to travel between CPUs, storage and other computing resources. The company also announced it was partnering with Red Hat to ensure these technologies have Linux compatibility. |
today's howtos
|
Recent comments
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago