Language Selection

English French German Italian Portuguese Spanish

Beta Testing 101

Filed under
Howtos

Ever have one of those days when you can't stand end users? I know I have.

There are several expectations end users have for people creating software, or even Linux distros. They want it to work on their hardware, they want it to be stable, and they want it right now. But, at the same time, new releases shouldn't come too often, or it messes up the feng shui of their systems.

But you can't get them to beta test. I say this from experience in multiple projects, both those I lead and those I've observed. Oh, you can get them to agree to beta test, but I'm not sure that they understand how. So, here's a few tips from someone who's been on all sides of the equation.

1. Get a jumpdrive and backup all your logs. They've come way down in price over the past few years, and it's not prohibitively expensive to buy. And buy you should... get a separate drive just for backing up logs while beta testing. The least experience Linux user in the world can be the best beta tester a project has just by making copies of their logfiles available.

2. In that vein, back up system information as well. Hardware-specific bugs are much easier to track down and kill when the bug-hunter can look for patterns. Make it a habit to save the output from lspci and lsusb into their own log files on that jumpdrive. Set up a folder for each session, cd to that folder, and simply lspci > lspci.log and lsusb > lsusb.log.

3. Further in the logging journey, learn to use dmesg and grep. To find out information in the system log about your agp setup, dmesg | grep agp. This will prove invaluable, and keep you from having to manually parse potentially hundreds of lines of text to find two lines of interest. Once again, you can concatenate the output to a logfile: dmesg | grep agp > agpinfo.log.

4. Finally, learn to use stdout and stderr. These are the output and errors that come from running a program. To help diagnose dependency errors and the like, the output from a malfunctioning program can be redirected to a log file like so: make > makelog 2>&1. This can turn a single run into a goldmine of information for the harried developer with people breathing down his neck about the next release.

5. Try things you normally wouldn't. Play the games. Write a letter to your Great Aunt Ruth. Listen to that Hanson cd you won't admit you still own. Burn copies of it and leave it unmarked in your friends' cars so they get curious and pop it in. Cross-compile the kernel for a 386. Test your dial-up modem. Scan for wireless networks. Any number of these things will be attempted by the end users of this project, and your input may keep a bad situation from ruining a release.

6. Go to Wal-Mart and buy one of those cheap marble composition books. Take notes in it while you use the computer. Having a hard copy in case of major system meltdown can be worth more than gold, and it will give you something to look back on later. Notes files get deleted, but you can keep notebooks for 20 years if you take care of them.

Well, that's it for this lesson. I hope that it's useful for potential beta testers, software developers, and even the end users (maybe now you can understand why some things aren't being released yet.)

And that's life when you're in spinlock.

More in Tux Machines

today's howtos

Leftovers: OSS

  • Report: If DOD Doesn't Embrace Open Source, It'll 'Be Left Behind'
    Unless the Defense Department and its military components levy increased importance on software development, they risk losing military technical superiority, according to a new report from the Center for a New American Security. In the report, the Washington, D.C.-based bipartisan think tank argues the Pentagon, which for years has relied heavily on proprietary software systems, “must actively embrace open source software” and buck the status quo. Currently, DOD uses open source software “infrequently and on an ad hoc basis,” unlike tech companies like Google, Amazon and Facebook that wouldn’t exist without open source software.
  • The Honey Trap of Copy/Pasting Open Source Code
    I couldn’t agree more with Bill Sourour’s article ‘Copy.Paste.Code?’ which says that copying and pasting code snippets from sources like Google and StackOverflow is fine as long as you understand how they work. However, the same logic can’t be applied to open source code. When I started open source coding at the tender age of fourteen, I was none the wiser to the pitfalls of copy/pasting open source code. I took it for granted that if a particular snippet performed my desired function, I could just insert it into my code, revelling in the fact that I'd just gotten one step closer to getting my software up and running. Yet, since then, through much trial and error, I’ve learned a thing or two about how to use open source code effectively.
  • Affordable, Open Source, 3D Printable CNC Machine is Now on Kickstarter
    The appeals of Kickstarter campaigns are many. There are the rewards for backers, frequently taking the form of either deep discounts on the final product or unusual items that can’t be found anywhere else. Pledging to support any crowdfunding campaign is a gamble, but it’s an exciting gamble; just browsing Kickstarter is pretty exciting, in fact, especially in the technological categories. Inventive individuals and startups offer new twists on machines like 3D printers and CNC machines – often for much less cost than others on the market.
  • Open Standards and Open Source
    Much has changed in the telecommunications industry in the years since Standards Development Organization (SDOs) such as 3GPP, ITU and OMA were formed. In the early days of telecom and the Internet, as fundamental technology was being invented, it was imperative for the growth of the new markets that standards were established prior to large-scale deployment of technology and related services. The process for development of these standards followed a traditional "waterfall" approach, which helped to harmonize (sometimes competing) pre-standard technical solutions to market needs.

Leftovers: BSD

  • The Voicemail Scammers Never Got Past Our OpenBSD Greylisting
    We usually don't see much of the scammy spam and malware. But that one time we went looking for them, we found a campaign where our OpenBSD greylisting setup was 100% effective in stopping the miscreants' messages. During August 23rd to August 24th 2016, a spam campaign was executed with what appears to have been a ransomware payload. I had not noticed anything particularly unusual about the bsdly.net and friends setup that morning, but then Xavier Mertens' post at isc.sans.edu Voice Message Notifications Deliver Ransomware caught my attention in the tweetstream, and I decided to have a look.
  • Why FreeBSD Doesn't Aim For OpenMP Support Out-Of-The-Box

Security Leftovers

  • FBI detects breaches against two state voter systems
    The Federal Bureau of Investigation has found breaches in Illinois and Arizona's voter registration databases and is urging states to increase computer security ahead of the Nov. 8 presidential election, according to a U.S. official familiar with the probe. The official, speaking on condition of anonymity, said on Monday that investigators were also seeking evidence of whether other states may have been targeted. The FBI warning in an Aug. 18 flash alert from the agency's Cyber Division did not identify the intruders or the two states targeted. Reuters obtained a copy of the document after Yahoo News first reported the story Monday.
  • Russians Hacked Two U.S. Voter Databases, Say Officials [Ed: blaming without evidence again]
    Two other officials said that U.S. intelligence agencies have not yet concluded that the Russian government is trying to do that, but they are worried about it.
  • FBI Says Foreign Hackers Got Into Election Computers
    We've written probably hundreds of stories on just what a dumb idea electronic voting systems are, highlighting how poorly implemented they are, and how easily hacked. And, yet, despite lots of security experts sounding the alarm over and over again, you still get election officials ridiculously declaring that their own systems are somehow hack proof. And now, along comes the FBI to alert people that it's discovered at least two state election computer systems have been hacked already, and both by foreign entities.
  • Researchers Reveal SDN Security Vulnerability, Propose Solution
    Three Italian researchers have published a paper highlighting a security vulnerability in software-defined networking (SDN) that isn't intrinsic to legacy networks. It's not a showstopper, though, and they propose a solution to protect against it. "It" is a new attack they call Know Your Enemy (KYE), through which the bad guys could potentially collect information about a network, such as security tool configuration data that could, for example, reveal attack detection thresholds for network security scanning tools. Or the collected information could be more general in nature, such as quality-of-service or network virtualization policies.
  • NV Gains Momentum for a Secure DMZ
    When it comes to making the shift to network virtualization (NV) and software-defined networking (SDN), one of the approaches gaining momentum is using virtualization technology to build a secure demilitarized zone (DMZ) in the data center. Historically, there have been two major drawbacks to deploying firewalls as a secure mechanism inside a data center. The first is the impact a physical hardware appliance has on application performance once another network hop gets introduced. The second is the complexity associated with managing the firewall rules. NV technologies make it possible to employ virtual firewalls that can be attached to specific applications and segregate them based on risk. This is the concept of building a secure DMZ in the data center. The end result is that the virtual firewall is not only capable of examining every packet associated with a specific application, but keeping track of what specific firewall rules are associated with a particular application becomes much simpler.