August 25, 2005

How to read Linux reviews

Author: Jem Matzan

Once upon a time, reviews related to GNU/Linux and free software in open source community publications and Web sites were candy-coated. We were amazed that things actually worked, even if making it work required hacking makefiles and configuration files, compiling from source, and getting your hands dirty in other ways, and gave everything glowing reviews. Today, we expect that everything should work properly the first time, every time, but still, things don't always work out as planned. Yet some readers seem to expect journalists to hide the dirty laundry of poorly designed software and badly supported hardware.

When I choose a product to review, be it hardware or software, it's because I think it's cool. If I'm disappointed, it's my responsibility to say so. My intention is not to destroy it with a negative review. It's because readers deserve to know the product is not all it's cracked up to be.

When a product Web site, public relations person, press release, or product announcement pumps up my expectations of a product, I expect it to perform as claimed. If it doesn't, I tell the readers. If a product meets or exceeds the expectations that its propaganda inspires, it gets a positive review.

No professional reviewer owes the free software/open source/Linux/BSD/GNU/whatever community a softball review for the sake of advocacy. Their mission is not to get more people to use GNU/Linux and free software. No project deserves to be "applauded for trying," even though the software ends up being buggy or useless. If you feel the need to applaud, clap your hands.

Reviewers also have no obligation to provide advice or "constructive criticism," and we are under no obligation to file bug reports for problems we find in the course of review testing. A review is not a critique of someone's high school art project. Reviews are not written to make software better, but to inform readers. I write them to make money, not to advocate my favorite software.

Lighten up, Francis

On the other hand, just as readers should not expect blandly positive reviews, they shouldn't expect entirely negative reviews either. Every product has good and bad points. But even if the bad outweighs the good in one reviewer's eyes, it's not the end of the world. One negative review will not kill a project and drive all of its users away. When it comes to reviews of free (as in price) or low-cost products, any press is good press. A fact- or experience-based review, no matter the reviewer's conclusions, still shows people what the product is capable of and what it's about. It also makes more people aware of a project that they might think is cool, even if it doesn't work properly yet.

Reviewers sometimes get hate mail and nasty article comments. We have to learn to take it in stride. But all the same, it gets tiring. So let's make a deal -- our reviewers won't go overboard criticizing a project unjustly, and you shouldn't ask that they be fired when you disagree with their opinion.

GNU/Linux is not just for software hobbyists anymore. And neither are the reviews.

Click Here!