How to review a Linux distribution

New versions of Linux distributions crop up almost every week. On the one hand, it’s great that development is so active, but on the other it can be difficult to wade through the options and find the one that suits you best. Most of us do not have the time to try out every new flavor, so we come to rely on reviewers to boil down the new releases for us. As a site that runs frequent distro reviews, NewsForge would like to give some advice to new distro reviewers — pointers to get you started and help you write a concise, informative assessment that will benefit you and your readers alike.

Before you even pop open the CD tray, make sure you have configuration details from your current system written down on paper.

It sounds obvious, but it is also a good idea to have a second PC at hand and connected to the Internet if at all possible. Don’t use your main machine for review purposes; try a spare computer instead — perhaps an older, less powerful one. Laptops are good in case (worst case scenario) you need to take it somewhere else for help. Find something that you have not used in a while (maybe replaced when you upgraded to modern hardware), or if that is infeasible, look for an inexpensive one at a garage or yard sale.

Last but not least, get a stopwatch. Trust me, you’ll need it before the end. Once you’ve got everything together, hit the start button, fire up the installation CD, and let’s get to it.

Step one: pay appropriate attention to the installer

The installer is the first thing a new user is going to see, so it deserves your full attention. You will want to describe it in detail; make note of the names of screens and the options presented on each. As you move from screen to screen, keep track of how much time each step takes, and scribble down some “first reactions” as you make your selections.

When it comes time to write, budget as much of your article space as you can to your review of the installer. Draw on your notes, reporting the steps, the options presented, and which options you selected. At least half the review needs to deal with the installation process — more if possible.

If the installer has some interesting quirks (like while-you-wait ads or video games) you should allocate more space to discuss them. You can dedicate up to two-thirds of your article to the installer, depending on how in-depth you want to explore its shortcomings.

Step two: review the applications

The heart of an operating system is in its apps, so be systematic and review them. Do you like the default GNOME or KDE environment better? Do you prefer Firefox or Opera? What do you think about OpenOffice.org?

It’s important not to be shy. You are there to give your opinion, so if you dislike the choice of desktop environments or email clients, say so and tell us why. Are there better choices they should have included? If so, what are they? Has Firefox slowed down since the Mozilla Foundation took over? Get into it and speak your mind.

And I can’t say this emphatically enough: be confident! You are a smart person, you know software, and as a result, you have determined what your favorite apps are for different tasks. Anyone who disagrees — especially a Linux distro — needs to be told where they have gone wrong.

Step three: install something from way outside the distro

Much as the distros would like to ship every application and library they can find, space is limited even on DVD images, so users are going to be installing software from outside the included application set. A good review should include your experiences installing something external as well.

Find an application, attempt to install it, and tell us how it goes. Don’t call up the local Linux Users Group and ask for hand-holding; just try and do it and report what happens without all the sugar-coating. For this step, you could pick something at random, or you could pick something well-known to you from another distro or OS.

Obviously, an application or game that you are familiar with makes for the best candidate. You are comfortable with it and know it works elsewhere. Look around; is there a DirectX video golf game handy? Or a Windows 95 small business accounting package that you have used for years? They are perfect choices and — best of all — they are real-world examples. Try to install both of them, and without resorting to WINE.

Does it work? If not, put it in your review. Linux will not succeed on the desktop as long as end-users cannot install their own Windows software without help.

Step four: judge the aesthetics

Review the themes, color schemes, and icon sets. You have to stare at the monitor all day long, so it is important to pick a distro that looks good.

But remember, you are not passing judgment on Linux as a concept, you are judging how well it measures up against other operating systems. Draw on your own experiences. Did you like BeOS, and mourn its demise? Tell us. Are OS X icons pretty? Explain in detail. Have you seen screenshots of Windows Vista (or some other pre-release) that put the desktop wallpapers of this distro to shame? Now is the time to say so.

Don’t neglect other media, though. Music and multimedia are rapidly becoming the cornerstone uses of home PCs. Fire up the media players included on your distro, and try to play some MP3s or iTunes Music Store tracks. Do you like the interfaces that you see? Put your favorite movie in the DVD drive and watch it, or download some videos from the Internet. Are the multimedia apps easy to use? Are they as simple and intuitive as your TiVo or your VCR remote control? If not, explain why.

Step five: try to figure out what the distro builders are really up to

One of the most difficult aspects of a good review is seeing past the surface and detecting the ulterior motives that bring you a Linux distro — particularly what is behind the decisions that you dislike. You will have to read the developers’ minds, but it is necessary.

What are their motivations? Are they good or bad? Are they catering to Linux gurus only, or are they kowtowing to Grandma and Grandpa by dumbing everything down too far? Do they no longer care about users at all?

If the distro is commercially funded (as opposed to a community project), explore the implications and point out where you see evidences of the difference this makes. Are the release coordinators corporate sellouts? Do recent mergers or financial sponsorship deals suggest outside influence? Do they have an “attitude” that you just don’t like?

Part of being an objective journalist is not changing the story to spare people’s feelings. You have to say what you know. And if you are not sure of your facts, present all of the possible theories. It is better to imply that the big corporations are ruining Linux than to say nothing and let it just happen.

Step six: assess the impact of this distro on the future

Again, this is something that probably doesn’t need to be said, but reinforcement never hurts: based on your experiences with this distro, predict the future of Linux. Will it succeed in “the marketplace”? Will it succeed on “the desktop” or in “the enterprise”? Tell us.

No one wants to install a dying operating system, so if that is what people are in danger of doing, you need to warn them.

Step seven: quit while you’re ahead

Above all else, be sure not to let time completely get away from you. I know, I know — the preceding steps cover a lot of ground. But your time is your own, so don’t go overboard just for the sake of one Linux distro review. Do you remember that stopwatch I mentioned in the first subsection? Here’s where it comes in handy.

You started it when you began the installation process, and you have (hopefully) been writing down the times when everything occurs — finishing the install, logging in, opening the main menu, etc. But it is important to set limits and know when to stop. Pick a time, and when the stopwatch reaches that time, just quit testing the distro wherever you are and start writing.

If you are new to the process, pick a longer limit — say, 90 minutes. As you get more familiar with reviewing Linux distributions, though, you will find yourself working faster and the judgments coming to mind almost as fast as you can write them down. With practice, you can get your overall review time down to well under an hour.

So what are you waiting for? Get out there, find an ISO, and give that distro what’s coming to it!