Author: Joe Barr
Prior to speaking with the author of the story, I learned that the study and the report had not been funded by Microsoft, and that the raw data came from 1,422 respondents — all from firms with less than one billion dollars a year in revenue.
To gauge the validity of research based on a poll, you have to know certain things about the poll itself, not just the published results. Key things to know include:
- How many were polled
- Who was polled
- How they were selected
- The questions asked
- How they were asked
Just as with trial juries, selection of the members of the poll group can bias the results. A better example might be a political poll where respondents may be taken primarily from one party or another, but the results presented as if they were representative of the population as a whole.
In this case, all 1,422 of the respondents were existing Info-Tech customers who were asked by email to participate. Whatever bias that selection presents is reflected in the results. The results of this study, however, are presented as if the respondents are representative of the market segment as a whole, not as a subset of invited customers.
A question about the questions
When I spoke to Info-Tech Director of Research Frank Koelsch, the primary author of the report, I asked if the questions asked were available. He told me that they do not publish the questions, just the results. Then he said, “They’re pretty apparent, though, from the results themselves.” If that were true, then why won’t Info-Tech reveal them?
I asked Dan Kusnetzky, a vice president at analysis heavyweight IDC, to comment on the study’s finding that Linux is not of interest to mid-sized firms. He said, “It’s really impossible for me to comment. I haven’t seen the survey instrument. I don’t know how the survey was executed. I don’t know anything about the demographics of the respondents.” The point being, if a well-known industry analyst is unable to say anything about the validity of a study because the data he needs to make a judgment on it is missing, then the typical IT manager and decision-maker is not going to be able to weigh the credibility of the report either.
Is the report biased?
The report’s wording is often misleading, and sometimes it’s simply wrong. Let’s start with the headline: “Most Mid-sized companies not interested in Linux.” Not even the numbers in the study support that contention.
My first reading of the study caught a glaring contradiction between the words and the figures in the study. According to the original study text, “Of those not running Linux today: 48 percent are not interested…” According to my math, 48 percent of 73 percent is only 35 percent of the total, and that’s a long way from “most.”
Info-Tech thanked me for catching the error, and said the text would be corrected to remove the phrase “Of those not running Linux today.” So the 48 percent that are not interested is now officially 48 percent of the total respondents. But that’s still a minority view, not “most” as Info-Tech would have you believe.
I asked Koelsch why the misleading use of the term “most” for a minority position. He said he doesn’t believe that it is misleading. He simply adds some more to the “not interested” category, taking it from one of the other categories. He explains that just because they are in the “Unsure if they will implement Linux” category, that doesn’t mean they are interested.
Yet the respondents had the opportunity to state unequivocally that they are not interested. It’s called the “Not interested” category, and 48 percent of the respondents marked that on the questionnaire. It’s Koelsch who has decided that some of the other respondents are probably not interested either, not the actual respondents. It’s Koelsch who stretched the value of 48 percent over the 50 percent mark with his misleading text.
I also asked Koelsch why he described the 10 percent who have plans to implement Linux over the next three years as being a “tiny 10 percent.” He laughed and said, “Well, because when we are looking at a factor of one in 10, that by anyone’s calculation is, let’s say, small. Call me crazy, maybe I could have used small rather than tiny, but it’s small.”
But Koelsch contradicted himself. Earlier in the conversation, when he was justifying the use of “most,” he said “there’s another large portion, another 15 percent, that aren’t sure.” Never mind that he changed that portion size from 14 to 15 percent. Look at the way he describes it. In Koelsch’s world, 10 percent deciding to implement Linux is tiny, while 14 percent uncertain is large. The size seems to depend not on the actual percentage, but on what’s being sold.
I’m not the only one to notice the bias. Dana Blankenhorn illustrated it on ZDNet by writing “Info-Tech Research Group reports Microsoft could lose as much as 10% of its mid-sized business customers to Linux, in its best markets, over just the next three years.” The point Dana is making, of course, is to take the spin factor into account.
Where’s the beef?
So is Linux really stalled in the mid-size market as Koelsch asserts? Not according to others we asked. Here’s what Bill Weinberg, open source architecture specialist for OSDL had to say:
We don’t see any evidence that Linux usage is stalling in companies that have less than $1 billion in revenues. The IDC data released in December takes into account small, medium and large organizations and reflects the clear and pervasive trend towards Linux with worldwide revenues for desktops, servers and packaged software running on Linux forecast to grow to $35.7 billion by 2008, a compound annual growth rate of 25.9 percent.
Dan Kusnetzky had this to say:
IDC is observing an expected evolution in the market for Linux operating environment software. More and more of the time, organizations are purchasing Linux pre-installed on systems rather than acquiring the software and pre-purposing an older system to support a Linux-based function. This trend tends to produce the following changes in the market — 1) fewer copies of Linux are purchased, 2) increasingly the enterprise version (containing maintenance and support services) rather than plain distribution is selected, 3) the revenues produced by these copies of Linux are growing and 3) fewer but larger systems are put into use to run Linux-based functions.
What’s the point?
All the anti-Linux bias in the report culminates with the following words: “The truth does not bode well for Linux-related vendors considering entering or expanding their presence in the mid-market.”
Now think again of what classic FUD is all about. This report attempts to frighten Linux people away from the mid-sized market. In fact, the very words used to do so are classic. Back in the early ’90s, when OS/2 was the victim of Microsoft FUD attacks, one particular FUDster used the phrase “This does not bode well” so often that its abbreviation — TDNBW — became known as a codeword for FUD. It’s certainly appropriate for this report too.
Category:
- Linux
