By

Market research is a discipline unto itself, a science that should take into account biases and factors that could skew the results. And, of course, market research should be free of the taint of bias.

Unfortunately, when the boys at ENT went out in search of proof that IIS was the most popular Web server in the Fortune 500, they blurred the line between market research and advocacy. In the end, they released an article that is riddled with flaws, inaccuracies, and misleading statements.

Let's begin with the lead:

"The largest US corporations are bucking the trend toward open source Web server deployments, according to an ENT study of Fortune 500 Web sites."

OK, that seems valid: the Fortune 500 is usually very conservative and it wouldn't surprise me if IIS and Windows NT were used more there than in the general Internet world.

The writers then go on to explain their research method: they typed the main corporate brochureware site into the Netcraft search engine and then reported on what they found.

Hmm. You mean they didn't make a single phone call to a Fortune 500 company to see what Web servers were being deployed within the corporation? This was no scientific attempt to see what the Fortune 500 was deploying companywide, despite what the authors promised in the lead. Indeed. the purported "testing" was limited to "the corporate identity or brochure Web sites of Fortune 500 companies." As Keith Jackson would say, Whoa, Nellie! It seems to be a reach to claim that the Fortune 500 is bucking the trend toward Open Source technologies because many corporations place their brochureware on IIS and Windows NT. Indeed, I don't think I would consider brochureware to be a very valid test of any Web server, nor would I try to discern any trends based on brochureware.

But wait again: even within the ill-considered guidelines set up by the authors, there's even more inconsistency, probably due to sheer laziness. When the authors set out to see what server was running a company's brochureware site, they didn't stop to consider that corporations may have more than one brochureware site.

Take, for instance, the grocery giant Kroger's, which is actually a collection of chains, including Tom Thumb, Owen's, Ralph's, Fred Meyer, Pay Less, and more. If the ENT "researchers" had taken the time, they would have seen that some of the chains under the Kroger's umbrella are fueled by Netscape Enterprise Server running on AIX--a finding that didn’t seem to make it into their research. I am not surprised: ENT is a Web site devoted to touting Windows NT, and it would be awfully inconvenient to point out that AIX and Netscape were in wider use.

The bottom line is that the ENT research is worthless on many levels: no one bases their computing decisions based on what the Fortune 500 uses for brochureware, and the results are so flawed as to be utterly worthless.