IT research firm Gartner is being sued by ZL Technologies who claim that
Gartner’s famous Magic Quadrants are unfair in that they give an advantage to “larger
companies with large sales and marketing budgets”, who can afford to pay high
fees to Gartner.
It’s not the first time that Gartner has faced charges of showing favoritism to large clients. A prominent Route 128 VC once shared with me that he had analyzed all of their (tech) portfolio companies and had uncovered a “magic number” of $73,000 (this was in 2004). According to the VC, all of their portfolio companies who spent at least that amount with Gartner each year appeared in a Magic Quadrant, while those which spent less than that number were not. Of course, that’s an anecdotal story and there’s no public evidence to substantiate it. But, it’s hard to shake that perception when you generate revenues from the same companies which you rate.
In many ways, it’s the same issue that has tarnished the credit rating agencies, which generate revenue from rating debit issues, as well as by selling research. In the early days of IT research, firms tended to fall into one of two camps. Firms like Gartner made their money by selling research to corporate IT departments, while those like IDC generated revenues largely by providing services to technology companies. But over time, those delineations became fuzzy or nonexistent. Technology companies were encouraged to buy services from the traditional IT research firms; a selling point was that you could have briefing sessions with analysts. The firms also began to sponsor conferences and events designed to showcase emerging technology companies. Here, you could gain exposure to potential buyers or investors, all for a hefty exhibitor fee (typically $25-40k). Before long, firms like Gartner, Forrester and others were selling $100k per year or more in services to the subjects of its research.
So, what’s the solution? It would have to be a market-driven solution. If corporate CIOs deemed IT research to be “tainted” by these conflicts of interest, research firms might be forced to “pick a side” and give up revenue streams from the either vendors or clients. But that’s unlikely to happen, especially in current market conditions.
A good start would be transparency and a “Chinese wall” between vendor-research and client-side research. I doubt they would do this, but Gartner could gain credibility by having an independent firm “audit” their clients and analyze relationships between dollars and quadrant results. At minimum, it would help for all industry research shops to disclose the extent of its customer relationships with any client that it also includes in its research. Gartner claims that would violate its client's privacy, but I truly doubt any Gartner client would object to this type of disclosure. It seems more designed to protect Gartner's competitive position than any concern for its clients.
In a post on her blog, Gartner Ombudsman Nancy Erskine tries to allay any concerns about the relationship between research and client status (HT to Andrew Spender for the link). In a follow-up post a few weeks ago, she added that "Anyone, client or not, user or vendor, can come forward if they think Gartner has published content that is inaccurate or misleading, or demonstrates bias or unfair treatment." But I think that misses part of the issue - for many smaller technology companies, just getting into a quadrant is what matters, not whether Gartner is publishing misleading or inaccurate information.
In the meantime, IT research faces other challenges. While Gartner, Forrester and others still have a strong presence, they have largely ceded coverage of the emerging technology space to blogs like TechCrunch and Mashable. Meanwhile, GigaOm has launched GigaOm Pro, a subscription-based IT research product led by Michael Wolf. The GigaOm Pro model, with its $79 subscription price, is unlikely to displace traditional IT research, but it clearly changes the model for those firms as well. At the same time, many analysts have launched boutique firms, like Charlene Li’s Altimeter Group, which recently hired Jeremiah Owyang, Ray Wang and Deborah Schultz from Forrester.
For those who like to read legal docs, Dave Kellogg takes a deep dive into ZL’s court filing along with Gartner’s response, and has some practical analysis.
Meanwhile, SageCircle points out that Gartner has been criticized in the past and has used those complaints as "evidence" to bolster its credibility.
