Content Matters

20 Years In, Personalization Still Fails to Deliver Data's Holy Grail

From the earliest days of the Internet, personalization was viewed as the holy grail.

Sites like My Yahoo quickly allowed users to personalize their web experience and it felt good for a while. 

But as time went on, choosing general categories, or selecting a handful of stock tickers just wasn't enough. We wanted true personalization. New products came on the market - iTunes Genius and Pandora promised to personalize music. Yet they still disappoint.

For data publishers, personalization would seem to be the key way to remain relevant in a world where information is increasingly free. Yet in market after market, personalized data still falls short. Here are three spaces where personalization should have transformed the market, yet it hasn't happened.

College Search

As the father of a HS sophomore, I've started to plan a few summer college visits. Our goal was to pick a few cities and plan visits around them. The process seemed simple - I'd pick a geographic area, select a few fields of study, provide a level of selectivity (e.g. top quartile), and I'd have immediate access to lists of targets. 

Apparently not. I tried several of the leading college sites, including Naviance and Cappex. Neither could deliver the simple lists I was looking for.

Hotels

With all the travel sites out there, picking the right hotel should be a no-brainer, right? Yet hotel screening sites rarely enable screening by the criteria that matter most to me. For business travel, assuming a minimal floor for safety and location, my priorities are:

1. Good wifi
2. Accessible outlets near the bed (I am so tired of moving furniture to plug in devices)
3. Good gym
4. 24-hour room service

OK - so try to search on those using TripAdvisor, Hotels.com or any other travel site. You can't. 

Hiring

Job boards were one of the early winners of web 1.0 and the process has been broken ever since. The job boards removed any friction in applying for positions, resulting in floods of unqualified applicants. I gave up on them a decade ago. 

The solution

In each of these cases, it's really about having good enough metadata to make the data searchable. In some cases, the publisher is fully in control. It's not that hard to provide more granular data on colleges and universities. The data is there. You just need to compile it and build a decent search interface. 

For travel or jobs, it's a little harder, in that you need the reviewer or applicant to fill out a form, rather than just entering text. But it's not that hard. I've posted TripAdvisor reviews and would be happy to score the hotel on 10-15 criteria, as long as you provide a decent UI to do so. 

It's amazing to me that, two decades in, we've seen so little advancement in creating great personalization for big markets. There's still a huge opportunity to take market share for those who can execute. And as the world shifts to mobile, improved personalization becomes even more important, driving alerts and other value-add features.

 

Semantic Technology Driving Real Revenue for Publishers

For the longest time, semantic technology has seemed more hype than reality, more future than today. Well, that’s not really the case. Publishers have been tapping into semantic technologies for more than ten years.

Wednesday, the SIIA will be sponsoring a half-day event focused on exactly that:

Driving Revenues with Semantics: How Semantic Technologies Boost ROI for Digital Media Companies

Evan Sandhaus of the NY Times R&D Labs will provide the opening keynote.

The afternoon event will do a deep dive into two areas where publishers are leveraging semantic technologies: semantic advertising and content enrichment. The semantic advertising panel features speakers from Peer39, MediaMath and AppNexus, who will share case study examples of ways in which publishers and ad networks are increasing the value of their existing inventory.

I will moderate the second session, focused on generating new revenues through content enrichment. I’m thrilled to have three great panelists, each of whom is using semantic technology in a different space:

• Hitesh Chitalia has been spearheading use of semantic technology at various business units of McGraw-Hill
• Stewart Wills, of AAAS.org, publisher of the journal Science, has just recently begun a project that leverages semantic tagging, both to improve internal editorial processes and to enable better content discovery.
• Ken Judy, of Simon & Schuster, a pioneer in the ebook space, have begun to use semantic technology to improve discovery.

Daniel Mayer, of Temis, will then provide a quick overview of how content enrichment can be applied to improve search, navigation and knowledge discovery.

And then... COCKTAILS and networking.

I hope to see you there.

Get off my lawn… and stop stealing my news

Like the cranky old man on his porch yelling “get off my lawn”, Rupert Murdoch last week came out to chastise Google (NASDAQ:GOOG) for indexing News Corp (NASDAQ:NWS) content. Whether you love or hate Rupert Murdoch, you’ve got to give the guy credit; he can get an entire industry in a tizzy just by spouting off.

The latest frenzy came after Rupert bashed Google and other sites that “steal content” and subsequent rumors that Microsoft was prepared to offer News Corp a deal to provide their content exclusively to Bing, removing it from the Google index.

While the comments generated fodder for the blogs, the reality is that whether or not News Corp content remains in Google is pretty insignificant. And it does nothing to address the underlying issues confronting the news industry.

An exclusive deal with Bing might be lucrative for News Corp, particularly as a short-term way of recapturing some of the ad revenue lost during the downturn. But in the long run, I don’t think it has a huge impact on either company. For Microsoft (NASDAQ:MSFT), it could add some cache to Bing (and after spending $100M to launch the search engine, shelling out a few bucks to Murdoch & Co is not a big deal). But news is only a small piece of Google’s pie and it’s not the part that drives the big revenues. Is the overall plan for Bing to simply become a vertical search engine for news and travel?

More importantly, it does nothing to change the underlying fundamentals of the industry, notably:

• There is a massive amount of traffic generated by free news content
• But, there are multiple outlets for almost every story; no single general news source is a must-read. In fact, most readers don’t even know the brand of most of the stories they read on the web. Like it or not, news is a commodity and its getting less valuable each day.
• While there may not be enough advertising $ to support the entire newspaper industry, there is certainly enough online advertising dollars to support multiple news sites.

These last two points are why I think that these efforts to move general news content behind the pay wall are largely fruitless. (Note that I’m talking about general news content; for a handful of specialized news providers such as the FT and the WSJ, where many subscriptions are paid by companies, a paid model may be appropriate.) There is a huge demand for free online news and it will be lucrative for some providers to look to fulfill that demand by providing a quality, free product.

Suppose for a minute that all the current newspapers decided to put up a paywall on the same day (and let’s suppose they each made this decision independently, so we won’t call it collusion). For a short term, it would create a chaotic environment for consumers of online news. Some might decide to pay, while others would make do with alternative media and whatever information they could find. But very quickly, there would spring up some players to fill that void.  Those players might include a handful of traditional papers (which, upon seeing 95% of their competitors remove their product from mainstream access, would quickly put their own product out there) as well as alternative media like the Huffington Post. You would also see other news media jump on the opportunity to expand their presence – CNN, MSNBC and others. We would likely see other new providers come forth, perhaps leveraging a mix of philanthropic and private capital sources for funding.

And those new providers would find a vast group of professional editorial resources available to them. Reporters who have been laid off as well as those not pleased with seeing their readership slashed as their content is moved behind the pay wall would clamor to join TheNewNewsBusiness.com.

Would the resulting product be as good as the best of our current newspapers? Certainly not at first. But over time, I have little doubt that news organizations developed from scratch today, with a deep understanding of the online world could quickly compete with the quality of traditional news organizations.

Let’s think about it for a moment. Most of the major newspaper brands were launched in the mid-nineteenth century. Their business model was born at a time when the only economical ways to reach local eyeballs was with newspapers, radio or yellow page ads. Their editorial models were developed in a world before globalization, where issues in the local community were the most important ones. It may have once made sense that each of 100 local newspaper organizations would each cover the same stories, but in today’s global world, that makes no sense. Yes, it’s important to have coverage of local events and interpretation of the local impact of global events, but that hardly justifies a full news room in every city of 100,000 population. On top of this, they all have the huge capital and labor expenses involved with printing and distributing printed newspapers.

The result is an industry with huge inefficiencies, a glut of inventory and an irrelevant business model built for a world that no longer exists. So, let them play with pay walls while they rattle their sabers. In the meantime, I’m looking forward to the formation of News 2.0.

Dating, porn still key drivers for $15 billion online subscription biz

Anne Holland’s new venture, Subscription Site Insider, has just released an in-depth study of subscription and membership-based content sites. According to the study, it’s now a $15 billion market that continues to grow by $1 billion per year.

 

In the study, the market is split into three segments with b2c sites accounting for nearly $6 billion, b2b sites $5 billion and adult sites generating $4 billion in revenue each year. Dating and online gaming each contribute $1 billion to the b2c figure; the dating figure would be higher if you count the 4.3 million Classmates.com members, whose membership fees are often paid in hopes of rekindling a long-ago high school romance (note: author’s editorial comment, not taken from the study).

 

The study included market surveys of 600 industry executives, plus a research study of 550 subscription/membership sites, and also builds upon 3^rd party research.

 

In line with their focus, Subscription Site Insider identifies many opportunities for additional revenue from these sites. Of the sites they review, roughly half were not using their site to drive ancillary revenues (ad sales, affiliate programs, online or instructor-led training, consulting, events and more). According to the study, only 10% of b2b sites and 16% of b2c sites engage in aggressive search marketing, as defined by the authors.

 

Some of the survey results were surprising. In the survey, industry executives were asked about M&A and expansion investments currently under consideration. While half of those surveyed said they were looking to launch another subscription product (no surprise), only 14% indicated that they were currently looking at launching a mobile subscription offering. A full 58% said that they were not considering a mobile subscription offering.  Another 28% said they might do so “if a good deal fell into their lap”. I can understand that response when talking about M&A, but do they expect the iPhone Fairy to leave an app under their pillow?

Sounds to me as though three-quarters of those surveyed are pretty far out of touch with the what’s going on in the content world. I’m not suggesting that everyone rush out and launch an iPhone or Droid app today, but leaders not considering mobile as part of their delivery platform going forward could be considered negligent in my book.

The full report, Practical Research to Grow Subscription Site Profits, is available for purchase on the Subscription Site Insider website.

An executive summary is available for free download.

Associated Press to Internet: Drop Dead

The Associated Press's plan to more aggressively police use of its content on the Internet seems less about saving the newspaper industry and more about justifying their own existence.

Do the AP's owner-partner newspapers really want their content removed from the Google index? How much might that reduce their traffic? Depending upon the news outlet, I'd guess Google accounts for between 25-50% of the traffic. How will those rapidly falling CPMs look when they lose half their traffic?

I realize the A.P. is in a tough spot. Newspapers rely upon its syndicated content, especially as they cut their own editorial resources. At the same time, the newspapers have pressured the A.P. to cut its fees. But the A.P.'s approach here doesn't do anything to address the long-term problem.

The problem remains that the basic business model for newspapers - that of being a near-monopoly for local advertising - is gone. The classified business has moved to Craig's List and elsewhere and it's not coming back. And the basic concept that a local auto dealer or department store should fund the cost of an editorial news operation no longer makes sense.

So, should the AP be allowed to go after those who steal its content? There are spam and splog sites out there which duplicate the content in its entirety and provide no links back to the source pages. So, yes, AP should go after those sites and partner with sites like Google to ensure those spam sites are not included in the Google index. But if they go after search engines and aggregators who follow the "fair use" doctrine, I think the cure may be worse than the disease. We'll see AP content removed from the search engine indexes and see new competitors spring up to fill the void. Meanwhile, AP newspaper partners will see a drop in page views, further eroding advertising revenues.

The solution should lie in partnering with the search engines and aggregators to ensure that AP-syndicated content is shown in the best light and drives the most traffic to its partner sites. The AP should be providing widgets for its partners' use on social media platforms and mobile devices, all designed to help them grow their share of news content consumption. They could also work with aggregators to develop ways to generate shared revenues. Instead, they seem focused on a plan to make themselves irrelevant on the dominant platform for news - the Internet. Sure, they may generate some short-term licensing revenues, but in the long-term, they're destined for irrelevance.

Associated Press - be careful what you wish for.

Mary Meeker on the Economy and Internet Trends 2009

Love her or hate her, it's hard to ignore Morgan Stanley's Mary Meeker.
This week, she posted a massive, 147 slide deck entitled "Economy + Internet Trends".

Despite the massive size of the deck, it's worth taking 30 minutes or so to walk through. She looks at the impact of the current economic environment on Content and Internet companies. Whether you agree with her analysis, there's plenty of data there for you to chew on.

The first 20 or so slides provide some interesting views into how the economy got here; nothing new but some useful charts and perspectives on what's similar and what's different from past recessions. The economy has pushed ecommerce growth to negative (starting Q3, 2008) after consistent 20% annual growth rates from 2001-2007. The technology sector's five flat or down sequential quarters matches 2001, but the outlook this time is worse.

Key spending sectors of the S&P500 have taken a huge hit - financial services, industrials, consumer discretionary, materials, information technology, telecom and energy. Even utilities, health care and consumer staples stocks are down 30-40% from their pre-recession peak.

Then she digs into the slides that are of greatest interest to those of us in the content and technology space. The trends she highlights include:

Online advertising: ad spend is closely tied to GDP growth. On top of that, right now we're in a very difficult period as the disruption caused by social media, video and VOIP is not yet translating into significant revenues Online CPMs are a fraction of offline CPMs. Meanwhile, doing a regression analysis, she shows how a flat GDP in 2009 could lead to a 4% drop in ad spend.

One bright note is that in difficult times, consumers spend more time, rather than less, on the Internet. It's "a cheap/efficient/transparent thrill!" as she describes it. The Internet is becoming a necessity. Broadband Internet was the last thing to be cut in consumer survey (even after personal care, toiletries & cosmetics).

At the same time, undermonetized social media, video and VOIP creates an opportunity for marketers to capitalize on low CPMs.

Looking ahead five years, Meeker projects the online consumption mix will be what she describes as consumer-enhanced professional content: anchored with first class professional content; augmented with first class user-generated content that's ranked/reviewed/'edited', supplemented by all-comers user-generated content that's ranked/reviewed, all presented in a holistic/widgeted way.

Or, as she describes it, a "clean combo of WSJ.com+bbc.co.uk + digg + techmeme + youtube + nytimes.com + allthingsd.com + facebook + twitter"

Meeker drills in to online advertising, noting that Google's share has increased from less than 50% in Q4 2005 to 67% in Q4 of 2008. Search advertising continues to show growth and is taking budget from other forms of advertising, most notably print magazines, direct mail and newspapers.

There is still a huge advertising opportunity - today, online advertising spend is $288 per home vs. $818 per home for newspapers. While CPM's and CPC's may be under near-term price pressure, if targeting/ROI continues to improve (as she believes they should), there should be long-term upside.

Meeker remains bullish on the ecommerce space. While some items, such as computer equipment, event tickets, books, music and videos have seen huge adoption rates on the Web, overall US online ecommerce penetration remains only around 4-6%, though it continues to rise.

Amazon should continue to show growth as it has become the web's most effective commerce search engine and continues to provide new technologies for its users, such as the Kindle and mobile apps for research and purchasing.

Another strong trend for Meeker is the positive growth in mobile. New mobile products, from iPhones to the Kindle to Netbooks are driving incredible growth. The enablers for this growth are:
1. broadband and wireless infrastructure; the cloud
2. Hardware: small, fast and cheap storage, touch screens, GPS and more
3. Software - browser-based tools, widgets
4. Communications tools - messaging, social media, cameras, VOIP
5. Digital content - music, video, news, search, shopping, weather, maps and more

She notes that Google Voice provides Unified Communication for consumers - one phone # + VOIP + Voicemail Transcription + SMS. That's a pretty big game changer.

We are undergoing the greatest media transformation in history and Meeker asks "Where is the great creative" to drive it?

In closing, she reiterates her believe that companies with cogent business models that provide consumer value (Amazon, iTunes, etc) should survive and even thrive in this economy, noting "consumers need value more than they've needed it in a long time".

The full presentation is posted at http://www.docstoc.com/docs/5045631/Meeker-Tech-09.

Thanks, Paul Kedrosky, for the heads up on this.

Five Questions for Steve Arnold

Earlier this week I had the chance to catch up with Steve Arnold, author of the influential Beyond Search blog. He is the author of two major studies about Google and six other monographs, including Enterprise Search Report, 1st, 2nd, and 3rd editions, and Beyond Search, a study of 24 content processing vendors for The Gilbane Group. Information about Mr. Arnold is available at ArnoldIT. He can be contacted at seaky2000 at yahoo dot com.

Content Matters: 2009 will certainly have its share of challenges. What do you see for search in 2009?

Stephen Arnold: The challenges to search depends upon where one stands to view the industry. For licensees or customers, 2009 is going to be focused on cost control and balancing user demands with resources. I expect fee push back and probably more interest in the lower cost, less complex systems driven by the customer's chief financial officer and budget realities.

For small vendors, the challenge will be to find niches and make compelling use cases in order to win sales. I think that some smaller vendors will resort to marketing and PR tricks, not solutions. This means more confusion in the marketplace.

For the larger vendors, the world is breaking into two camps, and the contention between Microsoft's complexity and Google's simplicity will make it harder for the second tier players to get "easy" client wins. Microsoft and Google have significant clout, and it is not at all clear how Microsoft will handle its "low cost" and bundled search offerings. Google, on the other hand, chugs along with its simple, easy, cost predictable approach.

In short, 2009 is going to winnow the buyers, the vendors, and the pundits. Who knows what will unfold in the present financial climate.

CM: Twitter adoption continues to grow. Has Google missed the boat on real-time search?

SA: Real time search is useful for certain types of information retrieval. Google and Microsoft have not "missed the boat" because neither company is an early adopter of new technology. I think Microsoft may be forced to take a stake in Twitter as the company did with Facebook. Google, on the other hand, has the pieces available to deploy a real time search within its present infrastructure. My thought is that Twitter and other real time systems are providing useful information to the Google and Microsoft teams. This market remains attractive and, so far, somewhat fluid.

CM: Semantic technology has been long-hyped, but remains a niche application; do you think semantic search will ever become mainstream?

SA: I am on record as a strong advocate of semantic technology in any of its manifestations. The technology, however, is somewhat old hat. Google, for example, has had semantic components operating within its infrastructure for some time. Microsoft has a history of research in this field. The current buzz is one of those interesting characteristics of information retrieval. Search in many organizations is miserable and a supposedly "new "silver bullet comes along. Everyone says, "This will address problem X and problem Y." Semantic technology is not something that the casual user needs to worry about. Semantics will play an increasingly important role in the plumbing. Users will click on topics, see related topics, and have certain queries reformed on the fly. Google, in my opinion, is now in a position to become the "Semantic Web", which as you might imagine makes some people bristle.

CM: Amazon, Google and Apple are starting to stake out their positions for ebooks and book search. Do you think one company will take a dominant position in ebooks the way Apple has for music?

SA: eBooks are one component of what I call the "digital Gutenberg", which is the title of my forthcoming study of Google and other online services. The general user, including some online outfits, now realize that their systems are capable of end to end content operations. There will be other information services flowing from the Amazons, Googles, and Apples of the world. Infrastructure, reach, and comprehensive platforms make these eBooks and other products possible. The key point is that traditional publishing companies and small information outfits will have to rethink their business processes and business models to thrive and survive. Information about my new study is here.

CM: Historically, as the economy comes out of recession, it creates transformation and change. Which tech segments have the potential for transformation in an eventual recovery?

SA: I have been working in online for more than 35 years. The factors driving change are economics and demographics. The too expensive methods can't work in some business operations the way these methods did a year or two ago. Demographics means that those resistant to change will be pushed aside by the younger cohorts bubbling up in the workforce.  The shift will be painful to many organizations who have the chains of their existing methods and business models around their ankles. Many organizations will die, unable to adapt to the present ecosystem.

Yahoo Gets a CEO! Now, What?

To put it mildly, 2008 was not a good year for Yahoo. The first quarter was consumed by the on-again, off-again romance with Microsoft. Q2 was when they declared their independence. In Q3, the numbers proved that to be a mistake and kicked off the Jerry watch. Q4 began a long, drawn-out search for a new CEO.

Here we are in 2009 and Yahoo has announced their new CEO, veteran technology leader Carol Bartz. I think Bartz is a solid, if unspectacular choice. She's a grownup (as my old boss used to say) and clearly has a proven track record. While Jerry Yang seemed too much in love with the company he'd created to make difficult decisions, Bartz shouldn't have any problems doing so. Besides her 15 years leading Autodesk, Bartz has sat on numerous boards, including Cisco, Intel and NetApp. Were she running the show when Microsoft came calling last year, I've little doubt that the process would have been smooth and professional (well, as professional as dealings with Steve Ballmer could go).

With Bartz running the Company and Sue Decker leaving, the search deal with Microsoft should be back on, reports Kara Swisher. According to Kara, Microsoft has a proposal ready to be presented to the Yahoo board, with an announcement possible by month-end. That deal seems like a no-brainer right now.

Henry Blodget worries that Bartz has no consumer Internet experience, fearing that an outsider might not fully comprehend that the search business is lost and that improving their offering is not a viable option.

Jon Ogg at 24/7 Wall Street likes the choice of Bartz, noting that she is an operator and orchestrator alike, but notes that with real growth unlikely in the current economy, Wall Street will need to decide on new metrics to measure Yahoo's success.

In a related post, Doug McIntyre agrees that Yahoo should kill its search offering, but sees potential opportunities in the nascent mobile search market, in the U.S., Europe and Asia. He also notes that Yahoo's core display advertising business is still fairly immature and there remain tremendous  opportunities to gain on traditional media.

So, will Carol Bartz be able to revitalize Yahoo? It's a tough challenge, but doesn't have the structural challenges that turning around GM or Citigroup might. Let's give her some time to come forward with a plan and let's hope she can do it. Yahoo remains a top notch Internet brand and there are a lot of reasons to hope they succeed.

Powerset Plus Live Search: A Half Percent of Web Searches May Get Better Results

TechCrunch interviewed Powerset founder Barney Pell and Microsoft Live Search GM Ramez Naam yesterday, following the formal announcement of the acquisition. After a half-dozen or so questions about whether and when the deal would be formalized, Arrington dug in to the specifics of what the acquisition might do for Microsoft (NASD:MSFT) in their efforts to become competitive in the search market.

The key takeaway - according to Ramaz Naam, is that roughly 5% of searches now done on the Internet could be aided by the application of natural language processing. Considering that Microsoft has roughly 9% market share of the Internet search market, that means that roughly 0.45% of searches will get better results with Powerset.

Now, one could argue (and they do) that by returning superior results, users will move to Powerset/Live Search and their market share will grow. But, to make that happen, they need to show that they deliver radically better search results most of the time. You can't displace what's become a verb (Google) by offering a little better results once in a while.

I think that semantic search will, one day, play a role in improving search results. But my experience with semantic processing at ClearForest and in watching technologies such as Powerset and Hakia suggests that we are years away from seeing it deployed on a mainstream basis. So the fact that Microsoft has added a team of 50+ developers, focused on semantic search, to their Live Search team is a good thing. I just don't see it having any significant impact on Microsoft's ability to become a player in web search in the next three-to-five years.

The full TechCrunch interview is available as a blog post and also as a podcast.

Microsoft to acquire Powerset but is this the right move?

The rumors tonight were that Microsoft will acquire semantic search engine Powerset for north of $100 million. If the deal happens, it's not a total shock. There were rumors of a potential acquisition last month at the time Barney Pell and company rolled out their prototype search of Wikipedia.

I'm not sure that I understand the logic behind the deal, though perhaps it's simply intended to keep Google from acquiring the technology. But adding semantic search technology to their LiveSearch won't solve Microsoft's (NASD:MSFT) problems. Their problem in the search market is that no one uses their technology. Buying a lab experiment won't change that. I also remain skeptical that Powerset technology really scales. Having worked in the semantic tagging space for four years, I've seen the challenges in making that model work across disparate content sets.

Now, $100 million is not a significant acquisition for Microsoft, so perhaps they're just looking at it as a lab experiment. And it may be that over time they will find ways to leverage the Powerset semantic technology across Microsoft products, perhaps integrating it into the FAST platform for enterprise search. But for those who may tout this as a way for Microsoft to get back into the web search market, I think they'll be disappointed.

UPDATE: TechCrunch reports that the deal is done. Still doesn't do anything to help Microsoft's position in the web search space and I doubt that Powerset will ever get out of the sandbox, but $100M to MSFT is just not that big of a deal.