The future of RSS: smart search engines

Over at Mashable, Ben Parr wonders, "What is the future of RSS? Is social media a better alternative?" I believe the world is groping towards a better solution than either.

In two years I have switched several times between feedreaders like Google Reader and social news sites like Reddit, Twitter and Friendfeed. Feedreaders amplify the volume of my reading. Social news helps me find the highest quality stories. The tension between quality and quantity keeps me switching. With growing volumes of news in an increasingly-online world, feed readers and social news are each incomplete.

But I am an extreme case; most people read far less. Do power users matter? They can help highlight a trend. Every reader online wants relevant news; some just go to greater lengths than others. Perhaps the migrations of power users between feedreaders and social news sites can teach us how to serve all readers.

Feedreaders assume you want to read everything by everyone you subscribe to, and nothing by anyone else. Subscribe to too many, and those assumptions start to break down. Chronological ordering starts to suck. Frequent writers drown out the rest, regardless of who you care about. Sifting through the noise becomes a challenge.

The popularity of social news is largely explained by this challenge. 10% of the users on a social news site vote on stories, and only 1% comment. The rest of us are all using social news purely to find interesting stories, often because the feedreader didn't work out.

But using a social news site has its own drawbacks. You can't find as many stories. Stay at the front page or in a small community, and filtering works. Lower down the list, quality drops. A larger community provides faster turnover, but it's also susceptible to lowest-common-denominator effects - think pictures of lolcats or youtube videos.

Even when filtering works, you only find stories your friends find interesting. Over time, you start to ignore interests that don't overlap with your network. You risk spending time reading low-quality comments or flame wars. Echo chamber effects suppress dissenting voices, though those are often the most interesting.

In a healthy community people do their reading in private, and come together to discuss what they read.

If neither works, where does that leave us? Competing incomplete interfaces create false dichotomies. Asking readers to provide favorite sources is a good start, and so is voting on stories for your friends. Accepting recommendations from friends is a part of the puzzle, but a small part, lest you risk endlessly regurgitating each other's recommendations. There's no reason these signals can't be combined.

The essential property of both feedreaders and social news sites: they aggregate content from many sources before presenting it to the reader. We need a better aggregator, a feedreader that can handle firehoses. One that can rank stories smarter than just chronologically or alphabetically, perhaps even adapt to our changing interests.

What would such an aggregator look like? It would have scale, to discover feeds quickly, and to crawl all the feeds out there. It would have smarts, to connect you up with only the stories you find interesting, and to prioritize them. These are big changes; the new tool looks nothing like its forebears. What it resembles most is a search engine. It crawls and indexes everything it can find. Rather than responding to queries, it knows you and your tastes and alerts you to interesting pages. It can be consumed in multiple layouts wherever you go - facebook, twitter, your feedreader.

This is our vision at MeeHive.

permalink

Realism in simulation

"The goal of simulation is not to simulate reality as closely as possible. With an accurate model you cannot find commonalities."

— Tom Slee

permalink

Start a startup, become a low-income household

“You look at assets and liability to judge the strength of established companies, but when you have a startup, you look at cash flow. Low-income families are the same way. When we talk about poor people living on a dollar a day, you don't get a dollar every day. It is a lump sum, and it is very irregular. The poorest households can recall their cash flows a month ago.”

— Daryl Collins

permalink

“The general sequence of friction-reducing inventions is thought to have been runners, rollers, rollers held in place by guides, rollers held in place by guides and thickened on the ends to make them roll straighter, the wheel and axle. The wheel appears to have been first used in Sumer around 3500 BC, whence it spread across Europe, Asia, and North Africa. This orderly diffusion pattern makes it conceivable that all the wheels in use today are directly descended from the invention of a single gifted individual."
— Cecil Adams

permalink

Newspapers and personalization

Bruce Kasanoff thinks personalization could save newspapers, and here's how: Generate custom front pages for each user. Avoid echo chamber_i effects by providing alternate views. To make money, don't advertise. Instead, generate targeted leads for the most influential of your readers. Above all, focus on making your readers smarter. Remember information for them, not just about them.

Personalization holds promise for newspapers, and also feedreaders, online aggregators, and social news sites. Get it right and you can win the attention of lots of readers. It's a traumatic change for newspapers to attempt, though. People change; tuning for individual users implies adaptation and learning. Newspapers would require ongoing manual attention, expensive and culture-altering manual attention. Attention to take away from journalism and content-generation.

There may be an alternative. Personalization is important if you face customers, but perhaps newspapers needn't face customers. The alternative is a division of labor (and revenues) between writers and publishers. Let newspapers focus on original research and writing, and take a share of ad revenues from the aggregators that send them traffic. comments

  
Kartik Agaram, 2009-04-28: Hmm, newspapers may be forced into the division of labor anyway. And not even get a share of the revenues!   
Anonymous, 2009-05-01: Hi Kartik,
Thanks for the very interesting post. I totally agree with you that the future of newspapers is in creating a more personalized approach for each reader. In this rapidly growing information age, media providers of all sizes need to adapt their approach to the specific needs of each reader.
Moreover, major newspapers need to be held more accountable for the quality and accuracy of the news they produce. We at NewsCred are trying to deal with this aspect in particular and are seeking to engage more users in the debate on media credibility. Our ultimate goal is to become the online reference point for trustworthy news articles, authors and sources. I realize that we have grand objectives but we whole-heartedly believe in the importance of a free, transparent and accountable press and we must start somewhere!
One thing is for certain. We are experiencing a massive shift in the way news information is produced and consumed. It's high-time for the major newspapers to accept this transition and to work with it before they contribute to their own demise...
Thanks again for the interesting post! Alex
Community and Marketing Manager
http://www.newscred.com
  
  
Kartik Agaram, 2009-05-01: "..media providers of all sizes need to adapt their approach to the specific needs of each reader." My point was the opposite: perhaps providers don't need to adapt to readers.

permalink

Oswald Garrison Villard in 1918: Few industries maintain so many unprofitable enterprises as newspapers. A newspaper owner gets a place at every table, access to all the top politicians' ears, and the power to impose his worldview on his readers—or, at least, the illusion of such influence.

Clay Shirky: Society doesn’t need newspapers. What we need is journalism. And journalism has always been subsidized.

Jack Shafer: What's gotten many newspapers in trouble today is poorly timed acquisitions. At the height of the bubble, no price was too high if a newspaper claimed to be the dominant daily in a market. Newspaper owners who overburdened themselves with debt are more vulnerable than the medium itself. Most struggling newspapers will survive at lower but handsome profit margins.

via

permalink

Three books

Here are three books with the same fundamental substance: Vedanta, 7 habits, Flow. But there's huge variation in the form. Vedanta, written in antiquity, states the truth baldly in dictums. 7 habits, from the late 80s, avoids putting the reader off by phrasing its dictums as choices. Flow synthesizes cognitive research to describe how the changes we make within ourselves lead to external improvements. That's an appealing formulation to me after 20 years of the self-help cliches spawned by 7 habits.

These books exemplify a trend: knowledge gets progressively more accessible, not just by ubiquity but by the form in which it's consumed. When knowledge was scarce readers were supplicants, happy to take it in any form. Now it is abundant, writing is a buyer's market with greater emphasis on form. The filters readers create—against the preachy, against cliches—coevolve with the gimmicks of sellers and the skill of writers.

Credit: conversations with dad

permalink

“Children even as young as three and four years old often know that stealing is wrong, even without being explicitly told by adults. They don't need us to define the goal. That's easy. The problem is actually living by those values day to day. We need to help them develop a deep, abiding commitment to these values, a commitment that can override other needs and goals. The hard part isn't moral literacy; it's moral motivation."
— Richard Weissbourd

permalink

“GM might be the latest casualty of the distorted incentives created by our employer-based health care system. The whole idea of Social Security is that people do an inadequate job of saving when left to their own devices. But companies, even companies as big and proud as General Motors, are overly concerned with the present as well."
— Nate Silver on the limitations of corporations

permalink

Eliezer Yudkowsky: Our culture puts all the emphasis on heroic disagreement and heroic defiance. A group which can't tolerate disagreement [and criticism] is not rational. But if you don't tolerate agreement [and praise], you also are not rational.

Rational people often feel ashamed of strong feelings. The best informal definition I've ever heard of rationality is "That which can be destroyed by the truth should be." We should aspire to feel the emotions that fit the facts, not aspire to feel no emotion.

Carl Schulman: When we rate individuals highly the affect heuristic comes into play. Students can revere teachers simply because they only interact with the latter in domains where they know less.

A well-written piece that purports to summarize a field can leave you ignorant of your ignorance. Especially if it doesn't cite its sources. Without reading major alternative views you will tend to overrate whoever you came across first.

Eliezer Yudkowsky: In this world there are people who have created awesome things and it is okay to admire them highly.

permalink