Linux For Suits

December 2002

Ends and Means

The carrier and content industries aren't buying the Net's end-to-end argument. Here are a pair of open source ideas that might finally win them over.

Here are two questions to mull for the couple thousand words or so:

1) Why are whole industries, including the biggest carriers and content producers, still clueless about the Net (and Linux along with it)?

2) What open source projects will help those industries not only understand the Net and Linux, but start innovating and making money with them?

The Internet has been around for a long time. The ideas behind it have been around even longer. Many of those ideas were shared with UNIX, which has co-evolved with the Net since 4.2BSD was released in 1983. Once that happened, Peter Salus says in Casting the Net (Addison-Wesley, 1995), "Solid protocols on a flexible operating system at low cost was an unbeatable combination. OSI never had a chance."

OSI was an abstract architecture for open systems that turned out to be no less monolithic, and far less appealing, than any vendor's "complete network solution". Oceans of ink were spilled in the 80s on the subject of OSI. Many vendors signed on to endorse and co-develop it (as the Corporation for Open Systems -- COS) in 1985: but ultimately the OSI model's only real success wa getting everybody to separate topics into "stacks" (with hardware on the bottom and applications on top).

OSI failed, John Quarterman told Peter Salus, because

What got implemented instead was UNIX and the Net. Together they're still getting implemented -- with Linux playing a leading role. But many of the early lessons still haven't sunk in. In fact, the very nature of the Net remains a mystery to many -- perhaps most -- people who aren't familiar with the seminal ideas behind it. Let's look at three expressions of those ideas.

The first is from End-to-End Arguments in System Design, the 1984 essay by Jerry Saltzer, Dave Clark and David P. Reed. This was the argument that framed the Internet's architecture, and blew away the whole centralized concept of the "stack", where ownership of lower levels gave control to everything above:

The second is something David Isenberg wrote in The Rise of the Stupid Network, a widely circulated internal document that cost David his job at AT&T in 1997:

The third is from the preamble to The Cluetrain Manifesto, which David Weinberger, Chris Locke, Rick Levine and I wrote in early 1999:

Here's a way of tying all three together: Markets are made where the Net's ends meet. Supply and demand, production and consumption, seller and buyer, vendor and customer -- all those polar forces whose dialog we call economics -- are put right next to each other by the Net's empty (stupid) middle.

The Net's end-to-endedness supports countless new efficiencies that go without notice because they happen in the background. GE Global eXchange Services (GXS). for example, processes over billion transactions worth over a trillion dollars a year -- all over the Net.

A couple years ago, GXS president and CEO Harvey Seegers told me Cluetrain's key insight was that the net evened the balance of power between supply and demand. In the past GE was in command of its relationships with its customers, he said; but that ended with the Net. Now customers have real power. Even giant companies like GE are coping with a new reality where they can't muscle their customers any more -- and liking it, because now they have much healthier relationships with those customers.

After Cluetrain came out we heard the same kind of thing from lots of other big companies, including Wal-Mart, Johnson & Johnson, Prudential and Ericsson.

But a conspicuous silence came from the companies that actually carry the Internet to homes and businesses. Verizon, AT&T, Qwest, Time Warner Cable, Comcast, Cox and SBC all try to talk Internet jive, but they all still seem to be dreaming about metered traffic and subscription valves on content spigots.

This dream has been a nightmare to Internet veterans ever since carriers first began to build broadband services around the same supply/demand power asymmetries established by consumer marketing. To them the Net was all big-to-small and few-to-many, just like retailing, and just like television.

It so upset Gordon Bell that he circulated an email in January, 1995 with the title "Building Cyberspace with One-way Streets -- bad idea? conspiracy? shortsightedness? incompetence?" In it he pleaded with his correspondents to put their "bodies in front of the backhoes that are installing asymmetric networks that simply mimic cable TV."

The big "content" companies from the entertainment industry shared those fantasies. Unfortunately, they did get the Net's founding clues -- and hated them. The last thing they wanted to see was billions of dumb consumers turned into smart customers -- or worse, smart suppliers. Or, worse than that, both. Which is exactly what Napster showed them. That made Napster very unlucky.

But Napster was just one mutant animal. It wasn't a species, and it wasn't a habitat. That's why peer-to-peer (P2P) MP3 file sharing didn't go away after the RIAA put Napster through the chipper. It couldn't go away, because the RIAA's problem wasn't Napster, but its habitat: the Net itself.

In the end-to-end ecology of the Net, everybody can share anything with anybody, including ideas. Or, in business parlance, innovations. Good ideas beget better ideas. One innovation suggests another and another.

That's why Napster's chopped up remains came to life like the army of brooms in The Sorcerer's Apprentice. These include not only countless Napster-inspired alternatives, but a whole new end-to-end file sharing protocol: Gnutella.

As of September, tallied 2,525,791 FastTrack (a proprietary Gnutella-like service) users online. Slyck says that's 25% more than Napster had at its peak. iMesh has 836,385. Gnutella has 130,556. Slyck also lists six Napster clones, plus 191 MP3 file servers on five networks that range from 3.4 million to 8.3 million songs per server.

The RIAA has taken some of these guys to court, of course, and they've talked Congress and the Copyright office into taxing the U.S.-based Internet radio business out of existence. But they can't stop resourceful companies from taking advantage of the Net's end-to-end magic, which ignores national boundaries. Here's Dean Landsman, a veteran broadcaster and programming consultant:

In the long run (and it might be very long), the Net will win. When that happens (hopefully sooner than later) the question for the carrier and content companies will be no different than it was in 1995: How can I win here?

David Isenberg sets up the answer with a few clues from his original Stupid Network paper:

The "network services" David talks about are "stacks" owned by the carriers. They are not part of the Internet's infrastructure.

The Internet, on the other hand, is comprised of many "stupid" protocols that each provide infrastructural network services. Mail routing. DNS lookup. Hypertext. File transport. Among those, thanks to Gnutella, we now have peer-to-peer file sharing as well. In fact, Gnutella is so peer-to-peer that crossloading seems a better term than of "uploading" and "downloading".

"Crossloading" is a transport concept that was plainly implied by TCP/IP but not implemented until Napster prototyped it. In other words, Napster, in spite of its centralized and proprietary design, did a fine job of making the end-to-end argument anyway.

This argument is still so deeply anathema to both the carrier and the content industries that its success at taking advantage of both is an irony of the first order. So is the fact that the Net withstands continued dysunderstanding by the Hollywood types who desperately want to regulate it away. And so is the fact that Congress almost unanimously lets Hollywood do the thinking for both of them.

Thanks to years of constant lobbying (going back to '95 and earlier), Congress today shares Hollywood's understanding of the Net, and its paranoia as well. Tom Poe, who runs a free recording studio in Reno, recently sent a helpful suggestion to Senator Harry Reid of Nevada. Here is how Senator Reid replied:

Note the language. Senator Reid sees the Net as "Intellectual property" real estate that must be "protected". Stuff made unimportant by lack of ownership is "public domain content" to which "ordinary Internet users" should be "allowed" to have "access". To gauge the ubiquity of this default understanding in Congress, consider this: the DMCA was passed by a unanimous vote.

It's going to take a lot of counter-lobbying to even begin changing all those minds. The short-term prospect isn't good. In his keynote speech at the O'Reilly Open Source Convention last summer, Lawrence Lesssig offered little hope for reasoning with people like Senator Reid. Lessig told the story of Congressman J.C. Watts, who recently resigned his seat because "If you are explaining, you are losing".

So arguing isn't going to do the job with Congress, and it probably won't do the job with business, either. Hey, if they haven't grokked end-to-end by now, how many decades is it going to take?

What we need is implementation. The questions are who, what and how? The how part is easy: The Net is free and open, so the development process needs to involve free software and open source methods.

For who, I see just two populations:

  1. The free & open source developer community; and
  2. Businesses that are motivated to innovate on new Internet services

For what I have two suggestions. One is instant messaging (IM). The other is identity (ID). Neither is in the suite of protocols that define the Net's infrastructure, and all kinds of business opportunities open up once those services are ready to support new kinds of functionalities and new business innovations based on them.

First, IM.

The Net's founding hackers didn't come up with an IM protocol, but Jeremie Miller and the guys at did, starting in '99. Jabber is widely used now. IBM and Earthlink both reportedly use it extensively on an internal basis. So do uncounted other organizations. Meanwhile, the familiar IM systems from Microsoft, AOL and Yahoo all continue to lock their users into closed systems that don't interoperate with each other. Until one or more of them gets into alignment with the Net's stupidity, we won't see ubiquitous IM.

Second, ID.

To guage the importance of ID, consider the matter of your own identities: those representations of yourself in the business world. How many of them are granted to you by outside organizations? How many are mostly beyond your control? Is it not close to 100% in both cases?

What new business possibilities would open if your suite of identity representations were fundamentally your own and under your own control? What if you, as a customer in the business world, were fundamentally autonomous in respect to every business, including all those businesses that relate to you as a name with a number that they assign?

What if your identity operated in the networked world according to protocols that grant your "end" just as much power as the ones that make and distribute the products you buy, and the ones that intermediate your transactions?

What new businesses would be possible if your identity suite had its own open APIs?

What new innovations would be called forth if business conversation about your personal information -- your location, your destinations, your interests, your preferences for anything and everything -- was primarily about what you as a customer want to do with that information, and about the voluntary relationships you want to have with businesses that provide desired products and services to you?

What kinds of existing businesses will suddenly find all kinds of productive new ways to relate to customers once the Net supports a public digital identiy infrastructure that gives all ends on the Net equal powers as peers?

What kinds of businesses would go away once markets turn into countless sets of real conversations between fully empowered customers and the companies that relate to them on a fully permitted basis?

It's easy to imagine countless new businesses -- and a lot less unwanted advertising, PR, promotion, junk mail and other annoying guesswork about what you, "the consumer", might want.

Making this happen is what has been up to in the year since it was founded by Andre Durand, who also co-founded Like Jabber, PingID has a .org counterpart -- -- that handles open source development. (Disclaimer: I am on the advisory boards of both and is working on a peer-to-peer browser or thin (embedded) client-based public digital identity infrastructure. It makes heavy use of XML, SOAP, XML-RPC and other standards that Jabber also trafficks in, which makes for many useful synergies. For example, the Jabber protocol not only handles instant messaging, but also presence. When you show up somewhere, for example, your presence can trigger an IM message in XML that could set all kinds of useful activities in motion.

At the helm of's efforts is Bryan Field-Elliot. I recently asked Bryan how it's going. Here's what he told me:

Again, the choice Bryan presents is about implementation. But motivation to implement gets a lot easier once people start imagining all the cool ways this stuff can be put to use. Here are seven that have come up for Andre Durand, just through his conversations with businesses that have taken an interest in building out ID infrastructure:

  1. Identity Service Providers (basically identity hosting services),
  2. Identity Verification/Authentication Services (Certificate/Signatory Services).
  3. Identity Guardian Services - companies that protect/monitor the use of your identity information.
  4. Identity Interoperability Services - companies that help transport an identity in one system to an identity in another system.
  5. Identity Reputation Companies - companies that help you maintain and monitor your digital reputation to ensure it is clean and accurate.
  6. Identity Networks - this is what PingID Network is building.
  7. A Branded, quality assured Network which facilitates business-to-business and business-to-consumer identity interaction.
  8. Identity Registrars - companies that 'issue identities'.

"In addition to this", Andre says, "I imagine that there will be tons of 'identity infrastructure integrators', identity security monitoring companies, identity theft insurance products, etc."

There are also existing business categories that can make estensive use of a public ID infrastructure: banks, cell phone companies, credit card companies, retailers... and our original two -- carriers and content providers.

With an end-empowering ID infrastructure in place, many new carrier services to customers suddenly become imaginable, because each customer is much more than a name, a number and an address -- a passive "consumer." Their identities are not only far more rich and detailed, but their behavior is far more active and involves far more choice. Rather than threatening carriers, it gives them an API to which they can address far more granular and flexible services, including bandwidth as symmetrical or asymmetrical as the customer requires.

When preference-rich and fully-empowered customers show up in the content marketplace, eager to develop relationships with the artists who make movies and recordings, all the DRM issues that hog conversational airtime today get exposed as desperate attempts by producers and distributors to control "consumer" behavior.

A sufficiently end-empowering digital identity infrastructure will make consumers evolve into customers. When that happens, all arguments about controlling consumer behavior become moot.

And the end-to-end argument finally wins, big time.

Doc Searls is Senior Editor of Linux Journal.