[This was a fast paced conference, and my computer ran out of power, so much of this material was transcribed by hand. As such, this will be a partial paraphrase, partial direct quote affair. -Dana]

O'Reilly: Welcomes everyone to the conference, and introduces the other panel members.

Talks a bit about the way that the current attention being paid to Linux can hide the fact that Open Source is absolutely key to the Internet infrastructure. Programs like Bind (Berkeley Internet Name Daemon, the servers behind the Domain Name System), Apache, and Sendmail, as well as the TCP/IP stacks used in just about everyone's internet software, all came out of the open source community. Most of them out of Berkeley UNIX.

Olson: According to a recent survey, sendmail handles 78.1% of Internet SMTP traffic. Allman and Olson founded Sendmail, Inc in March of 1998. From the start, this new company was committed to supporting Open Source Software.

One goal of Sendmail, Inc, was to solve the biggest problem with sendmail: it is very difficult to configure. The commercial, value added piece that the company sells is added usability and easy of configuration.

Olson noted the tremendous synergy between the Open Source efforts of the Internet and the work of the company.

Beckemeyer: Spoke briefly of EarthLink in general. EarthLink is one of the largest Internet service providers. They service one and a quarter million mail boxes.

Massive growth of the company started in 1994 and 1995. At that time, OSS implementations of various protocols, such as network news and e-mail, were by far the most scalable. It became apparent, though, that even OSS implementations weren't scalable enough for EarthLink.

EarthLink began to make modifications to the free software to add even more scalability. Another modification made to sendmail was anti-spam improvements. This was rolled back into the OSS movement.

[After the initial brief presentations, the panel went to a Q&A/Interactive format, with the three speakers commenting back and forth.]

O'Reilly: This illustrates the key advantage of OSS: 'roll your own.'

Beckemeyer: Getting 'under the hood' was the key to scalability, as well as high availability.

Olson: OSS is the tool that enables innovation.

O'Reilly: There's a downside here, though. Earthlink has extended sendmail, but how much of its' special purpose extensions have gone back into the Open Source community? As large end users build on this software, are they continuing to participate in Open Source development efforts? For example, 70% of all of the software at amazon.com is Open Source. Even a larger percentage of the software at Yahoo is Open Source. Many companies build a lot of proprietary stuff on top of OSS. When does that diverge from the spirit of OSS?

Beckemeyer: This is a business question. Is there more pay-back from the competitive advantage, or is there more pay-back from feeding the improvements back to the community.

He also talked about why some of the extensions were not stuff that other people would necessarily want to use. After all, how many users need over a million sendmail mailboxes? And they did write up the basis for their modifications in a paper that was delivered at Usenix.

Olson: Every generation of sendmail is always Open Source. There is no test lab that can match the Internet. The GUI part is not part of the Open Source distribution. [This is the part that Sendmail, Inc, sells.]

O'Reilly: Would it not be useful to the free software community in general, and Linux in particular, to have a free, powerful and friendly sendmail configuration program?

Olson: As the Open Source Software community changes, so will the company change. [I lost the bulk of this. I believe he implied that such a system would be released Open Source as the demand for it appeared. ]

O'Reilly: Is INN, one of the more popular Open Source network news server implementations, falling behind proprietary offerings?

Beckemeyer: INN development is moving slowly. There isn't much activity. The key is the critical mass; that generates excitement and innovation. One of the risks of Open Source Software is that the community may lose interest in a given project.

O'Reilly: Open source has played a major role in preserving open standards on the Internet. For example, Sendmail has preserved SMTP, as Apache has preserved HTTP. His point is that these Open Source applications have implemented the standards with real rigor, and their massive installation base has locked those standards into place.

Olson: Standards are critical; they work best with a large, sharing community. This has been a key driver over the last 15 years. See the SQL 'standards': a series of competitors creating an almost unworkable 'standard'.

Standards on the Internet really do work through broad implementation.

Beckemeyer: The next paradigm shift that happens on the Internet will not be driven by vendors.

O'Reilly: When IBM first opened up the PC hardware standard, there was an explosion of innovation, with lots of small companies entering the market. Hardware companies like Compaq and Dell. Software companies like Microsoft and Lotus. You no longer had to do everything yourself, so the barriers to entry were lower.

Companies like Dell have realized that they aren't in the hardware business at all, at least the way computer companies used to be. They realized that they were a sales, marketing, and distribution company. RedHat is doing the same thing now with Linux. Bob Young is very explicit about that.

So if you're looking for RedHat to become the next Microsoft, you're looking in the wrong place. The next Microsoft is more likely to be Amazon. (It must be something in the Seattle water.)

Just like Microsoft took an open hardware standard and built proprietary software on top of it, the big web sites are taking open source software and building proprietary e-commerce businesses on top of it.

Question from the audience: Regarding GPL: doesn't it require the re-release of all derivative offerings?

O'Reilly: GPL doesn't apply to Amazon. All of their 'stuff' is internal only. If they were distributing software, they'd have to release it, but they are just using it. My point is that it would be good if they were distributing some of what they are building. But no license would make them do that, even the GPL. And anyway, there are a lot of other licenses that are less restrictive. Sendmail isn't GPL: it is Berkeley. Modifications don't have to be released.

Beckemeyer: Yes. It is important that the company is an active sponsor of Open Source Software. There is a real synergy at work.

O'Reilly: On the brighter side, the closed culture is changing at a lot of big companies. Companies are starting to loosen the restrictions on releasing software Open Source.

Question from the audience: to EarthLink and others: internally, how do you manage software projects?

Beckemeyer: Historically, managers are very uncomfortable with the OSS development style. It's an unknown quantity if the critical mass will appear and stay around long enough to create a viable product. It's very risky.

O'Reilly: There are a lot of myths about Open Source Software. A myth is that Open Source development is faster. I don't think that's true. Like a great wine, they are maturing in the bottle.

Have you ever heard of Sturgeon's law? When someone told science-fiction writer Theodore Sturgeon that 95% of science fiction was crap, he replied, '95 percent of everything is crap.' This has gone down in science fiction circles as Sturgeon's law. Yes, it's true: 95 percent of Open Source Software is crap. It's also true that 95 percent of commercial software is crap. The difference is that in the open source world, no one uses the crap. In the commercial world, they spend marketing dollars to get people to eat it. [General laughter]

Olson: A lot of the more mundane problems, such as making sendmail easy to configure, never got the attention of the Open Source community, and it probably never will.

Question from the audience: Besides free labor, what good qualities of the Open Source Software movement can be injected into internal projects?

O'Reilly: Good design is good design. Linus says that Linux succeeded partially because it has a clean design, from the get go. Microsoft Windows could never have succeeded as Open Source because it's not a clean design. The most successful Open Source projects are basically very clean designs.

Linus makes this point in his essay in our Open Sources book: part of the success of Linux is based on good fundamental design decisions. Larry Wall makes the same point in his essay in the book about Perl--it succeeded because it was well designed for its purpose.

Related points: Fred Baker, chair of IETF: 'One of the things we did really well is that we agreed that we would standardize on the absolute minimum of things we needed to standardize on, and leave the rest to evolution and chance.' A kind of a minimalism is a guiding principle. That's part of the reason it works: programs have known inputs and outputs. It being UNIX. That's part of the reason UNIX has been been such a great basis for open source projects.

Another piece is that OSS is not a magic bullet. Most of these projects didn't start out with someone trying to write software for other people. They wrote the software to solve their own problem. Eric Allman created sendmail in the early days of the Arpanet because it was easier for him to relay people's mail than to give 700 other people at UC Berkeley login accounts on his machine.

Larry Wall had a problem. He wrote simple code to solve his problem. The same is true of Perl. Larry Wall wrote it to solve some problems he was struggling with. Then he gave it away because he thought other people might have the same class of problems.

I like to say that your "return on investment" is the solution to your own problem. What you get back from other people is just an extra dividend. EarthLink is a case in point: they've done a lot of custom design to solve their own problems. Open Source may flourish the best in arenas where there isn't a lot of competitive advantage to be had.

Olson: The strongest aspect of the Open Source paradigm is the huge and broad peer review.

Question from the audience: It sounds like we're equating the Open Source movement with free labor. Isn't Bind a project where most of the labor is not free?

O'Reilly: Free labor is one of the big curses of the movement. Richard Stallman may say "Think Free Speech, not free beer" but an awful lot of people do like the beer. And that confuses people. Free (gratis as opposed to libre, as Linus likes to say), is a good thing, but it's not core to the movement.

Now, a lot of the free software developers are paid to develop free software. Most of the members of the core Apache group had a vested, professional interest to work on Apache. They needed it for their businesses. The free labor is incidental.

Beckemeyer: EarthLink has a lot of people on staff that are paid to develop Open Source.

Olson: My company also pays some people to actively work in the Open Source process. The key point is the economy of participation is not for money. As Eric Raymond describes it, it's a gift economy. You don't charge for the various parts of the software development cycle.

Beckemeyer: We do it because we get something back.

O'Reilly: I do a lot of Open Source activism because it's good for my business. But there's a slippery slope here. Malcolm Beattie was working to release Perl 5.005. It was taking a lot of time to get it out. We gave him a sabbatical to get it finished. Then a lot of other people started asking for the same thing.

Beckemeyer: It's hard for a CFO [Chief Financial Officer] to find cause and effect in these development environments. It's critical to convince the senior management that there is benefit to this system.

O'Reilly: Microsoft has made billions on Open Source Software, since the Internet is Open Source, yet they haven't given anything back. It's important to give something back.

Cisco is a good example: they don't use Open Source Software, but they're a contributing member of IETF. They're close to Microsoft in aggressiveness, but they know how much benefit they've received from open standards so they are good stewards of the community. O'Reilly likened Microsoft's activities to clear cutting forests.

Question from the audience: [very very long, hard to understand question.]

O'Reilly: The Open Source community is very much focused on winning 'the last war'.

As I was saying earlier, I do worry that we might beat Windows on the desktop, only to find that we should have been building open source ecommerce software for the web, because that's where the key battleground is now.

EarthLink: Yes, there is something of a lag on commercial offerings.

Question from the audience: How do we get IBM and others to contribute?

O'Reilly: One wing of the free software movement is the FSF, headed by Stallman. Non-free software is immoral. That group will probably never get along with the business world. The pragmatic wing, Open Source Software, is winning out. IBM and HP are on the Open Source track.

Question from the audience: Regarding Open Source business models. How does a manager go about implementing a strategy?

Olson: It is difficult to take something with a pure commercial identity into an Open Source identity. It's ten times harder to change once the name has been established. There are a large number of commercial models for Open Source software.

O'Reilly: Look at the history of the PC industry. There is a spectrum of open and proprietary. In the past, the two ends have danced.

RedHat is interesting because they understand the model. They are assembling a set of components into a commodity. They claim to not be a software company, but a packaging and distribution company. I see business models all over. The key is to get out of the box.

Question from the audience: Is Open Source Software anti-competitive? Isn't it really 'dumping'?

O'Reilly: I think that is far fetched.

Beckemeyer: It's a valid question. It's never been done.

O'Reilly: It can't really be anti-competitive, since all of the competition has access to the source.

Olson: Open Source Software is not about being free of charge: it's about free distribution. It's about liberty.