Commodifying Usenet and the Usenet Archive or Continuing the Online Cooperative Usenet Culture?

Ronda Hauben

pdf (68 Kb)

[Originally published in Science Studies, vol. 15, n. 1 (2002), pp. 61-68]

This article explores the conflict between the cooperative online culture of users who have created Usenet and the corporate commodification of Usenet posts by companies archiving the posts. The clash of decision-making processes is presented through the details of how Usenet users choose to petition a company to provide protection for the public archives it had collected. The company disregarded the petition and the archives were sold to another company. The new company has begun to put its own copyright symbol on the posts in its archives. How will such a commodification affect the cooperative nature of Usenet itself and the continuing vitality of Usenet's cooperative culture? The article explores this culture clash and considers possible consequences.

Keywords: commodification, electronic communication, usenet

Commodification of knowledge is a trend in modern societies (Suarez-Villa,2001). A close look at individual cases shows, however, that this process is contentious. The transformation of a public into a private good provokes resistance by those who contributed to the production of that good. If they are now prevented from using it free of charge and from having free access to that good, they may even regard commodification as expropriation. The collaboration that produces a public good in science or technical research is an important process to understand and to protect. Such collaboration has made it possible to create theInternet and Usenet. Researchers creating these important online developments needed the input and contributions from as many people as possible. A recent example from the Usenet world illustrates the tensions and conflicts which result when corporations become involved and begin to commodify a public good.

Usenet is a worldwide distributed online newsgroup and discussion forum. Contributions to it consist of short or long opinions, comments, articles, questions, or answers typed into the system through computers and then distributed from host site to host site until they have traversed all sites that subscribe to the newsgroup to which they are directed. Each such contribution is called a "post". (Hauben & Hauben, 1997) Contributors are sometime called posters. This article examines the corporate archiving of Usenet posts, which then become subject to commodification. These posts are contributed freely by Usenet users. Recently a corporation doing the archiving has put its copyright notice on the posts in this archive. It is unlikely that most contributors have agreed to have their posts archived or to have the copyright of a company appear on the posts.

A Public Good in Corporate Hands

On February 12, 2001, those accessing the archive of Usenet posts collected and archived by the company Deja.com (Deja), learned the archive had been transferred to another company, Google, Inc. (Google). In a press release announcing the acquisition, Google indicated that the archive would be made available to the public in a few months. Google said it "bought" the archive but the price was not indicated. It is likely that Google expected acclaim for acquiring the archive from Deja. The archive had many users and Deja was going bankrupt at the time and either selling or auctioning off its assets.

Among those in the online Internet community, some users welcomed the Google purchase and urged patience to see what would develop. There was also another response. A number of people online were concerned that Google had taken offline the five years of Usenet posts that Deja had collected and substituted a much smaller archive that Google had been collecting on its own. An article appeared in "The Register", a British online publication on February 13, 2001. The article expressed concern that Google had not maintained the Deja interface and the online availability of the archive until they perfected their own interface. Subsequent articles on February 14 and February 15, 2001 included comments by the then chief executive officer (CEO) of Google, Larry Page, promising that some of the archive would be back online in a month and the rest in three months.

There were other concerns expressed both by users online and in the online press during this period. Among these were references to a petition that eventually contained almost 4000 signatures and many comments. The petition had been initiated a few months earlier to appeal to Deja to safeguard the Usenet archive. After collecting Usenet posts from 1995 to 2000 and making them available online, Deja cut back access from five years of posts to only the past year. Included in the petition were several comments describing the archive as a public good that had somehow fallen into private hands. One comment in the petition urged that the, "USENET archive... should *never* have been in private/ corporate hands... give it to an appropriate educational establishment" (comment by Brian McNeil).

To understand the controversy around the corporate archiving and copyrighting of Usenet posts, it is necessary to know something about the origins of Usenet and of archiving Usenet. The collaborative process was crucial for the origins and development of Usenet. A distributed form of archiving was developing as Usenet developed. The open and collaborative process that marked the development of both Usenet and the Google search engine, which was originally developed as a research project, is a process that facilitates the development and implementation of new concepts in technology. Cooperation and collaboration are the processes that generate new knowledge and ways of developing technical processes. The give and take among researchers in the open process where they share knowledge and problems, makes possible ever new developments and improvements.

A proprietary process, is the opposite. It limits the source of input. This tends to narrow the development and change to incremental changes, rather than qualitative leaps. Eventually a proprietary process freezes what is developed for various reasons, amongst which is the need to realize the profit to pay for previous development. When technical pioneers are forging a brand new process or technology, they need the input and support of all who can contribute to the new development. This article will not only explore the collaborative process essential to the development of qualitatively new technologies like Usenet and the Internet, but it will also consider the nature of the efforts to commodify these new developments, such as the archiving of Usenet posts by corporations or the transformation of a publicly funded search engine research project into a private company, like Google.

Unix, Usenet, Internet

Usenet grew up as part of the Unix community. Unix was created in 1969 at Bell Labs, the research arm of the US publicly regulated phone company, AT&T (cf. Holtgrewe & Werle, 2001). Researchers Ken Thompson and Dennis Ritchie, among others at Bell Labs had been part of a broader research project working with Project MAC at the Massachusetts Institute of Technology (MIT). They experienced the close communication that was possible through the new form of programming environment being developed at MIT known as time-sharing. At MIT this was originally the Compatible Time Sharing System (CTSS), and subsequently research was begun to create a more advanced system called MULTICS. AT&T, however, withdrew from the MULTICS collaboration at MIT. Its Bell Labs researchers set out to create their own version of a time-sharing system to be used at AT&T. They called their system Unix (Hauben & Hauben, 1997: 131-134).

Dennis Ritchie, one of the creators of Unix, wrote that Unix was created at Bell Labs by programmers hoping that a "fellowship would form" (Hauben & Hauben, 1997: 51). AT&T (the home of Bell Labs) was a government-regulated corporation subject to the 1956 Consent Decree that restricted it to the telephone business. It was therefore not allowed to commercialise software. The researchers at Bell Labs who created Unix were able to make it available to other researchers and academic institutions for a minimal fee for the tape. There was, however, no technical support from AT&T. Unix users were on their own to solve any problems. From this situation a community grew up to support each other. They formed an association of academic and research users of Unix called Usenix.

By 1979, UUCP (Unix to Unix CoPy Program) was being distributed with the Unix code. UUCP allowed computers using Unix to communicate with each other over telephone lines. From this context Usenet evolved. Usenet was conceived in 1979 by Duke University graduate students Tom Truscott and Jim Ellis. They were active in the Unix community and wanted to contribute a means to create an online Usenix newsletter. In collaboration with others, they developed early versions of the Usenet software and explored its capability. In the January 1980 Usenix meeting, the software was made available to those who were interested.

Usenet was a grassroots network. The users would contribute "posts". Each post would circulate to other users via Usenet software using UUCP. In this way the users created the content and the form of the developing Usenet. It soon spread from the US to Canada, and then to Europe and Australia (Hauben & Hauben, 1997: chapters 2, 3 and 10).

An important aspect of the contributed posts was that they circulated until their expiration date. Each site could set its own date for the expiration of the posts, but they all expired. Consequently, a user would contribute a post and it would be sent out across the globe, but it would expire and disappear from each node on the network on different but set dates.

On Usenet, the posts would be grouped according to particular topics in "newsgroups". A newsgroup like sci.econ was the place where a user would post about an economics topic. News.misc was a newsgroup about Usenet. By the early 1990s, individual Usenet participants archived the posts of some Usenet newsgroups. An index was maintained online which provided the addresses of the sites for the archived newsgroups. A Canadian Usenet pioneer, Henry Spencer, maintained an archive of most Usenet posts through the 1980s. The earliest two or three years of these posts were made available online on certain occasions.

Increasingly, Usenet was being transported via the Internet rather than predominantly via UUCP and phone lines. For a period in the 1980s and into the early 1990s, the U.S. National Science Foundation (NSF) provided support for an NSF backbone for the U.S. portion of the Internet. Traffic on this backbone was required to adhere to the NSF's Acceptable Use Policy (AUP) until 1995 when the NSF backbone was privatised (Hauben & Hauben, 1997: 219-220). There was an AUP because the NSF backbone was initially founded and for many years financed by public funding. The AUP was the means of protecting the public interest in the network. The AUP explained: "(1) NSFNET Backbone services are provided to support open research and education in and among U.S. research and instructional institutions, plus research arms of for-profit firms when engaged in open scholarly communication and research. Use for other purposes is not acceptable." The AUP then explained in more concrete terms how this applied in specific situations. For example, with regard to uses by the international research community, the AUP stated that among the "specifically accepted uses were:... (2) Communication with foreign researchers and educators in connection with research or instruction, as long as any network that the foreign user employs for such communication provides reciprocal access to U.S. researchers and educators."

Because the AUP required that the international research community could use the NSFNET backbone as long as net- works created in their countries provided reciprocal communication access, the protection provided to the U.S. research community for non-commercial use of the NSFNET extended to other countries. The AUP forbade commercial use of these networks except under certain specified circumstances that would serve the research community.

Privatisation and the Clash of Cultures 

With the privatisation and commercialisation of the U.S. portion of the Internet, companies like Deja were created which began to archive Usenet posts. Several users report discontinuing their own archiving when it appeared that these companies were maintaining a large archive. Some users, however, did not want their posts archived. They were concerned about the effect on the continuing development of Usenet from archiving by private companies. The entities that have done archiving like Alta Vista and Deja and now Google are private companies. The decisions about the nature and goals of their archiving activity have been and are under their control. This differs from the practice on early Usenet, when the online community determined the important aspects to be considered when a policy decision was needed (Hauben, 2001). Private corporate decision-making and cooperative online decision-making represent two different cultures. For example, in the early development of Usenet, new software was being created to transport Usenet. Mark Horton and Matt Glickman were creating the new software and Horton considered changing the name of Usenet. He explained his intentions to the online community of Usenet users, asking them for their consideration of his proposal. There was extensive discussion of the reasons that Horton proposed to justify such a change. As a result of the discussion, the decision was that the name Usenet should remain and that Horton's reasons for a change were not adequate. This was an example of how decision-making can be enhanced through an online cooperative process.

Corporate decision-making, on the contrary, is often centralized and focused on short-term goals. It is also often difficult to have divergent opinions expressed in an unprotected corporate environment where one can lose one's position or even job if one speaks in a way that is not appreciated by senior management. Often the views of all involved are not heard or even if they are heard, they can only take a secondary place to the more immediate profit orientation or fiduciary requirements of management. In such a situation, as with Deja and the Usenet archives created from the contributed postings of users, the users have little ability to affect the corporate decision making process. On the surface, it may seem an anomaly to have Usenet users write a petition to a corporation. Petitions are most often thought of as being the right of citizens with regard to their government officials. Usenet users, however, accustomed to be participants, acted to express their views, signing and writing comments in a petition to the company Deja. The request of a number of users to have the archive put into a public repository received no response from Deja, the corporate holder of the archive.

What is the effect on the online community and what are the legal implica- tions of the clash of cultures that results from a private company collecting and then maintaining an archive of public and contributed posts? To gain some grasp of the issues, it is helpful to stress the public origins of the private company Google. Graduate student researchers funded by public research funds under the Digital Libraries Initiatives (DLI) developed the Google search engine. In a paper presented in 1998, Sergey Brin and Lawrence Page, who worked in a DLI initiative at Stanford University in the U.S., describe the harmful effects of the commodification of search engine technology and emphasize the need for public technology research and development. They write (Brin & Page, 1998):

"Up until now most search engine development has gone on at companies with little publication of technical details. This causes search engine technology to remain largely a black art and to be advertising oriented.... With Google we have a strong goal to push more development and understanding into the academic realm."

They continue explaining their strategy to de-commodify such research:

"Another goal we have is to set up a Spacelab-like environment where researchers or even students can propose and do interesting experiments on our large-scale web data."

Their plan was to create a public research database as a laboratory for web search engine research. In their article they acknowledge the public funding in the context of the Stanford Integrated Digital Library Project in which industrial partners are also involved.

The plan of Brin and Page was not implemented. Instead of creating the public web search engine laboratory, those working on the Google search engine were encouraged to create a private company, which would become part of the "black art of proprietary search engine technology" that Brin and Page critiqued. The incentives were set by the funding agency, the NSF, which at the time of the creation of Google, testified to the US Congress that the "transfer to the private sector of 'people' – first supported by the NSF at universities – should be viewed as the ultimate success of technology transfer." (1) For the NSF, Google is the company which provides "an excellent example of knowledge transfer from NSF investment in people." As a consequence, the search engine Google, originally created as part of a public research project, was transformed into the product for a private company. The private company's mission "to organize the world's information, making it universally accessible and useful" was generally welcomed. But how does this corporate goal compare with the goal of Usenet users to communicate?

Towards a Commercial Usenet Culture?

Usenet was created to facilitate communication. There is an unwritten agreement that people who post on Usenet are willing to cooperate in effecting that communication (Hauben & Hauben, 1997: 52). Archiving the posts was not explicitly intended. It was seen by some users as a means of dealing with people's contributions to Usenet in a way that differed from their intentions. This may be tolerated as long as the archive can be accessed by the Usenet community free of charge and without any copyright restricting the use of the archive. In the Google archive the posts are initially individually presented separate from the discussions. There is a provision for viewing the discussion, but that is an option not the default. Some users even welcomed archives because they could help to preserve Usenet's heritage: the cooperative and communicative tradition of the community.

But how long will users tolerate the fact that their contributed posts are copyrighted by a company? Google is moving exactly in this direction. Google is no longer only the private holding company for a public archive but has started to put a © Google 2002 copyright notice after each post in its Usenet archive. Traditionally under the Berne Convention, which the U.S. joined in 1989, users are accorded copyright ownership of their creations, as soon as they are created. Google has not requested that users turn over their copyrights to Google, yet the company is copyrighting the posts. Google's CEO expressed some concern about the copyright of the posts in its archive. Since Google did not ask Usenet users before its decision to put its copyright on users' posts, it did not have a way to take into account users' views. If Google does not create a means for the Usenet community to discuss or to be involved in decisions regarding how the archive is handled who will be responsible for safeguarding the public nature of the archive? (Hauben, 2001)

Any company declaring that it has the right to the ownership of these posts, or to buy or sell a compilation of such posts, presents a serious challenge to Usenet's cooperative culture. Such actions can have a chilling effect on users. Usenet, as a cooperative culture, requires a process with provisions for public discussion and decision making to determine and then safeguard the public interest.

Already the archiving of Usenet and the commercialisation of the Internet has changed Usenet in subtle ways. In the past diverse views were cherished and discussion between those with differences was welcomed. If there was any harassment of those with a minority point of view, other users would speak up in support of the person being abused. More recently, with the archiving of posts, there is less defense being provided for minority or unpopular views on some newsgroups. Consequently, there is less interest in these newsgroups when the range of discussion is narrowed in this way. Traditionally, Usenet provided an environment that welcomed differences. This is the treasure that Usenet has provided to users. If archiving interferes with this environment, it becomes a serious problem for the continued development of Usenet. In the past posters would add their ideas to a discussion, no matter how brief, often saying this was their 2 cents. With the archiving presenting posts as individual works, there is less of an incentive to make a small contribution.

Usenet has been affected by the archiving of its posts. Some users who know about the archiving have chosen to write "x-no-archive: yes" in the first line of the post, with nothing else on the line to prevent them from being made available to others in the archive. Other users, however, do not know about this possibility, nor about the archiving of their posts in general. Usenet itself can be affected in a serious way if the problems that develop with archiving are not treated cooperatively and sensitively. Recently Google has created a place for users to post comments on its web page, but how Google will respond to these comments is not yet known. Various decisions made by Google in the past differ significantly from the way Horton made a proposal to users, and solicited their input before making a decision that would affect them. There are users who stress that Usenet is more important than any archive of Usenet posts and that if the archiving hurts Usenet, it is a serious loss.

In the short term, Google may seem to be providing a valuable service in gathering and making available an extensive Usenet archive. But in the long term – given Google's copyright policy and their method of decision-making – the continued development of Usenet and of the ability of users to communicate is jeopardized. It appears to be essential that public entities provide for the safeguarding of the Usenet archive and of the Usenet decision making process, and that Google learn to understand the importance of responding to the needs of Usenet and the Usenet community in a way that they don't perceive of as in competition with but as complementary to Google. Hopefully, this article will help serve as a catalyst for discussion and research in this vein. (2)

Acknowledgements

The author wishes to thank Raymund Werle for his helpful comments.

References

Brin, S. & Page, L.
1998 "The Anatomy of a Large-Scale Hypertextual Web Search Engine." WWW7 / Computer Networks 30(1-7): 107-117.

Hauben, R.
2001 "Usenet and the Usenet Archives: The Challenge of Building a Collaborative Technical Community", Paper presented at the conference "Innovations for an e-Society." Oct 17-19, Berlin.

Hauben, M & Hauben, R.
1997 Netizens. On the History and Impact of Usenet and the Internet. Los Alamitos, CA: IEEE Computer Society Press.

Holtgrewe, U & Werle, R.
2001 "De-Commodifying Software? Open Source Software Between Business Strategy and Social Movement". Science Studies 14 (2): 43-65.

Suarez-Villa, L.
2001 "The Rise of Technocapitalism". Science Studies 14 (2): 4-20.

Ronda Hauben, Columbia University, USA.