From: hauben@news.cs.columbia.edu (Michael Hauben) Newsgroups:alt.folklore.computers Subject: APRANET History of Network Working Group and NCP. Part 1/4 Date: 23 Jan 1994 17:15:54 -0500 Comments and Criticisms Welcome. This paper will be in 3 or 4 parts depending on if I decide to post parts of the appendices in a 4th part. Behind the Net: The untold history of the ARPANET By Michael Hauben hauben@columbia.edu The global Internet's progenitor was the Advanced Research Projects Agency Network (ARPANET) of the U.S. Department of Defense. This is important to remember, because the support and style of management by ARPA to its contractors was crucial to the success of the ARPANET. As the Internet develops and the struggle over the role it plays unfolds, it will be important to remember how the network developed and the culture with which it was connected. As a facilitator of communication, the culture of the Net is an important feature to acknowledge. The ARPANET Completion Report, as published jointly by Bolt, Beranek and Newman (BBN) of Cambridge, Mass., and ARPA concludes by stating: ...it is somewhat fitting to end on the note that the ARPANET program has had a strong and direct feedback into the support and strength of computer science, from which the network itself sprung. (Chapter III, pg.132, Section 2.3.4) In order to understand the wonder that the Internet, and various parts of the Net, represent, we need to understand why the ARPANET Completion report ends with the suggestion that the ARPANET is fundamentally connected to and born of computer science, rather than of the military. PART I: The history of ARPA leading up to the ARPANET A climate of pure research surrounded the entire history of the ARPANET. The Advanced Research Projects Agency was formed to fund research, and thus was not oriented to a military product. The formation of this agency was part of the U.S. reaction to the then Soviet Union's launch of Sputnik in 1957. (ARPA draft, III-6). ARPA was assigned to research how to utilize the mili- tary's investment in computers via Command and Control Research (CCR). Dr. J.C.R. Licklider was chosen to head this effort. Licklider came to ARPA from Bolt, Beranek and Newman, (BBN) in Cambridge, MA in October 1962. (ARPA draft, III-6) He came to ARPA from a background of combining engineering studies and physiological psychology. This provided Licklider with an unusual prospective uncommon among engineers. From Licklider's arrival, the department's contracts were shifted from independent corporations towards "the best academic computer centers" (ARPA draft, III-7). The then current method of computing was via batch processing (i.e., input via stacks of punched cards, and output: the results, or lack of them, made known one or more days later.). Licklider saw improvements could be made in CCR only from work on advancing the current state of computing technology. He particularly wanted to move forward into the age of interactive computing, and the current contractors were not moving in that direction. In an Interview, Licklider told the interviewee that SDC (Systems Development Corporation) "was based on batch processing, and while I was interested in a new way of doing things, they [SDC] were studying how to make improvements in the ways things were done already." (An Interview with J.C.R. Licklider conducted by William Aspray and Arthur Norberg on October 28, 1988 Cambridge, Mass. CBI Univ of Minn., Madison) The office "developed into a far-reaching basic research program in advanced technology." (ARPA draft III-7) Licklider's Office was renamed Information Processing Techniques Office (IPT or IPTO) to reflect that change. The Completion report states that "Prophetically, Licklider nicknamed the group of computer specialists he gathered the 'Intergalactic Network'." (ARPA draft, III-7) Before work on the ARPANET began, the very idea of the network was planted by the creation of the Information Processing Techniques Office of ARPA. Robert Taylor, Licklider's successor at the IPTO, remembers why this was true because of Lick's interest in interconnecting communities: Lick was among the first to perceive the spirit of community created among the users of the first time-sharing systems... In pointing out the community phenomena created, in part, by the sharing of resources in one timesharing system, Lick made it easy to think about interconnecting the communities, the interconnection of interactive, on-line communities of people, ..." (ARPA draft, III-21) The "spirit of community" was related to Lick's interest in having computers help people communicate with other people (Licklider and Robert Taylor, "The Computer as a Communication Device") Licklider's vision of an "intergalactic network" con- necting people represented an important conceptional shift in computer science. This vision was also an important beginning to the ARPANET. After the ARPANET was up and running, the computer scientists using it realized that assisting human communication was a major fundamental advance that the ARPANET made possible. As early as 1963, a commonly asked question of the IPTO directors by the ARPA directors about IPTO projects was "Why don't we rely on the computer industry to do that?", or occasion- ally more strongly, "We should not support that effort because ABC (read, "computer industry") will do it - if it's worth doing!" (ARPA draft, III-23) This question leads to an important point - this ARPA research was different from what the computer industry had in mind to do - or was likely to undertake. Since Licklider's creation of the IPTO, the work supported by ARPA/IPTO continued his explicit emphasis on communications. The Completion Report explains, The ARPA theme is that the promise offered by the computer as a communication medium between people, dwarfs into rela- tive insignificance the historical beginnings of the comput- er as an arithmetic engine." (ARPA draft, III-24) The Completion Report goes on to differentiate ARPA from the computer industry: The computer industry, in the main, still thinks of the computer as an arithmetic engine. Their heritage is reflect- ed even in current designs of their communication systems. They have an economic and psychological commitment to the arithmetic engine model, and it can die only slowly..." (ARPA draft, III-24) The Completion Report further analyzes this problem by tracing it back to the nation's universities: ...furthermore, it is a view that is still reinforced by most of the nation's computer science programs. Even univer- sities, or at least parts of them, are held in the grasp of the arithmetic engine concept.... (ARPA draft, III-24) ARPA's IPTO was responsible for the research and development which led to the success of first the ARPANET, and later the Internet. Without the commitment that existed via this support, such a development might never have happened. One of ARPA's criterion for supporting research was that the research had to be of such a level as to offer an order of magnitude of advance over the current state of development. As most research and develop- ment is not immediately profitable, there is a need for organiza- tions which do not pursue profit as their goal, but rather work on furthering the state of the art. What is very telling is that computer networking spread widely without profit being involved. Others have understood the communications promise of comput- ers. For example, in RFC 1336, David Clark, senior research scientist at MIT's laboratory for computer science, is quoted, It is not proper to think of networks as connecting comput- ers. Rather, they connect people using computers to mediate. The great success of the internet is not technical, but in human impact. Electronic mail may not be a wonderful advance in Computer Science, but it is a whole new way for people to communicate. The continued growth of the Internet is a technical challenge to all of us, but we must never loose sight of where we came from, the great change we have worked on the larger computer community, and the great potential we have for future change. Various research predating the ARPANET had been done by Paul Baron, Thomas Marill and others. [End note 1] This led Lawrence Roberts and other IPTO staff to formally introduce the topic of networking computers of differing types (i.e.: incompatible hardware and software) together in order to share resources to the early 1967 meeting of ARPA's Principle Investigators (PI). In the spring of 1967 at the University of Michigan, ARPA held its yearly meeting of the Principle Investigators from each of its university and other contractors. (ARPA draft, III-25) Results from the previous year's research was summarized and future research was discussed, either introduced by ARPA or the various researchers present at the meeting. Networking was one of the topics brought up at this meeting. (ARPA draft, III-25) At that meeting, it was decided that there had to be agreement on conventions for character and block transmission, error checking and retransmission, and computer and user identification. These specifications became the contents of the inter-host communication's "protocol." Frank Westervelt was chosen to write about this protocol and a communication group was formed to study the questions. (ARPA draft, III-26) In order to develop a network of varied computers, two main problems had to be solved: 1. To construct a 'subnetwork' consisting of telephone circuits and switching nodes whose reliability, delay char- acteristics, capacity, and cost would facilitate resource sharing among computers on the network. 2. To understand, design, and implement the protocols and procedures within the operating systems of each connected computer, in order to allow the use of the new subnetwork by the computers in sharing resources. (ARPA, II-8) After one draft and additional work on this communications position paper was completed, a meeting was scheduled in early October 1967 by ARPA at which the protocol paper and specifica- tions for the Interface Message Processor (IMP) were discussed. A subnetwork of IMPs, dedicated mini-computers connected to each of the participant computers, was the method chosen to connect the participants's computers (hosts) to each other via phone lines. This standardized the subnet to which the hosts connected. Now, only the connection of the hosts to the network would depend on vendor type, etc. ARPA had picked 19 possible participants in what was now known as the "ARPA Network." From the time of the 1967 PI Meeting, various computer scientists who were ARPA contractors were busy thinking about various aspects which would be relevant to the planning and development of the ARPANET. Part of that work was a document outlining a beginning design for the IMP subnetwork. This speci- fication led to a competitive procurement for the design of the IMP subnetwork. By late 1967 ARPA had given a contract to the Stanford Research Institute (SRI) to write the specifications for the communications network they were developing. In December of 1968, SRI issued a report "A Study of Computer Network Design Parame- ters." Elmer Shapiro played an important role in the research for this report. Based on this work, Roberts and Barry Wessler of ARPA wrote the final ARPA version of the IMP specification. (ARPA draft, III-32) This specification was ready to be discussed at the June 1968 PI meeting. The Program Plan "Resource Sharing Computer Networks" was submitted June 3, 1968 by the IPTO to the ARPA Director, who approved it on June 21, 1968. It outlined the objectives of the research, and the plan of how the objectives would be fulfilled. The purposed network was impressive as it would prove useful to both the computing research centers which connected to the network and the military. The proposed requirements for the research would provide immediate benefits to the computer centers the network would connect. (ARPA draft, III-35) ARPA's stated objectives were to experiment with varied interconnections of computers and sharing resources in an attempt to improve produc- tivity of computer research. (ARPA, II-2) Justification was drawn from technical needs in both the scientific and military environ- ments. The Program Plan developed into a set of specifications. These specifications were connected to a competitive Request for Quotation (RFQ) to find an organization which would design and build the IMP subnetwork. Following the approval of the Program Plan, 140 potential bidders were mailed the Request for Quotation. After a bidders conference, 12 proposals were received and from them ARPA nar- rowed the bidders down to four. BBN was the eventual recipient of the contract. (ARPA draft, III-35) The second technical problem, as defined by the ad hoc Communications Group, still remained to be solved. The set of agreed upon communications settings (known as a protocol), which would allow the hosts to communicate with each other over the subnetwork, had to be developed. This work was left "for host sites to work out among themselves." (ARPA draft, III-67) This meant that the software necessary to connect the hosts to the IMP subnetwork had to be developed. ARPA assigned this duty to the initially designated ARPANET sites. Each of the first sites had a different type of computer to connect. ARPA trusted/knew the programmers at each site would be capable of modifying their operating systems in order to connect their systems to the subnetwork. In addition the sites needed to develop the software necessary to utilize the other hosts on the network. (ARPA draft, III-39) ARPA's assigning of responsibilities made the academic computer science community an active part of the ARPANET develop- ment team. (Interview with Alex McKenize, Nov, 1 1993) Steve Crocker, one of graduate students involved with the development of the earliest ARPANET protocols, associates the placement of the initial ARPANET sites at research institutions to the fact that the ARPANET was ground-breaking research. He wrote in a message responding to my questions on the COM-PRIV mailing list: During the initial development of the Arpanet, there was simply a limit as to how far ahead anyone could see and manage. The IMPs were placed in cooperative ARPA R&D sites with the hope that these research sites would figure out how to exploit this new communication medium. (Crocker, 1993A) The first sites of the ARPANET were picked to provide either network support services or unique resources. The key services the first four sites provided were UCLA - Network Measurement Center SRI - Network Information Center UCSB - Culler-Fried interactive mathematics UTAH - graphics (hidden line removal) (Cerf, Vinton 1993) Steve Crocker also recounts that the reason for selecting these particular four sites was because they were "existing ARPA computer science research contractors." This was important because "the research community could be counted on to take some initiative." (RFC 1000, pg 1) The very first site to receive an IMP was UCLA. Professor Leonard Kleinrock of UCLA was involved with much of the early development of the ARPANET. His work in queuing theory gave him a basis to develop measurement techniques used to monitor the ARPANET's performance. This made it natural to make sure that UCLA received one of the first nodes as it would be important to measure the network's activity from early on. In order for the statistics to have correct data and analysis purposes - one of the first two or three sites had to be the measurement site. Sure enough UCLA was assigned to be the Network Measurement Center (NMC). [END NOTE 2] Part II. The Network Working Group Once the initial sites were picked, representatives from each site gathered together to start talking about solving the technical problem of getting the hosts to communicate with each other. The ARPA Completion report tells us about this beginning: To provide the hosts with a little impetus to work on the host-to-host problems. ARPA assigned Elmer Shapiro of SRI "to make something happen", a typically vague ARPA assign- ment. Shapiro called a meeting in the summer of 1968 which was attended by programmers from several of the first hosts to be connected to the network. Individuals who were present have said that it was clear from the meeting at that time, no one had even any clear notions of what the fundamental host-to-host issues might be. (AC Draft III-67 1.4.1.7) We see that this group, which came to be known as the Network Working Group (NWG), was exploring new territory. The first meeting took place several months before the first IMP was put together and they had to think from a blank slate. Throughout the existing recollections of the important developments the NWG produced, (especially RFC 1000) the reader is reminded that the thinking involved was groundbreaking and thus exciting. Steve Crocker remembers in the RFC Reference Guide (RFC 1000) that the first meeting was chaired by Elmer Shapiro, who initiated the conversation with a list of questions. (Crocker, 1993b) Also present were Steve Carr from University of Utah, Stephen Crocker from UCLA, Jeff Rulifson from SRI, and Ron Stoughton from UCSB. These attendees are the programmers referred to in the ARPANET Completion Report. According to Steve Crocker, this was a seminal meeting. The attendees could only be but theoretical, as none of the lowest levels of communication had been developed yet. They needed a transport layer or low-level communications platform to be able to build upon. BBN would not deliver the first IMP until August 30, 1969. It was important to meet before this date, as the NWG "imagined all sorts of possibilities." (Rfc1000) Only once their thought processes started could this working group actually develop anything. These fresh thoughts from fresh minds helped to incubate new ideas. The ARPANET Completion Report properly acknowledges what this early group helped accomplished: "Their early thinking was at a very high level." (ARPA draft, III-67) A concrete decision of the first meeting was to continue holding meetings similar to the first one. This wound up setting the precedent of holding exchange meetings at each of the sites. Steve Crocker, describing the problems facing these network- ing pioneers, writes: With no specific service definition in place for what the IMPs were providing to the hosts, there wasn't any clear idea of what work the hosts had to do. Only later did we articulate the notion of building a layered set of protocols with general transport services on the bottom and multiple application-specific protocols on the top. More precisely, we understood quite early that we wanted quite a bit of generality, but we didn't have a clear idea how to achieve it. We struggled between a grand design and getting some- thing working quickly. (Crocker,1993c) The initial protocol development lead to DEL (Decode- Encode-Language) and NIL (Network Interchange Language). These languages were more advanced than what was needed or possible at the time. The basic purpose was to form an on-the-fly description that would tell the receiving end how to understand the informa- tion that would be sent. These first set of meetings were ex- tremely abstract as neither ARPA nor the universities had deemed any official charter. However, the lack of a charter allowed the group to think broadly and openly. BBN did submit details about the host-IMP interface specifi- cations from the IMP side. This information provided the group some definite starting points to build from. Soon after BBN provided more information, on Valentine's Day, 1969, members of the NWG, members of BBN and members of the Network Analysis Corporation (NAC) met for the first time. [Endnote 3] As all the parties had different priorities on mind, the meeting was a difficult one. BBN was interested in the lowest level of making a reliable connection. The programmers from the host sites were interested in getting the hosts to communicate with each either via various higher level programs. And BBN also did not turn out to be the "experts from the East" that Steve Crocker wrote the members of the NWG expected. He continues by writing in RFC 1000 that they constantly thought that "a professional crew would show up eventually to take over the problems we were dealing with." A step of incredible importance and openness occurred as a result from a "particularly delightful" meeting that took place a month later in Utah. (RFC1000) The participants decided it was time to start recording their meetings in a consistent fashion. What resulted was a set of informal notes titled "Request for Comments." Steve Crocker writes about their formation: I remember having great fear that we would offend whomever the official protocol designers were, and I spent a sleep- less night composing humble words for our notes. The basic ground rules were that anyone could say anything and that nothing was official. And to emphasize the point, I labeled the notes "Request for Comments." I never dreamed these notes would distributed through the very medium we were discussing in these notes. Talk about Sorcerer's Apprentice! (Crocker, RFC 1000, pg 3, 1987) Crocker replaced Shapiro as the Chairman of the NWG soon after the initial meeting. He describes how they wrestled with the creation of the host-host protocols: Over the spring and summer of 1969 we grappled with the detailed problems of protocol design. Although we had a vision of the vast potential for intercomputer communica- tion, designing usable protocols was another matter. A custom hardware interface and custom intrusion into the operating system was going to be required for anything we designed, and we anticipated serious difficulty at each of the sites. We looked for existing abstractions to use. It would have been convenient if we could have made the network simply look like a tape drive to each host, but we knew that wouldn't do. (Crocker, RFC 1000, pg. 3) The first IMP was delivered to UCLA in late August, 1969. The next was delivered to SRI a month later in October. [Endnote 4] Once more than one IMP existed, the NWG had to implement a working communications protocol. This first set of pairwise host protocols included remote login for interactive use (telnet), and a way to copy files between remote hosts (FTP). Crocker writes: In particular, only asymmetric, user-server relationships were supported. In December 1969, we met with Larry Roberts in Utah, [and he] made it abundantly clear that our first step was not big enough, and we went back to the drawing board. Over the next few months we designed a symmetric host-host protocol, and we defined an abstract implementa- tion of the protocol known as the Network Control Program. ("NCP" later came to be used as the name for the protocol, but it originally meant the program within the operating system that managed connections. The protocol itself was known blandly only as the host-host protocol.) Along with the basic host-host protocol, we also envisioned a hierarchy of protocols, with Telnet, FTP and some splinter protocols as the first examples. If we had only consulted the ancient mystics, we would have seen immediately that seven layers were required. (RFC 1000, pg 4) After Robert's guidance, the Network Working Group went forward in developing the protocols necessary to make the network viable. The group swelled in attendance as more and more sites connected to the ARPANET. The group became large enough (around 100 people) that one meeting was held in conjunction with the 1971 Spring Joint Computer Conference in Atlantic City. A major test of the NWG's work came in October 1971, when a meeting was held at MIT. Crocker continues the story, [A] major protocol "fly-off" - Representatives from each site were on hand, and everyone tried to log in to everyone else's site. With the exception of one site that was com- pletely down, the matrix was almost completely filled in, and we had reached a major milestone in connectivity. (Crocker, RFC 1000, pg. 4) The NCP was created as what was called the "host to host protocol." Explaining why this was important, the authors of the ARPA draft write: The problem is to design a host protocol which is suffi- ciently powerful for the kinds of communication that will occur and yet can be implemented in all of the various different host computer systems. The initial approach taken involved an entity called a "Network Control Program" which would typically reside in the executive of a host, such that processes within a host would communicate with the network through this Network Control Program. The primary function of the NCP is to establish connections, break connections, switch connections, and control flow. A layered approach was taken such that more complex procedures (such as File Trans- fer Procedures) were built on top of similar procedures in the host Network Control Program. (Arpa draft, II-24) As the ARPANET grew, the number of users bypassed the number of developers. This signaled the success of these networking pioneers. Steve Crocker appointed Alex McKenize and Jon Postel to replace him as Chairmen of the Network Working Group. The Comple- tion Report details how this role changed: McKenzie and Postel interpreted their task to be one of codification and coordination primarily, and after a few more spurts of activity the protocol definition process settled for the most part into a status of a maintenance effort.(ARPA draft,III-69) ARPA was a management body which funded academic computer scientists. ARPA's funding paved the way for these scientists to create the ARPANET. BBN helped via developing the packet switch- ing techniques which served as the bottom level of transmitting information between sites. The NWG provided an important develop- ment in its "Request for Comments" documentation which made possible the developing the new protocols. PART III. About RFC's as "Open" Documentation The openness initiated from the very first meeting of the Network Working Group continued on in a more informal formalized manner in the Request For Comments. As meeting notes, the RFCs were meant to keep members updated on the status of various developments and ideas by the development community. They were also meant to gather responses from people. The Documentation Conventions RFC (RFC 3) documents the "rules" governing the production of these notes. Heading the page were the open distri- bution rules: Documentation of the NWG's effort is through notes such as this. Notes may be produced at any site by anybody and included in this series. These opening sentences invite anyone willing to be helpful in the protocol definition process. This is important because all restrictions are denied by these words, allowing for the best possible developments. The guide goes on to describe the rules concerning the contents of the RFCs: The content of a NWG note may be any thought, suggestion, etc. related to the HOST software or other aspect of the network. Notes are encouraged to be timely rather than polished. Philosophical positions without examples or other specifics, specific suggestions or implementation techniques without introductory or background explication, and explicit questions without any attempted answers are all acceptable. The minimum length for a NWG note is one sentence. The RFC continues to explain the philosophy behind the perhaps unprecedented amount of openness represented: These standards (or lack of them) are stated explicitly for two reasons. First, there is a tendency to view a written statement as ipso facto authoritative, and we hope to pro- mote the exchange and discussion of considerably less than authoritative ideas. Second, there is a natural hesitancy to publish something unpolished, and we hope to ease this inhibition." (Crocker, RFC 3 - 1969) [The entire RFC is reproduced in Appendix B.] This openness led to the exchange of information. Technical development is only successful when information is allowed to flow freely and easily between the parties involved. These open principles are what made the development of the Net possible. Statements like the ones contained in RFC 3 are very pro- gressive in their openness. The late 1960's was a time alive in popular protest for freedom of speech and people demanding more of a say in how their country was run. The openness applied in trying to develop new technologies fits well with the cry for more democracy which students demanded throughout the country and the world. What is amazing is that the collaboration of the NWG (mostly graduate students) and ARPA (a component of the mili- tary), seems to be contrary to the normal atmosphere of the times. Robert Braden of the Internet Activities Board reflects on this collaboration: For me, participation in the development of the ARPAnet and the Internet protocols has been very exciting. One important reason it worked, I believe, is that there were a lot of very bright people all working more or less in the same direction, led by some very wise people in the funding agency. The result was to create a community of network researchers who believed strongly that collaboration is more powerful than competition among researchers. I don't think any other model would have gotten us where we are today. (RFC 1336) These ideas point to a reason why the work of these computer scientists founded what has led to be one of the most amazing and democratic bodies (i.e.: The Net and the culture attached to it) to emerge in a long time. The community that has developed and the tools which accompany it form an important democratic force. [See endnote 5.] The idea of calling these notes a "Request for Comment" set a fascinating tradition. It predates the Usenet Post, which in a fashion could be called a "request for comment" as it is the presentation of a particular person's ideas, questions or com- ments, to the general public (of those who read that newsgroup) for comments, criticism or suggestion, or just plain to further the readers' knowledge. Other Early RFCs echo this reality. There are plenty of RFCs which are in response to a previous RFC. Following are some examples, more are contained in the appendix. 1 Crocker, S. Host software 1969 April 7 65 Walden, D. Comments on Host/Host Protocol document #1 36 Crocker, S. Protocol notes 1970 March 16 38 Wolfe, S. Comments on network protocol from NWG/RFC #36 39 Harslem, E.; Heafner, J. Comments on protocol re: NWG/RFC#36 33 Crocker, S. New Host-Host Protocol 1970 February 12 47 Crowther, W. BBN's comments on NWG/RFC #33 1970 April 20 Part IV: Conclusion How were the developments of the ARPANET made possible? This question appears from the very problems that the various contri- butors to the ARPA project faced themselves. None of the partici- pants had the solutions to any of the tasks they approached before putting much thought and work into their research. As the resulting ARPANET was tremendously successful and fulfilled the project ARPA presented, it is important to see what can be learned from the research out of which it emerged. Bernie Cosell, who worked at BBN during this early period, describes the impor- tance of openness in a developmental situation: *no*one* had the necessary expertise [and vision] to figure any of this out on their own. The cultures among the early groups were VERY different multics, sigma-7, IBM ... at Rand, ... PDP-10s at BBN and SRI... [and possibly] UCSB and Utah had pdp-10's, too. The pie-in-the-sky applications ranged over a WIDE landscape, with no one knowing quite where it would lead. Some kind of free, cross-cultural info/idea exchange *had* to happen. (Cosell 1993) The computer scientists and others involved were encouraged in their work by ARPA's philosophy of gathering the best computer scientists working in the field and supporting them: IPT usually does little day-to-day management of its con- tractors. Especially with its research contracts, IPT would not be producing faster results with such management as research must progress at its own pace. IPT has generally adopted a mode of management which entails finding highly motivated, highly skilled contractors, giving them a task, and allowing them to proceed by themselves. (ARPA draft, III-47) The work of the Network Working Group was vital to the development of the ARPANET. Vint Cerf, another of the graduate students involved with the early protocol development and still closely connected to the Internet, echoed this sentiment when he opened his paper "An Assessment of ARPANET Protocols," by writ- ing: The history of the Advanced Research Project Agency resource sharing computer network (ARPANET) is in many ways a history of the study, development, and implementation of protocols." (Cerf, _An Assessment of ARPANET Protocols_) Cerf supports Cosell's opinion about the uncertainty and newness of the entire project when he continues in his paper by writing: The tasks facing the ARPANET design teams were often un- clear, and frequently required agreements which had never been contemplated before (e.g., common protocols to permit different operating systems and hardware to communicate). The success of the effort, seen in retrospect, is astonish- ing, and much credit is due to those who were willing to commit themselves to the job of putting the ARPANET togeth- er. (Cerf, IBID.) The NWG's work blazed the trail which the developers of the TCP/IP suite of protocols (Transport Control Protocol/ Internet Protocol) followed to success when the need to expand and include other networks based on other technologies than NCP arose. The principles embodied by RFC 3 and open RFC documentation provided a strong foundation which began with NCP and was continued by the work on TCP/IP. NCP was developed in the field and versions of it were released early in its development so various programmers could work on implementing and improving the protocol. In addi- tion all specifications were available for free and easily available for people to examine and comment on. Through this principle of early release the problems and kinks were found and worked out in a timely manner. The future developers of TCP/IP learned from the developers of NCP a practice of developing from the bottom up. The bottom-up model allows for a wide-range of people and experiences to join in and perfect the protocol and make it the best possible. The public funding of the ARPANET project allowed its documentation to be open and available. This documentation was neither restricted nor classified. The possibility of communi- cation represented by openness was necessary for these pioneers. Research of new fields of study require that researchers cooper- ate and communicate in order to share their expertise with the larger body of people conducting research. This openness is especially critical when no one person has the answers in ad- vance. Larry Roberts of ARPA explained in an article: "Since the ARPANET was a public project connecting many major universities and research institutions, the implemen- tation and performance details were widely published." ("The Evolution of Packet Switching", 267) The people at the forefront of development of these proto- cols were the members of the Network Working Group, many of whom came from academic institutions, and who therefore had the support and time needed for the research. In summing up the achievements of the process that developed the ARPANET, the ARPANET Completion Report draft explains: The ARPANET development was an extremely intense activity in which contributions were made by many of the best computer scientists in the United States. Thus, almost all of the "major technical problems" already mentioned received con- tinuing attention and the detailed approach to those prob- lems changed several times during the early years of the ARPANET effort. [II-24] Fundamental to the ARPANET, as explained by the Completion Report, was the discovery of a new way of looking at computers. The developers of the ARPANET viewed the computer as a communica- tions device rather than only as an arithmetic device. (draft, III-24) This new view made the building of the ARPANET possible. This view came from the research conducted by those in academic computer science. The shift in the understanding of the role of the computer is fundamental to advancing computer science. The ARPANET research has provided a rich legacy for the further advancement of computer science and it is important that the significant lessons learned be studied and used to further advance the study of computer science. END - NOTES 1. This history is covered well in the article "From ARPANET to USENET" by Ronda Hauben. Also in Chapter III, section 1.1.2 starting on page III-9 in the published ARPANET Completion Report. 2. These quotes show some of the perspective chosen to pick the initial ARPANET sites. III - 689 "CCN's [The Campus Computing Network of UCLA] chance to obtain a connection to the ARPANET was a result of the presence at UCLA of Professor L. Kleinrock and his students, including S. Crocker, J. Postel, and V. Cerf. This group was not only involved in the original design of the network and the Host protocols, but also was to operate the Network Measurement Center (NMC). For these reasons the first delivered IMP was installed at UCLA, and ARPA was thus able to easily offer CCN the opportunity for connection." pg II-16 " In a somewhat less structured way, the research groups receiving ARPA IPTO support were then encouraged to begin consid- ering the design and implementation of protocols and procedures and, in turn, computer program modifications, in the various host computers in order to use the subnetwork. Several specific responsibilities were arranged: UCLA was specifically asked to take on the task of a "Network Measurement Center" with the objective of studying the performance of the network as it was built, grown, and modified; SRI was specifically asked to take on the task of a "Network Information Center" with the objective of collecting information about the network, about host resources, and at the same time generating computer based tools for storing and accessing that collected information. Beyond these two specific contracts, some rather ad hoc mechanisms were pursued to reach agreement between the various research contractors about the appropriate "host protocols" for intercommunicating over the subnetwork. The "Network Working Group" of interested individuals from the various host sites was rather informally encouraged by ARPA. After a time, this Network Working Group became the forum for, and eventually a semi-official approval authority for, the discussion of and " III - 60 1.4.1.5 The Network Information Center The accessibility of distributed resources carries with it the need for an information service (either centralized or distribut- ed) that enables users to learn about those resources. This was recognized at the PI [ed. Primary Instigators] meeting in Michi- gan in the spring of 1967. At the time, Doug Engelbart and his group at the Stanford Research Institute were already involved in research and development to provide a computer-based facility to augment human interaction. Thus, it was decided that Stanford Research Institute would be a suitable place for a "Network Information Center" (NIC) to be established for the ARPANET. With the beginning of implementation of the network in 1969, construc- tion also began on the NIC at SRI." 3. The NAC was contracted by ARPA to "specify the topological design of the ARPANET and to analyze its cost, performance, and reliability characteristics. (ARPA, III-30) 4. RFC 1000 reports on the process of the installation of the first IMP. "[T]ime was pressing: The first IMP was due to be delivered to UCLA September 1, 1969, and the rest were scheduled at monthly intervals. At UCLA we scrambled to build a host-IMP interface. SDS, the builder of the Sigma 7, wanted many months and many dollars to do the job. Mike Wingfield, another grad student at UCLA, stepped in and offered to get interface built in six weeks for a few thousand dollars. He had a gorgeous, fully instrumented interface working in five and one half weeks. I was in charge of the software, and we were naturally running a bit late. September 1 was Labor Day, so I knew I had a couple of extra days to debug the software. Moreover, I had heard BBN was having some timing troubles with the software, so I had some hope they'd miss the ship date. And I figured that first some Honeywell people would install the hardware -- IMPs were built out of Honeywell 516s in those days -- and then BBN people would come in a few days later to shake down the software. An easy couple of weeks of grace. BBN fixed their timing trouble, air shipped the IMP, and it arrived on our loading dock on Saturday, August 30. They arrived with the IMP, wheeled it into our computer room, plugged it in and the software restarted from where it had been when the plug was pulled in Cambridge. Still Saturday, August 30. Panic time at UCLA. The second IMP was delivered to SRI at the beginning of October, and ARPA's interest was intense. Larry Roberts and Barry Wessler came by for a visit on November 21, and we actually managed to demonstrate a Telnet-like connection to SRI." 5. This democratic community is in danger of being fundamentally altered. This study of the history of the development of the ARPANET in conjunction with my paper, "The Social Forces Behind the Development of Usenet News" are meant to help people under- stand where the Net has come from, in order to defend it, and try to fight to keep it open and democratic - the seventh wonder of the world as a recent ad called the Internet, misdirected as it was - but correct any way. I hope to make this analysis available in RFC form as a comment on RFC 1000. Bibliography Special Thanks to Alexander McKenizie of BBN, Stephen Crocker of TIS, and Vinton Cerf of CNRI for making research materials available. ARPANET COMPLETION REPORT DRAFT , September 9, 1977, unpublished. Cerf, Vinton G., private corespondence, dated Nov 27, 1993. Subject: "Re: Early Days of the ARPANET and the NWG" Cerf, Vinton G., "An Assessment of ARPANET Protocols." Infotech Education Ltd. Stanford University, California, 21 pages Cosell, Bernie "Re: RFC1000 - Questions about the origins of ARPANET Protocols 2/2" Article: 54310 of alt.folklore.computers, Nov. 23, 193. Crocker, Stephen D., 1993A email message to Com-Priv mailing list (com-priv@psi.com) Subject "Re: RFC1000 (Partial response to part 1)" Date: Nov 27, 1993. Crocker, Stephen D., 1993B email message to Com-Priv mailing list Subject: "Re: RFC1000 (End of response to part 1)" Date: Nov 27, 1993. Crocker, Stephen D., 1993C email message to Com-Priv mailing list Subject "Subject: Re: RFC1000 (Response to part 2)" Date: Nov 27, 1993. Crocker, Stephen D., RFC 3, DOCUMENTATION CONVENTIONS. Crocker, Stephen D., RFC 1000, RFC Reference Guide. Heart, F, McKenzie, A., McQuillan, J., Walden, D., ARPANET Completion Report, Washington, 1978. Licklider, J.C.R., Interview conducted by William Aspray and Arthur Norberg on October 28, 1988 Cambridge, Mass. CBI Univ of Minn., Madison. Licklider, J.C.R. and Robert Taylor, "The Computer as a Communi- cation Device" from "In Memoriam: J.C.R. Licklider 1915-1990," Aug. 7, 1990, p. 40; reprinted by permission from Digital Research Center; originally published as "The Computer as a Communication Device," in "Science and Tech- nology", April, 1968, pg. 40 Mckenzie, Alexander, Interview with Nov 1, 1993. Roberts, Lawrence Member IEEE, Invited Paper, "The Evolution of Packet Switching", Proceedings of the IEEE Volume 66, Number 11, November 1978, pages 1307 - 1313 From: rh120@ciao.cc.columbia.edu (Ronda Hauben) Newsgroups: alt.folklore.computers Subject: Wanted:Info to Add to History of Usenet Article Date: 28 Feb 1995 17:41:20 GMT A few years ago I gave a talk about the early evolution of Usenet News and put together the following talk. I would like to develop this history more and welcome suggestions as to what else should be included in it. Ronda ------- THE EVOLUTION OF USENET NEWS: THE POOR MAN'S ARPANET by Ronda Hauben au329@cleveland.freenet.edu Usenet News was born in 1979 when Tom Truscott, and Jim Ellis, graduate students at Duke University and Steve Bellovin, a graduate student at the University of North Carolina conceived of creating a network to link the computers at their different schools together. Using homemade auto dial modems and the Unix to Unix copy program (called UUCP), that was being distributed with the UNIX operating system, version 7, Steve Bellovin, one of the students, wrote some simple shell script programs in Unix to have the computers call each other up and search for changes in the files and then copy the changes. The program was very slow, however, and so the students enlisted Stephen Daniel, also a graduate student at Duke, to rewrite the program code in the C programming language. Other people at Duke and the University of North Carolina took part in getting the network debugged. Once the programs were functioning on their respective machines, Jim Ellis went to a meeting of the academic Unix users group called Usenix. In the following account, Tom Truscott describes what happened: "James Ellis (jte) gave a short talk and handed out a 5 page "Invitation to a General Access UNIX Network" at the January 1980 Usenix Conference in Boulder Colorado. We made up 80 copies and they were gobbled up (not surprising, there were a record- smashing 400 attendees.)....Afterwards, jte mentioned that the audience particularly enjoyed his description of Duke's two home- built 300 baud autodialers." (from email message, Oct 12, 1992) The invitation they distributed explains: "The initially most significant service will be to provide a rapid access newsletter. Any node can submit an article, which will in due course propagate to all nodes. A "news" program has been designed which can perform this service. The first articles will probably concern bug fixes, trouble reports, and general cries for help. Certain categories of news, such as "have/want" articles, may become sufficiently popular as to warrant separate newsgroups. (The news program mentioned above supports newsgroups.) "The mail command provides a convenient means for responding to intriguing articles. In general, small groups of users with common interests will use mail to communicate. If the group size grows sufficiently, they will probably start an additional news group... "It is hoped that USENIX will take an active (indeed central) role in the network. There is the problem of members not on the net, so hardware newsletters should remain the standard communication method. However, use of the net for preparation of newsletters seems like a good idea. The Invitation urged: "This is a sloppy proposal. Let's start a committee. No thanks! Yes, there are problems. Several amateurs collaborated on this plan. But let's get started now. Once the net is in place, we can start a committee. And they will actually use the net, so they will know what the real problems are." (from "Invitation to a General Access Unix Network" by Tom Truscott, Duke University.) Several months later, the software for the A News program for Usenet News was put on the conference tape for general distribution at the Delaware Summer 1980 Usenix meeting. The handout distributed at this conference explained: "A goal of USENET has been to give every UNIX system the opportunity to join and benefit from a computer network (a poor man's ARPANET, if you will)...." (copy in the Usenet History Archives) One of the students, Stephen Daniel, who wrote the C program for A News, explains why the term "poor man's ARPANET" was used. He wrote, "I don't remember when the phrase was coined, but to me it expressed exactly what was going on. We (or at least I) had little idea of what was really going on on the ARPANET, but we knew we were excluded. Even if we had been allowed to join, there was no way of coming up with the money. It was commonly accepted at the time that to join the ARPANET took political connections and $100,000. I don't know if that assumption was true, but we were so far from having either connections or $$ that we didn't even try. The `Poor man's ARPANET' was our way of joining the CS community (Computer Science -ed), and we made a deliberate attempt to extend it to other not-well-endowed members of the community. It is hard to believe in retrospect," he writes, "but we were initially disappointed at how few people joined us. We attributed this lack more to the cost of autodialers than lack of desire." (from email dated Jan 25, 1993, Usenet History List) The ARPANET that Daniel is referring to pioneered the network technology that serves as the foundation of today's global internet. The first host connected to the ARPANET was the SDS Sigma-7 on Sept. 2, 1969 at the UCLA (University of California in Los Angeles) site. It began passing bits to other sites at SRI (SDS-940 at Stanford Research Institute), UCSB (IBM 360/75 at University of California Santa Barbara), and Utah (Dec PDP-10 at the University of Utah). There were many unexpected problems and obstacles, but through the collaborative work by the participants using the net, the number of sites steadily expanded and by 1977 the ARPANET extended to more than 50 sites from Hawaii to Norway. Originally funded under the Department of Defense's program for Advanced Research Projects Agency, the ARPANET was an experimental network that was set up by the U.S. military for those university computer science departments and private research institutes with DoD funding. The sites were involved in the research of getting the network to function and they had access to the advantages of the network to help their research. But academic computer science departments without DoD grants had no means of access to the ARPANET and had no access to the advantages that it provided for collaborative research. Usenet News, however, was available to all who were interested as long as they had access to the Unix operating system (which in those days was available without charge to the academic community.) And posting and participating in the network was possible at no cost to the individuals who participated besides the cost of their own equipment and the telephone calls to receive or send Netnews. Therefore the joys and challenges of being a participant in the creation of an ever expanding network, the experience available to an exclusive few via the ARPANET, was available via Usenet News to those without political or financial connections -- to the commonfolk of the computer science community. As Daniel notes, Usenet pioneers report that they were surprised at how slowly Usenet sites expanded at first. But when the University of California at Berkeley (UCB) joined Usenet, links began to be created between Usenet and the ARPANET. University of California at Berkeley was a site on the ARPANET. At first, it is reported, mailing lists of discussions among Arpanauts (as they were called by Usenet users) were poured into Usenet. This first connection between the ARPANET and Usenet News, Daniel reports, only contributed to "the sense of being poor cousins." Daniel explains: "It was initially very hard to contribute to those lists, and when you did you were more likely to get a response to your return address than to the content of your letter. It definitely felt second class to be in read-only mode on human-nets and sf-lovers. (Those were two popular ARPANET mailing lists. Mailing lists are a way of having all the people who wanted to be part of a discussion receive all the mail that any person sent -ed) Daniel also clarifies the different philosophy guiding the development of Usenet as opposed to that of the ARPANET. Daniel explains that "Usenet was organized around netnews, where the receiver controls what is received. The ARPANET lists were organized around mailing lists, where there is a central control for each list that potentially controls who receives the material and what material can be transmitted. I still strongly prefer the reader-centered view," he concludes. With the increasing connections to the ARPANET from Usenet, the numbers of sites on Usenet grew. Describing the gatewaying between Usenet and the ARPANET, Steve Bellovin, another Usenet pioneer, explains: "The first gateway of ARPANET mailing lists to Usenet was an early force to have gateways with ARPAnet. Gateways," he continues, to ARPAnet were on the side things and in all likelihood not officially sanctioned. However, this provided the impetus for future gateways into ARPAnet. This was the first pressure on the ARPAnet to provide service to a larger number of people -- a first step to transforming of the ARPAnet to become a part of the backbone on the Internet." The original creators of Usenet explain that they didn't know about the existence of the ARPANET mailing lists until the ucbvax at Berkeley, which was also on the ARPANET, joined Usenet. "Only when ucbvax [Unversity of California at Berkeley -ed] joined the net," explains Truscott, did "fa" (from arpa-ed) appear. Indeed I was unaware of the ARPANET mailing lists such as human-nets until ucbvax enlightened us." (comment from Tom Truscott, Sept. 25, 1990, Usenet History Archives). And Lauren Weinstein, a pioneer of both the ARPANET and of Usenet, describes how the restrictions of both the technical and the military environment of the ARPANET had an impact on the continued technical expansion of the network. Weinstein writes: "Greetings. It's all too easy to forget, even for those of us who were there all along, how "small" it all started. When I was at UCLA-ATS (ARPANET site 1) in the early 70's, even small mailing lists could cause concern. I still distinctly remember the concerns regarding network loading from Geoff Goodfellow's NETWORK-HACKERS mailing list (this was in the days when "hacker" didn't have the negative meaning it has picked up since then) as the list passed *100* addresses. A list about wine (WINE-TASTERS, I believe it was called) which was mentioned in "Datamation" magazine caused memos to be sent out from the powers-that-be about "official use" of the net. There was also a lot of hand-wringing about the 255 site limit (that is, a limit on the number of IMPs (Interface Message Processors) in the network topology under NCP [Network Control Program-ed].) It's quite remarkable how much we accomplished on what by today's standards were slow machines with "tiny" amounts of memory, running with a 56 Kbit network backbone!" (from Nov. 23, 1992) Another ARPANET pioneer Bernie Cosell described the problem that ARPA faced with the burgeoning mailing list discussion groups that had developed. He writes: "Well the influence of Arpa may have had a fair bit to do with it. In fact, mailing lists, themselves, almost went away. The problem is that the ARPANET was funded for two, and only two purposes: 1) to provide a vehicle for doing networking research [routing algorithms, queueing theory, congestion control, protocols, etc] and 2) to provide a means for having ARPA contractors collaborate electonically. [Also, the ARPANET was originally funded by the Department of Defense to create a working network. And the original vision conceived of it as a means of saving the DoD money by making it possible for the military to utilize remote computer and human resources and thereby eliminate the need to duplicate such resources - ed.] Now," Cosell explains, "ARPA was fairly liberal within those limits, but they did occasionally put their foot down. The `mailing list' problem [which predated USENET] happened with SF- Lovers, about the first [along with HUMAN-NETs] really large- scale mailing list. BUT...unlike HUMAN-NETs, SF-LOVERS could show *NO* legitimate reason for using "ARPA bandwidth" and so actually got shut down for a couple of months [it was eventually reinstated when Roger Duffy managed to make the case to ARPA that email was an important use of their network, and SF-LOVERS was providing valuable research and experimentation in the administration and operation of large mailing lists]." "Up to the day that the ARPANET was decommissioned there were explicit [but poorly enforced] use policies. In fact, when CSNET (a net that was created under pressure from some of those academic computer and science departments who were excluded from the ARPANET-ed) wanted to use the ARPANET as an email link between two disconnected parts of CSNET, the NSF actually had to seek [and got] permission from DARPA for that use of the net." "The use of the ARPANET for distribution of netnews was always a tricky matter and I don't know how the guys at BRL (U.S. Army Ballistics Research Laboratory-ed)managed to get away with it -- but I think that the idea was that they managed to convince their management, and then DCA (Defense Communications Agency-ed) [who was operating the ARPANET at the time] that a `significant percentage of the usenet traffic they were expending computer resources on and gobbling ARPANET bandwidth *was* legitimate `research' traffic and that the interests of that research were best served by quietly ignoring the `abuse' around the edges [like net.jokes and such]." (from communication from Bernie Cosell, Jan. 26, 1993) Some excerpts from the Human Nets mailing list show the kind of discussion that it encouraged. One writer describes the unique nature of computer facilitated communications. He writes: "I think/feel that computer communications (done between humans via computers) lie somewhere between written and verbal communications in style and flavor. There is an ambience of informality and stream-of-consciousness style that pervades it but coupled with ideas that are well thought out (usually) and deeper in insight than average verbal communications. Does this make any sense to anyone 'sides myself?" from 5/15/81 FFM@MIT-MC Subject: English Murdering & flame about human telecommunications In another Human Nets article, the writer describes the advantages he has experienced from his participation on the ARPANET: "Ever since I first `found' the ARPANET, some 3 years ago, I had considered it a playground. It is also a place where quite a bit of work gets done, but I think the `playground' atmosphere really encourages the work, since if you can make your work fun then you will want to work harder, increasing productivity, but also increasing addiction." (Jonathan Alan Solomon,"Computer Network Addiction," ) There were many problems and difficulties that Usenet participants faced, but they worked to help each other gain and continue access to Usenet News and email. There are many stories of frustrations along the way but one of the most outstanding is told by Amanda Walker who shows how it was necessary to send email across the continent to get email to the computer center on the Case Western Reserve University campus. Describing the problem, Walker writes: "Indeed. I suspect that there are any number of examples of this, but the most egregious in my experience was at CWRU. The ECMP department had a VAX 11/780 on Usenet ("cwruecmp"), and the campus computer center had a DEC-20 in the room next door. The machines were separated by a grand total of about 30 feet and a piece of wallboard, but the computer center was not at all interested in "catering" to "those CS types" by stringing an RS- 232 line between them. So it was possible to send mail between them, but only by sending via a route resembling: cwruecmp => ucbvax (UUCP) ucbvax => columbia (CU20A, I think) (ARPANET) columbia => cmu-cs-c => cwru 20 (CCnet) [i.e. the mail went from Case Western Reserve University in Cleveland, Ohio to the University of California in Berkeley via uucp; from the University of California Berkeley to Columbia University, in New York City, via the ARPANET; from Columbia University in New York City to Carnegie Mellon, in Pittsburgh, Pa, and from Carnegie Mellon University in Pittsburgh, Pa to Case Western Reserve University in Cleveland, Ohio via ccnet -ed] "Yup," writes Walker, "that's three networks, and two coasts just to get through a piece of sheet rock :-). Took about a week, too." Despite such frustrations, there were also those who helped Usenet News to grow and develop. Unix enthusiastists and pioneers at some large companies like AT&T's Bell Labs did whatever they could to provide support for Usenet. And at one point AT&T realized that it would save millions of dollars if it worked out the bugs to have internal email and in the process it gave support to the pioneers of Usenet who were trying to develop more efficient email routing programs. Also, Digital Equipment Corporation supported Usenet in various ways and the spread of Usenet and Unix encouraged the sale of the Unix based computers from DEC. Also, Usenet newsgroups provided needed technical help for the folks using Unix and Unix based computers. By 1982, the continuing explosion of Usenet News surprised even its most dedicated fans. Gregory Woodbury, describes the shock that was experienced when the pioneers realized how Usenet News was not mainly developing as a means connecting spatially nearby localities sharing news, as they had envisioned, but instead, was taking a totally unexpected course of development. He writes: "I do not recall that anyone was quite expecting the explosion that followed," recounts Woodbury, "What developed took everybody by surprise. When the direction of evolution took an unexpected turn, and a continental network emerged, spanning the continent from California to North Carolina, and Toronto to San Diego, it was sort of a shock to realize what had happened." (from Gregory G. Woodbury, "Net Cultural Assumptions") Statistics Gene Spafford presented at an IETF meeting in 1988 show the tremendous growth and development that NetNews, as its founders like to call Usenet, experienced. 1979 3 sites ~2 articles a day 1980 15 sites, ~10 articles a day 1981 150 sites, ~20 articles a day 1982 400 sites, ~50 articles a day 1983 600 sites, ~120 1984 900 sites, ~225 1985 1,300 sites, ~375 articles per day, 1+Megabyte per/day 1986 2,500 sites, ~500, 2MB+ 1987 5,000 sites,~1000, 2.5MB+ 1988 11,000 sites,~1800, 4MB+ (from Gene Spafford, Usenet History Archives - from the Mailing List, Gene Spafford, Oct. 11, 1990 based on stats from Adams, Spencer, Horton, Bellovin and Reid) Today Usenet News continues to grow in both the number of sites and in the size of posts it carries and the number of newsgroups. Usenet is transported by uucp connections and via nntp (Net News Transfer Protocol) along the internet, which is the child of the old ARPANET. Many times pioneers of Usenet have been convinced that the load of posts or the number of sites was becoming too great and that it wouldn't be sustainable. The fear is now facetiously referred to as "the imminent death of the net is predicted." For though each time the problems have seemed insurmountable, they have been investigated and solutions found to deal with them through the hard work of many net participants (sometimes referred to on Usenet as "net.citizens" or netizens). In the past few years a system of FreeNets has begun to develop utilizing the Netnews software to make Usenet News available to community people who don't have access to a university or industry site. Cleveland FreeNet, sponsored by Case Western Reserve University and other community organizations in Cleveland, Ohio was the first FreeNet. They used the Netnews software to create a set of local newsgroups reflecting the different community services in the Cleveland area like the hospitals, public schools, public libraries, museums, etc. Users also have access to the worldwide newsgroups of Usenet News. The FreeNet software makes it relatively easy to read and post on Usenet, and is a bit more manageable than most of the public domain software used in the Universities. There are now FreeNets across the U.S. and Canada and even a few in Europe. The ARPANET pioneered important breakthroughs in computer network technology. It also pioneered the ability to collaborate and to utilize dispersed resources -- both people and computers. Usenet represents the continuation of this tradition by making access to these collaborative research relationships available to the commonfolk. The extension of Usenet has also required a great deal of pioneeering effort and technical development, but the folks participating in Usenet have been there to solve the problems. Writing in 1968 before the ARPANET network began, J.C.R Licklider, who has been called the Father of the ARPANET, and Robert W. Taylor predicted the challenge that would face society with the development of computer networks. They wrote: "First, life will be happier for the on-line individual because the people with whom one interacts most strongly will be selected more by commonality of interests and goals than by accidents of proximity. Second, communication will be more effective and productive, and therefore more enjoyable. Third, much communication and interaction will be with programs and programming models, which will be (a) highly responsive, (b) supplementary to one's own capablities, rather than competitive, and (c) capable of representing progressively more complex ideas without necessarily displaying all the levels of their structure at the same time -- and which will therefore be both challenging nad rewarding. And, fourth, there will be plenty of opportunity for everyone (who can afford a console) to find his calling, for the whole world of information, with all its fields and disciplines, will be open to him, with programs ready to guide him or to help him explore." "For the society," they continued, "the impact will be good or bad depending mainly on the question: Will `to be on line' be a privilege or a right? If only a favored segment of the population gets a chance to enjoy the advantage of `intelligence amplification,' the network may exaggerate the discontinuity in the spectrum of intellectual opportunity." "On the other hand," they continued, "if the network idea should prove to do for education what a few have envisioned in hope, if not in concrete detailed plan, and if all minds should prove to be responsive, surely the boon to humankind would be beyond measure." "Unemployment would disappear from the face of the earth forever, for consider the magnitude of the task of adapting the network's software to all the new generations of computer, coming closer and closer upon the heels of their predecessors until the entire population of the world is caught up in an infinite crescendo of on-line interactive debugging." [from "In Memoriam: J.C.R. Licklider 1915-1990," Aug. 7, 1990, p. 40; reprinted by permission from Digital Research Center; originally published as "The Computer as a Communication Device," in "Science and Technology", April, 1968, p. 40) The vision of the masses being needed to participate in the debugging and development of the network that will make a new world possible is still a helpful vision. Thus I want to invite you to the Wonderful World of Usenet News. It's a world that needs and will reward your participation. Editor's Notes: * This and following excerpts describing responses to the student are from the Appendix to "The Social Forces Behind the Development of Usenet News", by Michael Hauben, Fall 1992. **The Unix Operating system was created by computer programmers at Bell Labs. They explain that it was created to help develop a community of programmers. With the growth and development of Usenet News it has demonstrated that it has achieved that purpose. Also special thanks to Bruce Jones and the Usenet Pioneers for the history materials they have gathered and made available. -- Ronda Hauben, rh120@columbia.edu or ronda@umcc.umich.edu "The Netizens and the Wonderful World of the Net: An Anthology on the History and Impact of the Net" via http://www.columbia.edu/~hauben/project_book.html Also http://scrg.cs.tcd.ie/scrg/u/rcwoods/netbook/contents.html From: rh120@ciao.cc.columbia.edu (Ronda Hauben) Newsgroups: alt.folklore.computers Subject: Wanted:Info to Add to History of Usenet Article Date: 28 Feb 1995 17:41:20 GMT A few years ago I gave a talk about the early evolution of Usenet News and put together the following talk. I would like to develop this history more and welcome suggestions as to what else should be included in it. Ronda ------- THE EVOLUTION OF USENET NEWS: THE POOR MAN'S ARPANET by Ronda Hauben au329@cleveland.freenet.edu Usenet News was born in 1979 when Tom Truscott, and Jim Ellis, graduate students at Duke University and Steve Bellovin, a graduate student at the University of North Carolina conceived of creating a network to link the computers at their different schools together. Using homemade auto dial modems and the Unix to Unix copy program (called UUCP), that was being distributed with the UNIX operating system, version 7, Steve Bellovin, one of the students, wrote some simple shell script programs in Unix to have the computers call each other up and search for changes in the files and then copy the changes. The program was very slow, however, and so the students enlisted Stephen Daniel, also a graduate student at Duke, to rewrite the program code in the C programming language. Other people at Duke and the University of North Carolina took part in getting the network debugged. Once the programs were functioning on their respective machines, Jim Ellis went to a meeting of the academic Unix users group called Usenix. In the following account, Tom Truscott describes what happened: "James Ellis (jte) gave a short talk and handed out a 5 page "Invitation to a General Access UNIX Network" at the January 1980 Usenix Conference in Boulder Colorado. We made up 80 copies and they were gobbled up (not surprising, there were a record- smashing 400 attendees.)....Afterwards, jte mentioned that the audience particularly enjoyed his description of Duke's two home- built 300 baud autodialers." (from email message, Oct 12, 1992) The invitation they distributed explains: "The initially most significant service will be to provide a rapid access newsletter. Any node can submit an article, which will in due course propagate to all nodes. A "news" program has been designed which can perform this service. The first articles will probably concern bug fixes, trouble reports, and general cries for help. Certain categories of news, such as "have/want" articles, may become sufficiently popular as to warrant separate newsgroups. (The news program mentioned above supports newsgroups.) "The mail command provides a convenient means for responding to intriguing articles. In general, small groups of users with common interests will use mail to communicate. If the group size grows sufficiently, they will probably start an additional news group... "It is hoped that USENIX will take an active (indeed central) role in the network. There is the problem of members not on the net, so hardware newsletters should remain the standard communication method. However, use of the net for preparation of newsletters seems like a good idea. The Invitation urged: "This is a sloppy proposal. Let's start a committee. No thanks! Yes, there are problems. Several amateurs collaborated on this plan. But let's get started now. Once the net is in place, we can start a committee. And they will actually use the net, so they will know what the real problems are." (from "Invitation to a General Access Unix Network" by Tom Truscott, Duke University.) Several months later, the software for the A News program for Usenet News was put on the conference tape for general distribution at the Delaware Summer 1980 Usenix meeting. The handout distributed at this conference explained: "A goal of USENET has been to give every UNIX system the opportunity to join and benefit from a computer network (a poor man's ARPANET, if you will)...." (copy in the Usenet History Archives) One of the students, Stephen Daniel, who wrote the C program for A News, explains why the term "poor man's ARPANET" was used. He wrote, "I don't remember when the phrase was coined, but to me it expressed exactly what was going on. We (or at least I) had little idea of what was really going on on the ARPANET, but we knew we were excluded. Even if we had been allowed to join, there was no way of coming up with the money. It was commonly accepted at the time that to join the ARPANET took political connections and $100,000. I don't know if that assumption was true, but we were so far from having either connections or $$ that we didn't even try. The `Poor man's ARPANET' was our way of joining the CS community (Computer Science -ed), and we made a deliberate attempt to extend it to other not-well-endowed members of the community. It is hard to believe in retrospect," he writes, "but we were initially disappointed at how few people joined us. We attributed this lack more to the cost of autodialers than lack of desire." (from email dated Jan 25, 1993, Usenet History List) The ARPANET that Daniel is referring to pioneered the network technology that serves as the foundation of today's global internet. The first host connected to the ARPANET was the SDS Sigma-7 on Sept. 2, 1969 at the UCLA (University of California in Los Angeles) site. It began passing bits to other sites at SRI (SDS-940 at Stanford Research Institute), UCSB (IBM 360/75 at University of California Santa Barbara), and Utah (Dec PDP-10 at the University of Utah). There were many unexpected problems and obstacles, but through the collaborative work by the participants using the net, the number of sites steadily expanded and by 1977 the ARPANET extended to more than 50 sites from Hawaii to Norway. Originally funded under the Department of Defense's program for Advanced Research Projects Agency, the ARPANET was an experimental network that was set up by the U.S. military for those university computer science departments and private research institutes with DoD funding. The sites were involved in the research of getting the network to function and they had access to the advantages of the network to help their research. But academic computer science departments without DoD grants had no means of access to the ARPANET and had no access to the advantages that it provided for collaborative research. Usenet News, however, was available to all who were interested as long as they had access to the Unix operating system (which in those days was available without charge to the academic community.) And posting and participating in the network was possible at no cost to the individuals who participated besides the cost of their own equipment and the telephone calls to receive or send Netnews. Therefore the joys and challenges of being a participant in the creation of an ever expanding network, the experience available to an exclusive few via the ARPANET, was available via Usenet News to those without political or financial connections -- to the commonfolk of the computer science community. As Daniel notes, Usenet pioneers report that they were surprised at how slowly Usenet sites expanded at first. But when the University of California at Berkeley (UCB) joined Usenet, links began to be created between Usenet and the ARPANET. University of California at Berkeley was a site on the ARPANET. At first, it is reported, mailing lists of discussions among Arpanauts (as they were called by Usenet users) were poured into Usenet. This first connection between the ARPANET and Usenet News, Daniel reports, only contributed to "the sense of being poor cousins." Daniel explains: "It was initially very hard to contribute to those lists, and when you did you were more likely to get a response to your return address than to the content of your letter. It definitely felt second class to be in read-only mode on human-nets and sf-lovers. (Those were two popular ARPANET mailing lists. Mailing lists are a way of having all the people who wanted to be part of a discussion receive all the mail that any person sent -ed) Daniel also clarifies the different philosophy guiding the development of Usenet as opposed to that of the ARPANET. Daniel explains that "Usenet was organized around netnews, where the receiver controls what is received. The ARPANET lists were organized around mailing lists, where there is a central control for each list that potentially controls who receives the material and what material can be transmitted. I still strongly prefer the reader-centered view," he concludes. With the increasing connections to the ARPANET from Usenet, the numbers of sites on Usenet grew. Describing the gatewaying between Usenet and the ARPANET, Steve Bellovin, another Usenet pioneer, explains: "The first gateway of ARPANET mailing lists to Usenet was an early force to have gateways with ARPAnet. Gateways," he continues, to ARPAnet were on the side things and in all likelihood not officially sanctioned. However, this provided the impetus for future gateways into ARPAnet. This was the first pressure on the ARPAnet to provide service to a larger number of people -- a first step to transforming of the ARPAnet to become a part of the backbone on the Internet." The original creators of Usenet explain that they didn't know about the existence of the ARPANET mailing lists until the ucbvax at Berkeley, which was also on the ARPANET, joined Usenet. "Only when ucbvax [Unversity of California at Berkeley -ed] joined the net," explains Truscott, did "fa" (from arpa-ed) appear. Indeed I was unaware of the ARPANET mailing lists such as human-nets until ucbvax enlightened us." (comment from Tom Truscott, Sept. 25, 1990, Usenet History Archives). And Lauren Weinstein, a pioneer of both the ARPANET and of Usenet, describes how the restrictions of both the technical and the military environment of the ARPANET had an impact on the continued technical expansion of the network. Weinstein writes: "Greetings. It's all too easy to forget, even for those of us who were there all along, how "small" it all started. When I was at UCLA-ATS (ARPANET site 1) in the early 70's, even small mailing lists could cause concern. I still distinctly remember the concerns regarding network loading from Geoff Goodfellow's NETWORK-HACKERS mailing list (this was in the days when "hacker" didn't have the negative meaning it has picked up since then) as the list passed *100* addresses. A list about wine (WINE-TASTERS, I believe it was called) which was mentioned in "Datamation" magazine caused memos to be sent out from the powers-that-be about "official use" of the net. There was also a lot of hand-wringing about the 255 site limit (that is, a limit on the number of IMPs (Interface Message Processors) in the network topology under NCP [Network Control Program-ed].) It's quite remarkable how much we accomplished on what by today's standards were slow machines with "tiny" amounts of memory, running with a 56 Kbit network backbone!" (from Nov. 23, 1992) Another ARPANET pioneer Bernie Cosell described the problem that ARPA faced with the burgeoning mailing list discussion groups that had developed. He writes: "Well the influence of Arpa may have had a fair bit to do with it. In fact, mailing lists, themselves, almost went away. The problem is that the ARPANET was funded for two, and only two purposes: 1) to provide a vehicle for doing networking research [routing algorithms, queueing theory, congestion control, protocols, etc] and 2) to provide a means for having ARPA contractors collaborate electonically. [Also, the ARPANET was originally funded by the Department of Defense to create a working network. And the original vision conceived of it as a means of saving the DoD money by making it possible for the military to utilize remote computer and human resources and thereby eliminate the need to duplicate such resources - ed.] Now," Cosell explains, "ARPA was fairly liberal within those limits, but they did occasionally put their foot down. The `mailing list' problem [which predated USENET] happened with SF- Lovers, about the first [along with HUMAN-NETs] really large- scale mailing list. BUT...unlike HUMAN-NETs, SF-LOVERS could show *NO* legitimate reason for using "ARPA bandwidth" and so actually got shut down for a couple of months [it was eventually reinstated when Roger Duffy managed to make the case to ARPA that email was an important use of their network, and SF-LOVERS was providing valuable research and experimentation in the administration and operation of large mailing lists]." "Up to the day that the ARPANET was decommissioned there were explicit [but poorly enforced] use policies. In fact, when CSNET (a net that was created under pressure from some of those academic computer and science departments who were excluded from the ARPANET-ed) wanted to use the ARPANET as an email link between two disconnected parts of CSNET, the NSF actually had to seek [and got] permission from DARPA for that use of the net." "The use of the ARPANET for distribution of netnews was always a tricky matter and I don't know how the guys at BRL (U.S. Army Ballistics Research Laboratory-ed)managed to get away with it -- but I think that the idea was that they managed to convince their management, and then DCA (Defense Communications Agency-ed) [who was operating the ARPANET at the time] that a `significant percentage of the usenet traffic they were expending computer resources on and gobbling ARPANET bandwidth *was* legitimate `research' traffic and that the interests of that research were best served by quietly ignoring the `abuse' around the edges [like net.jokes and such]." (from communication from Bernie Cosell, Jan. 26, 1993) Some excerpts from the Human Nets mailing list show the kind of discussion that it encouraged. One writer describes the unique nature of computer facilitated communications. He writes: "I think/feel that computer communications (done between humans via computers) lie somewhere between written and verbal communications in style and flavor. There is an ambience of informality and stream-of-consciousness style that pervades it but coupled with ideas that are well thought out (usually) and deeper in insight than average verbal communications. Does this make any sense to anyone 'sides myself?" from 5/15/81 FFM@MIT-MC Subject: English Murdering & flame about human telecommunications In another Human Nets article, the writer describes the advantages he has experienced from his participation on the ARPANET: "Ever since I first `found' the ARPANET, some 3 years ago, I had considered it a playground. It is also a place where quite a bit of work gets done, but I think the `playground' atmosphere really encourages the work, since if you can make your work fun then you will want to work harder, increasing productivity, but also increasing addiction." (Jonathan Alan Solomon,"Computer Network Addiction," ) There were many problems and difficulties that Usenet participants faced, but they worked to help each other gain and continue access to Usenet News and email. There are many stories of frustrations along the way but one of the most outstanding is told by Amanda Walker who shows how it was necessary to send email across the continent to get email to the computer center on the Case Western Reserve University campus. Describing the problem, Walker writes: "Indeed. I suspect that there are any number of examples of this, but the most egregious in my experience was at CWRU. The ECMP department had a VAX 11/780 on Usenet ("cwruecmp"), and the campus computer center had a DEC-20 in the room next door. The machines were separated by a grand total of about 30 feet and a piece of wallboard, but the computer center was not at all interested in "catering" to "those CS types" by stringing an RS- 232 line between them. So it was possible to send mail between them, but only by sending via a route resembling: cwruecmp => ucbvax (UUCP) ucbvax => columbia (CU20A, I think) (ARPANET) columbia => cmu-cs-c => cwru 20 (CCnet) [i.e. the mail went from Case Western Reserve University in Cleveland, Ohio to the University of California in Berkeley via uucp; from the University of California Berkeley to Columbia University, in New York City, via the ARPANET; from Columbia University in New York City to Carnegie Mellon, in Pittsburgh, Pa, and from Carnegie Mellon University in Pittsburgh, Pa to Case Western Reserve University in Cleveland, Ohio via ccnet -ed] "Yup," writes Walker, "that's three networks, and two coasts just to get through a piece of sheet rock :-). Took about a week, too." Despite such frustrations, there were also those who helped Usenet News to grow and develop. Unix enthusiastists and pioneers at some large companies like AT&T's Bell Labs did whatever they could to provide support for Usenet. And at one point AT&T realized that it would save millions of dollars if it worked out the bugs to have internal email and in the process it gave support to the pioneers of Usenet who were trying to develop more efficient email routing programs. Also, Digital Equipment Corporation supported Usenet in various ways and the spread of Usenet and Unix encouraged the sale of the Unix based computers from DEC. Also, Usenet newsgroups provided needed technical help for the folks using Unix and Unix based computers. By 1982, the continuing explosion of Usenet News surprised even its most dedicated fans. Gregory Woodbury, describes the shock that was experienced when the pioneers realized how Usenet News was not mainly developing as a means connecting spatially nearby localities sharing news, as they had envisioned, but instead, was taking a totally unexpected course of development. He writes: "I do not recall that anyone was quite expecting the explosion that followed," recounts Woodbury, "What developed took everybody by surprise. When the direction of evolution took an unexpected turn, and a continental network emerged, spanning the continent from California to North Carolina, and Toronto to San Diego, it was sort of a shock to realize what had happened." (from Gregory G. Woodbury, "Net Cultural Assumptions") Statistics Gene Spafford presented at an IETF meeting in 1988 show the tremendous growth and development that NetNews, as its founders like to call Usenet, experienced. 1979 3 sites ~2 articles a day 1980 15 sites, ~10 articles a day 1981 150 sites, ~20 articles a day 1982 400 sites, ~50 articles a day 1983 600 sites, ~120 1984 900 sites, ~225 1985 1,300 sites, ~375 articles per day, 1+Megabyte per/day 1986 2,500 sites, ~500, 2MB+ 1987 5,000 sites,~1000, 2.5MB+ 1988 11,000 sites,~1800, 4MB+ (from Gene Spafford, Usenet History Archives - from the Mailing List, Gene Spafford, Oct. 11, 1990 based on stats from Adams, Spencer, Horton, Bellovin and Reid) Today Usenet News continues to grow in both the number of sites and in the size of posts it carries and the number of newsgroups. Usenet is transported by uucp connections and via nntp (Net News Transfer Protocol) along the internet, which is the child of the old ARPANET. Many times pioneers of Usenet have been convinced that the load of posts or the number of sites was becoming too great and that it wouldn't be sustainable. The fear is now facetiously referred to as "the imminent death of the net is predicted." For though each time the problems have seemed insurmountable, they have been investigated and solutions found to deal with them through the hard work of many net participants (sometimes referred to on Usenet as "net.citizens" or netizens). In the past few years a system of FreeNets has begun to develop utilizing the Netnews software to make Usenet News available to community people who don't have access to a university or industry site. Cleveland FreeNet, sponsored by Case Western Reserve University and other community organizations in Cleveland, Ohio was the first FreeNet. They used the Netnews software to create a set of local newsgroups reflecting the different community services in the Cleveland area like the hospitals, public schools, public libraries, museums, etc. Users also have access to the worldwide newsgroups of Usenet News. The FreeNet software makes it relatively easy to read and post on Usenet, and is a bit more manageable than most of the public domain software used in the Universities. There are now FreeNets across the U.S. and Canada and even a few in Europe. The ARPANET pioneered important breakthroughs in computer network technology. It also pioneered the ability to collaborate and to utilize dispersed resources -- both people and computers. Usenet represents the continuation of this tradition by making access to these collaborative research relationships available to the commonfolk. The extension of Usenet has also required a great deal of pioneeering effort and technical development, but the folks participating in Usenet have been there to solve the problems. Writing in 1968 before the ARPANET network began, J.C.R Licklider, who has been called the Father of the ARPANET, and Robert W. Taylor predicted the challenge that would face society with the development of computer networks. They wrote: "First, life will be happier for the on-line individual because the people with whom one interacts most strongly will be selected more by commonality of interests and goals than by accidents of proximity. Second, communication will be more effective and productive, and therefore more enjoyable. Third, much communication and interaction will be with programs and programming models, which will be (a) highly responsive, (b) supplementary to one's own capablities, rather than competitive, and (c) capable of representing progressively more complex ideas without necessarily displaying all the levels of their structure at the same time -- and which will therefore be both challenging nad rewarding. And, fourth, there will be plenty of opportunity for everyone (who can afford a console) to find his calling, for the whole world of information, with all its fields and disciplines, will be open to him, with programs ready to guide him or to help him explore." "For the society," they continued, "the impact will be good or bad depending mainly on the question: Will `to be on line' be a privilege or a right? If only a favored segment of the population gets a chance to enjoy the advantage of `intelligence amplification,' the network may exaggerate the discontinuity in the spectrum of intellectual opportunity." "On the other hand," they continued, "if the network idea should prove to do for education what a few have envisioned in hope, if not in concrete detailed plan, and if all minds should prove to be responsive, surely the boon to humankind would be beyond measure." "Unemployment would disappear from the face of the earth forever, for consider the magnitude of the task of adapting the network's software to all the new generations of computer, coming closer and closer upon the heels of their predecessors until the entire population of the world is caught up in an infinite crescendo of on-line interactive debugging." [from "In Memoriam: J.C.R. Licklider 1915-1990," Aug. 7, 1990, p. 40; reprinted by permission from Digital Research Center; originally published as "The Computer as a Communication Device," in "Science and Technology", April, 1968, p. 40) The vision of the masses being needed to participate in the debugging and development of the network that will make a new world possible is still a helpful vision. Thus I want to invite you to the Wonderful World of Usenet News. It's a world that needs and will reward your participation. Editor's Notes: * This and following excerpts describing responses to the student are from the Appendix to "The Social Forces Behind the Development of Usenet News", by Michael Hauben, Fall 1992. **The Unix Operating system was created by computer programmers at Bell Labs. They explain that it was created to help develop a community of programmers. With the growth and development of Usenet News it has demonstrated that it has achieved that purpose. Also special thanks to Bruce Jones and the Usenet Pioneers for the history materials they have gathered and made available. -- Ronda Hauben, rh120@columbia.edu or ronda@umcc.umich.edu "The Netizens and the Wonderful World of the Net: An Anthology on the History and Impact of the Net" via http://www.columbia.edu/~hauben/project_book.html Also http://scrg.cs.tcd.ie/scrg/u/rcwoods/netbook/contents.html