Subject: On the Early History of Unix and Unix Tools(pt1of5)draft
From: [email-masked] (Ronda Hauben)
Date: 10 Jan 1996 16:42:53 GMT
Newsgroups: alt.amateur-comp,comp.misc,comp.unix.misc,alt.folklore.computers,sci.engr,soc.history.science,comp.edu

An earlier draft of this article was posted in Summer 
1994. I have finally gotten to do further work on the draft
and welcome comments. It will be part of the netbook 
"Netizens: On the History and Impact of the Net".

Thanks for any comments, suggestions, etc.

Ronda 
[email-masked]

---------------
                 On the Early History and Impact of Unix
              Tools to Build the Tools for a New Millenium
				by Ronda Hauben
				[email-masked]

			(Part 1 of 5)

    "When the barbarian, advancing step by step, had discovered
    the native metals, and learned to melt them in the crucible 
    and to cast them in moulds; when he had alloyed native 
    copper with tin and produced bronze; and, finally, when by a 
    still greater effort of thought he had invented the furnace, 
    and produced iron from the ore, nine tenths of the  battle 
    for civilization was gained. Furnished, with iron tools 
    capable of holding both an edge and a point, mankind were 
    certain of attaining to civilization."
                                       Lewis Henry Morgan

     "When Unix evolved within Bell Laboratories, it was not a
     result of some deliberate management initiative. It spread
     through channels of technical need and technical contact....
     This was typical of the way Unix spread around Bell
     Laboratories....I brought out that little hunk of history to
     point out that the spread and success of Unix, first in the 
     Bell organizations and then in the rest of the world, was due
     to the fact that it was used, modified, and tinkered up in a 
     whole variety of organizations."
                                       Victor Vyssotsky

     "UNIX is a lever for the intellect."
                                       John R. Mashey

Iron Tools and Software Tools
 
     Our era is witnessing the birth of an important new technology, 
different from any in the past. This new technology is the technology of 
software production. The new tools of our era are tools that make it 
possible to produce software. Unlike the tools forged in the past, 
software tools are not something you can grab or hold. Similarly the 
processes required to develop this new technology are new. Knowing the 
details of how such new tools have been created will make it possible to 
better understand the promise they present for our times and for the 
future.
 
     Describing a previous technological leap, the American 
anthropologist Lewis Henry Morgan called the production of iron "the 
event of events in human experience....Out of it," he wrote, "came the 
metallic hammer and anvil, the axe and the chisel, the plow with an iron 
point, the iron sword; in fine, the basis of civilization which may be 
said to rest upon this metal." Morgan maintained that until the human 
species had mastered the skills and methods necessary for forging iron, 
social development had been paused awaiting the achievement of this next 
necessary technological advance.  "The want of tools," he wrote, 
"arrested the progress of mankind in barbarism. There they would have 
remained to the present hour, had they failed to bridge the chasm." 
Frustrated by the lack of knowledge about how iron was invented, Morgan 
lamented, "It would be a singular satisfaction could it be known to what 
tribe and family we are indebted for this knowledge and with it for 
civilization."(1)

     We are more fortunate than our ancestors. We can know some of the 
details of whom to credit for the technological achievements of the 
computer revolution. Documenting their contributions while many of the 
pioneers are still alive will provide future generations with knowledge 
of whom they are indebted to for this new and stunning advance. Like the 
invention of iron, the technology of software production promises to make 
possible an important new level of civilization. Such knowledge will also 
help those who want to build on what has already been achieved.

Creating a Technology of Software Development
 
     In Introduction to Computer Science, Alan Perlis, one of the early 
pioneers of computer science, observed that the nature of computer 
programming is too often misunderstood. "Although computer programming is 
often inaccurately considered an art," he wrote, "it is actually an 
infant technology."(2)
 
     Like Morgan, Perlis believed that the progressive and accelerating 
ability to use and produce tools represents a defining element in the 
advance of human society. He observed that the rapid increase in the 
ability of the human species to use and mold new tools was guided by two 
laws. The first law describes the rapid increase in the number of people 
who can wield the tools and in the number of tools available. The second 
describes how the rapid development of tools to create tools guarantees 
the continuing expansion of the ability to use the tools. Perlis wrote:
  
     "First, the increase in the population trained to use tools 
     and in the number of tools available for their use implies 
     that the potential for tool utility grows as a product rather 
     than as a sum.  Second, the amount of composition -- that is 
     the number and uses of tools to produce new tools -- grows as 
     a part of the product and assures an even greater overall 
     growth in tool use."(3)
 
     Among the tools Perlis saw coming into common use "are those which 
perform automatically at great speed, and whose management must perform 
in the same way. Here," he explained, "the computer is essential because 
of the need for quick response."(4)

The Creation of Time-Sharing
 
     The development of the modern computer raised the question of how to 
manage its operation more efficiently. An important solution to the 
problem of computer management proposed in 1959 was the sequential and 
automatic interspersing of many tasks known as time-sharing.(5) Among the 
earliest time-sharing systems were those developed at MIT. By 1963, there 
were two versions of the Compatible Time Sharing System (CTSS) operating 
at MIT on two IBM 7094 computers, one at the Computation Center, and 
another at MIT's Project MAC.

     Those using the early time-sharing systems at MIT and elsewhere soon 
discovered the delights of interactive computing made possible by time-
sharing.(6) Describing the advantages of interactive computing, time-
sharing pioneers Robert Fano and Fernando Corbato, wrote:
 
    "For professional programmers the time-sharing system has come to 
    mean a great deal more than mere ease of access to the computer. 
    Provided with the opportunity to run a program in continuous dialogue 
    with the machine, editing, `debugging' and modifying the program as 
    they proceed, they have gained immeasurably in the ability to 
    experiment. They can readily investigate new programming techniques 
    and new approaches to problems."(7)
 
     The results of this programming flexibility led both to a bolder and 
more flexible approach to problem solving and to undertaking new areas of 
research. Fano and Corbato reported that users not only would build on 
each other's work, but also they would come to depend more and more on 
the computer to facilitate their work. The most surprising development 
that they encountered, however, was the fact that the users themselves 
created many of the programming commands used in the system, instead of 
needing professional programmers. While at the conventional computer 
installation, they noted, "one hardly ever makes use of a program 
developed by another user, because of the difficulty of exchanging 
programs and data," in the Project MAC time-sharing environment, "the 
ease of exchange has encouraged investigators to design their programs 
with an eye to possible use by other people. They have acted essentially 
as if they were writing papers to be published in technical journals."(8)
 
     Fano and Corbato envisioned that time-sharing systems would 
have a profound impact on the future. "Communities will design 
systems," they predicted, "to perform various functions -- 
intellectual, economic and social -- and the systems in turn 
undoubtedly will have profound effects in shaping the patterns of 
human life."(9) "The coupling between such a utility and the 
community it serves," they discovered, "is so strong that the 
community is actually a part of the system itself." They foresaw 
the development of a symbiotic relationship between the computer 
systems and its human users which "will create new services, new 
institutions, a new environment and new problems." Among these, 
they proposed, would be the question of access. "How will access 
to the utility be controlled?" they asked, "To what ends will the 
system be devoted, and what safeguards can be devised for its 
misuses? It is easy to see," they concluded, "that the progress 
of this new technique will raise many social questions as well as 
technical ones." (10)
 
     Others during this period were concerned with the impact the 
computer would have on current society. For example, John McCarthy 
predicted that, "The computer gives signs of becoming the contemporary 
counterpart of the steam engine that brought on the industrial 
revolution."(11) Unlike the steam engine, however, the utility of the 
computer was dependent on the successful development of software programs 
written to direct it. Therefore, along with the increasing speed and 
capacity of computer hardware, came the increase in the demand for and in 
the cost of software. By the mid 1960's, the U.S. government was spending 
increasing amounts of money to create programs to utilize computer 
resources. The U.S. government, wrote McCarthy, "with a dozen or so big 
systems serving its military and space establishments, is spending more 
than half of its 1966 outlay of $844 million on software."(12)
 
     Pointing out the need for studying the processes of programming, 
McCarthy observed, "What computers can do, depends on the state of the 
art and the science of programming as well as on speed and memory 
capacity."(14) Computer pioneers like McCarthy recognized that the 
computer was more than an efficient bookkeeping machine. There was a need 
to discover what new applications were possible, and to create these new 
applications. Therefore, there would be a need for breakthroughs in the 
process of programming software. McCarthy believed that it was important 
for the user to be able to program in order to realize the potential of 
the computer. He pointed out that programming was a skill that was not 
difficult to learn, and that it was more important to understand the task 
being automated than to master programming languages. "To program the 
trajectory of a rocket," McCarthy offers as an example, "requires a few 
weeks' study of programming and a few years' study of physics."(14)
 
     These early explorations in time-sharing prepared the foundation for 
an important development in the process of creating software. Once the 
discovery was made that simple programming tools could be created to aid 
in the process of software development, and that such tools could help 
those who understood the tasks to be automated, a needed leap could be 
made in how to develop software. Such a program was to be carried out by 
research programmers and developers at Bell Labs in the 1970's and early 
1980's, building on the principles developed by the pioneers of time-
sharing and Project MAC.
 
The Multics Collaboration

     In 1964, MIT joined with GE and AT&T in a project designed to 
implement time-sharing by developing a new computer and a new operating 
system. The joint research project among GE, MIT, and AT&T was created to 
extend time-sharing techniques from a pilot program into a useful 
prototype for the future information utility.(15) The researchers 
realized that there was no existing computer that would meet the demands 
of time-sharing. Therefore part of the goal of their collaboration was to 
make it possible to develop a new computer as well as a new operating 
system. 
 
     The collaborative project was called Multics [Multiplexed 
Information and Computing Service] and was to be implemented on the GE 
645 computer.(16) Technical leadership of the project included F. J. 
Corbato from MIT and V.A. Vyssotsky from Bell Labs. "One of the overall 
design goals is to create a computing system," they wrote, "which is 
capable of meeting almost all of the present and near-future requirements 
of a large computer utility.  Such systems, must run continuously and 
reliably 7 days a week, 24 hours a day in a way similar to telephone or 
power systems, and must be capable of meeting wide service demands: from 
multiple man-machine interaction to the sequential processing of 
absentee-user jobs..."(17)
 
    The goal of the research was to produce a prototype time-sharing 
system. Berkley Tague, one of the Bell Labs researchers involved in the 
Multics project writes, "The Multics Project was a joint project of Bell 
Labs, the GE Computer Systems Division, and MIT's Project MAC to develop 
a new computer and operating system that would replace MIT's CTSS system, 
Bell Labs BESYS, and support the new GE machine."(18) Though AT&T 
withdrew from the project in 1969, the joint work achieved significant 
results. Summarizing these achievements, Tague writes, "Multics was one 
of the seminal efforts in computing science and operating system design. 
It established principles and features of operating system design that 
are taken for granted today in any modern operating system."(19)
 
The Need for Operating System Research at Bell Labs 

     Even though AT&T withdrew from the research collaboration on 
Multics, computer scientists at Bell Labs wanted some way of continuing 
the advanced form of programming research that their work with CTSS and 
Multics had made possible. As early as 1957, Bell Labs had found they 
needed an operating system for their inhouse computer center which was 
then running lots of short batch jobs. Describing the situation then 
facing the Labs, Vyssotsky explains, "We just couldn't take the time to 
get them on and off the machine manually. We needed an operating system 
to sequence jobs through and control machine resources."(20) The BESYS 
operating system was created at Bell Labs to deal with their inhouse 
needs. When asked by others outside the Labs to make a copy available, 
they did so but with no obligation to provide support. "There was no 
support when we shipped a BESYS tape to somebody," Vyssotsky recalls, "we 
would answer reasonable questions over the telephone. If they found 
troubles or we found troubles, we would provide fixes."(21)

     By 1964, however, Bell Labs was adopting third generation computer 
equipment and had to decide whether they would build their own operating 
system or go with one that was built outside the Labs. Vyssotsky recounts 
the process of deliberation at the time, "Through a rather murky process 
of internal deliberation we decided to join forces with General Electric 
and MIT to create Multics." He explains that the Labs planned to use the 
Multics operating system, "as a mainstay for Bell Laboratories internal 
service computing in precisely the same way that we had used the BESYS 
operating system."(22)

Unix Is Born and the Introduction of Pipes
   
     When AT&T made the decision to pull out of the Multics 
collaboration, they took the research operating system off their GE 645 
computer and put up the GECOS operating system. Though GECOS was adequate 
for applications, it was "nowhere near as satisfactory if you were trying 
to do things that were technically difficult and imperfectly defined," 
explained Vyssotsky, "which is the main task of research."(23)

      For the pioneering work of Bell Labs research programmers like Ken 
Thompson and the research purposes of the Labs, an operating system more 
like what Multics had promised was needed. Along with the advantages of 
immediate feedback which time-sharing provided, the Bell Labs researchers 
wanted to continue the ability to work collaboratively which time-sharing 
had made possible. 

     "What we wanted to preserve," one of the creators of Unix, Dennis 
Ritchie writes, "was not just a good programming environment in which to 
do programming, but a system around which a fellowship could form. We 
knew from experience that the essence of communal computing, as supplied 
by remote-access, time-shared machines, is not just to type programs into 
a terminal instead of a keypunch, but to encourage close 
communication."(24)

     Ritchie describes how an informal group led by Thompson had begun 
investigating alternatives to Multics before the GE-645 Multics machine 
had been removed from the Labs.(25) Thompson and Ritchie presented Bell 
Labs with proposals to buy them a computer so they could build their own 
interactive, time-sharing operating system. Their proposals weren't acted 
on. Eventually, Thompson found a little used PDP-7 computer. According to 
Vyssotsky, the orphaned PDP-7 computer was a machine, "more in the class 
of a Commodore 64 than the class of a PC-AT."(26)

     Ritchie, Thompson, and Rudd Canady, who had been part of the Multics 
project, applied the lessons they had learned to the design of a file 
system for an experimental operating system. Writing on a chalk board, 
they created a file system design based on "the particularly simple way 
of viewing files that was suggested by the Multics I/O system."(27)
 
     "Soon," Ritchie recounts, "Thompson began implementing the paper 
file system (perhaps `chalk file system' would be more accurate) that had 
been designed earlier." Thompson was eager to get a working model so he 
could test it out. He proceeded to create the other aspects of an 
operating system. "A file system without a way to exercise it was a 
sterile proposition," notes Ritchie, "so he [Thompson] proceeded to flesh 
it out with the other requirements for a working operating system, in 
particular the notion of processes."(28)

     Describing the primitive conditions that Thompson faced, Ritchie 
writes, "At the start, Thompson did not even program on the PDP itself, 
but instead used a set of macros for the GEMAP assembler on a GE-635 
machine."(29) A paper tape was generated on the GE 635 and then tested on 
the PDP-7 until, according to Ritchie, "a primitive Unix kernel, an 
editor, an assembler, a simple shell (command interpreter), and a few 
utilities (like the Unix rm, cat, cp commands) were completed. At this 
point, the operating system was self-supporting, programs could be 
written and tested without resort to paper tape, and development 
continued on the PDP-7 itself."(30)
   
     Ritchie describes how Thompson's PDP-7 assembler was a model of 
simplicity. "There were no libraries, no loader or link editor," he 
writes, "the entire source of a program was presented to the assembler, 
and the output file -- with a fixed name -- that emerged was directly 
executable."(31) Once the assembler was completed, "the system was able 
to support itself. And thus the operating system we now call UNIX was 
born," notes Ritchie.(32)

    Among the other active contributors during this period were 
Bell Labs researchers Rudd Canady and Joe Ossanna. The 
researchers were anxious to continue their work on a more 
advanced computer than the PDP-7. However, their efforts to get 
AT&T to buy them a more advanced computer for their time-sharing 
research hadn't succeeded. With the help of Ossanna and another 
Bell Labs researcher Lee McMahon, they were finally able to 
convince management to buy them a new PDP-11 computer. To obtain 
this agreement, though, the researchers promised to create a text 
processing system. Doug McIlroy explains that "typists everywhere 
were seen as potential customers of the promised document-
preparation program. Only later did Ossanna spot the patent 
department as a ripe candidate. By then there had already been 
clients in the telephone-operations business."(33) By Spring 
1971, the Bell Labs Unix pioneers had a text formatter, 
fulfilling their promise. They had translated one they had 
created for the PDP-7 from a program McIlroy had written in the 
BCPL language while working on Multics. The text formatter was an 
assembler program for the PDP-11.(34)

     One of the important developments in Unix was the introduction of 
pipes. Pipes had been suggested by McIlroy during the early days of 
creating Unix. Ritchie explains how "the idea, explained one afternoon on 
a blackboard, intrigued us but failed to ignite any immediate action. 
There were several objections to the idea as put....What a failure of 
imagination," he admits.(35) McIlroy concurs, describing how the initial 
effort to add pipes to Unix occurred about the same time in 1969 that 
Ritchie, Thompson and Canaday were outlining ideas for a file system. 
"That was when," he writes, "the simple pipeline as a way to combine 
programs, with data notationally propagating along a chain of (not 
necessarily concurrent) filters was articulated."(36) However, pipes 
weren't implemented in Unix until 1972. "Thanks to McIlroy's 
persistence," Ritchie writes, "pipes were finally installed in the 
operating system (a relatively simple job), and a new notation was 
introduced."(37) Several of the old commands had to be changed to make 
them usable with pipes, notes Ritchie. Summarizing how pipes found their 
way into Unix, Vyssotsky notes that Thompson put them in, but "it was 
McIlroy who said, `look you ought to do it.' Pipes, like most things in 
Unix were not a radically new idea."(38) He describes how similar ideas 
had appeared in other languages like SIMULA as early as 1967.
 
     Dick Haight, a Unix pioneer who helped develop the 
Programmer's Workbench of Unix tools, was present the day that 
pipes were introduced into Unix. He describes what happened:
 
     "I happened to have been visiting the research crew the
     day they implemented pipes. It was clear to everyone 
     practically minutes after the system came up with pipes 
     working that it was a wonderful thing. Nobody would ever go 
     back and give that up if they could help it."(39)
 
Also describing the day pipes were introduced, McIlroy writes:
 
   "Open Systems! Our Systems! How well those who were
    there remember the pipe-festooned garret where Unix
    took form. The excitement of creation drew people to
    work there amidst the whine of the computer's cool-
    ing fans, even though almost the same computer ac-
    cess, could be had from one's office or from home.
    Those raw quarters saw a procession of memorable
    events. The advent of software pipes precipitated a
    day-long orgy of one-liners....As people reveled in
    the power of functional composition in the large,
    which is even today unavailable to users of other
    systems."(40)

The Software Tool
 
     Pipes had been created by the time the Version 3 Unix Manual 
appeared in February 1973. The date listed for the creation of pipes is 
January 15, 1973(41). Not only were pipes a significant addition to Unix, 
but according to McIlroy, pipes made possible a subsequent important 
discovery. "In another memorable event," he writes, "the unarticulated 
notion of software tools, which had been bolstered by pipes, was finally 
brought home by the liberation of the pattern matching program grep from 
within the editor."
 
     McIlroy describes how he asked Thompson to create a program to help 
him with some work that he was trying to do. This program resulted in the 
invention of the software tool `grep.' Following is McIlroy's account of 
how `grep' was taken out from the editor, leading to a clearer 
understanding of the notion of a software tool. He writes:
 
     "Grep was invented for me. I was making a program to read 
     text aloud through a voice synthesizer. As I invented 
     phonetic rules I would check Webster's dictionary for words 
     on which they might fail. For example, how do you cope with 
     the digraph `ui', which is pronounced many different ways: 
     `fruit', `guile', `guilty', `anguish', `intuit', `beguine'? 
     I would break the dictionary up into pieces that fit in ed's 
     limited buffer and use a global command to select a list. I 
     would whittle this list down by repeated scannings with ed 
     to see how each proposed rule worked."
 
     "The process was tedious, and terribly wasteful, since the 
     dictionary had to be split (one couldn't afford to leave a 
     split copy on line). Then ed copied each part into /tmp, 
     scanned it twice to accomplish the g command, and finally 
     threw it away, which takes time too."
 
     "One afternoon I asked Ken Thompson if he could lift the 
     regular expression recognizer out of the editor and make a 
     one-pass program to do it. He said yes. The next morning I 
     found a note in my mail announcing a program named grep. It 
     worked like a charm. When asked what that funny name meant, 
     Ken said it was obvious. It stood for the editor command 
     that it simulated, g/re/p (global regular expression print)."
 
     "Progress on my talking program accelerated dramatically.  
     From that special-purpose beginning, grep soon became a 
     household word. (Something I had to stop myself from writing 
     in the first paragraph above shows how firmly naturalized 
     the idea now is: `I used ed to grep out words from the 
     dictionary.') More than any other single program, grep 
     focused the viewpoint that Kernighan and Plauger christened 
     and formalized in `Software Tools': make programs that do 
     one thing and do it well, with as few preconceptions about 
     input syntax as possible."(42)

     Grep is listed in the Manual for Version 4 Unix which is dated 
November, 1973. The date given for the creation of grep is March 3, 1973, 
following the creation of pipes.(43) The creation of grep, McIlroy 
explains, was followed by the invention of other special purpose software 
programs that could be used as tools. He writes:
 
     "A while later a demand arose for another special-purpose 
     program, gres, for substitution:  g/re/s.  Lee McMahon 
     undertook to write it, and soon foresaw that there would be 
     no end to the family:  g/re/d, g/re/a, etc.  As his concept 
     developed it became sed, a tool of remarkable utility that 
     is largely unappreciated today, because it capitalizes on 
     the perfect familiarity with ed that was universal ten years 
     ago, but no more.  Sed covers a lot of needs.  For example, 
     we have never seen fit to adopt the widespread `head' 
     utility because `sed 10q' does just as well."(44)
 
     What McIlroy refers to as "the unarticulated notion of 
software tools...brought home by the liberation of...grep from 
within the editor" has become one of the significant achievements 
of the Bell Labs research work on Unix. By making it possible to 
use `grep' to search for a data pattern in a file, without having 
to use an editor to go inside the file, the Bell Labs researchers 
discovered that they could create a plethora of software tools 
to be used in varying combinations, thus facilitating the 
customized application by a user, a goal sought by those doing 
research in programming.(45)
 
     McIlroy explains that the notion of "software tool" only became 
articulated among the Bell Labs researchers with the publication of the 
book "Software Tools" by Kernighan and Plaugher. "Still unnamed in our 
circles," McIlroy notes, "until Kernighan and Plauger wrote their book, 
the idea nevertheless became a guiding principle."(46) McIlroy adds that 
"We were definitely building tools before K&P, though we lacked the 
suggestive word."(47)

     Describing how the notion of software tools helps to create an 
effective programming environment, Brian W. Kernighan and Rob Pike, 
authors of The Unix Programming Environment, explain that each tool is 
designed to be used with other tools and achieving this end is more 
important than how each is designed internally.(48) The most important 
aspect considered with each tool is the function that the tool is to 
have. New tools are designed once a new function is intended.

     "The guiding principle," they write, "for making the 
choice should be that each program does one thing. Options are 
appropriately added to a program that already has the right 
functionality. If there is no such program then a new program is 
called for. In that case the usual criteria for program design 
should be used: the program should be as general as possible, its 
default behavior should match the most common usage and it should 
cooperate with other programs."(49)
 
     Thus Unix, according to Kernighan and Pike, created "a
new style of computing, a new way of thinking of how to
attack a problem with a computer. This style," they explain,
"was based on the use of tools: using programs separately or
in combination to get a job done, rather than doing it by
hand, by monolithic self-sufficient subsystems, or by
special-purpose, one-time programs."(50)
 
     The philosophy of using software tools that developed from 
research in Unix is outlined in the "Foreward" to the special 
issue of The Bell System Technical Journal published in 1978, 
on "The Unix Time-sharing System." Describing the principles 
which they have found to be an important component of the Unix 
philosophy of software design, the researchers write:
 
     "UNIX utilities are usually thought of as tools -- sharply 
     honed programs that help with generic data processing tasks. 
     Tools were often invented to help with the development of 
     UNIX programs and were continually improved by much trial, 
     error, discussion, and redesign, as was the operating system 
     itself. Tools may be used in combination to perform or 
     construct specific applications."(51)
 
     They explain that a distinctive style evolved as part of 
Unix research. "Unix software works smoothly together; 
elaborate computing tasks are typically composed from loosely 
coupled small parts, often software tools taken off the 
shelf."(52)
 
     "Sophisticated tools to make tools have evolved," they 
observe. (53) Software development tools such as "nroff"
and "troff" were created. Not only was it important to create 
tools, but soon tools to create tools, like "yacc" and "lex",
were developed. "Yacc" and "lex" were used to create numerous 
little languages and applications like eqn and awk that greatly 
enhanced the popularity of Unix. 

     The evolution of the tool "diff" created by McIlroy, a commonly used 
software tool, is an example of how tools were continually improved, 
based on the experience gained using them. McIlroy reports how he based 
his work on algorithms created by others and then he tried three 
different algorithms before settling on the one finally used.(54) Other 
Unix tools were created through a similar process. Each program was 
created to fulfill some simple capability and was called a tool. The 
programs were designed to be fun to use and to be helpful to programmers. 
Among the principles guiding the toolbuilders were:

     "(i) Make each program do one thing well. To do a new job, 
     build afresh rather than complicate old programs by adding 
     new features.

     (ii) Expect the output of every program to become the input 
     to another, as yet unknown, program. Don't clutter output 
     with extraneous information. Avoid stringently columnar or 
     binary input formats. Don't insist on interactive input.

     (iii) Design and build software, even operating systems, to 
     be tried early, ideally within weeks. Don't hesitate to 
     throw away the clumsy parts and rebuild them.

     (iv) Use tools in preference to unskilled help to lighten a 
     programming task, even if you have to detour to build the 
     tools and expect to throw some of them out after you've 
     finished using them."(55)
 
     "Our goals throughout the effort," write Ritchie and 
Thompson, in describing the research objectives of their work on Unix,
"when articulated at all, have always been to build a comfortable 
relationship with the machine and to explain ideas and inventions 
in operating systems and other software." (56)
 
     Frederick P. Brooks Jr is another computer pioneer who recognized 
the importance of pipes and combining single function programs into 
what he calls "a unified programming environment" to make the 
work of programmers more productive. Brooks describes Unix as one 
of "the first integrated environments to come into widespread 
use," which "improved productivity by integral factors."(57)

     In explaining how such an environment functions, he writes:
 
    "They attack the accidental difficulties that result 
     from using individual programs together, by providing 
     integrated libraries, unified file formats, and pipes and 
     filters. As a result, conceptual structures that in 
     principle could always call, feed, and use one another can 
     easily do so in practice." (58)
 
     Following the development of single function software tools 
which could be used together via pipes in a programming 
environment, came the development of whole toolchests and 
workbenches of tools to rationalize the work of programmers.  

Open Source Code and the Early Development of Unix

     Since the Unix programming environment was used by the 
researchers themselves, they came to learn its weaknesses and 
problems and were encouraged to make needed improvements. 
Contributing to the value of Unix during its early 
development, was the fact that the source code was open and 
available. It could be examined, improved, and customized.
In an article "Can UNIX Survive Secret Source Code?", Mike Lesk, 
then a Bell Labs researcher, observes how only when computer 
source code is open and can be modified, will it be developed and 
vibrant.  He gives the example of COMIT, the string processing 
language. At one point, its owners decided there would no longer 
be any modifications in the code, and so only distributed it in 
binary form. "You haven't heard of COMIT since," he notes. He 
describes how the same fate befell TRAC, a language close to 
FORTH. "Software is more attractive to hackers," Lesk maintains, 
"if it can be changed. As more and more UNIX suppliers restrict 
access to source code, fewer and fewer advanced research shops 
will be attracted to UNIX."(59)
 
     Commenting on the importance of open source code in the 
early years of Unix development at Bell Labs, Thompson and 
Ritchie write, "Because all source programs were always available 
and easily modified on-line we were willing to revise and rewrite 
the system and its software when new ideas were invented, 
discovered, or suggested by others."(60)
 
    Not only was the source code open and available to the Bell 
Labs researchers developing the system, but the Labs also made 
the sources available on tape at the request of academic 
colleagues. Robert Fabry of the University of California at 
Berkeley was able to get a tape of Version 4 Unix and that began 
the long and important role played by faculty and students at the 
University of California at Berkeley in the development of 
Unix.(61)

     Source code tapes were made available to interested 
researchers in the academic and computer science community for a 
nominal fee. For example, when John Lions, a Professor of 
Computer Science at the University of New South Wales in 
Australia read a paper published by Thompson and Ritchie in mid 
1974, he wrote them for a copy of the Unix tape. After signing a 
license agreement with the Labs, and a token payment of $110 
Australian ($150 U.S.), the Unix Edition 5 tape and manuals 
arrived, "as a late Christmas present," in December, 1974, 
remembers John Lions.(62)
 
     While Bell Labs made the tape and manuals available,  
they did so with no support. Berkley Tague explains the 
release was "`caveat  emptor' -  i.e. dollars on the table up 
front and no support promised."
 
     Henry Spencer, a Unix pioneer from the University of 
Toronto in Canada, and one of the programmers of C News, 
describes how early users of Unix in the academic community
had to provide their own support. He explains:
 
    "It was very common at the time. This was in the
    days when UNIX was still treated by the Bell System
    as, `Oh, just something we happen to do for our own
    internal use. You can have a copy if you want, but if
    you got problems, don't bother us.' And the result
    was if you wanted UNIX support you did it yourself
    or it didn't happen."(63)
 
    Lions agrees, "We needed help," he notes, "but we
couldn't get any from outside sources so we ended up
generating our own expertise."(64) Not only did those working 
on Unix implementation at the University of New South Wales
have access to the code, but Lions explains how Ian Johnstone,
the tutor working with him in his Operating Systems class, 
suggested making some of the code for the Unix kernel available
to the students in his class. "I think it was in 1975," 
remembers Lions, that Ian Johnstone asked, "`Why don't we run 
off a few of the source files for the kernel and ask the students 
to take a look at them? Then we can ask them some questions; 
maybe it will be interesting'."(65) Lions took Johnstone's 
suggestion and made some of the Unix source code available to his 
class, but his students complained. They felt they needed to see 
the source code for the whole kernel in order to make sense out 
of any part.
 
     Taking their suggestion, Lions decided to make a large part of the 
source code available to his class. "The following year," he recounts, "I 
prepared a booklet containing the source files for a version of Edition 6 
UNIX that could then run on a PDP-11/40 system." Lions followed the book 
of source code with a book of "explanatory notes that was intended to 
introduce students to the code."(66) Lions explains that working on his 
book, "`A Commentary on the UNIX Operating System' was a real learning 
experience. By slowly and methodically surveying the whole kernel he 
notes, "I came to understand things that others had overlooked."(67)
 
     When he read the manual and wasn't quite certain about his 
interpretation, Lions would read the code. Through this process, 
he was able to determine that the manual was "really quite 
accurate in its description of what a program actually does. In 
the Thompson/Ritchie era," he observes, "words were carefully 
chosen."(68) Lions writes, "In our opinion, it is highly 
beneficial for students to have the opportunity to study a 
working operating system in all its aspects."(69)
 
     "Moreover," he adds, "it is undoubtedly good for students majoring 
in computer science to be confronted at least once in their careers with 
the task of reading and understanding a program of major dimensions."(70) 
Lions found that, "On the whole the authors of UNIX, Ken Thompson and 
Dennis Ritchie, have created a program of great strength, integrity and 
effectiveness," which he urged his students to "admire and seek to 
emulate."(71) Not only did students in Lions' class read and study the 
Unix source code and Lions' Commentary on the source code, but Lions 
describes how he sent more than 200 copies of his book to Bell 
Laboratories. Eventually, the Labs took over distribution of the book.
 
     Tague relates how Lions' book of commentary and the 
Unix source code were used at AT&T "as part of the documentation 
package for those who wanted to understand or modify the UNIX(r) 
source code that the USG [Unix Support Group] shipped."(72)
Even after Unix V6 had been replaced by Unix V7, Tague explains 
that Lions' Commentary continued to be useful as an introduction 
to Unix. Tague writes:
 
     "It outlined the conceptual architecture, very clearly in 
     the short form of the system before it had accreted all the 
     minor changes and feature additions that disguised the  
     original clarity of its structure. All new people were given 
     a copy when they joined the USG. And I suspect most 
     development groups did the same."(73)
 
     Pioneers like Henry Spencer describe how important it was to 
those in the Unix community to have the source code. He notes how 
having the sources made it possible to identify and fix the bugs 
that they discovered, "There is something the UNIX community has 
always been fairly strong on," he explained during an interview, 
"admitting things you know don't work about the software." Even 
in the late 1970's and early 1980's, remembers Spencer, 
"practically every UNIX site had complete sources."(74)
 
     One of the early functions of Usenet, the early online community
of Unix systems begun in 1979, according to Spencer, was to provide
cooperative software support for the Unix community. He elaborates:
 
     Well, for one thing, Usenet predated a lot of company bbs's 
     and the like. It was basically a cheap way to hear about 
     things fast and this was at a time when practically every 
     UNIX site had complete sources and so a bug report often 
     came with a fix. It was a way of finding out what people had 
     discovered and what fixes they'd worked out for it. Quickly 
     and easily. And for that matter, if you ran into something 
     that you couldn't solve yourself, putting out an inquiry to 
     a bunch of fairly bright people who were fairly familiar 
     with the code, often got a response, `O Yeah. We solved that 
     one' or `You're right. There's a bug. Here's how to fix it' 
     or sympathy even if no one had a fix for it."(75)
 
     Another Unix pioneer, Dick Haight, corroborates the 
important role open source code played for those in the Unix 
community: 

     "That by the way, was one of the great things about UNIX in 
     the early days: people actually shared each other's stuff. 
     It's too bad that so many sites now have purposefully turned 
     off the read privileges in order to keep their ideas from 
     being stolen. Not only did we learn a lot in the old days 
     from sharing material, but we also never had to worry about 
     how things really worked because we always could go read the 
     source. That's still the only thing that matters when the 
     going gets tough."(76)
     
     Unix was continually used and improved by its creators. A growing
community of programmers, system administrators, and users, both at
Bell Labs and at academic and research sites outside the Labs, also
used, developed and debugged the code. The fact that the source code
was open and available made this possible. The result was the
creation of Unix as a powerful and flexible programming environment.

     Though Unix was primarily designed to "help build research  
software," Bell Labs software developers soon found that, "what 
worked well in the programming laboratory also worked well on 
modest projects to develop minicomputer-based systems in support 
of telephone company operations."(77)

AT&T Automates Internal Operations

    In the late 1960s, there had been an upsurge in demand for 
telephone company service which taxed all AT&T's resources.

     The 1970 annual report of AT&T described the crisis:
 
    "The company concluded that the convergence of a number of adverse 
    factors contributed to the crisis, including an unforecasted rapid 
    upsurge of demand for business traffic on a poor switching system, 
    faulty trunk maintenance and administration, inadequate training of 
    maintenance personnel and poor main distributing frames 
    administration."(78)
 
     Section 1 of the U.S. Communications Act of 1934 charges the Federal 
Communications Commission (FCC) with promoting a "rapid, efficient, 
nationwide and worldwide wire and radio communications service with 
adequate facilities at reasonable charge."(79) This legislative 
obligation put pressure on AT&T to create and support the Bell Labs 
efforts to push forward the forefronts of computer science, since the 
most advanced technology and science were needed for AT&T to meet its 
obligations as a regulated communications utility.
 
     Under pressure from both the New York Public Service 
Commission (NYPSC) and the FCC, AT&T had to find a way to upgrade 
their service.(80) In response to these requirements, a push
began within AT&T to "use the computer to support the operation
of the Bell System Network."(81) Berkley Tague, who would be
instrumental in implementing this change at AT&T, describes
the broad scope of the effort that AT&T instituted to deal with
the service crisis. He lists: "Monitoring and alarms, maintenance
and staff management, inventory and order control, revenue data
collection and billing, traffic measurement and control, circuit
provisioning,(82) etc." and he provides the following illustration:
 
    "The data that has to be collected to bill for calls was 
     being stored on a variety of media -- e.g. paper AMA tape, a 
     punched paper tape a few inches wide, IBM compatible mag 
     tape, and probably a few others. These tapes had to be 
     mounted, saved, trucked to DP centers for processing, 
     etc. They could be lost or damaged in the handling. An 
     obvious solution was to send the data over datalinks -- our 
     own network -- to the DP centers. Minicomputer-based 
     systems were designed to collect, preprocess and 
     transmit this data in some cases; in others, the   
     electronic switches themselves were programmed to do the 
     job. The goal was to eliminate the need for people  
     handling physical media -- a process that is both expensive 
     and error-prone."(83)
 
     Software was needed to facilitate these changes. Tague explains
how the Unix system, originally created for computer research, also
served the needs of program developers. "The difference," he explains,
distinguishing between the needs of researchers and the concerns of
developers involved "support, stability and reliability." He describes
how developers have to meet deadlines and so only tolerate absolutely
necessary changes to the system. A researcher, on the other hand, 
according to Tague, will "tolerate a fair amount of upset and 
change and revision if it advances the toolkit significantly, 
even if it means you must recode your work."(84)
 
    The use of Unix by developers for support operations began 
in the 1972 period. Tague describes how "a number of developers 
were planning to build their own operating systems - typically 
their first system and often their first major programs."(85)
Having experience with Unix, he advocated Unix for the job. It was 
an uphill battle to convince developers to adopt Unix when there 
was no offered support and it was made up of 12000 lines of 
undocumented code.
 
     In 1973, Tague pointed out the need for a central support
group and volunteered to form the group. He was given the job.
The goal of the USG which Tague created in September 1973 was to 
"provide the stability and support that would buffer developers 
from the experiments that researchers were daily imposing on the 
evolving UNIX system." He describes how Dennis Ritchie promised 
him a portable version of Unix, which was delivered as promised 
by October, 1973.(86) Recoding the Unix kernel from assembler 
language, which was specific to each computer, into the C 
programming language, was an important feat. Once the Unix kernel 
was coded in C, it would become possible to implement Unix on 
different computers without having to rewrite the majority of the 
code.
 
     The effort to make Unix machine independent was desirable, so that 
AT&T wouldn't be dependent on any one vendor for its computer hardware or 
software or service. He explains, "A goal of my effort was to remove the 
vendor lock so we could get competitive bids on the volume buys when we
deployed these mini-based systems across the Bell System."(87)
 
     Every project adopting Unix, Tague reports, added their own features 
and function to it and it became the responsibility of the USG to choose 
among or merge the variations. The University of California at Berkeley, 
according to Tague, performed a similar job with the variations of Unix 
developed by academic computer scientists outside of AT&T.
 
Creating a Programmer's Workbench

     While Tague was working on Unix Support Operations, other Bell Labs 
researchers were planning another major project that would impact on 
software development and on the use of a new form of tool.
 
     Rudd Canaday (who had designed the original Unix file system 
with Thompson and Ritchie), Dick Haight, Ted Dolotta, John Mashey, 
and others formed the Programmer's Workbench Group (PWB) within the 
Business Information Systems (BIS) project.(88) Tague explains that 
the "BIS problem was to get a common `workbench' that would drive 
code onto any of the three or four commercial vendor's mainframes 
that was used by BIS." Tague writes that "By putting a UNIX(r) 
system in front of the large mainframe systems, developers got the 
advantages of UNIX(r) in developing code and a place they could 
store debugged standard command sequences that drove development 
and testing on the target mainframes."(89)
 
   David Nowitz, a Unix pioneer who helped with the development
of UUCP, and then went on to collaborate in the creation of 
honeydanber uucp, explains that when he hired into Bell 
Labs in 1969, there were programmers working on many different 
computers with different languages.(90) The problem these 
different computers represented for those at AT&T, is described 
by Haight:
 
    "We had a real problem to solve at the time. For one thing, 
     we had a fairly large group of software developers at the 
     Labs working on several different main frames. The biggest 
     group, of course, consisted of people working on the IBM 
     360s or 370s. The programmers working on the IBM 360 or 370 
     had to contend with batch processing and IBM's Job Control 
     Language (JCL) problems. Other programmers were working with 
     the UNIVAC and another group with xerox computers. Only a 
     few were using early but expensive time sharing like TSO 
     (IBM's timesharing) and Univacs Remand Service."(91)
 
     "These systems," Haight relates, "not only offered very 
different programming environments, but proved to be very 
expensive to use and very unfriendly." Describing the task they 
were charged with, Haight continues, "Basically, we ended up trying 
to give all these people a cheap text-editing front end that they 
could use for interactive program entry. For that to happen, we 
needed to develop a programming environment that was consistent 
with what they already were using."
    
     They developed a software subsystem which included five 
components that all of the programmers working on the different 
computers could use. The subsystem which they created was called 
the PWB. It included RJE (Remote Job Entry), an editor, a program 
for tracking version changes, a word processor and software to 
simulate test input.(92) Haight believed that Unix was "the best 
program development system around" and that this "development 
environment notion was the very concept that got PWB going back 
in '73."(93)
 
     The vision of the PWB/UNIX was conceived of in mid April 
1973 and installed on the first workbench machine in October 1973 
on a PDP 11/45. By 1977 the PWB was serving 1000 users.(94) The 
programmers working on the PWB rejected the top down software 
design philosophy, i.e. of creating a fully integrated facility 
and implementing it, and then making it available to users. 
Instead they adopted a bottom up philosophy. They designed 
independent single function tools and made them available to 
users as early as possible. Users were encouraged to provide 
feedback. The PWB programmers had the actual feedback to help 
them determine what modifications were needed to improve the 
tools. Evan Ivie, a Bell Labs researcher who was one of the 
creators of the PWB, describes why the PWB/UNIX group adopted the 
bottom up philosophy of software design:
 
     "The idea of designing and building a complete and fully 
     integrated workbench system was rejected for a number of 
     reasons, not the least of which is the fact that no one 
     knows what that system should look like at this point in 
     time. Instead, every effort was made to identify some of the 
     immediate needs of potential users and to develop pieces of 
     the workbench that could satisfy those needs quickly. This 
     approach provided the workbench designers with much valuable 
     user feedback quickly, and it allowed the projects to start 
     picking up pieces of the Workbench to satisfy their most 
     critical needs immediately and thus to start a phased 
     transition to the complete workbench."(95)
 
     The design for implementing Unix on the IBM System/370 was 
done in 1979, coding was done in 1980, and by early 1981, the 
first production system IBM 3033AP was installed at Bell Labs 
at Indian Hill. Unix had been suggested as the development system 
for programmers for the switching system software for the 5ESS 
switch because the Programmer's Workbench provided "editors, 
source code control and software generation systems."(96)
 
     In 1981, after a production Unix system was running on an 
IBM 3033AP in Indian Hill, the application software tools were 
ported from the PDP-11/70 computer to the 3033AP. Shell
scripts were carried over and then users were transferred.(97)
 
     By 1984 the largest user of the Unix system for the 
IBM System/370 was the development project for the 5ESS Switch.
The creation at AT&T Bell Laboratories of software for
the 5ESS switch required millions of lines of code.(98) The Unix 
operating system with the Programmer's Workbench software was 
chosen for the 5ESS switch software development project because 
the researchers felt that Unix "provided a better development 
environment than any other operating system available. In 
addition, the developers were all trained in using 
this system and all the software tools had been developed."(99)

     The elaborate process of applying Unix to the IBM 3033AP is 
described by the researchers:
 
     "Over 300 tools, written in both C and shell command 
     language, were identified and examined. After careful study, 
     almost half of the tools were found to be little-used and 
     were eliminated for porting to the 3033AP. The C programs 
     required recompiling to generate objects that would run on a 
     3033AP; in general, they compiled without problems. The 
     shell scripts were carried over with almost no problems. 
     (After testing) System testing, which consisted primarily of 
     generating the system software for the 5ESS switch was then 
     done. The effort to port the application tools was small and 
     again proved the strength and computer independence of the 
     UNIX operating system and the associated application 
     programs."(100)
 
     The rationale leading to the creation of the Programmer's 
Workbench, is outlined by Ivie. He wrote:
 
     "Although the computer industry now has some 30 years of 
     experience, the programming of computer-based systems 
     persists in being a very difficult and costly job. This is 
     particularly true of large and complex systems where 
     scheduled slips, cost overruns, high bug rates, insufficient 
     throughput maintenance difficulties, etc., all seem to be 
     the rule instead of the exception. Part of the problem stems 
     from the fact that programming is as yet very much a trial 
     and error process."(101)
 
      There are at this point, he observed, "only the beginnings of 
a methodology or discipline for designing, building and testing 
software. The situation is further aggravated by the rapidly 
changing hardware industry and by the continuing evolution of 
operating systems which continues to nullify much of the progress 
that is made in the development of programming tools. What can be 
done," he asked, "to move the programming industry toward a more 
professional and stable approach to software development?"(102)
 
     After enumerating several of the alternatives, he explained that his 
proposal involved "a very different approach to improving the development 
process." His recommendation was "... that the programming community 
develop a program development `faculty' (or facilities) much like those 
that have been developed for other professions (e.g. carpenter's 
workbench, dentist's office, engineer's laboratory)." Such an approach, 
he explained, "would help focus attention on the need for adequate tools 
and procedures; it would serve as a mechanism for integrating tools into 
a coordinated set; and it would tend to add stability to the programming 
environment by separating the tools from the product (the current 
approach is equivalent to carpenter's leaving their tools in each house 
they build)."(103)
 
     The Business Information Systems Programs (BISP) area of Bell 
Labs which was to use the Workbench program had among the computers
it used: an IBM 370/158 and 168, a Univac 1108 and 1110, two Xerox
Sigma 5's and several minicomputers.(104) The Workbench 
encouraged the development of machine independent programming 
tools. "Each tool must now function," Ivie explained, "for 
programmers developing code for a number of different vendor 
machines. There is no room for shortcuts which are dependent on 
the idiosyncrasies of a given machine (or a given project.) One 
is thus forced into a more stable and generalized software 
development approach which should be more applicable to new 
machines."(105)
 
     Summing up the rationale of the Programmer's Workbench 
philosophy, Ivie wrote that "the programming profession has yet
to produce a software development methodology that is sufficiently
general so that it can be transferred from one project to another
and from one machine to another." With the development of a 
Programmer's Workbench, he hoped, "a machine dedicated to the 
software development and maintenance function can serve as a 
vehicle for the development of such a methodology."(106)

Rationalizing the Process of Software Production

     In their article published in Scientific American in 1966, Fano and 
Corbato noted that the creation of a working computer in 1944 was 
followed by a series of quantum leaps in both the speed of the 
electronics and the development of special languages which facilitated 
human-computer communication. In another article in the same special 
issue of Scientific American, John McCarthy identified the importance of 
being able to program for making use of the computer. He also noted that 
spending time mastering a programming language was less important than 
understanding the functions the software is to serve. Their descriptions 
demonstrate that the computer is an all-purpose machine that can be 
directed to do whatever tasks humans can conceive of programs to 
describe. This poses a dilemma. Since humans are still needed to 
create the directives for the computer, the challenge raised is 
how can the process of programming be made more rational and 
productive.

     Perlis had observed that the technical progress of a society 
was marked by the development of tools and the ever developing 
set of people capable of wielding those tools. He noted the 
continuing evolution of the process of producing tools. In the 
evolution of Unix, the need for software tools became evident and 
a series of tools was developed to meet these needs. The creation 
of time-sharing and the development of Unix as a time-sharing 
system has helped to create a community of people who not only 
use Unix, but who also continue its development. The 
contributions to the evolution of Unix by researchers at Bell 
Labs and then by those at the University of California at 
Berkeley, and other users and programmers around the world who 
continued the development of Unix, are an important achievement 
in the development of the process of software creation.

    Writing in 1975, one of the team of researchers creating the 
PWB, P.J. Plauger, observed that "building on the work
of others is the only way to make substantial progress in any 
field."(107) Describing the process of creating software tools, 
Plauger explained that each tool is a small programming project 
requiring anywhere from a few man months to a few hours to 
complete. He proposed that these tools need to be designed in a 
standard way so that they can interface with and be combined in a 
useful way with other tools for a variety of different purposes. 
"The programmer working this way becomes a tool builder, and his 
impact extends far beyond the task he may originally have set out 
to solve," he concluded.(108)
 
     Placing the creation of software tools into the broader context of 
tool production provides a basis to evaluate the achievements and 
problems of this new stage. Sigvard Strandh, in the History of 
the Machine, describes how mechanical hand tools had acquired a 
stable form at an early stage. "From the Middle Ages until the 
era of industrialism," he writes, "the tools of the carpenter, 
joiner and smith remained more or less the same." With the 
publication of the Great French Encyclopedia (between 1751 and 
1780), by Diderot and d'Alembert, however, Strandh observes that 
the development of these tools was given a substantial boost. 
Publication of the descriptions of the tools used in all the 
trades extent at the time, in great detail, in the Great French 
Encyclopedia, made it possible for those interested to learn how 
to build the tools, "so as to establish himself in one of the 
trades described."(109)

     The publication in the Great French Encyclopedia made 
possible the constant improvement of the old hand tools, and thus 
made it possible to manufacture the new tools needed for the new 
technical developments. "Industrialization," Strandh writes, 
"means the manual methods of manufacture are mechanized, that is, 
taken over by machines. The machine tool is the local outcome of 
the process. Mass manufacture of machine tools," which he writes, 
"has been called 'the industry of industries', didn't become 
widespread until the 1870's."(110)

     The conditions described by Strandh made possible advances in the 
technology of tool production in the 19th Century. These conditions 
provide a helpful perspective when examining the achievements of the Unix 
community in developing the software tool creation process. Bell Labs 
researchers early on recognized that software production required text 
editing in place of soldering, and that this substitution represented a 
significant challenge. The creation by Bell Labs programmers of software 
tools, making it possible to produce better software more efficiently, is 
an important development. This development is similar to the ability to 
improve hand tools that Morgan documents as the technological advance 
represented by the invention of iron. The success of writing the code for 
the 5ESS switch at AT&T demonstrates the achievement these software tools 
have made possible.(111)
 
     Generating software is different from creating hardware and it 
presents a particular challenge. You can't see software tools or touch 
them or handle them as you can mechanical tools. "Software is invisible 
and unvisualizable," Brooks points out, since "the reality of software is 
not inherently embedded in space." Therefore, he observes, "In spite of 
the progress made in restricting and simplifying the structures of 
software, these structures remain inherently unvisualizable and thus do 
not permit the mind to use some of its most powerful conceptual tools. 
This lack not only impedes the process of design with one mind, it also 
severely hinders communication among minds."(112)
 
     Therefore, there is a particular problem posed when creating, 
describing, or representing software tools. Diderot was able to catalog 
the tools and industrial processes of his times, but such a task is much 
more difficult when it comes to creating such an encyclopedia or 
presentation of software tools, or in creating the software component 
catalog that McIlroy once proposed. The development of UUCP and then 
of Usenet, as a means of cheap communication among those of the Unix 
community, made it possible to surmount some of the difficulties inherent 
in the process of software production.(113)

     The Usenet community was able to pioneer the creation of a 
cooperative network that helped facilitate communications among the Unix 
software community. "Though large institutions have been able to avail 
themselves of communications networks such as ARPANET, the UNIX community 
has made inexpensive electronic communication available to all its 
members via Usenet," remarked John Stoneback, a faculty member at 
Moravian College, in an article describing the important role 
that Unix played for members of the academic community.(114)

     Describing the importance of the creation of the Unix communication 
program UUCP, McIlroy writes that "before uucp remote login was the only 
kind of electronic connection between Unix systems. With uucp, every PDP 
11 with dial-up (not even dial-out) access became connected. E-mail and 
file transfer previously available only among the fortunate few sites 
on ARPANET, were democratized overnight. This was the brainchild of Mike 
Lesk."(115) Lesk's original plan in creating UUCP was to create an 
automatic software updating system.(116) That, however, was fraught with 
difficulties, and instead, with modifications to UUCP by David 
Nowitz, it became the basis for a broad ranging online computer 
network that came to be called Usenet. McIlroy, describing how 
uucp was used by Tom Truscott and other Usenet pioneers to create 
an online community, writes, "Soon Truscott saw that the new 
medium could be used for automatic dissemination of information, 
and netnews was born."(117)
 
    The Unix system is based on a methodology that helps the 
user to learn the new language. The methodology is similar to the 
way children learn to talk. As Unix pioneers Brian Kernighan and 
John Mashey, write, "The code that people see, adapt, and imitate 
is usually well structured. People learn to code well in the same 
way that they learn to speak their native language well by 
imitation and immediate feedback."(118)
 
     "People often look at each other's code," they explain,
"comment on it in person and through interuser communication 
facilities, and take pieces of it for their own use. The ideas of 
programming teams and egoless programming fit into the Unix 
environment well, since they encourage sharing rather than 
isolation." (119) Given the frequent sharing and building on each 
others code, though, "some programs have always been `owned' by 
one or two people, many others," however, they note, "have been 
passed around so much that it is difficult to tell exactly who 
wrote them...."  Also, they point out that "Tools have been built
to automate many common programming tasks."(120)
 
     Kernighan and Mashey explain how in its early days, up 
through 1974, "Unix best supported a single, cooperative, tightly 
coupled group of people on each machine." By 1975, the PWB/UNIX 
made possible support for a larger number of users. They describe 
how there have been large groups of over 200 users working on 
different machines, but that users preferred "to share a machine 
so they could share procedures and data bases."(121) They 
describe how the concept of software tools spread from the Unix 
Programming environment and PWB/UNIX to more generalized 
applications described by Kernighan and Plauger in their book 
Software Tools (1976), then to further applications that were 
ported to other machines, and then to the more generalized 
applications of the Lawrence Berkeley Laboratory (LBL).(122)

Documenting Technological Progress

    In writing about the role that technology plays in the 
development of society, Morgan observed that oral or written 
documentation helps to facilitate technological progress. 
Morgan described how the Greek poet Homer by documenting the 
civilization that existed prior to his times, provided future 
generations with a yardstick by which they could measure their 
own accomplishments.(123)
     
   So, too, documentation of the achievements of the Unix pioneers and of 
the tools they created, is an important step to be taken by those 
wanting to understand the progress made in our times. When 
amateur scientists created the Royal Society in Great Britain in 
the 1660's, one of their early efforts was to invite Rev. Spratt 
into the Society and to support him to write a history of the 
documentation of technology from Homer to their own times. 
Diderot's work on the Great French Encyclopedia, in the years 
leading up to the French Revolution, played such a role 
summarizing the knowledge, including, most importantly, the 
summary of all the trades and industries with their tools 
existing up to his time in France. 
   
     Unix pioneers often emphasize that scientific progress can only be 
made by building on what has already been achieved. It is important, 
therefore, to recognize the legacy of how Unix made it possible to 
achieve programming projects involving millions of lines of code by 
creating a workbench of software development tools. The creation of a new 
form of tool requires a new form of tool production process and 
though such technology is only in its infancy, there are examples 
to be studied to know what we are building on. As Henry Spencer 
and Geoff Collyer recognized when they were working on increasing 
the performance of Netnews software, it was necessary to 
systemmatically record what was happening to be able to see what 
progress was being made, "To know how to get somewhere, you must 
know where you are starting from."(124) Similarly, it is 
important to systemmatically document the history of the 
significant developments of the software tool revolution, so that 
there will be a yardstick by which to measure the accomplishments 
and to determine the new questions to be explored.(125) 
-----------------------------------

[Research for this article was begun as part of an Independent Study in 
Spring 1994 at the University of Michigan Dearborn on Unix tools with 
Dr. Narasimhamurthi.  It was good to take a course which made possible 
not only doing some interesting and important research, but also where 
it was possible to design and use the tools that Unix has made possible 
to help with the research. Begun to help commemorate the 25th anniversary 
of Unix in 1994, further work on the article was done during Summer 1995.
Thanks to Doug McIlroy, David Nowitz, John Lions, and Berkley Tague for 
making available information that was helpful in doing the research for 
this article and thanks to Doug McIlroy, Alfred Aho, Tom Truscott,
Eric Allman and Mark Horton for commenting on earlier versions-
Ronda  [email-masked]]
--------------------------------------------------------------------
      This draft for comment is Chapter 9 of the draft netbook 
	    Netizens: On the History and Impact of the Net
               http://www.columbia.edu/~hauben/netbook
        	Notes to the draft can be found there.