[ Site Index] [ Linux Index] [ Feedback ]
Kissing Cousins: A brief history of UNIX and Linux
This year, UNIX turns 30. By some estimates, that makes it middle-aged. Linux, however, is still less than ten -- a turbulent tearaway child. What lessons can it learn from its elder sibling's past?
The history of UNIX reads a bit like one of the more monotonous passages from the Book of Genesis, full of " ... begats ... " and cryptic names of long-dead people and companies. According to legend, it all started because Ken Thompson wanted to play Space Wars, the first graphics-based game. He managed to find a tiny (for two metre high and three hundred kilogram heavy values of tiny) Digital PDP-8 minicomputer gathering dust in a corner of his department at Bell Labs; but it had no operating system. Ken -- and a number of his colleagues -- had been working on the Multics operating system, a vastly ambitious scheme to build a system that would be to on-line computing about the same as a national grid is to electricity: when AT&T pulled out of Multics in 1969 (leaving Honeywell to finish the job), Ken was at a loose end for a bit.
One thing led to another, and Ken began building a very simple filesystem and program loader for his PDP-8. It's name, UNIX, was a pun on Multics, the multiple computing system. (UNIX started out as a single-user system on the nearest thing to a microcomputer that existed in 1970.) His colleagues Dennis Ritchie and Brian Kernighan joined in, along with other staff members. In 1971 they suddenly discovered that their department was supposed to be producing a word processor (for AT&T's patent department), so they added some typesetting tools and shipped UNIX -- now running on PDP-11's and able to multi-task -- to various departments around the company.
AT&T was, at the time, bound by a consent decree -- a government anti-trust lawsuit that was brought against the telephone company because it was operating a monopoly. Under the terms of the settlement, AT&T were forbidden from selling software: so the UNIX team gave copies away to universities, for academic study. With a re-write of the kernel in C (a language they'd developed for their own purposes), it was reasonably portable. Moreover, most university computer science courses wanted to teach operating systems theory to their students, and UNIX was the only readily available example (with source code) that wasn't a crawling horror from the nethermost pits of hell and didn't cost more than an executive jet.
In the early days, UNIX was sometimes disparagingly referred to as a virus; it infected computer scientists and spread from mainframe to minicomputer like a horrible plague, driving out proprietary operating systems. The University of California at Berkeley had an active computer science department; they took the UNIX source code and developed it, pumping out releases -- the standing joke was that the UCB staff would lock six hackers in a room, slide pizzas under the door, and every six months tape up whatever came back out and ship it as a new release of Berkeley UNIX.
In the late seventies, AT&T woke up and realised that they'd given away the crown jewels. The in-house UNIX team were told to develop something saleable: they came up with UNIX release 7, which was licensed to other software companies (most notably as Xenix by Microsoft, who in turn sold it on to SCO), and then with System V UNIX. These products weren't sold to the public by AT&T -- they were licensed to software houses who in turn sold them, and paid a royalty to AT&T, thus circumventing the consent decree.
Meanwhile, Berkeley UNIX kept on going. It was licensed to various companies under favourable (cheap) terms, including a small bunch of hackers from Stanford University who had the intriguing idea of building a workstation -- a powerful networked computer with a megapixel monitor, running a multi-tasking operating system. The Sun workstations ran SunOS, a thinly-disguised version of BSD UNIX, and blazed a trail through academia, where over the course of the 1980's UNIX workstations rapidly drove out the proprietary operating systems favoured by the likes of Apollo. And all the time, the cost of a UNIX system dropped; from the order of $100,000 in the 1970's, to $10,000 in the 1980's ...
Also in the 1980's, an interesting but initially obscure set of networking protocols was developed (mostly on BSD UNIX systems connected to an obscure government-funded wide-area network called ARPAnet). ARPAnet was interconnected to other networks, and the TCP/IP protocols it was based on became the backbone of the young internet.
Some commercial UNIXes, such as SunOS, were based on BSD 4; others (such as SCO UNIX) were based on AT&T System V Release III. The two lineages cross-fertilized each other; SVR3 acquired the BSD TCP/IP stack and virtual memory subsystem, while BSD acquired a violent aversion to doing anything the AT&T way -- by the late eighties the talented hackers who'd written the original UNIX had moved on to other things, and the resulting mish-mash of features was largely the product of deranged marketing flaks. Meanwhile, the marketing war continued, leading to fragmentation.
The late eighties UNIX scene was a mess of conflicting standards. In general, each AT&T licensee was convinced that theirs was the One True UNIX, bound to conquer the marketplace. Although they were nominally committed to the idea of open standards (write software on one machine, run it on any other by simply recompiling the source code), all of them really wanted to lock their customers into their own system. The marketing theory was that customers wanted a system that complied with open standards (such as POSIX), but that it was better for the vendor to addict the customers to their particular system's extensions -- so a sort of arms race resulted, with OEMs taking a core UNIX version and adding all sorts of weird bells and whistles.
By 1990, the UNIX market was in crisis. Microsoft was making steady in-roads into territory that had formerly been dominated by workstations and minicomputers. In an attempt to make their wares appetising, the UNIX vendors had cross-licensed lots of extra tools, lumbering themselves with high royalty payments. A weird ecology of licensing developed, with companies inventing new subsystems, hyping demand for them, then licensing the software to each other: a single commercial UNIX sale would thus trigger royalty payments from the vendor to twenty or thirty software companies who had contributed to the product.
SCO, for example, attempted to sell Open Desktop -- SCO Unix plus X11, Motif, IXI's X Desktop, the Ingres database, Locus Merge (a DOS run-time environment), and hordes of extra tools -- as a PC operating system. But on every copy of Open Desktop 3.0 that SCO sold, SCO had to pay $100 in license fees, and another $50 in administration overheads, while Microsoft could sell Windows 3.11 and DOS 5.0 for less than $20 to a VAR, or for $100 to an end user. (Incidentally, these licensing fees drove the cost of a desktop UNIX system so high that, it 1991, a Finnish student called Linus got annoyed and started writing his own UNIX-like kernel. What happened next is still earning me a living.)
The result was that commercial UNIX systems were seen as expensive, over-specified, lumbering brutes that simply couldn't compete on the desktop; and there matters stood until around 1994-95.
The log-jam was finally broken by Linux, although few people in the commercial UNIX field noticed it. I first ran across Linux in 1992, an interesting freeware download on the internet: it fitted on two floppy disks, it only supported a standard VGA display and a couple of types of hard disk controller, and if you wanted to make it run on your hard disk you had to patch the kernel image by hand and then compile and install all the tools you needed (starting with gcc). But by 1995, Linux had matured radically. The Linux kernel was a functional clone of UNIX, with the essential features of all the available commercial variants rolled in. Distributions now existed: while Slackware 3.0 and Red Hat 2.0.2 were a tad harder to install than SCO OpenServer 5.0, at least you didn't need to hand-patch the kernel, and you had free clones of almost all the UNIX utilities you were used to. Moving to it as a former heavy user of SCO Open Desktop 3 (and some internal development releases of Open Server 5), I found it a breath of fresh air. Much of the weird gunk imposed by marketing fiat that had lumbered down SCO or SunOS was missing. Instead, there was a fairly complete set of GNU utilities which matched (and in many cases outperformed) their commercial equivalents, and an attitude among the developers that was contagious and subversive: Linux came with a whiff of revolutionary enthusiasm that had all but vanished from the computing scene a decade earlier.
More to the point, even in those days the feature set offered by a Linux distribution was comparable to a commercial UNIX kit -- because the UNIX vendors had unbundled many essential components (such as compilers, document typesetting tools, and the graphical desktop environment), making them separate purchaseable items. (This was a brain-dead attempt at cutting the end-user cost of a UNIX system by divesting it of many of the licensing encumbrances. Instead, it cut the useability of a UNIX system, making it less desirable.) A corollary of this is that the commercial UNIX vendors were reluctant to use free tools, even when open source solutions were available off the shelf; the result was that their products began losing functionality even though free software was available to plug the gaps. I remember being told in 1994 that SCO wasn't going to include GNU Emacs in Open Server 5.0 -- because they might be asked to support it, and they hadn't written it! For similar reasons tools like Perl were ignored, even for in-house jobs that screamed for it.
Linux was already much more useable than a lightweight desktop UNIX system in 1995; today, the usability gap between an entry-level Solaris or SCO system and a modern Linux distribution is striking. (Which is probably why Linux now out-sells the entire commercial UNIX field by 50%.) I recently had the unpleasant experience of setting up a Sun Ultra 5 workstation running Solaris 2.7. It was like stepping back into the stone age; a graphical desktop (CDE) based on Motif, missing tools, no compiler, no perl, no bash, brain-dead defective vi implementation, and so on. Solaris hadn't progressed, in user interface terms, from what was current in the world of Windows circa 1992. Linux in contrast has been making accelerated progress in user interface design; KDE, Gnome, and now the new generation of forthcoming desktop developments (such as Nautilus, or Corel's UI integration work) have given Linux a slicker feel than just about any commercial UNIX system except SGI's Irix. (And SGI have always been the Apple of the UNIX world, marching to a subtly different beat.)
The fragmentation that crippled the commercial UNIX industry can't easily happen to Linux. It is possible to add extensions to a linux distribution that make it more desirable, or better, than a competitor; it just isn't possible to monopolise those extensions. Good proprietary ideas will be cloned as fast as possible. More importantly, the business model all the Linux distributors are adopting makes proprietary ownership of source code irrelevant; they have to make money by selling support services, so locking up their extensions doesn't achieve anything much other than to get the Taliban riled at them. If the bells and whistles aren't proprietary, there's no way for a fragmented market to develop: those pundits who predict that the increasing number of linux distributions will fragment the market don't understand why UNIX fragmented in the first place. Having a multitude of distributions is good insofar as it ensures that no one company or group can mess up the future of Linux beyond hope of recovery, while promoting competition to keep them on their feet and striving to make things work better.
This is not to say that there are no specialised niches in the Linux universe; but such niches relate to providing support for Linux on different hardware platforms or in different environments. Linux now runs on everything from a Palm Pilot to an IBM S/390 mainframe, and it's no surprise that the requirements of supporting it on these platforms are different.
UNIX was effectively strangled by copyright restrictions and royalty payments, two features which are singularly absent from the Linux universe. About the only thing that could fragment the Linux world today would be for a sufficiently big company -- Microsoft springs to mind -- to bring out their own distribution, adding some "killer" functionality (ability to seamlessly integrate and run Windows 2000 applications, perhaps?), and simultaneously refuse to share it -- instead, dragging it back to the proprietary software model. And Microsoft isn't about to get into the Linux distribution business while there's still hope of salvaging the Windows 2000 business (into which they've sunk billions of millions of dollars of development money). Realistically: Microsoft Linux simply isn't going to happen this year or next -- by the time it happens, Microsoft will no longer be dominant.
Meanwhile UNIX is dying, but it doesn't matter. Happy thirtieth birthday, UNIX. We've learned from your mistakes -- and this time round, we're going to get it right.
DeCSS Revisited
In Shopper 146 I wrote about DVD-ROM and Linux, about a piece of software called DeCSS, and the actions of some despicable would-be monopolists who have been molesting teen-agers, persecuting the innocent, and generally doing their best to make enemies out of anyone who believes that if they buy a copy of a movie to watch on their TV they should be allowed to watch it on any model of TV, anywhere in the world -- or on any operating system, for that matter -- without additional payments. (And, oh, little things like the legal right to reverse-engineer software for the purpose of being compatible with some proprietary file format -- a right that's built into copyright law both in the USA and in Europe, although you wouldn't know it if you learned about copyright by reading the RIAA's account of the Linux DVD dust-up.)Well, the DVD Copy Control Association managed to get a court injunction upheld against a number of web sites. The injunction bans those web sites from publishing the DeCSS source code. More worryingly, this gagging order -- issued by a New York court -- bans them from publishing the location of the DeCSS source code.
I don't know where the DeCSS source code is stored; I don't have a copy and I don't have a link for you to follow.
However, I do know how the domain name system works. DNS is a hierarchical network of database servers that map domain names (internet addresses) to TCP/IP machine addresses; when you try and lookup a host by name (e.g. www.antipope.org), your browser queries its local DNS server. If the DNS server doesn't already know where that host is, it figures out which server adjacent to it knows about antipope.org, or failing that, the top-level .org domain; the requests are forwarded until something comes back, at which time it is cached in your local DNS server (and passed on to you).
It is of course against the wishes of a rather blinkered New York judge for me to point you at a machine where DeCSS is available for download. However, I can tell you how to ask DNS to find the source code and give it to you, because a highly imaginative hacker has put it in the database of a DNS server in France, which has got to qualify as the second-most bizarre misuse of the domain name system that I know about. When you query the DNS system appropriately, it returns the source code to DeCSS, wherever you are -- and incidentally propagates it to several new computers (and one trusts that the DVD CCA aren't about to try and serve court orders on every machine on the internet):
dig @138.195.138.195 goret.org. axfr | grep '^c..\..*A' | sort | cut -b5-36 | perl -e 'while(<>){print pack("H32",$_)}' | gzip -dDig, the Domain Information Groper, is a DNS query tool; in this weird-looking sequence of commands dig is told to go to the server at 138.195.138.195 (a raw IP address, right?) and ask for information about the domain goret.org. Dig is asking for a special type of information called a zone transfer -- a complete dump of everything the server knows about that domain. The output from the zone transfer is piped into grep, sorted, mangled, decoded and uncompressed ... and turns out to be the source code to the banned DeCSS DVD encryption cracker, buried in a database that normally maps hostnames to IP addresses.
If this doesn't illustrate the stupidity of trying to ban a piece of free software once it's gotten onto the internet, nothing will. (Oh, and good luck to the DVD CCA, going after the owners of a French DNS server -- given the general attitude of French courts towards American companies persecuting their citizens for no very good reason.)
Caldera v Microsoft: the outcome
It's wrap-up time: the lawsuit is over. I wrote about the Caldera v. Microsoft lawsuit in Shopper 144; this lawsuit is now settled, out of court, and very much in Caldera's favour.
To rake over old history: ages ago, Novell purchased Digital Research (and their operating system, DR-DOS). Novell discovered that Microsoft appeared to be engaging in dirty tricks to dislodge DR-DOS from those VARs who were selling it; so they fired off a lawsuit. Time passed, and Novell sold DR-DOS (and the lawsuit) to Caldera Inc, which continued it. Caldera pushed the lawsuit, alleging that Windows 95 was deliberately fixed so as not to load on top of DR-DOS, thus freezing DR-DOS out of the market: they sued Microsoft for $3 billion, and got as far as a courtroom in Utah.
Just before the case came to trial in front of a jury (who, in Novell's home town, may be presumed not to be well-disposed towards Microsoft), Microsoft and Corel announced an out of court settlement. No figure was announced, but Microsoft admitted that they would take a one-time hit on dividends in the first quarter of 2000, amounting to roughly $160 million.
Some minor facts need to be borne in mind when thinking about this, however. For starters, Caldera Inc isn't the same Caldera as the Linux distributor; Caldera Systems Inc was spun off a year or so ago, as was another company, Lineo Inc, which inherited the DR-DOS and embedded systems business. Caldera Inc is effectively a holding company that's been pursuing the lawsuit, and most of the shares are owned by Novell. For seconds, it's unlikely that the payout will have been only $160m, given the sum in jeopardy and the fact that Caldera Inc had a very strong case: my bet is that Microsoft gave Caldera a rather larger sum from their cash reserves (which are huge), and announced the hit on dividends to mislead onlookers into thinking that it was a relatively small settlement.
A third interesting fact: Caldera is still a thorn in Microsoft's side. Lineo are working on an embeddable Linux environment tailored for Windows-CE-like palmtops and appliances. Most interestingly, they're talking about providing a Windows API to ease porting of manufacturer's CE-specific applications. Lineo Embedix PDA, due to debut next year, should allow a palmtop manufacturer to switch from CE to Linux with ease. The porting layer is to be based on Willows originally a commercial -- now GPL'd -- attempt at reverse engineering the Windows APIs. In view of the number of PDA makers now looking at Linux as a cheap,lightweight solution to their OS needs -- lightweight compared to Windows NT, that is -- this is a development that deserves watching. Especially if some of the money the parent company is rolling in ends up filtering back to fund Lineo's R&D ...
[ Site Index] [ Linux Index] [ Feedback ]