Prologue

[ Site Index] [ Attic Index] [ Perl/Tk Index] [ Feedback ]

[ Intro ] [ Chap 1 ] [ Chap 2 ] [ Chap 3 ] [ Chap 4 ]

  • Introduction

    Introduction

    This book covers a specialized field; how to design and program graphical applications using Perl and the Tk toolkit. A specialized book deserves a specialized introduction: if nothing else, some justification of the choice of language and graphics extensions is called for. More fundamentally, I had to make a number of decisions about the content of the book before I was able to begin writing, and it seems only fair to share them with you before we go any further.

    Why Perl?

    Perl has been with us for less than a decade, and has already gathered a dedicated following among working programmers and internet specialists. It is largely the product of one man's search for a better tool: Larry Wall, a prolific and inventive UNIX systems programmer. In 1987, Larry was asked to write a package for man1aging several remote systems. None of the off-the-shelf tools seemed sufficiently powerful or flexible, and programming the entire system in C appeared unrewarding. So Larry bolted together a new language, and called it Perl. Perl had powerful report formatting features, because Larry needed to print system administration reports. It had pattern matching features similar to those of awk, the standard UNIX pattern-matching language, and somewhat better facilities for handling large quantities of data and lots of files. It had interprocess communication and internet sockets, to make it easier to talk to other programs and computers. And it had various other linguistic bells and whistles that struck Larry as useful.

    It should be noted at this point that Perl's initial design was not as arbitrary as this description may imply; Larry is a linguist by training, and he has some theories about human use of computer languages. Human languages tend to spawn a hodge-podge of sometimes inconsistent features and found words; they assimilate anything that facilitates their primary purpose, communication. Perl is similarly pragmatic, being designed to make certain types of programming task easy. Thus, although Perl may sound somewhat confused when compared to a language designed by computer scientists or theoreticians, it is above all designed to be useful, rather than to conform to any theory of usefulness.

    In due course, Larry released Perl to the Internet, in hope that other programmers would also find it useful. The response was surprising; UNIX system administrators rapidly jumped on the bandwagon, using Perl as a replacement for the traditional UNIX shell programming tools (for tasks such as user and process administration). They found that they could accomplish entirely within Perl tasks which otherwise entailed the laborious glueing together of many other programs. And Perl was not only a more convenient, unifying paradigm; it was faster. Although Perl was implemented as an interpreter, it was semi-compiled; the interpreter pre-compiled the Perl program into an intermediate form, then executed it on a byte-code interpreter. (In this way, Perl is best compared to those compiled languages -- like UCSD Pascal, or Visual Basic, or Java -- which require a runtime platform to run on.)

    Certain curious characteristics of Perl became evident. Perl is syntactically dense; as Larry Wall remarked, ``there's more than one way to do it'' (whatever 'it' may be). Perl provides very flexible syntax, and some of its features are very powerful; it is frequently possible to embed and conflate expressions until a four or five line subroutine is compressed into a single cryptic statement. Thus, it has been characterized as 'semantically dense'; one statement in Perl may correspond to five or ten lines of C. Perl belongs to a category of languages (along with Tcl, Python, and others) which are collectively referred to as VHLL's -- Very High Level Languages.

    With Perl 4, released in 1991, Perl acquired a certain respectability and ubiquity in the UNIX world. Perl 4 was a powerful UNIX scripting language, suitable for just about any task which would formerly require a shell script. Modified versions of Perl could interface directly to databases such as Oracle, and provide powerful reporting facilities. And Perl could be used to write internet client/server applications. But Perl 4 was handicapped by several problems. For one thing, it was difficult, if not impossible, to manipulate complex data structures in it. It was not amenable to sophisticated software engineering; although a package mechanism existed, it was not particularly flexible or widely used. Perl 4 could not easily be embedded in other applications (unlike Tcl). And it was not particularly extensible.

    Efforts got under way to create a new version of Perl, Perl 5; this saw the light of day in 1994. And not a day too soon; for the wildfire spread of the World Wide Web thrust Perl right into the limelight. World Wide Web servers can invoke other programs through an API called the Common Gateway Interface. Such programs need to format their output in HTML, Hypertext Markup Language, which the web server then sends back to the user's browser. Perl, by virtue of its good string processing capabilities, fast execution speed, and powerful pattern matching, rapidly became the language of choice for CGI scripting.

    Perl 4 was useful for quick hacks and simple maintenance scripts. While some fairly large programs were written in Perl 4, its lack of a good mechanism for splitting up programs into sub-sections and its poor handling of variable scoping made it unsuitable for large scale software development. Perl 5, in contrast, provides excellent solutions to both these problems. Consequently, although it is still possible to write a quick hack in Perl 5, it is also possible to build a large and sophisticated piece of software in it.

    With the release of Perl 5, Perl matured from a high performance scripting language into a general high-level application development system. Perl 5 was a radical re-write of Perl 4. Although most Perl 4 applications could run unchanged under Perl 5, Perl 5 had large numbers of additional features; object orientation, support for complex data structures, a more powerful modularization facility, better interfaces to externally loaded libraries, and a cleaner language design. Even more importantly, the designers of Perl 5 achieved a near-miracle -- a better language version that is backwardly compatible, has more features, and is actually smaller and faster than its precedessor!

    In summary, the features to note are:

    Why Tk?

    Tk is a high-level graphical application toolkit developed by Dr John Ousterhout, initially at the University of California at Berkeley but later with funding from Sun Microsystems. Tk was originally developed as a graphical front-end system for Ousterhout's Tcl (Tool Command Language) on UNIX and X windowing system platforms, but has been ported to a variety of other operating systems (MacOS, Windows) and languages (Perl, Python).

    Tk provides a partial solution to the problem of writing software for windowing user interfaces. Such programs require a radically different architecture from traditional programs executed as a batch job or interactively at a terminal. Traditional UNIX (or other command-line interface programs) usually operate on one or more streams of input data, reading it in and generating one or more output streams that are transformed in some way. They are initiated by a typed command, process a stream of information, then terminate. Graphical user environments (GUIs) are fundamentally different. They are usually 'event driven'; some stimulus (the user moving a mouse, for example, or pressing a key) generates an 'event', which causes the program to respond in some way (by moving an on-screen pointer to track the mouse motion, or by triggering a subroutine to service some task).

    Writing event-driven programs has often been seen as a difficult task. The programmer needs to write software that can respond to a variety of changing stimuli: worse, the nature of the stimuli it can respond to may vary depending on the current state of the program (for example, which windows are open on-screen at any given time). While command-line programs are frequently linear in structure (insofar as they slurp data in at one end, do something to it, then emit transformed data from the other end), graphical applications are by their very nature non-linear; they need to pay a lot of attention to their own internal state, and contain subroutines for dealing with varied contingencies.

    Tcl is a minimal, but extensible, language: like Perl, it belongs to a category sometimes known as VHLLs (Very High Level Languages). Ousterhout originally wrote Tcl because he perceived a need for a general- purpose, compact, embeddable language -- a language which could be built into other programs in order to provide them with a uniform command-driven control interface. His goal was to encourage interoperability of software, initially on the UNIX operating system, by giving programs a common vernacular with which to send one another commands.

    As a procedural, string-oriented language, Tcl was reasonably suited for controlling traditional command-line applications; however, its weaknesses became apparent in the early 1990's with the trend towards graphical programs. Allowing programs to manipulate each other via Tcl commands (or complex Tcl-based driver harnesses, like Expect) was not enough to make applications development easy. By 1990, the X windowing system had become the de-facto standard for graphical application development under UNIX; any solution Ousterhout came up with had to address the demand for graphical applications. But unlike the Macintosh OS or Windows environments, the X windowing system does not provide a uniform set of user interface resources. Each graphical application running on an X windowing system needs a set of 'widgets' (graphical design elements such as menus, scroll bars, and windows); there is no common toolbox available on all X systems. Simply giving X applications a common command language didn't solve the initial problem Ousterhout had set out to tackle.

    Ousterhout therefore began development work on the Tk graphical toolkit extensions to Tcl.

    Tk is a set of graphical 'widgets' that provide a uniform-looking set of furniture for X applications (similar in appearance to the Motif graphical environment). Tk widgets can be bolted together like Lego to create complex interfaces. Their semantics are quite rich; graphical elements can be nested inside one another and combined into composite widgets that serve specialized purposes. Furthermore, Tk provides mechanisms for linking in fragments of Tcl code, so that when an event occurs, the appropriate action is executed. In effect, Tk provides a magician to hide behind the curtain of a graphical application, pulling the levers that despatch functional subroutines and relieving the programmer of much of the work involved in developing a graphical application.

    Although Tk started life as a tool to make research projects easier, it rapidly gained popularity because it was far easier to use than conventional X windowing system application development tools. (A simple program to generate a pushbutton saying 'press me' that exits on a button-press can be written using about two lines of Tcl/Tk; the same application, using the Motif, Xt and Xlib libraries and the C programming language, takes approximately 120 lines of code to express.) It is fair to say that today, more than half of all software developed for X windows and UNIX is written using Tk. Sun Microsystems adopted Tcl/Tk as part of their portable scripting language initiative; SCO Inc. based their ScoAdmin system administration toolkit on an early colateral development, SCO Visual Tcl. Tk, although initially intimately spliced into Tcl, was delicately disconnected from its original language system; today it is available as a set of extenstions for other VHLLs -- including Python and Perl.

    There are other solutions for graphical application development in Perl, of course. Perl has no built-in graphical system that bears the same relationship to it that Tk bears to Tcl; instead it uses external, loadable libraries. The UNIX 'curses' library for manipulating dumb text-only character terminals is available. There are high-level windowing libraries based on top of Perl's curses support. There is also an interface to the 'Xforms' graphical system, another X windows programming toolkit that follows a somewhat different paradigm from Tk. (We will briefly look at Xforms later in this book.) But the most mature and comprehensive graphical programming library for Perl is the Perl/Tk system, which provides Perl 5 with an object-oriented graphical programming environment suitable for developing X windows applications.

    Why Rapid Application Development?

    It has become something of a cliche to say that modern software has become bloated; even though today's computers are massively faster than those of a decade ago, software has grown faster than the supporting hardware.

    One of the problems associated with large software projects is the issue of software maintenance. Many of the costs incurred in writing a program are encountered later on in its life, in adapting it to run on new platforms or under changed conditions, or in adding additional features. Programs get flabby and inefficient with middle age, and become harder to maintain.

    Software engineers have developed a number of techniques for treatig this disease, which seems to be a universal constant of programming (regardless of the methodology du jour which the project management mandates). Big programs take longer to write -- therefore any trick which makes a program smaller makes it faster and cheaper to develop. Even if the end result runs more slowly, improvements in hardware performance will catch up eventually; thus, keeping development costs down may be seen as more important than optimizing for efficiency. A Very High Level Language like Perl has obvious appeal; anecdotal reports and experience suggest that Perl programs are much more semantically dense than C or C++, so that a Perl program of about 1000 lines may actually be functionally equivalent to a slab of 5000 lines of C. Similarly, a high level widget toolkit like Tk may massively reduce the difficulty of developing a graphical application.

    These wins are not necessarily counterbalanced by a corresponding loss in efficiency. Perl is remarkably efficient for a VHLL; it is tighly coded internally, and provides a rich toolkit of high-quality software that less skilled programmers may not be able to equal. A well-designed C program will usually outperform a well-designed Perl program, but a badly-designed C implementation may be drastically slower than a well-thought-out Perl application. In general, programmer ability can often make up for any speed hit resulting from the use of the higher-level environment. Moreover, a compact Perl application may be easier for a programmer to understand -- and thus, to get right -- than a big C application that is replete with arcane graphical API calls that have no bearing on the high level logic of the application domain.

    Rapid Application Development (RAD) is a software engineering technique that capitalizes on the availability of high-level languages and interface toolkits. The keynote of RAD is software development through iterative approximation; the construction of a series of 'working models' of the finished program that gradually approach the final specification in terms of functionality and performance. Starting with a crude painting of a user interface, the developers test, redesign, and reimpliment the program, taking feedback from each stage of the process and feeding it into the next implementation. Prototypes are built, modified, and thrown away when they outlive their usefulness, until the 'final' prototype is sufficiently stable, bug-free, and functional to actually qualify for a product roll-out.

    This sounds inefficient, and in some respects it is. Code is written to be thrown away. However, in approaching a final product through successive iterations, a RAD project closely mimics the process by which people establish their requirements for a piece of software; it is more intuitively obvious than the traditional analysis/design/specification processes of other methodologies. It also has the political advantage that programming work may be seen to start sooner rather than later; 'real work' starts early, when a traditional project would still be working on the requirements documentation.

    RAD also works particularly well with graphical applications. Traditional software engineering methodologies are weighted towards consideration of the underlying data manipulations. GUI applications, in contrast, pay a hitherto inordinate ammount of attention to the user interface; by some estimates up to 90% of the programming work in a GUI application is focussed on making it more useable. By using a series of prototypes for testing interface concepts and discarding non-viable alternatives, a RAD project may come up with a better end-product than one that was specified in minute detail at the outset.

    A specific requirement for successful development of graphical applications using rapid prototyping is an object-oriented, high-level graphical toolkit. Each chunk of the programs visible interface should ideally be a separate, modular component which is as independent of the rest of the program as a Lego brick is independent of the other contents of a toy box. You can take any Lego brick of a given size and shape and replace it with any other, changing the colour of whatever toy it is part of; you can even replace it with a sub-structure made of different components that fit in the same gap. Changing a Lego brick in a model ship shouldn't make the bridge sprout glossy black fur or become transparent -- alterations to the ``look and feel'' of a graphical application should likewise not have unforseen effects elsewhere in the program. This requirement can be met by using a toolkit like Tk (albeit with due care and attention); it's somewhat harder to achieve if the interface is constructed in an ad-hoc manner and bolted onto the underlying application without any systematic attempt at functional compartmentalization.

    RAD languages are not necessarily restricted to building pretty graphical interfaces. The main emphasis is on -- naturally enough -- making it easier for the programmers to experiment. The long edit/build/test cycles of the traditional 'C' programming environment (and other compiled systems) militate against experiments; for this reason, most RAD languages are interpreted; however, compilers are usually available for the finished product. Complex sets of tools are a common feature of RAD environments: both toolbox libraries full of useful high-level routines, and tools for making the development process easier (for example, graphical interface prototyping tools like SpecTcl/SpecJava). Very often, database products and 4GLs are presented as rapid prototyping environments. However, this is not the case; they may be intended to support the rapid development of a graphical front-end to their own data repository, but they are not true general purpose environments. A better example of a RAD tool par excellence would be Microsoft's Visual Basic: a high-level language with a very sophisticated graphical application development environment that takes care of most of the legwork of setting up a GUI front-end and leaves the programmer free to concentrate on the actual meat of the program -- the routines where the program actually has to do some work. Another RAD tool to note is HyperCard; although this Macintosh application was widely promoted as a hypertext browser, it is actually much closer to the object-oriented high-level RAD philosophy. (HyperCard has a high-level language, HyperTalk, and can communicate with other applications or plug-in extension commands.)

    Perl was not designed as a RAD language; it was designed as a high-performance UNIX scripting and text processing system. However, with the addition of a modular graphical front-end based on pTk it becomes a formidable tool for rapidly assembling X windows applications. The high speed of Perl, and the availability of the compiler module, make it suitable for developing commercial applications; and better still, it does not tie programmers to any one RAD methodology or graphical toolkit. Perl is more flexible than most purpose-designed RAD environments because it wasn't designed for the job but grew into it naturally.

    Why this book?

    This book exists because, when I first wanted it, I couldn't find it. At the present time there are no other explanatory books specifically about Perl/Tk. There is plenty of reference documentation -- the Perl/Tk kit comes with several hundred pages of command references -- but trying to learn a programming language through the reference material is a painful exercise, even for the experienced. I wanted to learn Perl/Tk, but could find no books to make my journey easier: which is why I decided to write this one.

    This book is not a one-stop shop for Perl and Tk. The Perl bible, ``Programming Perl'' (REF), runs to over seven hundred pages. The Tk bible is ``Tcl and the Tk Toolkit'' (REF), and it runs to over four hundred pages (although it covers Tcl along the way). The reference man (manual) pages for pTk occupy another hundred and fifty files; the Perl and pTk man pages, when printed out, fill nearly a ream of paper. Obviously, it is not sensible to reproduce the existing reference documentation in this volume. Therefore in this book I concentrate on providing an overview and tutorial in programming in Perl/Tk, referring to the following sources where appropriate:

    Perl:

    Tk:

    (You do not need copies of 'Programming Perl' or 'Tcl and the Tk Toolkit' to use this book, but they are generally helpful as supplementary reading and shed light on those corners of the language which are not covered here. You do need copies of the man pages; see (REF) for details of how to obtain them.)

    Perl/Tk is, in principle, a cross-platform programming environment. Tk has been ported to Windows and MacOS, as well as UNIX; Perl runs on a variety of operating system, including virtually every UNIX there is, and Windows and MacOS by way of VAX/VMS and other, more obscure, platforms. pTk, the portable Tk library, is in principle available on any computer that has an operational GUI. However, Perl/Tk has deep roots in the UNIX culture; it was the first platform for which Perl and Tk were developed, and is the only platform which unequivocally supports all features of the Perl/Tk system. (For example, file pathnames differ on MacOS or Microsoft operating systems; MacOS, for fundamental reasons, cannot handle the fork() system call in a UNIX-like manner, while Windows has differing, comparable limitations.)

    In writing this book I have made the simplifying assumption that you are using Perl/Tk on a UNIX-like platform. In other words, if you are using a PC then you are running Linux or FreeBSD or SCO OpenServer rather than Windows; if you are using a Macintosh, you are running it under MkLinux or MachTen. Furthermore, for the sake of simplicity, I also assume that your UNIX system is reasonably 'normal'. UNIX has a long and hoary history: the location, name, and syntax of even the commonest commands may differ. For example, the syntax of ps(1) on UNIX systems of Berkeley ancestry differs radically from that of ps(1) on AT&T-descended platforms. The standard commands also live in different places; /bin/gcc on Linux with /opt/SUNWspro/cc on Solaris are both standard locations for the system C compiler!) Consequently, wherever possible, instead of resorting to external UNIX commands, I try to do everything within Perl; this has good implications for portability and efficiency, as well as making life easier. Be warned, however, that if you try to use this book in conjunction with Perl running an Atari ST (or a deviant UNIX clone where ls, rm, and chmod have all been moved to /etc/backscratch/version/5/0/0/stuff and modified to take new and exotic options) you may not get the results you expect.

    (As a point of reference, the scripts in this book have been developed on a PC running Linux (specifically, RedHat Linux 3.0.3, laid out in accordance with the Linux Filesystem Standard) with a totally 'vanilla' Perl and Tk installation. This mimics the layout of a traditional (pre System V Release 4.0) UNIX system, with some characteristics of both BSD4.4 and System V Release 3.2. In addition, scripts have been tested on Solaris 2.5 (and, in some cases, on other platforms). The filesystem layout required for a suitable Perl/Tk installation on UNIX is covered in (REF: installing Perl/Tk).

    How to use this book

    This book divides evenly into two sections:

    The first section is inescapable. Perl/Tk builds upon some fairly advanced Perl programming concepts, including the use of complex data structures, objects, and modules. The first section therefore contains all the foundation material necessary to go from a standing start (with no prior Perl experience) to programming Perl in an object-oriented fashion. Simple programming questions are provided, to help you gauge your familiarity with the material in each section. (Some answers and illustrative discussions are provided at the end of the book.)

    Not every programmer (by a long way!) is familiar with object-oriented programming (OOP). Therefore the basic concepts of OOP are explained in parallel with Perl. The easiest way to learn OOP is by using someone else's pre-canned class libraries; therefore the Combined Perl Archive Network (CPAN) is introduced early and used often, both for building blocks and exemplary colour.

    Some features of Perl are not explained in any great detail, if at all. Perl's report-generation facility, for example (centred around the format command) is oriented towards producing formatted ASCII dumps of text files or databases. While this is a valuable feature, it is orthogonal to the concerns of a GUI developer -- in a graphical environment, very different tools exist for doing much the same job. At the other end of the spectrum, advanced features are omitted: the XS interface to 'C' applications, the low-level nitty-gritty of inter-process communication, and tools for embedding Perl in other programs, are not really appropriate to this level of discussion. In any event, comprehensive coverage of these topics can be found in 'Programming Perl'.

    A lot of attention is paid to understanding variables and data structures in Perl. GUI applications are basically complex hierarchies of data structures; without a clear understanding of references it is not possible to write a non-trivial Perl GUI application.

    A tour of the CPAN is also necessary. Larry Wall's famous recipe for a great programmer is one part each of laziness, hubris, and impatience: laziness because they don't want to waste time reinventing the wheel, hubris because they are only interested in tacking the big jobs, and impatience because they want to get things done rapidly. CPAN is a great tool for great programmers: a well-integrated library of Perl modules structured by programming task. By learning to use CPAN properly you can save yourself from having to reinvent the wheel at length, and aim at the big problems without tripping over mole-hills along the way. We examine CPAN early (in the next chapter), and revisit it frequently throughout the book.

    The second section follows a similar pattern. Tk is a friendly graphical environment, but relies on several concepts which are totally unfamiliar to a programmer raised in the traditional command-line programming paradigm. Tk applications are hierarchical structures of 'widgets' (graphical elements such as windows, buttons, slider bars, and so on) bound together by callbacks -- subroutines which are triggered when a widget receives some kind of 'event'. (An event can be any interaction the application experiences -- a mouse movement, button click, keystroke, or a command issued by another program.) Learning to design and assemble hierarchies of widgets, then attach appropriate callback scripts to them, calls for a different programming discipline from the standard data-driven application.

    As Tk programming is introduced, Rapid Prototyping is used to elaborate a user interface; this demonstrates the use of a dummy test-bed, its conversion into a partial working model, and its replacement by a better model at a later time.

    Finally, we put the concepts together and walk through the development of a medium-scale application that uses a Tk interface to manipulate some complex data structures. Hopefully, by this stage you will have enough confidence to start exploring Perl/Tk on your own!

    • [ Site Index] [ Attic Index] [ Perl/Tk Index] [ Feedback ]

    [ Intro ] [ Chap 1 ] [ Chap 2 ] [ Chap 3 ] [ Chap 4 ]