The shell game

Software QA Magazine, Vol. 3, No. 4, August 31, 1996 Tester's Toolbox column

by Danny Faught
Copyright 1996, Danny Faught

Shells are a common tool in a Unix test developer's toolbox. Just one example that David Korn told me about is a system of test scripts consisting of some 50,000 lines of shell code. Scripts are popular for testing tasks that don't require a more cumbersome compiled language, and shells are often the scripting languages of choice because of their ubiquity.

In this article I briefly mention the most common shells, whet your appetite with a short snippet of code, and mention a rather heated argument between two camps. I list several shells that will run on non-Unix systems. At the end of the article I'll point you to several different resources for learning more about shells.

A guided tour

A shell is the command-line user interface, but it can also be a scripting language, and that scripting ability will be the focus here. There are at least nine different shells that you're likely to find on Unix systems (and several on other systems as well).

Here's a brief tour of the most common shells. One of the earliest shells was the Bourne shell (often referred to by its filename "sh"). It remains the shell that's most likely to be found on any Unix system, and it's the first of the largest category of shells. Later shells to appear that are sh-compatible are the Korn Shell (ksh) and the Public-Domain Korn Shell (pdksh), the GNU Born-Again Shell (bash), and Z-shell (zsh). Some of these shells are compliant or mostly compliant with the POSIX.2 standard, and you also may find a separate shell called only the "posix" shell.

The C-shell (csh) was introduced with Berkely Unix. In modern Unix systems, csh is as omnipresent as sh. The other shell in the csh category is tcsh, which includes bug fixes and enhancements such as interactive features borrowed from the Tenex system. You'll also find that some vendors have added similar features to csh without changing the name.

The Plan 9 shell, "rc", is sometimes considered a category of its own, though some place it in the sh category. Rc was written for the Plan 9 operating system and has been ported to Unix. An enhancement to rc is "es".

Sample code

Below is an interesting snippet of Bourne shell code that I wrote as part of a front end for a Unix kernel reliability test (with line numbers added):

 1    (
 2    while /bin/true
 3    do
 4        /bin/sleep 300
 5        /usr/bin/uptime
 6    done
 7    ) &
 8    uptime_pid=$!
 9    /bin/renice -n -10 $uptime_pid > /dev/null
10    reliability_test &
11    test_pid=$!
12    wait $test_pid
Lines 1-7 are run in a subshell, due to the enclosing parentheses. It's an endless loop that spits out the time and load average every 5 minutes. The ampersand on line 7 causes it to run in the background, so the parent script continues from there immediately. It might have been better to put the uptime loop in a separate script so it could be identified more easily in a process listing. An interesting thing I observed is that the uptime loop itself identified several system problems, often slowing down or hanging entirely while the rest of the system appeared on the surface to be healthy.

Line 8 records the process id of the subshell. This is used in another part of the script to kill the subshell when the parent script exits. Then line 9 lowers the "nice" value of the subshell, giving it a better chance to run while the system is under stress. I could have put a nice command somewhere in the while loop instead, but it would have run each time through the loop and thus would have been less efficient.

Line 10 starts the test program in the background, and line 11 records the process id. Finally, line 12 causes the script to pause until the program is done. I didn't run the test program in the foreground because I set up a signal handler (not shown here) that kills the child processes, and I couldn't figure out an easy way to determine the process id of the script's current foreground process.

Note that the script has a race condition. If it catches a signal between lines 7 and 8, or lines 10 and 11, the child process will be started but its process id will not be saved, so the signal handler will incorrectly assume that the child process hasn't started yet. I would probably have to use a more advanced language to fix this problem.

Why not csh?

A very popular opinion is that sh is a good shell for programming but not very tolerable for interactive use, while csh is better than sh for interactive use but not so good for writing scripts. For interactive use, there are now many shells that are better than the both sh and csh (not counting the various enhanced csh's). But the shell programming issue is still hot. Probably the most well-known crusader against csh programming is Tom Christiansen, the author of the csh-whynot Frequently Asked Question (FAQ) file.

The more systems you deal with, the more likely you are to run into trouble with csh. However, some of the bug information in csh-whynot is out of date, especially with respect to more recent incarnations of csh. Legend has it that Christiansen started his crusade while working in the System Software Test Group at Convex (I started working with the group after he had moved on). Csh use was common, and so was the use of csh scripts. Christiansen became frustrated with the problems and limitations of csh. Eventually he convinced the group that perl was the way to go. To this day, perl use is widespread, and the csh scripts that remain continue to give us problems. When we use shell scripts, sh is preferred, though we're likely to migrate to ksh in the future.

When Christiansen posts the csh-whynot file, he often incites arguments. One objection raised by Doug Hamilton, the author of the Hamilton C-shell, was that csh-whynot caused some readers to conclude that all C-shells shared the same defects. Practically all of the csh problems that Christiansen cites have been fixed in Hamilton's shell, and csh-whynot now contains a grudging reference to this fact. However, it should be noted that Christiansen is complaining about the Unix csh, and Hamilton's shell currently isn't supported under Unix.

Unix shells aren't just for Unix

You don't have to use Unix to be able to use a Unix shell. It didn't take too much effort for me to find a number of shells that run on systems such as MS-DOS, 16- and 32-bit MS-Windows, OS/2, and AmigaDOS.

The MKS Toolkit seems to be a well-respected commercial solution for getting a Unix-like environment on a non-Unix system. The flagship is MKS KornShell, but it's also accompanied by a long list of other utilities that Unix programmers are accustomed to using. The MKS Toolkit is supported on MS-DOS, OS/2, and Windows NT.

Another Korn shell you can buy comes from the efforts of David Korn himself. Global Technologies Ltd., Inc. is offering ksh93, the latest release of which they call "The Official KornShell", available for BSDI, Linux, DG-UX, SCO Unix, HP-UX, Solaris, Windows 95, and Windows NT. This incarnation of the KornShell is an evolution from ksh88. ksh93 is positioned as an alternative to Tcl and perl, and it seems to have the features to make a good run at this goal. ksh93 can dynamically link C functions. Two examples of extensions that take advantage of this are the Desk Top KornShell (dtksh) and tksh. dtksh provides a ksh interface to the Motif library, while tksh is linked with the Tk graphics library and can also run Tcl code. Personally, I'd be interested to see if ksh beats perl to having an "expect" extension so "expect" users have a choice besides Tcl.

For a nice overview of ksh93 and how it fits into the shell history, see "The New KornShell ksh93" in the June 1996 edition of the Linux Journal.

Another commercial shell that is available for Windows NT, Windows 95, and OS/2 is the Hamilton C shell by Hamilton Laboratories. This shell is a complete rewrite from the original csh implementation. It also adds a number of features, such as many built-in Unix utilities and tcsh-style enhancements. The author reports that most of the problems traditionally associated the C-shell are fixed in his implementation.

You might want to look at 4DOS for MS-DOS, 4DOS/NT for Windows NT, and 4OS2 for OS/2 from JP Software. While not compatible with any Unix shells, these offerings are also cheaper than the other commercial shells.

There are a number of freeware shells out there. What follows are just a few that I found. One I was pointed to was bash for Windows NT and Windows 95, available at ftp://ftp.cygnus.com/pub/gnu-win32/latest/i386/bash.zip. Several OS/2 shell ports are available at ftp://hobbes.nmsu.edu/os2/unix/. In this directory you'll find pdksh (ksh527rt.zip), bash (bash_112.zip), zsh (zsh01.zip), and tcsh (tcsh6061.zip). There's a Korn shell clone for MS-DOS, OS/2, and Windows NT, called the "MS-DOS Shell", available at ftp://ftp.simtel.net/pub/simtelnet/msdos/sysut/ms_sh23b.zip. Also in the same archive is csh4.zip, which contains a shell.exe program for MS-DOS that is patterned after the C-shell (it seems to be very old, though).

There are also some shell ports available for AmigaDOS. You'll find sh and pdksh for the Amiga in the Amiga Developers Environment archives under ftp://ftp.ninemoons.com/pub/ade/, along with many other tools. A csh-like shell is in the Aminet archives, at ftp://ftp.netnet.net/pub/aminet/util/shell/csh548.lha.

References

Many books about shells are available, though it's much easier to find books for some shells than others. Personally, I learned the hard way, so I don't have any particular books to recommend. Start with the man pages, which tend to be fairly good references (for example, type "man 1 sh"). Question 1.5 of the Unix FAQ mentions two places to find Unix book lists, but the first place is a few years out of date, and the other one is defunct. General books about Unix would also be helpful, since many of the utilities you use in a shell script aren't a part of the shell itself.

There are several places to find most of the FAQ files listed below. See the file "Introduction to the *.answers newsgroups" in the news.answers newsgroup to learn how to find them. I list the FAQ titles as they appear in news.answers, and I list archive names in parenthesis for searching the ftp archives.

A nice treatise on the various shells is the "UNIX shell differences..." (unix-faq/shell/shell-differences) FAQ file. This FAQ contains a brief history of the shells, and a feature matrix that makes comparing features easy.

Another good place to look is the "Unix - Frequently Asked Questions" FAQ files (unix-faq/faq/), (the "Unix FAQ"). Part 5 contains a categorization of the shells and a nice list of the various "dot" files used by the shells. Part 5 also has a pointer to another document containing a feature comparison.

The comp.unix.shell newsgroup is a very active interchange for information about shells. Read the "Welcome to comp.unix.shell" FAQ (unix-faq/shell/intro) and heed its advice before posting.

I was able to find a few FAQs for specific shells. There is the "Z-shell Frequently-Asked Questions" (unix-faq/shell/zsh) file. And Chet Ramey posts the "BASH Frequently-Asked Questions" file to comp.unix.shell. You won't find the bash FAQ in the news.answers archives, but it is available at ftp://slc2.ins.cwru.edu/pub/bash/FAQ.

Tom Christiansen's "Csh Programming Considered Harmful" file (unix-faq/shell/csh-whynot) doesn't get posted regularly, but you'll still find it in the FAQ archives (version 1.7). An older version (1.6) is available at ftp://perl.com/pub/perl/versus/csh.whynot.gz

I have compiled a more verbose list of information about non-Unix shells and proposed that it be added to one of the comp.unix.shell FAQ files. Contact me via Ridgetop Publishing if you'd like a copy.

Here's how to contact the vendors mentioned herein:

Global Technologies Ltd., Inc.
5 West Avenue, Old Bridge, NJ 08857
PO Box 489, Amherst, MA, USA 01004-0489
908-251-2840

Hamilton Laboratories
21 Shadow Oak Drive
Sudbury, MA 01776-3165
508-440-8307, hamilton@bix.com

JP Software
P.O. Box 1470
East Arlington, MA 02174
617-646-3975, service@jpsoft.com, http://www.jpsoft.com/

Mortice Kern Systems, Inc.
185 Columbia Street West
Waterloo, Ontario, N2L 5Z5, Canada
800-265-2797, sales@mks.com, http://www.mks.com


Back to the home page