Making the Digital Environment Talk SHEEPby Simon Goodwin and Robert Bryant
Ed: The code samples in this article may contain indentation errors. Please download the PDF version of this edition of Amiga World for the correct code samples.
Scripting is a great strength of the Classic Amiga, because it allows users to combine and automate operations from all different programs in the system, passing data automatically back and forth, using the best features of each to get results that no one program, however bloated, could match.
The new central scripting language of the Amiga Digital Environment (DE) is code-named SHEEP (This is an internal working name for this language. The final name will be different). It handles scripting, querying, Shell and inter-object communication, and general programming. It enables the user to interact with objects in the AmigaDE, and allows objects within the DE to control each other. It is the glue that holds the flexible Digital Environment together and makes everything universally scriptable, configurable and modifiable.
Having a language like SHEEP as an integral part of a system enhances its usefulness, as most applications will then be written with scriptability in mind. That integration allows users to harness the functionality available in the OS or applications and combine it with custom code to fill in what's missing. SHEEP will handle both GUI and command-line interaction and make it all as painless as possible.
"If you want to get some job done on your computer, either it is very easy or it is extremely difficult," says Wouter van Oortmerssen. The decisive factor, he says, is whether you have an application suited to the job at hand. "And even when you have a GUI app that can do what you want to do, GUI's are traditionally very inflexible, especially for repetitive tasks. For those [times], there are very few alternatives to using a scripting language," he says.
Heading up the SHEEP project for Amiga is Wouter van Oortmerssen, a programmer with a stellar reputation on the Classic Amiga who has jumped at the chance to help design the new one. Wouter is best known for his concise yet powerful Amiga-E language, yet this is just one of more than a dozen languages he's designed en route to his doctorate in Computer Science. He's now working full time for Amiga Inc., developing SHEEP.
Tim Sweeny, creator of the 3D shooter Unreal and programmer extraordinaire, summed up his thoughts of Wouter's talents this way: "Wouter is a brilliant programming language designer, and possibly the most prolific one on Earth--he wrote the E programming language back in the early Amiga days, and has since implemented a ton of imperative, functional, visual, reactive, linear, and other crazy languages. We email once in a while, and whenever I describe some new language idea I have, he's like: 'I implemented that 5 years ago, and here are the problems you run into...'"
The Classic Amiga scripting language was ARexx, a standard component since release 2 a decade ago. ARexx was derived from Rexx, the original IBM mainframe version, with many Amiga-specific extensions. Versions of Rexx have since been ported to many other systems, like OS/2, Unix and Qdos. Rexx is a general-purpose high-level language, versatile yet easy to learn.
The true significance of ARexx stemmed from the way it became a standard part of the Classic Amiga. Almost every system component, from applications to the desktop, and even emulators for other systems, had an ARexx port, so any operation that could be performed manually with the mouse or keyboard could be automated under ARexx control. In some cases this allowed operations or custom configurations that were only feasible with ARexx support.
There was no need for each program to have its own arcane scripting language built-in, as has led to notorious compatibility and security problems in clumsier systems. While the idea was brilliant, well-executed and integrated, ARexx was not perfect-it dated back to the 1980s, and could be slow, prolix or clumsy at times. It did a lot, but could have done more.
Before ARexx became part of the system, old Amigas were shipped with an implementation of Microsoft BASIC. As on many home computers, users were encouraged to write their own programs in this dialect, extended with support for Amiga multimedia. While superior to their other offerings, Microsoft BASIC was slow, big and buggy. It was inflexible and incompatible with 32-bit systems, so third party packages like Amos, Blitz, Maxxon and HiSoft BASICs arrived in the 1990s to fill the gap between ARexx and professional programming languages like C, Oberon and hardcore assembly language.
SHEEP fills that niche, as well as the need for a standard, powerful scripting language. It's quite possible to write applications and utilities in SHEEP, and is in fact a very efficient approach for building prototypes or solving the sort of problems that might keep one programmer busy for anything from a few minutes to a few days. While compilers for Java, C, C++ and our ultra-efficient portable VP-code are more suitable for larger projects or routines likely to be run many times, SHEEP's interpreted, functional design delivers results sooner, if not always more quickly. It's ideal for interactive development and testing, an area where compiling languages cannot compete.
ARexx was tightly integrated with the Amiga OS and applications and "allowed you to do the kind of programming that just isn't possible in any other way," says van Oortmerssen. "Besides, ARexx was also a nice design that fitted its purpose, simple and friendly so everyone could pick it up and do useful things with it quickly."
You can go from editing a SHEEP program to running it, and back into the editor of your choice, immediately. Future versions will allow your SHEEP code to be compiled after it has been tested, and to make secure, stand-alone programs that outperform Java and match the speed of compiled C++ in many cases.
SHEEP is ideal for installation scripts, communication between programs-much like ARexx, including access to ARexx in emulated Classic Amiga environments-and the sort of hacks that would be written in shell script, BASIC, E, Perl or Python on old systems. However, SHEEP is far more secure than ARexx, C or BASIC. There's no possibility of accidentally overwriting memory or clobbering some system resource unrelated to your program. SHEEP forswears the dangerous POKEs and pointers that fill the gaps in other languages, yet it can manipulate complex data structures simply and efficiently.
SHEEP can be used for database queries, as SQL (Structured Query Language) is on big systems. The new Amiga boasts immensely powerful ways of storing and organizing data; SHEEP provides a clear window on the Digital Environment.
Breeds of SHEEP
SHEEP can be interpreted, for ease of maintenance, or compiled into fast VP code. SHEEP is a synthesis of several Classic Amiga concepts and good ideas from elsewhere, into a powerful yet simple language that is easy to read and write, without the arbitrary and arcane punctuation of many other languages.
SHEEP works without the 'garbage collection' that unpredictably wastes time and memory in most other languages that allow dynamic data-structures. It does this by using new functional programming principles and automatic memory management, without forcing the programmer to use it any differently from conventional languages.
SHEEP is polymorphic so you can write programs oblivious of the exact types of data they will manipulate, and re-use code without making type-specific versions or complicated, slow and error-prone run-time tests. SHEEP lets you specify data types when it could make programs more efficient if they were preordained, but it doesn't force you to do so in more general cases. New types can be defined, with any combination of data inside. Types can be a superset of several types. Thus a tree with any number of branches or leaves can be defined in one line:
This defines two types, 'branch' and 'leaf,' and a supertype, 'tree,' which can be used to refer to the value of either of those types. The vertical bar indicates that a tree (or subtree, implicitly) can be either a branch made up of further (sub) trees or a leaf, with a 'data' property.
Every value has a type, but it has the most general type 'any' unless you specify otherwise. If a general type is passed into a more specific context, as when a string of characters is passed to a routine expecting an integer, SHEEP coerces the string to a number, and raises a dynamic type error if it won't fit. If both types are strictly defined, the compiler detects mismatches before the program runs.
"The new Amiga OS, is much more based on objects," says van Oortmerssen, "which will automatically expose their programmable interfaces to the world. The potential for ARexx-like scripting is therefore far greater." SHEEP provides a window to the AmigaDE and its immensely powerful ways of storing and organizing data.
SHEEP reports errors in context. If in doubt, it issues a warning. SHEEP has built-in error-tracking and exception handling, so if something goes wrong you can identify the problem and fix it, or alter the program so that the special case will be spotted and resolved automatically, without cluttering up the source with conditional tests. This gives beginners support and reassurance, without cramping the style of experts.
SHEEP uses the concept of 'vectors' to express structures normally stored in static arrays or dynamic lists. SHEEP vectors combine the efficiency of arrays with the dynamism of lists. Complete or partial vectors may be compared, extended or replaced.
Functions may take any number of parameters, applying defaults or patterns if necessary, and may return any number of values. You can tag parameters when you call a function, so you need only specify the parameters you care about, and they may be in whatever order you like. Thus you could specify desired details of a new screen, letting others default, by calling an 'openscreen' function like this:
You can define several instances of a function, depending on the type or value of parameters you want to deal with. SHEEP automatically chooses the right one to match a particular function call. Gratuitous brackets are not needed, either in the call or the function definition.
Pattern matching is a key feature of SHEEP, simplifying programs. SHEEP can search out patterns within lists, splitting them according to the position of the pattern. It can just as readily merge patterns and lists. Pattern matching also allows particular values to be extracted from user-defined data types.
There are several neat ways to test large amount of data in one step. 'Find' performs a conditional search through a vector, returning an index of the first match. 'Filter' is similar but returns a vector denoting all the matches. 'Fold' combines all the matches, accumulating a total.
Sheep is a no-limits language, with no restriction other than available memory on the length of strings, vectors, identifiers, programs or individual lines.
SHEEP notes the indentation of the first line in a list. If a new line is indented the same amount as the previous one, it's a new statement. If it's indented more, it's taken as a continuation, allowing long statements to be presented neatly. If it's indented less, the next token implies the end of the list.
One striking feature of SHEEP is the way that you can teach it rules in a simple format, and expect it to find and apply them in context. You can add functions, or definitions, without having to wrap them in obscure syntax. To take a very simple example, imagine you need to report the number of files a program has copied in a human-friendly way. Rather than resort to a sequence of tests, or sub-literate laziness like "n file(s) copied," you can write:
Here's a slightly more complicated example-an implementation of the QuickSort algorithm, which requires a couple dozen lines of code in many languages. SHEEP can do it in just four (the lines starting with two dashes are comments) with no weird symbols apart from brackets, plus signs and arrows, all of which have relatively obvious meanings.
You'll notice there's no need to declare the types of data items like x, rest or pivot. This is not sloppiness-the data type simply doesn't matter to SHEEP. The same code will happily and efficiently sort text, whole numbers, or fractions, whereas old-fashioned languages would need a separate routine for each.
In this case,
You don't have to warn the program about the length of the vector or check it inside the program. One size fits all, as the program is recursive-it works by calling itself repeatedly, splitting the list into smaller parts until they're all in order, but you don't have to worry about programming or checking the stack-SHEEP looks after that for you.
SHEEP delivers some of the same excellent properties as ARexx: "It's simple and friendly right from the start," says van Oortmerssen. "ARexx, however, had the problem that people easily outgrew it: As soon as they mastered the language, they noticed that slightly larger scripts would run dreadfully slow and easily become messy. SHEEP is designed to last you a lot longer," he says.
The definition of what order the sorted list should be delivered in is similarly flexible. The term 'before' in the listing could have any name, and any definition associated with it which can determine the order of two items. If we want to sort a vector of whole numbers, known as int(egers), we define 'before' like this:
This says that when 'x' and 'y' are integers, 'before' is true if 'x' is less than (<) 'y.' Equivalent rules may be defined for other types of data, and you need not be limited by them-if you want punctuation at the end of a sorted list of words (rather than in ANSI or Unicode order), you may define an arbitrary function to put things the way you want them.
The 'filter' operation scans through a list, assigning each value in turn to 'x' and asking 'before x pivot?' If this is true, the value 'x' precedes the value 'pivot', so 'x' is moved to the result. Once all the 'rest' has been scanned, the list is more nearly in order, as early values have been removed. 'Quicksort' calls itself for progressively shorter lists, until every value is in its proper place.
Splitting and Joining SHEEP
The expression '[pivot] + rest' shows how lists of values (or 'vectors') may be split and joined in SHEEP. The expression takes a vector and assigns the first element to 'pivot' and the remainder to 'rest.' The '+' operation can both join and split patterns. It allows us to create vectors like this:
This leaves 'a' holding the vector [1,2,3,4,5,6]. The '+' operation constructs vectors, while pattern matching inspects and deconstructs them. Imagine a function 'chop,' which is passed a vector. The following instruction takes the vector apart:
The pattern matching operation splits the parameter in one vector of length two, which must contain the integers 1 and 2, and puts the remainder of the vector into 'v,' so 'v' becomes [3,4,5,6] if 'chop' is passed the value assigned to 'a.'
The last example shows how SHEEP can express a classic graphics algorithm. It assumes that a device with appropriate width and height is ready to accept output from the 'plot' statements.
This example shows that the functional sophistication of SHEEP is not an obstacle to conventional programming. Apart from the lack of syntactical clutter, the Mandelbrot listing could be in any block-structured language. It can be run and edited as readily as interpreted BASIC, yet has the speed and structure of a modern compiled dialect:
-- SHEEP Example Mandelbrot fractal generator
When there are already thousands of computer languages in existence, you need special reasons to propose a new one, let
alone develop something as sophisticated as SHEEP, which is simple on the surface yet built on cutting-edge Computer
SHEEP borrows concepts from classics like ARexx, Prolog, BASIC and Perl, and newer languages like Python and Java, and
benefits from the practical experience of 15 years of Classic Amiga product development and integration. SHEEP adds
significant new ideas, eliminates dangerous or over-complicated gimmicks, and has the cohesion that comes from one
brilliant mind, rather than a committee of lobbyists.
SHEEP demonstrates that if a job's really worth doing, it's worth approaching afresh. SHEEP not only is something new
and exciting-it allows and facilitates great things that would not otherwise be practical. It is a key component of the
new Amiga, and nothing else can match it.
When there are already thousands of computer languages in existence, you need special reasons to propose a new one, let alone develop something as sophisticated as SHEEP, which is simple on the surface yet built on cutting-edge Computer Science within.
SHEEP borrows concepts from classics like ARexx, Prolog, BASIC and Perl, and newer languages like Python and Java, and benefits from the practical experience of 15 years of Classic Amiga product development and integration. SHEEP adds significant new ideas, eliminates dangerous or over-complicated gimmicks, and has the cohesion that comes from one brilliant mind, rather than a committee of lobbyists.
SHEEP demonstrates that if a job's really worth doing, it's worth approaching afresh. SHEEP not only is something new and exciting-it allows and facilitates great things that would not otherwise be practical. It is a key component of the new Amiga, and nothing else can match it.
© 1996 - 2000 Amiga Inc. | firstname.lastname@example.org