Home | FAQ | Mailing Lists | Download | Printing | Developers | Bugzilla | CVS | GAR Doc
 





 

 





 

LNX-BBC: GAR Architecture


  1. Overview
  2. User's Perspective
    1. Building packages with the system
    2. Configuring the GAR system
  3. Maintainer's Perspective -- Basics
    1. An Example
    2. Dependencies
    3. Checksum file
  4. Maintainer's Perspective -- When defaults aren't enough
    1. pre- and post- rules
    2. Overriding default behavior
    3. Putting the two together
    4. Patching
    5. Providing nonexistent behavior
    6. Manifest-based installs
  5. Hacker's Perspective
    1. gar.conf.mk
    2. bbc.gar.mk
    3. bbc.gar.lib.mk

1
Overview

The GAR system is a similar system to the BSD ports collection. It's a mechanism for automating the compilation and installation of third-party source code. It appears in the form of a tree of directories containing Makefiles and other ancillary bookkeeping files (such as installation manifests and checksum lists).

Of interest to people wishing to learn the GAR system is this overview of the ports collection.

2
User's Perspective

From the User's perspective, the GAR system may well be a tree of carefully-maintained source code, ready to compile. The reality is that the system automatically downloads and customizes third-party source code archives. It builds necessary dependencies in the right order, and generally takes care of a lot of the work associated with compiling and installing software.

2.1: Building packages with the system

The GAR system provides seven basic targets for each package:

  1. fetch
  2. checksum
  3. extract
  4. patch
  5. configure
  6. build
  7. install
The user who is building a package will use the system by entering your package's directory and running "make install" or "make patch" (for example).

In the base GAR directory are a number of files that can mostly be ignored. The really interesting stuff is in the subdirectories. There is one subdirectory for each category of package (shells, editors, development, games, etc), and inside each category directory is a directory for each package in that category. Thus, the shells/ dir will have directories named bash/, tcsh/, zsh/, ash/, sash/, and so on.

To build and install a bash, for example, you simply cd into shells/bash/ and run "make install". Couldn't be simpler, right?

2.2: Configuring the GAR system

If you wish to configure the way the GAR system behaves, this is usually done by editing the file gar.conf.mk, which lives in the base GAR directory. In here you'll see a number of basic variables for things like the installation prefix, build prefix (for things like header files that are used as part of the build process but may not need to be installed), CFLAGS, a local file repository, and master backup site for source downloads.

For most cases, only the prefix and exec_prefix variables will need to be changed.

3
Maintainer's Perspective -- Basics

From the package maintainer's perspective, the GAR system is a library of make targets that can be configured by setting certain special variables or overridden file-by-file. To make a GAR package, you make a directory for a package inside the directory representing its category. Inside this directory, you create a Makefile describing your package.

3.1: An Example

The best way to understand the process of making a new package is to look at an example:

GARNAME = grep
GARVERSION = 2.4.2
CATEGORIES = utils
MASTER_SITES = ftp://ftp.gnu.org/pub/gnu/grep/
DISTFILES = $(DISTNAME).tar.gz 

DESCRIPTION = grep
define BLURB
 The GNU family of grep utilities may be the "fastest grep in the west".
 GNU grep is based on a fast lazy-state deterministic matcher (about
 twice as fast as stock Unix egrep) hybridized with a Boyer-Moore-Gosper
 search for a fixed string that eliminates impossible text from being
 considered by the full regexp matcher without necessarily having to
 look at every character. The result is typically many times faster
 than Unix grep or egrep. (Regular expressions containing backreferencing
 will run more slowly, however.)
endef

CONFIGURE_SCRIPTS = $(WORKSRC)/configure
BUILD_SCRIPTS = $(WORKSRC)/Makefile
INSTALL_SCRIPTS = $(WORKSRC)/Makefile

CONFIGURE_ARGS = $(DIRPATHS)

include ../../bbc.gar.mk

The first few lines set some basic bookkeeping info. The GARNAME and GARVERSION refer to the package name and version string as the GAR system will manipulate them. They are also used to create the convenience variable DISTNAME, which defaults to $(GARNAME)-$(GARVERSION) (since most GNU or automake-using packages name their tarballs in that fashion).

The fetch target will, by default, try to grab $(MASTER_SITES)$(DISTFILES) for each of the space-separated entries in either variable.

Next are some basic description variables. The DESCRIPTION variable should be a short, one-line description of the package, while the BLURB is a longer multi-paragraph description. This BLURB is taken straight from the debian package for grep.

The actual build of the system depends on the CONFIGURE_SCRIPTS, BUILD_SCRIPTS, and INSTALL_SCRIPTS variables. These point to a space-separated list of files that are essential to the configure, build, and install steps (respectively).

The GAR system is pretty smart, and will know what to do with most types of scripts. It knows to just run CONFIGURE_SCRIPTs named "configure", and to run "make" or "make install" in the directory where a Makefile lives. It also knows that Imakefiles use xmkmf, and so forth. For most packages, no special magic needs to happen here.

We usually, however, need to specify the CONFIGURE_ARGS to include the directory settings that we define in gar.conf.mk. This currently requires setting it to include $(DIRPATHS). This may, in future, not be necessary for packages that use autoconf-generated configure scripts. Some older configure scripts don't accept all of the standard directories, so the package maintainer must set them by doing something like:

NODIRPATHS += --bindir --sbindir --libexecdir --datadir --infodir --sysconfdir --sharedstatedir --localstatedir --libdir --includedir --mandir

But this is uncommon.

Finally, we include the bbc.gar.mk library. This takes all of the variable setting we've done and puts it to good use. This allows your little teeny Makefile to provide the seven basic targets described in the User Perspective section. It must come last (or, at least, after all of the variables have been set) in order to function properly.

3.2: Dependencies

XXX: write this section.

3.3: Checksum file

The package isn't complete until there is a checksum file. Finish up your Makefile, and then run "make fetch". If all the files look correct to you, run "make makesum" and check the resulting "checksums" file into CVS alongside the Makefile. This checksums file will be consulted during the "make checksum" phase.

4
Maintainer's Perspective -- When defaults aren't enough

Quite often, the default behavior isn't what one would hope. You can pass parameters to your configure script with CONFIGURE_ARGS, but what if the GAR system doesn't know about your configuration script type? What if the configuration steps aren't enough? What if they're flat-out wrong? What if the same is true for the fetch, extract, or install rules?

Fortunately, the system provides for this in a number of ways. What follows are a few of the mechanisms that a package maintainer can use to override or enhance the default behaviors.

4.1: pre- and post- rules

For each of the seven basic targets, there exist slots for per-package pre- and post- rules. That means that the package maintainer can specify work to be done immediately before or after a rule.

As an example, let's consider the util-linux package. It doesn't use a standard autoconf-style configure script, but it can be configured by setting a variable at the top of the MCONFIG file. Thus, the end of our utils/util-linux/Makefile looks like the following:

CONFIGURE_SCRIPTS = $(WORKSRC)/configure
BUILD_SCRIPTS = $(WORKSRC)/Makefile
INSTALL_SCRIPTS = $(WORKSRC)/Makefile

CONFIGURE_ARGS = $(DIRPATHS)

pre-configure:
	echo "DESTDIR=$(prefix)" > $(WORKSRC)/MCONFIG.NEW
	cat $(WORKSRC)/MCONFIG >> $(WORKSRC)/MCONFIG.NEW
	mv $(WORKSRC)/MCONFIG.NEW $(WORKSRC)/MCONFIG
	$(MAKECOOKIE)

include ../../bbc.gar.mk

Thus, before the configure script is run, the package-defined pre-configure rule adds code setting DESTDIR to the $(prefix) variable in MCONFIG.

As another example, the Bourne Again SHell (bash) can be linked to with the name "sh" in order to cause it to behave (somewhat) like a POSIX Bourne Shell. Thus, the end of our shells/bash/Makefile looks like:

CONFIGURE_SCRIPTS = $(WORKSRC)/configure
BUILD_SCRIPTS = $(WORKSRC)/Makefile
INSTALL_SCRIPTS = $(WORKSRC)/Makefile

CONFIGURE_ARGS = $(DIRPATHS)

post-install:
	(cd $(bindir); ln -sf bash sh)
	$(MAKECOOKIE)

include ../../bbc.gar.mk

Thus creating the symbolic link to sh.

4.2: Overriding default behavior

Sometimes it's not enough to merely enhance the behavior of a rule. Sometimes the default handler for a given script is dead wrong, or the script has no default handler. In this case, it is easy enough to provide your own handler.

For example, the Perl package has a file called configure.gnu that needs to be run. The GAR system doesn't know that it's just like a standard configure script, so we have to tell it like so:

CONFIGURE_SCRIPTS = $(WORKSRC)/configure.gnu
BUILD_SCRIPTS = $(WORKSRC)/Makefile
INSTALL_SCRIPTS = $(WORKSRC)/Makefile

NODIRPATHS += --exec_prefix --bindir --sbindir --libexecdir \
  --datadir --infodir --sysconfdir --sharedstatedir         \
  --localstatedir --libdir --includedir --mandir

CONFIGURE_ARGS = $(DIRPATHS)

configure-$(WORKSRC)/configure.gnu:
	mkdir -p $(COOKIEDIR)/configure-$(WORKSRC)
	cd $(WORKSRC) && $(CONFIGURE_ENV) ./configure.gnu $(CONFIGURE_ARGS)
	$(MAKECOOKIE)

include ../../bbc.gar.mk

There's some magic surrounding this that just has to be taken on faith. First, one must make a directory in $(COOKIEDIR) that has everything but the filename from the rule's name (unless the rule doesn't have any slashes in it). This isn't that clean, and may be unnecessary in future.

Next, you do the work you wanted to do. In this case, we do exactly what happens in the standard library routine for configure scripts, but we use configure.gnu. Notice that we put CONFIGURE_ENV in even though we don't use it.

Finally, we run $(MAKECOOKIE) to signify that this step is complete. This lets GAR know where to pick up where it left off if the build only gets partway through.

4.3: Putting the two together

The following example is from the net-tools package, which has a bizarre interactive configuration system. The maintainer automated this by setting some variables at the top of the package's Makefile and then piping the output of the "yes" program into the configurator in order to accept all defaults.

CONFIGURE_SCRIPTS = $(WORKSRC)/Makefile
BUILD_SCRIPTS = $(WORKSRC)/Makefile
INSTALL_SCRIPTS = $(WORKSRC)/Makefile

pre-configure:
	echo "BASEDIR=$(prefix)" > $(WORKSRC)/Makefile.NEW
	cat $(WORKSRC)/Makefile >> $(WORKSRC)/Makefile.NEW
	mv $(WORKSRC)/Makefile.NEW $(WORKSRC)/Makefile
	$(MAKECOOKIE)

configure-$(WORKSRC)/Makefile:
	# if you want non-default answers to the "make config"
	# questions, you'll have to go through and answer the
	# questions the way you want, and then preserve the
	# resulting file (instead of the action below, which
	# simply uses the defaults every time)
	yes '' | make -C $(WORKSRC) config
	$(MAKECOOKIE)

include ../../bbc.gar.mk

4.4: Patching

Sometimes the system simply won't build from pristine upstream sources, no matter how much fiddling you do in pre- and post- rules. For instances like these, we provide patches with our packages.

To make a patch, run "make extract" and then go make your changes. Once you're done making changes, run "make makepatch". This will create a file named "gar-base.diff" in the files/ dir. Check this patch into cvs and add it to the PATCHFILES variable in your Makefile.

In addition, the system will download any third-party patches from the list of MASTER_SITES if it can't find them on the local system.

Here's the util-linux package again as an example. We made the file utils/util-linux/files/no-chown.patch as a patch to the original sources. Then in our Makefile we put:

PATCHFILES = no-chown.patch

4.5: Providing nonexistent behavior

Suppose you want to perform a configure step, but there is no actual program or script associated with the step of configuration. You could conceivably want to run a series of shell commands to create a configuration file. Since there is no actual script to run, you would specify "none" as the CONFIGURE_SCRIPTS.

For example, look at the way the tracesroute package performs its build step. There was no makefile, so the maintainer had to provide a method for the compilation commands to be run semi-manually. Thus, the BUILD_SCRIPTS is set to "none", and a build-none target is created.

WORKSRC = $(WORKDIR)

BUILD_SCRIPTS = none

build-none:
	$(CC) $(CFLAGS) -o $(WORKSRC)/tracesroute $(WORKSRC)/traceroute.c -lresolv -lm
	$(MAKECOOKIE)

include ../../bbc.gar.mk

It is worth noting that WORKSRC is set to WORKDIR because the .c file is extracted via extraordinary means.

It is the opinion of the GAR authors that any package supplying a -none rule should properly provide a generalized rule for inclusion in the bbc.gar.lib.mk file. It may or may not be useful; however, for there to be a generalized default rule that turns .c files into executables.

4.6: Manifest-based installs

One of the most common occasions where a package maintainer would like to use a -none rule is when installing software that has no built-in installation mechanism. Moving files from location to location is common enough that GAR provides a general mechanism for this.

A package maintainer can specify the locations, permissions, and modes of a file (or collection of files) in a file called manifest. This file contains a list of installation tuples (one per line) of the following format:

source:destination[:mode[:owner[:group]]]
An installation tuple may use shell variables taken from the GAR environment, but must enclose them in curly braces instead of parentheses. Thus, a sample manifest file may contain the following:

${WORKSRC}/nwall:${bindir}/nwall:2755:root:tty
${WORKSRC}/src/foo:${sharedstatedir}/foo
${WORKSRC}/yoink:${sysconfdir}/yoink:0600

Note that all options after the destination location for the file are optional, but one must provide all preceeding options in order to define a given optional field. Thus to specify the owner of a file, one must also specify the mode.

5
Hacker's Perspective

5.1: gar.conf.mk

5.2: bbc.gar.mk

5.3: bbc.gar.lib.mk