Welcome to Sleepycat. Sign in or register.

Open Source

Sleepycat's Berkeley DB is very popular in the open source community. Berkeley DB can be found in many open source projects, including all versions of Linux and BSD UNIX, OpenLDAP, sendmail, Kerberos, various Apache projects, Subversion, MySQL, OpenOffice and many others.

A partial list of open source projects using Berkeley DB is provided here:

Operating Systems

All distributions of Linux, including Red Hat, Novell/SuSE, Debian GNU, Gentoo, Mandrake, TurboLinux, MontaVista and others, use Berkeley DB.

All variations of BSD UNIX, including FreeBSD, NetBSD, OpenBSD and others, use Berkeley DB.

Infrastructure

OpenLDAP is an open source implementation of the Lightweight Directory Access Protocol (LDAP). LDAP allows users on a network to do simple query operations quickly, without the overhead of a heavyweight relational database server and complex query language. OpenLDAP directory entries are stored in Berkeley DB, and the OpenLDAP engine converts queries into operations on the Berkeley DB database.

Red Hat Directory Server is an LDAP-based server that centralizes application settings, user profiles, group data, policies, and access control information into an operating system-independent, network-based registry. The Red Hat Directory Server is based on Berkeley DB, as are the LDAP directories from AOL and Sun Microsystems.

Apache uses Berkeley DB for basic HTTP authentication. Apache Directory Server is an embeddable LDAP server written in Java. It uses Berkeley DB Java Edition. SpamAssassin is the leading open source spam filter and uses Berkeley DB to store address lists. The Apache Jakarta Java Caching System (JCS) is a distributed caching system for server-side Java applications. It uses Berkeley DB Java Edition for managing disk storage.

Kerberos is a network authentication protocol. It is designed to provide strong authentication for client/server applications by using secret-key cryptography. A free implementation of this protocol is available from the Massachusetts Institute of Technology. Kerberos is available in many commercial products as well.

MySQL, the popular open source SQL database system, allows users to create tables that are managed by Berkeley DB. The combination of the MySQL query engine with Berkeley DB's storage services gives MySQL users fast, reliable, transaction-protected data management services with all the power and flexibility of the SQL query language.

Netatalk is a freely-available, kernel level implementation of the AppleTalk Protocol Suite, originally for BSD-derived systems. A *NIX/*BSD system running Netatalk is capable of serving many Macintosh clients simultaneously as an AppleTalk router, AppleShare file server (AFP), *NIX/*BSD print server, and for accessing AppleTalk printers via Printer Access Protocol (PAP).

ActiveMQ is an open source, Apache 2.0 licensed, high performance Messaging Fabric consisting of a scalable cluster of Message Brokers and a complete JMS 1.1 provider which integrates seamlessly into Java EE (J2EE) containers, light weight containers and any Java application. ActiveMQ uses Berkeley DB Java Edition for persistence support as it is much faster and has better recovery than alternatives.

Blitz is an open source JavaSpaces implementation designed to ease development and deployment of JavaSpaces technology. It is Jini 2.0 enabled, and uses established VM principles. It also implements smart indexing, tunable persistence, and active/passive lease cleanup. Blitz uses Berkeley DB and Berkeley DB Java Edition for persistence, making it easier to install by eliminating the need for external databases.

Libferris is a virtual file system that runs in the user address space, making the file system and other hierarchical storage systems easier to use. Libferris presents a uniform interface to hierarchical data. Libferris persists data in many ways, including Berkeley DB XML.

Noid is a tool for efficiently generating, tracking, and binding unique identifiers, which are produced without replacement in random or sequential order, and with or without a check character that can be used for detecting transcription errors. Noid creates minters that can bind identifiers to arbitrary element names and element values that are either stored or produced upon retrieval from rule-based transformations of requested identifiers, the latter having application in identifier resolution. Noid minters are very fast, scalable, easy to create and tear down, and have a relatively small footprint. Noid uses Berkeley DB as the underlying database.

Email

Sendmail is the most widely deployed Mail Transport Agent (MTA), or email server, on the Internet. Sendmail is estimated to handle 75% of the world's Internet email traffic.

Postfix provides an open source Mail Transport Agent (MTA) for UNIX-like systems. Postfix uses Berkeley DB to make access control decisions, to transform mail addresses, and to make routing decisions.

Cyrus IMAP server is a high performance, freely available mail server that supports both IMAP4 and POP3. Berkeley DB is used in a number of support roles. The databases that keep track of quota, messages delivered, and the list of folders available on the IMAP server can all use Berkeley DB as the store.

Exim is an open source Message Transfer Agent (MTA). It runs on most Unix-like operating systems, and is included as the default MTA in some Linux distributions. Exim uses Berkeley DB to store information about temporary delivery problems to certain hosts, or routing problems for certain addresses, in order to implement a sensible pattern of retrying. It also uses Berkeley DB for storing aliases and other information that is used during the delivery of email.

SquidGuard is a free, flexible and ultra-fast filter, redirector and access controller plugin for Squid. Together, Squid and SquidGuard give systems administrators a powerful tool for combating spam on the email systems that they manage. SquidGuard lets administrators define multiple access rules with different restrictions for different user groups on a squid cache. SquidGuard stores blacklists and other information in Berkeley DB for efficiency, and performs well even with a million or more entries.

SpamProbe relies on a Bayesian analysis of the frequency of words used in spam and non-spam emails received by an individual person. The process is completely automatic and tailors itself to the kinds of emails that each person receives. Written in C++ for good performance, database access uses Berkeley DB for quick startup and fast term count retrieval.

Tools

RPM, the package manager from Red Hat, is a powerful command line driven package management system capable of installing, uninstalling, verifying, querying, and updating computer software packages, such as Red Hat Linux and other Linux distributions. RPM uses Berkeley DB from Sleepycat Software to store the information about the installed software base.

Subversion is a version control system for managing source code and other files. Subversion has been built from the ground up as a set of libraries encompassing both server- and client-side functions. Built on top of these libraries are the command line and GUI clients. The server uses Berkeley DB as a robust storage system for the version repository. This allows Subversion to provide features such as atomic commits, hot backup, and safety against file corruption.

XEmacs is an open source text editor, but more than that, an application platform. The intent is that anything you can do with your computer, you can do from within XEmacs. For this reason it has C-level support to access functions provided by many external libraries, such as Berkeley DB. A list of external C libraries that XEmacs can be built with is provided on the project Web site's download area.

ht://Dig is a complete World-Wide Web indexing and searching system for a domain or intranet. This system is not meant to replace the need for powerful Internet-wide search systems like Lycos, Infoseek, Google and AltaVista. Instead, it handles searches for a single company, campus, or even a particular subsection of a Web site, and can span one or many webservers over multiple protocols. The ht://Dig project and the related mifluz project use Berkeley DB as the core backend. Berkeley DB provides high-performance, reliability, and great flexibility in data format.

dsniff provides a collection of tools for network auditing and penetration testing. Dsniff, filesnarf, mailsnarf, msgsnarf, urlsnarf, and webspy passively monitor a network for interesting data (passwords, e-mail, files, etc.). Arpspoof, dnsspoof, and macof facilitate the interception of network traffic normally unavailable to an attacker (for example, due to layer-2 switching). Sshmitm and webmitm implement active monkey-in-the-middle attacks against redirected SSH and HTTPS sessions by exploiting weak bindings in ad-hoc PKI. Dsniff automatically detects and minimally parses each application protocol, only saving the interesting bits, and uses Berkeley DB as its output file format, only logging unique authentication attempts.

Cfengine is an autonomous agent and a policy language for building expert systems which administrate and configure large computer networks. Cfengine uses the idea of classes and "intelligent" principles to define and automate the configuration and maintenance of system state, for small to huge configurations. Cfengine uses Berkeley DB to record the state of each host using changing resource measurements and cryptographic checksums so that anomalous states can be detected.

Applications

OpenOffice is a free productivity suite, including word processor, spreadsheet, presentation program, drawing package and database tools. It is compatible with all major office suites, such as Microsoft Office and StarOffice.

Chandler is a next-generation Personal Information Manager (PIM), integrating calendar, email, contact management, task management, notes, and instant messaging functions. Chandler is a product of the Open Source Applications Foundation (OSAF).

Citadel is an open source collaboration and messaging platform, suitable for groupware, email, and BBS applications. Berkeley DB is used throughout the system for all of its databases, including the message store. Citadel's use of Berkeley DB makes it easy to install and administer while providing scalable, high-performance service.

Syncato is a micro content management system and blogging software designed to extract the maximum potential from the content of your posts. All posts in Syncato are stored as XML within the Berkelely DB XML data manager and are searchable using XPath queries. This includes the ability to execute XPath via a URL from within your browser.

KLibido is a KDE program to download encoded articles from the usenet news service, using the nntp protocol. KLibido uses kde and the kde libraries for the interface, Berkeley DB for storing the newsgroups' headers, and UUDeview for decoding the posts.

Enlightenment is a Windows Manager and Desktop Shell for X Windows based systems. One of the foundation libraries is Edb, providing a database convenience library wrapped around Berkeley DB. It is intended to make accessing database information portable, easy, fast and efficient.

Webalizer is a web server log file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser. Webalizer uses Berkeley DB for DNS support.

Programming Languages

Perl is a popular, cross platform scripting language for web applications. There are two Perl modules, BerkeleyDB and DB_File, which provide Perl interfaces to Berkeley DB. Both act as wrappers to the Berkeley DB API. They differ primarily in the complexity of the interface they provide the user. The DB_File module provides a Berkeley DB 1.x compatible interface, even if Berkeley DB 2.x or better is being used behind the scenes. It is intended to be used in simple applications where transactions and locking are unnecessary. The BerkeleyDB module provides an interface to Berkeley DB 2.0 and above. It does not support Berkeley DB 1.x. This module exposes most of the API provided by Berkeley DB, including transactions, recovery, and fine-grained locking for concurrency. Both Perl modules are included in the Sleepycat source release, in the subdirectories perl.BerkeleyDB and perl.DB_File. The DB_File module is also included in the standard Perl source release. In addition, the current versions of both modules are available on CPAN, the Comprehensive Perl Archive Network of mirrored FTP sites.

Python is an object-oriented scripting language often compared to Tcl, Perl, Scheme or Java. PyBSDDB is a set of Python wrappers for the Berkeley DB library, allowing the use of most of the features of DB from the Python language. The DB objects can be used in a variety of ways, from simple persistent dictionaries all the way up to full transaction-protected databases. The Python wrappers allow you to store Python string objects of any length, keyed either by strings or integers depending on the database access method. With the use of another module in the package standard shelve-like functionality is provided allowing you to store any pickable Python object, allowing you to easily create an Object Database.

PHP is a widely-used, general-purpose scripting language that is especially suited for Web development and can be embedded into HTML. Berkeley DB XML provides an XML document store searchable with XQuery, something otherwise unavailable within PHP5.

Ruby is an object oriented scripting language. The following link provides a Ruby interface to Berkeley DB and Berkeley DB XML.

ScriptBasic is a script interpreter for the BASIC language type, provided under the GNU LGPL. There is a command line version of the interpreter, and third-party versions embedding the language into applications. The interpreter is modular and has a clean and well-defined C API either to embed or to extend the language. One module available for ScriptBasic allows developers to use Berkeley DB databases from the BASIC programming language. The Berkeley DB API is simplified, but is sufficient for most BASIC programs. It includes record-handling as well as transaction support.

The GNU C Library is primarily designed to be a portable and high performance C library. It follows all relevant standards (ISO C 99, POSIX.1c, POSIX.1j, POSIX.1d, Unix98, Single Unix Specification). It is also internationalized and has one of the most complete internationalization interfaces known. The code for the database library libdb comes from Berkeley DB.


Open Sources 2.0
by Chris DiBona, Mark Stone, and Danese Cooper, published by O'Reilly. Sleepycat's CEO, Mike Olson, wrote the "Dual Licensing" chapter. You can get a copy from either Amazon or O'Reilly.

Sign up for the
Sleepycat newsletter
Email

Categories