IBM is donating its Journaled File System (JFS) technology to the Linux open source community with the goal of completing a port to Linux. The JFS technology, which is currently used in the IBM OS/2 Warp Server, is now an open source project on developerWorks (see Resources). The code is available under the GPL license.
IBM has started the port to Linux and is making the source code available so the community can participate in the effort. Key functions such as format, file system mount/unmount, and directory list are operational on Linux today.
"We've ported the code to the point where it is a buildable work environment for the community," said Mark Peloquin, file systems specialist in the IBM Linux Technology Center. The code is not fully operational on Linux; rather it is at what the team considers the minimum level to enable the community to build the code, evaluate it, and grow the effort.
"We'll continue to drop code as we develop it," said Steve Best, team leader for the project at IBM. "Getting the community involved is the best way to get feedback on what we're doing."
Efforts to improve Linux scalability and failure recovery have taken on a new urgency as Linux grows in popularity and scope. In other words, no one can afford to wait hours to recover from a system crash, which can be the case for a large system running a non-journaled file system today. JFS can restore a file system to a consistent state in a matter of seconds or minutes.
JFS provides improved structural consistency and recoverability and
much faster restart times than
non-journaled file systems such as DOS FAT, OS/2 HPFS, traditional
UNIX file systems, and ext2, which is commonly used in Linux today. These
other file systems are subject to corruption in the event of system
failure because a logical file write operation
often takes multiple I/O operations to accomplish and may not be totally
reflected on the media at any given
time. These file systems rely on restart-time utilities that examine all of the file system's meta-data (such as directories and disk addressing structures) to
detect and repair structural integrity problems.
"This is a time-consuming and error-prone process, which,
in the worst case, can lose or misplace data," said Best. "In contrast, JFS uses techniques originally developed for databases to log
information about operations performed on the file system meta-data as
atomic transactions. In the event of a system failure, a file system is restored to a consistent state by replaying the log and applying
log records for the appropriate transactions."
The recovery time associated
with this log-based approach
is much faster. The replay utility need only examine the log records
produced by recent file system
activity rather than examine all file system meta-data.
The significance of having a journaling file system in Linux goes beyond a new and improved failure recovery mechanism. "Such file systems are the first baby steps to sophisticated clustering systems," said Dan Kuznetsky, an analyst with International Data Corp. "Journaling file systems will allow Linux to build a clustering technology step by step: making data more available, applications more available, and improving performance, availability, and common management systems."
The origins of the IBM technology
The JFS open source code on developerWorks originates from that currently shipping with the OS/2 Warp Server for e-business. As a fairly mature IBM technology, JFS provides the high-performance capability needed for fast system restart and file system restore.
The JFS technology used in OS/2 today can be restarted quickly, is transaction oriented, and highly scalable. It is tailored primarily for the high throughput and reliability requirements of servers, from single processor systems to
advanced multi-processor and
clustered systems, but can also be used in client configurations where performance and reliability needs are high.
JFS was designed to have journaling fully integrated from the start, rather than adding it to an existing file system. When introduced in OS/2 in 1999, JFS expanded file system capacity limits, improved the recoverability of the system in case of server failure, supported larger partitions as well as dynamic expansion, and allowed more flexibility in data storage and management.
The technology has been well-received in the OS/2 world, and IBM hopes to transfer the knowledge and techniques as a contribution to the Linux community.
Bringing journaling file systems to Linux
Several efforts are underway to bring journaling file system technology to Linux. One effort led by Stephen Tweedie, a leading Linux kernel developer, puts journaling into ext2. Called ext3, it is available in alpha form. Namesys has a journaling file system under development called ReiserFS. SGI has announced its intention to port its XFS file system to Linux. (See Resources for all links.)
The IBM team has additional work items in mind beyond merely porting the OS/2 technology to the Linux environment. "For example, Linux is in the process of transferring from a buffer cache technology to a page cache technology," said Peloquin. "We're porting our JFS with buffer caching but are going to make that transition to page caching. We're hoping we can contribute to the design of page caching in Linux to make it as high performing as possible."
IBM will be
working with members of the other journaling file system projects on common design issues, so that a single set of kernel patches will result.
"Our primary goal is to get the best journaling file capability in Linux," says Peloquin, "whether it's ours or aspects of ours in combination with others. We're offering this technology in the hopes that it can be used -- and the community will decide."
About the author
Jeanne Murray is editor of the developerWorks open source zone. Her decade of experience in software design and development at IBM in Research Triangle Park, NC, was preceded by years of toiling as a consultant and freelance writer in the Washington, D.C., area. She can be reached at