Guideline

Home > About Us > What We Publish > Guidelines > Guideline

Safeguarding Australia’s Web Resources: Guidelines for Creators and Publishers

BACKGROUND

On-going access to information resources published on the web is under threat due to changes in the computer technologies that are needed to use them. Dependence on particular hardware and software may result in resources becoming unusable when their support technology becomes obsolete. Australian web resources form a significant part of our documentary heritage and action must be taken now to safeguard them for use into the future.

Preservation solutions for web resources are still evolving. There are as yet no internationally agreed approaches for ensuring the resources will remain available for use over the long term although possible solutions are being explored. However, there are steps that can be taken now to ensure resources are appropriately organised, documented, and managed for needs beyond immediate access. These steps are an essential pre-requisite to later preservation action that might be required when solutions are available.

Creators, publishers, libraries and other information repositories all have a vital role to play in ensuring web resources remain available for use.

These guidelines are intended primarily to assist those creators and publishers who do not already have well established digital data management procedures in place. They could, for instance, be working in the academic, government and not-for-profit sectors, and include independent creators and newly established web publishers. Types of resources include those deemed by their creators to have lasting research or information value, and which are made available for public use - for example, research papers; electronic journal articles; government reports; creative writing; statistical databases and business or service delivery web sites. The resources will be varied in nature, ranging from static, plain text documents to dynamic, interactive, and/or multi-media resources whose content is updated periodically or evolves over time.

It should be noted that government agencies creating web resources, whether primarily for public information or as part of internal business operations, must comply with relevant state, territory or Commonwealth legislation concerning record keeping responsibilities and public accountability requirements. Record keeping and archival standards set by archives organisations should also be followed. Further information is available from Archives of Australia from the National Archives of Australia site.

GUIDELINES

The guidelines provide practical advice on creating, describing, naming and managing web resources to facilitate their on-going use. It is recommended that they become standard practice, built into the planning stage for all web resources created and into existing information management strategies for information in other forms. In this way, it will be easier to carry out preservation action that might be required at a later stage to maintain access to those resources judged to have lasting value.

Creating resources

Before a resource is created thought needs to be given to the technology used to publish it and to view it. This includes software and hardware, file formats, data standards, and how the resource will be described. Unless there are strong reasons to do otherwise, a policy of using very widely accepted standards is highly recommended. This approach will support the widest possible current access as well as making long-term access more likely.

Other points to consider include the following:

• Online resources intended for public access should be accessible to commonly used browsers.
• Standardised file formats in common use (eg. ASCII) are more likely to have affordable preservation paths developed for them than are non-standard formats.
• The simpler the data format, the easier it will be to preserve access to the data. For example, information in simple HTML should be relatively easy to preserve, whereas features such as plug-ins and Java script could make it more difficult. (However, it is reasonable to expect that in time developments in technology will overcome such preservation difficulties.)
• Standard formats (such as html) should be used in ways that comply with the standard. There are a number of online validation services available that will help minimise idiosyncrasies likely to cause problems when the resource is migrated to other versions and read by other browsers. (See the World Wide Web Consortium site and the Bobby service).
• Open, non-proprietary formats are less likely to encounter on-going access difficulties than are proprietary formats.

A useful source of information on best practices for creating web resources is the National Initiatives for Networked Cultural Heritage site.

Describing resources

Metadata is data that records information about a resource. It is essential to support discovery, use and management of the resource. For instance, information about the title can support searching; information about terms and conditions associated with use of the resource can determine how it can be used; and information about its technical characteristics can support preservation decisions.

It is desirable that metadata schemas in general use or which are recognised standards in particular fields, are used for all purposes. This will achieve consistency of description and searching of resources from different sources, and facilitate the exchange of metadata.

Metadata can be recorded by:

• embedding it within the resource to which it relates; eg., as HTML metatags
• storing it in a data management database separate from but linked to the resource
• including it in a resource discovery service, such as a library catalogue, with a link to the resource.

To support resource discovery and use:

• A basic level of descriptive metadata (for instance, title, creator name, publisher, subject terms and intellectual property details) should be recorded.
• More reliable and focused searching will be achieved by including the descriptive metadata in a resource discovery service such as a library catalogue or public database, than can be provided by web search engines. (If you are associated with a library, ask them to catalogue your resource.)
• Schemas such as Dublin Core, AGLS (for government information), EdNA (for educational resources), and a range of other subject-specific metadata sets, are available for use. The Meta Matters site provides more information about these schemas.

To support resource management:

• Additional metadata about the technical nature of a resource, such as resource type, file formats, data structures, processes used in its creation, and software required to operate it, is needed.
• Information about the changes and processes that have been applied to a resource throughout its life, is also needed to support preservation actions.
• Most resource discovery metadata schemas allow some, but not all of this information to be recorded. A number of specific administrative and preservation metadata models are currently being developed to fill this need. The Archives Recordkeeping Metadata Standard for Commonwealth Agencies (1999) is one example. The PADI (Preserving Access to Digital Information) site at provides information about others.

Maintaining links to resources

The name or identifier commonly used for web resources is the URL (Uniform Resource Locator). This identifies the domain name of a server on which a resource is housed and the place of the resource in the file structure of that server. The location-based nature of the URL is its major weakness as a resource identifier as the identifier will change if the resource is relocated. When this happens all existing links to the resource will be broken, including those embedded in other resources or, for instance, found in catalogues, indexes and listings that provide resource discovery services.

To maintain access to web resources over time it is necessary to ensure that links to them are not broken, even if their location on the web changes. The maintenance of links to resources requires careful planning and management of local web sites. There are a number of ways to make sure links continue to work. These can be grouped into three categories:

• Using a persistent identifier. (A persistent identifier is a name for a resource which will remain the same regardless of where the resource is located - thus links to the resource will continue to work even if it is moved as long as it can be resolved.)
• Organising your website to reduce the need to move material around
• Keeping older material available on your web site

The National Library is currently investigating options for implementation of a persistent naming scheme that will serve its own business needs and that can also be adopted as a national scheme. A report on a consultancy conducted for the National Library is available at www.nla.gov.au/initiatives/persistence/PIcontents.html

Additional information about managing web resources for persistent linking, is available in a companion document Managing web resources for persistent access

Managing resources

Creators and publishers can make sure their web resources are managed appropriately for ongoing access by:

• managing the resources themselves
• where relevant, influencing the way the resources are managed by their organisation, or
• arranging for a host site that is prepared to provide the necessary management conditions, to manage their resources for them.

The conditions needed for managing online resources for ongoing access include:

• Having regular back-up procedures in place, if possible using secure off-site storage and at least two different media (such as disc and tape) to avoid the complete loss of data. (Maintaining multiple copies greatly increases the chance of information surviving.)
• When data is copied (such as in making back-ups) checking the data to validate the copy.
• Maintaining software required to operate the files in the resource. (Although the maintenance of obsolete hardware is not workable in the long-term, maintaining the necessary software may be necessary at least until other strategies are in place.)
• Moving resources of long-term value to an archive - a stable technical environment where they are protected from sudden loss pending preservation action that will provide on-going access. The archive needs to include reliable programs to regularly “refresh” the data by copying it to new carriers; to update the metadata record, documenting any changes in the resources and how they are managed; and if possible, to maintain manageable versions of resources (such as linear versions of streaming files, or data and software for dynamically delivered databases).
• Addressing other issues such as deciding whether older versions of resources also need to be maintained, and how they will be distinguished from each other.
• Maintaining the metadata associated with the resource.

Creators who are not able to archive their own resources or have them archived by their organisation, are encouraged to make arrangements with an appropriate repository. For instance, the National Library and most State and Territory libraries accept responsibility for online publications that are in-scope for their selection criteria; and the National Archives of Australia accepts responsibility for selected Commonwealth government online records. It is expected that other bodies will accept similar roles over time.

ROLE OF THE NATIONAL LIBRARY

Archiving

The National Library has been archiving selected Australian online publications (individual documents, collections, e-journals and web pages) in its PANDORA Archive since 1996. However, the Library is able to archive only a portion of what is being produced online. It recognises that it must work with others and facilitate action to ensure a wider range of information resources is saved and accessible for future use. It does this by:

• contributing to the international development of relevant standards and promoting their use
• managing the National Bibliographic Database, a national resource discovery service for information resources held in Australia
• cataloguing the online resources that it archives in PANDORA onto the National Bibliographic Database
• providing an online registration form for notification of online publications that might be eligible for archiving in PANDORA
• Collaborating with State and Territory libraries and ScreenSound Australia to build the National Collection of Australian Online Publications

Preservation

The National Library accepts a national leadership role in the area of digital preservation and is also a participant in several international projects. At a practical level, it intends to preserve for long term access the digital resources archived in PANDORA. To this end, it is conducting trials to test migration of data to new formats and operating systems, so content can still be used when older formats and technologies become obsolete; and it is testing emulation of obsolete technologies – ie. making new generations of hardware and software function as if they were the technology originally used to provide access.

Developments in digital preservation techniques and relevant standards and best practices are reported on the Library’s PADI (Preserving Access to Digital Information) website.

CONTACT DETAILS

For more information or assistance with any matter raised in these guidelines, please contact:

Manager
Digital Archiving
National Library of Australia
Canberra ACT 2600

email: elecunit@nla.gov.au
fax: 02 6273 4322
phone: 02 6262 1140

June 2002