1. Purpose

This statement outlines the National Library of Australia’s policy on preserving its digital collections, and collaborating with others to preserve digital information resources.

The Library’s digital preservation program forms part of its overarching Preservation Program which covers all formats of material the Library collects. This policy should be read in conjunction with the Library’s general Preservation Policy and other policy documents and guidelines relevant to the way digital resources are created, selected, acquired, described and accessed. These can be found on the Library’s website at Policy & planning and Policy and practice statement. Of particular relevance are the Library’s Collection Development Policy, Collection Digitisation Policy and Collection Preservation Intent Statements.

2. The objectives of the Library’s digital preservation activities

The Library’s preservation role is guided by our key statutory responsibilities to develop and maintain a national collection of Australian published and unpublished information resources to ensure they remain available for current and future use. The Library recognises digital information resources as a critically important and increasingly large component of our collecting and preservation responsibilities.

The Library has been active in developing infrastructure to collect, manage, preserve and keep our digital collections available. This is recognised as ongoing core business for the Library and is critical to its future relevance.

The Library defines the primary objective of digital preservation activities as maintaining the ability to meaningfully access digital collection content over time. The primary concern is preserving the ability to access the Preservation Master File from which derivatives files may be created or re-created over time. To this end, preservation of digital library material includes:

  • Bit-level preservation of all digital objects which means keeping the original files intact;
  • Ensuring that authenticity and provenance is maintained;
  • Ensuring that appropriate preservation information is maintained;
  • Understanding and reporting on risks which affect ongoing access;
  • Performing appropriate actions on sets of digital objects to ensure that the objects continue to be accessible; and
  • Periodic review of preferred formats and digital metadata standards

3. The nature of the Library’s digital collections

The National Library began acquiring and creating digital information resources of enduring value around the mid-1980s. Subject to collecting and preservation agreements that the Library may enter into from time to time with other agencies, the digital information resources for which we currently accept some level of preservation responsibility include:

  • Australian web resources selected for inclusion in PANDORA -  Australia’s Web Archive, or gathered in periodic harvests of the Australian web domain or through other programs;
  • Preservation master copies produced by digitisation of analogue materials from the Library’s collections (and in some cases from the collections of other institutions), including the Newspaper Digitisation Program;
  • Australian digital publications (either directly deposited or originally published on physical carriers) acquired for the Library’s collections; and
  • Original unpublished digital files acquired or produced for inclusion in the Library’s Archival Collections (e.g. Manuscripts, Pictures, Maps).

Over time, this list of resource types can be expected to change in response to new business needs and opportunities, and reflect new forms of electronic publishing and communication.

This policy does not cover Oral History materials which are managed under a different policy.

4. The challenges of keeping digital information resources accessible

The Library’s digital collections are diverse in terms of types of collection materials, sources, and levels of control or influence over their creation. In addition, the Library’s digital collections are currently approaching petabyte scale, and are expected to continue growing.

Ongoing access to digital resources is threatened by many challenges. From a preservation point of view, the challenges include:

  • Media and technological obsolescence which affect collection access;
  • A wide range of file formats (and carrier formats);
  • Exponential scale and growth of digital collections;
  • A mix of materials needing to be kept in perpetuity, along with materials that can be discarded after specified periods or events;
  • Mixed levels of complexity in terms of object structure, relationships and dependencies;
  • Mixed levels of intellectual control;
  • Different levels of complexity in preservation planning and processing;
  • Different timetables for preservation action;
  • A need for different preservation approaches, often at different scales; and
  • A need for recurring – and possibly changing - preservation action cycles over time, using a changing suite of tools.
  • Access may be either online (the preferred delivery method) or limited to offline (local), depending upon the nature of the preservation action/s required and their associated technical constraints.

The Library’s digital preservation programs aim to address or at least take account of these challenges.

5. Broad directions for preserving the Library’s digital collections

Scope

It is the Library’s general intention to preserve the accessibility and usability of the majority of digital materials in its collections. The specific preservation intentions and responsibility for these digital materials has been expressed at a high level in various Collection Preservation Intent Statements. Preservation Intent can be described as the agreed expectations for preservation of different categories of content: for instance, whether content is to be preserved; who is responsible for preservation of the content; the period over which content must be preserved; and the required level of support for access to the content over time. However, because of resource pressures, the Library may have to allocate priorities for action. These will be decided based on the relative significance of particular materials and on the technical complexity of preserving access to them.

Preserving accessibility

Preservation of the Library's digital collections involves four main goals:

  • Maintaining access to reliable data at bit-stream level;
  • Maintaining access to content encoded in the bit streams;
  • Maintaining access to the intended contextual and intellectual meaning of the content; and
  • Maintaining the stated preservation intent for all digital material over time.

While specific preservation activities may focus on one or more of these goals, the Library’s preservation responsibility is only fulfilled when all four goals have been adequately addressed.

This responsibility applies across all digital collections, subject to curatorial and policy decisions for specific groups of digital objects.

The decision about what digital material is accepted into the collection will be the responsibility of the collection areas. Although the Library's collection development policy underpins this decision-making, collections staff will need to consider the Library's ability to provide access to this material over time. While there are no predetermined restrictions on the formats that will be accepted into the Library's repository, a decision about whether or not to accept the material will need to take into account the current level of support that can be provided for the known formats contained in the collection or collection item.

Formats that cannot be identified are not supported by the Library (unless subsequently identified in a later process). However, they will still be preserved in the repository with the standard integrity checks and backup support such that they can be reproduced at the bit-level at any time; however, there is no guarantee that access to the content can be provided at any time in the future, even though an exact copy of the original deposited files is still held in the repository.

The Collection area must decide, based on the available level of support, whether it is worth keeping this material, and should discuss with the depositor possible alternative formats for the content with a higher level of support [1].

Models and standards

The Library believes its digital preservation objectives will be best achieved by developing or adopting practices and infrastructure that comply with a coherent framework for reliable and manageable digital archives.

The Library uses the broad understandings and concepts embodied in the Open Archival Information Systems (OAIS) Reference Model [2] as a conceptual check for its own archive construction and management. In developing or adopting relevant systems and infrastructure, the Library aims to operate within the principles of reliable digital repositories as defined by international standards and best practices (e.g. PREMIS [3], Open Planets Foundation [4]).

The Library also recognises the value of practical experiments in developing useful new approaches. The Library will continue to develop process models addressing our particular business needs.

6. Policy implementation

Implementation of this policy will be contingent on the infrastructure provided by the Library and the availability of cost-effective solutions. These are influenced by expected rates of growth in the size of the Library’s collections, changes in technologies, and infrastructure replacement cycles. In implementing this policy with regard to its own collections, the Library will seek to apply certain principles.

The Library:

  • Recognises that preservation action is an inherent cost of collecting (unless there is an explicit intention not to preserve), and seeks to understand the costs associated with appraising, acquiring, managing and maintaining access to digital collection materials;
  • Attempts to allocate adequate and appropriate resources and infrastructure for sustained preservation action, but may need to tailor its collecting programs to take account of the ability to preserve and the availability of resources to do it; and
  • Attempts to ensure sufficient trained staff are available to manage the processes required to preserve our digital collections.

To implement the Policy the Library must:

  • Assist collection areas with analysis and assessment of digital material during the acquisition phase;
  • Understand what it holds in its collections;
  • Understand what its preservation intentions are for every digital object and what is required to realise those intentions;
  • Understand what is required to provide access, if there are any inhibitors to access, and the current level of support the Library is able to provide;
  • Evaluate and monitor the degree of risk arising from collection composition, preservation intentions and available level of support within the Library for digital collection content, and monitor for risk conditions arising during general Digital Library operations;
  • Anticipate the effects of changes in support;
  • Recognise planning triggers, and plan and take appropriate action on a scale appropriate to the size of the target; and
  • Audit the effectiveness of its preservation arrangements and modify the arrangements if necessary.

Systems and arrangements to address these preservation challenges must enable Library staff to deal efficiently, effectively and accountably with billions of objects, in diverse formats and collection contexts for access in wide ranging end user environments. Such systems and arrangements will need to support mass processes while offering the necessary flexibility to tailor both large- and small-scale preservation actions.

With regard to coordination and planning of the Library's preservation efforts, we:

  • Will take actions based on summary reports of digital content (composition and characteristics of our digital collections), metadata characteristics and risks across collections or defined sets of managed content;
  • Based on these reports we will compare metadata, preservation intent, significance and defined risks for managed content, in conjunction with detailed reference information and resource inventory knowledgebases (e.g. information about and associations between file formats and versions, software, hardware and levels of support) to evaluate risk and prioritise preservation planning and action;
  • Aim to plan our preservation action proactively, providing sufficient lead time for appropriate infrastructure to be put in place to support the required action;
  • Will consider the following broad preservation action approaches that are likely to be required:
    • Format migration at the point of collecting;
    • Format migration on recognition of risks;
    • Format migration at the point of delivery;
    • Emulation of various levels of software and hardware environments;
    • Maintenance or supply of appropriate software or hardware;
    • Documenting known problems for which no other action can be taken; and
    • Deaccessioning or deletion.

We expect to use a range of approaches to maintaining access to all the digital resources in our care. The Library will choose the most appropriate approaches to achieve our preservation objectives.

7. Research and standards development

The Library will continue to encourage and undertake research into areas in which we need more knowledge in order to manage our digital collections effectively. Current knowledge is not adequate to address these policy directions satisfactorily; and our needs will also change over time.

We will publicise our research priorities with a view to engaging the interest of others who may have a shared interest or be able to help.

The Library also believes that standards have an important role in reliable preservation procedures. The Library will continue to support standards development in Australia and internationally where there are potential benefits to our business needs.

8. Working with others to preserve the nation's digital information resources

The Library is one of many players with an interest in ensuring the national documentary heritage is preserved and remains accessible. The Library seeks to work with others who are taking, or could take, responsibility for preserving elements of Australia's digital information heritage. In working with such partners, the Library aims to:

  • Identify appropriate partners and stakeholders able to contribute to a national effort;
  • Establish agreements on responsibilities and roles that offer a reliable basis for ongoing accessibility;
  • Help develop and promote policies, procedures and tools to support such an aim;
  • Work with producers and re-users of digital content to encourage practices that will enable, rather than hinder, preservation; and
  • Work with governments to develop legislative and funding frameworks to enable cost-effective preservation.

9. Working with others to foster digital preservation

The Library places great importance on working with others in Australia and internationally to develop strategies and practices that support digital preservation. The Library aims to:

  • Freely share information on our approaches, systems, strategies, tools and experiences;
  • Learn from the experience of others as well as our own experience;
  • Support internationally useful mechanisms for comparing approaches and strategies, and for reviewing developments (e.g. IIPC PWG [5]); and
  • Participate in research and development projects where there is an identifiable benefit to the Library and to others involved in preserving Australia's digital information resources (e.g. NSLA [6]).

The Library has a special responsibility to foster digital preservation efforts in the South East Asia and Pacific regions as part of our role as a Regional Centre of the Preservation and Conservation Core Program of the International Federation of Library Associations (e.g. IFLA PAC [7]).

Contact

The Director of the National Library’s Web Archiving and Digital Preservation Branch is responsible for maintaining this Digital Preservation Policy and for leading its coordinated implementation.

Endnotes

[1]  Webb. C. 2012 Policy positions for DLIR – collecting, digitising, preserving, managing access for the National Library of Australia’s digital collections (v.06/16 Jan 2012). National Library of Australia internal report.

[2]  OAIS - Reference Model for an Open Archival Information System (viewed 24 May 2012)

[3]  PREMIS - PREMIS Data Dictionary for Preservation Metadata Version 2.1 (viewed 24 May 2012)

[4]  Open Planets Foundation (viewed 24 May 2012)

[5]  IIPC PWG - International Internet Preservation Consortium Preservation Working Group (viewed 24 May 2012)

[6]  NSLA – National and State Libraries Australasia (viewed 24 May 2012)

[7]  IFLA PAC – International Federation of Library Associations and Institutions, Preservation and Conservation (viewed 24 May 2012)

4th edition: February 2013 (1st edition: July 2001)

The National Library of Australia acknowledges Australia’s First Nations Peoples – the First Australians – as the Traditional Owners and Custodians of this land and gives respect to the Elders – past and present – and through them to all Australian Aboriginal and Torres Strait Islander people.

Cultural Notification

Australian Aboriginal and Torres Strait Islander people are advised that this website contains a range of material which may be considered culturally sensitive including the records of people who have passed away.

ENTER SITE