Guidelines for hardcopy newspapers digitisation

This document contains guidelines for scanning hardcopy newspapers for both preservation and access purposes. It was prepared by the National Library in response to requests from ANPlan partners for guidance on how to scan newspapers so that they can be added to Trove.

The National Library incorporates scanned newspapers into Trove to provide online access with full text search and correction capabilities. The newspapers presented are either out of copyright, or have had access granted by the copyright holder.

ANPlan libraries who wish to add their newspaper content to Trove must meet these guidelines.

The Australian Newspaper Digitisation Program (ANDP) uses the Newspaper Content Manager (NCM), an in-house developed system, to manage the scanning and OCR processes for newspaper digitisation. The requirements below therefore reflect what NCM is currently set up to receive. The requirements may change in the future as a result of software developments.

Quality of scanned image for newspapers

The purpose of digitising hard copy newspapers is to create a faithful representation of the original source material for both preservation and access purposes. The hard copies should be prepared in a similar way as for microfilming. While targets, such as those used in microfilming, are not required in the digitisation process, documentation detailing anomalies in the content, such as missing pages, is important.

The National Library current standards are to scan hard copy newspapers in colour. A bi-tonal image for each page is also required for OCR purposes.

The following features of the scanned image must be achieved in order to be acceptable:

  • Variation in the positioning or alignment of the image (skew) to be within ±1 degree from parallel.
  • All images should be of consistent size and dimension.
  • Each page should be captured in a separate image. However when text, maps, charts or other illustrations spread across two pages, both pages should be captured as one image so that the text or illustration is not cut in half.
  • Cropping to just inside edges of the page. This is the NLA standard, however other cropping standards may be negotiated.
  • Good contrast between the text and background that faithfully reflect the original material
  • No show-through which obscures legibility of the front of the sheet.
  • No overlapping documents.
  • No duplicate images.

The following requirements must also be met:

Attribute

Requirement

File format

TIFF 6.0

Scanning ratio

1:1

Resolution

400 ppi

Compression type

LZW for Colour Master image;

CCITT Group 4 for bi-tonal image

Colour rendition

If scanning in colour, both a 24 bit RGB colour and a bi-tonal image must be supplied for each newspaper page. The bi-tonal image is used for OCR purposes and therefore should be an exact copy of the colour image.

Technical metadata

Technical metadata must be included in the header of each image as described under Technical Metadata in this guideline.

File naming schema

  • File names must be unique for each image.
  • File names must follow the following convention:

nla.news-issn1234567X-s1-c.tif   (for the colour image)

nla.news-issn1234567X-s1-b.tif   (for the bitonal image)

  • ‘nla.news’ is the generic prefix for all digitised newspapers.
  • The issn number, issued by the Australian ISSN Agency, is used as a unique identifier for the digitised version of the newspaper.
  • Files must be saved into folders on a storage device (tape, hard drive, etc).
  • All images within the folder are numbered sequentially from –s1 without leading zeroes
  • Each folder should contain images for one year of publication.

 

 File naming scheme for newspapers.

 

 

Check file

The Library requires a Check file to be provided in the root directory of the storage device (tape, hard drive, etc).

The check file is a manifest of all the files on the device. It must be a CSV (Comma Separated Value) file and list for each file:

• the full path and file name

• the checksum type (MD5 or SHA1)

• the checksum

 

Technical metadata requirements

The National Library uses in-house software to ingest digital files of journal pages. In order for data to be accepted by the system, the following minimum metadata requirements must be met.

Tag

Name

Description

Example Data

256

ImageWidth

The number of pixels per row

5184 (pixels)

257

ImageLength

The number of rows of pixels in the image

7016 (pixels)

258

BitsPerSample

Number of bits per component

24 for colour images

259

Compression

Compression scheme used on an image data

1=uncompressed

262

PhotometricInterpretation

The colour space of the image data

2=RGB

271

Make

The scanner manufacturer

Icam Archive Systems Ltd

272

Model

The scanner model name or number

Guardian, SN# 415003

282

XResolution

Horizontal pixel count per resolution unit (inches, centimetres)

400

283

YResolution

Vertical pixel count per resolution unit (inches, centimetres)

400

305

Capture Software

Name and version number of the software package(s) used to create the image.

Phase One

306

DateTime

Date and time of image creation.

2014-07-02 17:01:56 (YYYY:MM:DD HH:MM:SS)

315

Artist / creator

Person who created the image.

[Name of Scanning bureau]

Digitisation terminology

  • Lossless compression: Data compression that is unlikely to result in loss of content.
  • LZW (or Lempel-Ziv & Welch): a common lossless compression algorithm used with data and image compression. Used extensively with image file formats such as GIF and TIFF.
  • Group 4 compression TIFF 6.0: a common format used by digital library projects for bi-tonal images, typically scanned pages of printed text.

In this document ‘scan’ and ‘digitise’ both have the meaning create a digital image from a physical object.