Digital is history: Preserving our digital collections

Hello, my name is Gareth, I’m the Digital Preservation Co-ordinator here at the National Library of Australia. The National Library of Australia acknowledges Australia’s first nations peoples, the first Australians, as the traditional owners and custodians of this land and gives respect to the elders past and present and through them to all Australian Aboriginal and Torres Strait Islander people.

One of the most challenging aspects of digital preservation is not the actual work but surprisingly it’s explaining what it is to people. Think of it as the digital equivalent of physical conservation so the Library has books, manuscripts, artworks and many other objects and they might require treatment from time to time if covers or spines are damaged or there’s rips and tears in paper. The Library has a dedicated team to treat these physical items to keep them stable and accessible to users. Likewise the Library has a growing collection of born digital publications, images and personal archives and you might not think it but digital objects in my opinion are even more fragile and require a lot more attention and treatment to help keep them stable and accessible.

My background is in linguistics and Mandarin Chinese so it even comes as a surprise to me that I’ve ended up here in digital preservation, more so because I didn’t even know the difference between a JPEG or a PNG file or an MP3 and MP4 so how did I possibly end up here? Well I was given an opportunity to define a few terms for a project that has become quite a significant area of work both here at the Library and in the field of digital preservation itself. This one little opportunity has opened my mind to the importance of digital preservation.

What we do is a balance of practical work, future planning and applied research to look after and ensure access to the Library’s growing digital collections. But before we can do any of that the most important thing is to get content off carriers as they come into the Library and into managed storage so that we do not lose the content. What I mean by physical carriers is CDs, DVDs, hard drives etc. Fortunately this is a shared responsibility across the Library and that frees up our time to focus on other important digital preservation work.

There are some pretty straightforward questions that guide digital preservation and in fact these are questions that you can ask at home as well. Which file formats exist in these digital collections? Can the Library provide access to the content of these file formats to users with the software that it has? Do we need additional software or professional software? Will the Library be able to continue to provide access to the content of these formats in the future? If not what do we need to do about it now?

The practical and applied research that we do go hand-in-hand. Our digital preservation system, Preservica, produces reports on the two million files it currently holds. We analyse these reports weekly to determine whether the Library holds new file formats, whether there are any corrupted files or any other issues that we need to be aware of. That involves a lot of research into understanding the file formats that we have, known issues that these file formats might have and the software that supports them.

So if I may paint a picture of what this looks like, cast your mind back to The Matrix movie. Can you see that green screen full of binary code and other symbols raining down? Well my monitors look a bit like that too, they’re just not green. To stress how cool the work is, we are time travellers, we are always jumping between earlier operating systems to run old applications from the years past. So just the other week I was in PC-Write from 1983 in an old DOS 3 environment.

The Library holds approximately 3 petabytes of digital content and that’s around 15 billion files. I should say that this number doesn’t include all the backups either. Three petabytes of digital content would be around the same as 3,000 1 terabyte hard drives or if we were to convert that to the average storage size of a phone today which is around 256GB we would have just under 12,000 phones of data. It’s amusing to remember, at least to me, that 10 years ago we weren’t talking in terabytes but now we all are. It’s only a matter of time before we’re talking about petabytes at home as well.

Some people might not realise that digital content that we create requires active management to ensure their longevity and a big part of this is looking after storage so what are some of the typical challenges? The volume of the digital content we create is huge, we are taking more photos and videos, creating more documents than ever before and the file sizes are getting bigger. Yes, storage might be relatively cheap but what are you going to do about the quality of your own collections? Are you deleting what you don’t want to keep? Are we really backing up our content? The impression I get when talking to friends and family is that they’re not backing up their content so precious photos and videos are stored solely on a mobile phone or on the computer and nothing else.

Now I might talk a bit about the rise of the convenient cloud storage. Yes, this is great. Be warned, those providers are not responsible if your content is lost or become corrupted so you need to do your own backups as well. Whilst we’re talking about backups, hard drives, they fail just like any other physical carrier so the best practice is to replace your backup hard drives with new ones once every five years or if you notice any performance issues sooner.

The Library has a large number of all of the common carriers you have around your home today in its collections so I’m talking about CDs, DVDs, USB sticks, hard drives among others. The Library also has a large number of legacy carriers too and I daresay many viewers will have these around their homes as well. They include 3.5 inch and 5¼ inch floppy disks. The Library also has some other rare carriers as well including Jaz disks, zip disks and SyQuest cartridges. As I’ve stressed before we ideally only access these carriers once to transfer the content to manage storage. Some carriers can be processed by us and colleagues on everyday computers. The 3.5 inch floppy disks can be processed on everyday computers with the help of the USB floppy drive. Other carriers however such as the 5¼ inch floppies requite legacy computer setups or professional equipment depending on the flavour of the floppy disks.

This is the perfect opportunity to tell everyone one simple message, get your content off those physical carriers that might be hiding around your house and garage and put it all into one managed storage and that’s typically your computer. If you don’t, those carriers will eventually fail and whatever content is on it will be lost as well. So let’s make it fun, let’s create a digital preservation at home. Round up all of the carriers you have and assess what kind of content they contain or might contain if possible and determine which carriers you want to process.

Now it’s time to transfer the content. Some carrier types will be easy to process yourself whilst others might require assistance. Fortunately some of the carriers at a greater risk of failure are the ones that are the easiest to process so let’s do them first. They are the CDs, DVDs, USB sticks and all of the other small hard drives that you have around the place.

Now what about the 3.5 inch floppy disks and zip drives? If you don’t have a computer with a 3.5 inch floppy drive or a zip drive to assist you there are plenty of USB 3.5 inch floppy drives and zip drives available online that you can purchase. You can plug these into any Windows computer via USB port or a Mac computer depending on the disk format. Gets a bit trickier with older carriers such as the 5¼ inch floppy disks and I’d say that most people wouldn’t have access to a working 5¼ inch drive. There are professionals who can transfer the content from these carriers for a fee so it’s worth considering.

Now that you have all the content that you wish to keep off those carriers and onto your managed storage it’s time to do an analysis of your own collection. Go back to the guiding questions I talked about at the beginning and do your best to answer them.

It's best to think of your phone and any memory card as being physical carriers because they are. These are carriers that can and will fail without warning so it is best to back up or transfer the content to your managed storage. Remember that’s your computer. I’ve heard of so many people losing all of their precious photos and videos because they’ve lost their phone or their phone has suddenly stopped working or is damaged. These are photos and videos that they will never be able to get back so please teach yourself how to back up your phone, ask the people around you how to do it if you're unsure. Most of the time all you need is a USB cable and a computer.

There are also cloud storage services like iCloud, OneDrive, Google Drive that automatically back up the photos and videos you take to cloud storage. However you will need to do a bit of research to make sure that this is set up in the right way for you and you need to be aware of the risks involved in using such services.

Lastly my top tips for looking after your digital content, if you want your digital content to last you will need to actively manage it. You can’t just stick it in a hard drive, put it in the cupboard and expect it to last. Keep your computer organised so I’m talking about consistent folder and filenames, perhaps ordered by year and project. Don't forget to delete what you do not want to keep. We also need a good backup system. I would recommend that you buy two external hard drives for backing up your main computer, one could be connected to the computer for automatic backups whilst the other could periodically back up your computer and be stored at your work or some other location. You might also want to consider that convenient cloud storage option as well in addition to your backups.

You should refresh your backup hard drives every five years so just remember like CDs and other physical carriers your hard drive does fail so it’s good to buy new ones every five years. Think about the file formats you have in your collection and the software needed to access them. If you have older file formats and with little software support you might want to consider moving the content to a different file format.