Trove API survey report

Executive Summary

The Trove API was first made available in April 2012. Since that time it has been used by a variety of users to display results from Trove on other websites, harvest Trove records for inclusion in other databases, retrieve tags or comments added to records contributed by organisations, conduct offline analysis, and create new tools and visualisations.

The National Library of Australia conducted a survey on the use of the Trove API from February 14 – 24, 2017. The purpose of the survey was to gather information from individuals and representatives of organisations on why they use the API, their experiences using the API, and suggestions for improvements. 198 complete responses were received; with 79% respondents identified as male, 17% female, and 4% other.

The survey results show there is dedicated group of technically proficient Trove users who are using the API regularly for the purposes it was intended. A second set of infrequent users are experimenting with the API as part of their studies. Examples of how individuals and organisations are using the API are included in this report.

In relation to improvements to the API, there were concerns about its stability and also requests to expand the range of metadata included. There is significant interest from the technically proficient users in being involved with the open development of the API.

‘There was one thing in particular I wanted to comment on which wasn't in the survey; it was about the process of development and maintenance of the API. What I would particularly like to see is this process being made open.’

‘I also think the source code for the system implementing the API should be open source so that external developers can review it and help to identify and correct errors.’

There are a number of specific Trove and Trove API development projects which will benefit from the information gathered in this survey. These projects will be rolled out in phases to suit the differing needs of the technically proficient users, students and organisations using the API to populate datasets and websites.

Specific actions in relation to improvements to the Trove API will be explored and prioritised following interviews with interested API users and a period of internal consultation.

Methodology

A total of 1,412 emails were sent to individuals who had registered for an API key for either commercial or non-commercial purposes.

The key focus areas of the survey were:

  • General questions about Trove including standard questions developed for use in all Library surveys;
  • Current use of the Trove APIs; and
  • Options for improvements to the Trove API.

Survey questions were developed in consultation with internal and external stakeholders, including members of the Trove Community of Practice (a NSLA CoP).

Potential survey participants were sent the survey link via email. A survey link was also placed on the API overview page of the Trove Help Centre. Two reminder emails were sent throughout the survey period.

The Trove team created and administered the survey using Qualtrics software.

From the respondents, a list of volunteers who indicated they would be interested in participating further in interviews about Trove and/or their API use was compiled.

Survey response rates

  • 198 valid responses were received
  • 62 emails bounced

The response rate, based on the total number of invitations sent (n=1,412) was 14% or 15% based on the known valid sample size (n=1,340).

The survey was structured to allow respondents to progress through the survey without answering all questions. Most questions were not mandatory, and negative feedback was received in response to mandatory questions.

A high number of free text comments provided examples of how individuals and organisations have engaged with, built on and shared digital content based on their use of the Trove API.

General questions about Trove

Overall, survey respondents were very positive about Trove, its services and its importance to Australian research. How this sentiment aligns with satisfaction with the Trove API is examined in more detail later in this report.

Satisfaction with Trove

Satisfaction with Trove was very high, with the majority of responses indicating an above average or excellent satisfaction rating. Only 3.5% of respondents rated their satisfaction with Trove as poor or below average.

Bar chart summarising satisfaction with Trove

The following quotes illustrate some of the reasons for this high level of satisfaction with Trove:

‘Directly relevant to the history curriculum (we are a K-12 School) and facilitating discovery and access to these resources. Authoritative source of truth for call numbers and other catalogue details, source of inspiration for subject headings and source of missing details and contextual information that helps us provide better catalogue records. Lets us know whether a title we hold should be retained or discarded, we try to retain items that Trove demonstrates may be rare or unique (i.e. no longer in circulation in Australia) which helps to protect national history.’

‘Trove has enabled my workplace to significantly improve the service we deliver to Aboriginal and Torres Strait Islander clients.’

Respondents were also satisfied with the information they receive from Trove, with a majority providing an above average rating. Comments by respondents indicated they would like to receive more content from the API, which currently does not include all the fields in Trove, and this may have affected the rating.

Bar chart summarising satisfaction with Trove information

‘PhD research has been made SO much more convenient, either through digitised newspaper collection, or in locating where various books are held, rather than having to search each library catalogue separately.’

‘TROVE displays our past corporate magazine and student newspaper thus making it possible for anyone, alumni included, from anywhere in the world access to the University's history...’

‘My interest is in early trials in NSW. Because official police court trial reports no longer survive for this time, Trove is the only location for any information about what happened there. There is no other source. (See my website http://nis.wikidot.com) The ability to follow someone around Australia or investigate non-city happenings so easily is very useful for me as I can work from home and don't need a trip to Sydney or Canberra to search newspapers.’

Improvements to Trove

Several themes emerged from the 74 free text comments about Trove. These including adding more digitised content, particularly digitised newspapers, changes to the Trove interface ranging from small functionality changes to overhauling the entire concept of zones, adding more content from cultural institutions, improving OCR quality, more engagement with users before making functionality improvements, and improved identification of open access and available online resources.

‘remove the concept of zones and make it as a normal facet. include more facets’

‘Keep digitising newspapers and more recent material. post 1950.’

‘Newspaper advertising 'articles' are composites of many advertisements. It would be good if each advertisement could be a separate sub-document. More newspapers scanned and made available.’

‘Better identification of open access resources -- search by licence etc. Better integration of people zone and other resources. Ability for users to define relationships between resources, and between people and resources.’

‘It would be good to see a simpler, more streamlined interface. I'd love more datasets from cultural institutions added. It would be excellent to get Trove and DigitalNZ talking to each other at the API level too.’

‘Improve the quality of the OCR so that more content can be found! And add more content - Australian cultural organisations should spend more of their budgets on digitisation and making content fully searchable.’

‘Better quality control on Trove version releases. Improved engagement with select public users for functionality of future releases.’

Sectors

Over one-third (37.01%) of respondents identified themselves as family historians, professional historians/researchers, and people conducting personal research. Respondents from this group used the API to populate websites with content, download the full text of digitised newspapers, and transfer metadata to a system they were comfortable working in for further analyses.

‘I refer clients to Trove all the time for research purposes, especially family history research. The newspaper zone is particularly rich pickings for researchers. I've used excerpts from digitised newspapers in Trove to make a video about a historical event for my workplace. Other staff have used it in similar ways.’

Screenshot from the VicFix campaign, which focuses on text correction of specific events in Victorian newspapers in Trove.

Screenshot from the VicFix campaign (https://www.slv.vic.gov.au/contribute-create/completed-vicfix-campaigns), which focuses on text correction of specific events in Victorian newspapers in Trove.

26% of respondents were fromuniversities. The Trove API had been used in several university courses to teach students about APIs and introduce them to working with large data sets. Extracting large data sets from Trove, especially full-text digitised newspaper articles was a common theme. The data was then used to build experimental interfaces, identify and analyse trends in digitised newspaper content, examine existing collections, and enrich other datasets, as illustrated by the following quote:

‘I am a historian working with criminal justice data in a longitudinal database covering all states and over more than 150 years. Extraction of data from digitised newspapers is essential to the enrichment of the data we use. And we have integrated a Trove API into our database management system to enable semi-automated discovery of newspaper reports related to our data (now at more than 250,000 cases). The scope of the project in time, breadth and depth would be impossible without Trove.’

24.41% of respondents identified as an employee at a library or cultural institution, or a vendor building software for this sector, with nine reporting they were Trove content partners. This group of users tended to use the Trove API to enhance their collection including cataloguing items, identifying gaps in their collections and holdings, and also promoting digitised newspapers from their local area.

‘I have been able to catalogue our library - and grab geographic distribution information about where our books are available elsewhere. This has been integral to understanding which items in our collection are more or less important.’

Another theme to emerge with the use of the API is play. There were several examples of users creating Twitter bots that tweet recipes, knitting patterns, or historical newspaper articles relating to current affairs. One participant in a GovHack event created an application that colourised black and white images.

//colourfulpast.org/), which colourises black and white images found through the Trove API.

 Screenshot from the Colourful Past website (http://colourfulpast.org/), which colourises black and white images found through the Trove API.

A further selection of products which use the Trove API is shown in Attachment A. The breadth of these products displays the diverse use of the API, as well as the diversity of audiences using the service.

Current use of the Trove API

Searching across the records in Trove is the most common use of the Trove API. This raises questions about whether current functionality for whole-of-Trove searching meets the needs of technically proficient users.

Participants were asked to list any issues they encountered while using the Trove API. The two common themes which emerged were:

  • The performance of the API; and
  • The completeness of metadata.

API Performance

Performance issues ranged from the service being unavailable, requests timing out, a high number of errors when searching digitised newspapers, and rate limiting. The following quotes provide an insight into respondent concerns:

‘Trove's current APIs are a good start, but could be improved in terms of reliability, performance, and rate limiting. We ended up taking the Trove API out of the public version of one project because it was too slow to return results compared to the other APIs we used (like the NZ equivalent, DigitalNZ). That said I also realise that API improvements take a lot of effort!’

‘Towards the end of the semester when all students were making calls from the finished apps, Trove servers just stopped answering for hours. At times the same call would have a very different answer, even when they were done just minutes apart.’

‘The rate limit that is currently in place would make my app non-responsive if a small number of users were using the app concurrently. That is a non-starter for me.’

Completeness of metadata

The second theme that emerged from responses is the completeness and quality of metadata returned via the API. Respondents expressed a desire for all metadata that is available via the Trove interface to be made available via the API. The following quotes illustrate users’ needs for the availability of all metadata:

‘Inconsistent/missing metadata (like extra notes and rights info) that was present on the trove front end for some records, but not available in the metadata from the Trove API for that same record.’

‘Not all data is exposed - more is exposed thru the human interface, so we used that instead. Functionality beats stability, as without that info, we can’t use the data.’

27 respondents indicated that they have screen scaped data from Trove because they could not access it in any convenient way.

We asked participants to indicate how they would like to see the Trove API help centre improved; the following suggestions were received:

‘I recall it being one very long page...should be broken up into easier to digest chunks.’

‘It's OK in my personal opinion. Basic formatting and sectioning could be cleaner and more distinct.’

‘maybe some short video/screen casts - suitable for beginners.’

‘Some functions of the API are not well documented in the Help centre, and there is no documentation of the People API in the help centre.’

Respondents were very positive about the option of introducing an API knowledge base and discussion forum where users could discuss solutions to problems.

Options for improvements to the Trove API

Participants were asked to comment on how they would like to see the Trove API improved. 46 responses were recorded, reflecting themes of needing to improve the performance of the API, data quality and format, and the availability of all metadata. Other useful suggestions included:

  • Sharing the API code base and allowing developers to contribute to API service development;
  • Increasing the data filtering options available via the API;
  • Investigating the possibility of allowing users to write data back to Trove (e.g. annotations);
  • Ensuring the API is available via https;
  • Providing an XML namespace and schema;
  • Providing CSV output;
  • Ability to get all results from all zones in a single set of data; and
  • Better pagination of search results.

Follow-up actions from Trove API survey

It is intended that further interviews will be conducted with Trove API users, from the list of volunteers compiled during the survey. These interviews will focus on gaining greater understanding of issues around performance and stability of the Trove API, metadata quality, and any potential solutions. Further opening up API development will also be considered, including using an open-source API framework and creating a forum and knowledge base for users. The perceived benefits of any proposed enhancements, including improved experience for select API users and potentially increasing user numbers overall, will be considered against the resources required to implement them.

Attachments