Skip to main contentSkip to navigationSkip to navigation

British Library adds billions of webpages and tweets to archive

This article is more than 10 years old
Every public tweet and Facebook entry in the UK could eventually be archived

Material on 1bn webpages from nearly 5m .uk websites, plus public tweets and Facebook entries, are to preserved for the historical record at six libraries in the UK and Ireland.

The archive project, aimed at preserving a digital record of events and cultural and intellectual works to match traditional print archives, begins on Saturday.

Its organisers say copies of every public tweet and Facebook entry in the UK could eventually be included. The moves will also cover ebooks and iPad editions of newspapers and other digital forms.

The British Library will begin the enormous operation a day after new regulations laid down by parliament come into force but already it is warning that key accounts of some events over the past 10 years have already been lost.

"If you want a picture of what life is like today in the UK you have to look at the web," said Lucie Burgess, head of strategy at the library. "We have already lost a lot of material, particularly around events such as the 7/7 London bombings or the 2008 financial crisis."

Social media reactions to the Queen's diamond jubilee celebrations were among other information that had fallen "into the digital black hole of the 21st century because we haven't been able to capture it," she said. "Most of that material has already been lost or taken down."

Burgess added: "We will have to distinguish between content published in the UK and elsewhere but in principle we will be able to archive the publicly available tweets of any individual, company or organisation."

Until now the British Library could only preserve a relatively small number of websites. The 2003 Legal Deposit Libraries Act paved the way for such information to be stored but copyright laws had forced it to seek permission each time it wanted to collect web content.

The Bodleian Library, in Oxford, Cambridge University library, the National Library of Scotland, the National Library of Wales and Trinity College library in Dublin, like the British Library, which is acting on behalf of all, have the right to receive a copy of every UK electronic publication.

The trawl will include embedded audio and video material but the regulations do not cover other mediums such as YouTube or Spotify. The project will later seek to identify UK sites in the .org and .com domains.

Burgess said that it had taken over 300 years to collect 750m pages from printed newspapers, yet the aim now was to collect 1bn webpages in a year and then conduct regular sweeps in future. "It is about taking a snapshot. It is not for us to say what is of interest. Researchers will be the judges of that."

Roly Keating, chief executive of the British Library, said that there had been " a very real danger" of millions of web pages, e-publications and other non-print items "falling through the cracks of a system that was devised primarily to capture ink and paper".

While the 2003 act established the principle that legal deposit "needed to evolve to reflect the massive shift to digital forms of publishing", the regulations would make "digital legal deposit a reality, and ensure that the legal deposit libraries themselves are able to evolve – collecting, preserving and providing long-term access to the profusion of cultural and intellectual content appearing online or in other digital formats."

Access to the material will be offered in reading rooms at each of the six libraries.

Comments (…)

Sign in or create your Guardian account to join the discussion

Most viewed

Most viewed