Greg Linden may not be a household name, but he changed the way we interact with culture and transformed retail forever. An engineer at Amazon in the late 1990s, Linden worked on a curious problem: how to recommend books without human intervention. Until then Amazon relied on editors who wrote hundreds of reviews every year. It was a costly and time-consuming process.
Automating recommendations proved trickier than anyone expected. Linden cracked it. He hit on “personalisation”, which paradoxically meant looking not at an individual’s purchasing history, but only at correlations among products. Regardless of what you had bought in the past, Amazon realised that if product A was often bought alongside product B, it meant almost anyone buying product A would also want product B. Amazon tested the results to see which method sold more books. No surprises: the editors were soon looking for new jobs. Humans out; machines in. Some estimates suggest a third of Amazon sales arise from these recommendations. Ever since, the rise of algorithms has been relentless. Now books, articles, music, films, not to mention holidays and clothes, are all suggested by machines.
Last year 1m new books were published in English. Since at least the ancient Greeks, people have believed there is too much to read; now they may be right. That, of course, doesn’t even count all the self-published works, the reams of news or the Borgesian vastness of the internet. By any measure, we have an astounding surplus of reading matter.
The more we have, the more we rely on algorithms and automated recommendation systems. Hence the unstoppable march of algorithmic recommendations, machine learning, artificial intelligence and big data into the cultural sphere.
Yet this isn’t the end of the story. Search, for example, tells us what we want to know, but can’t help if we don’t already know what we want. Far from disappearing, human curation and sensibilities have a new value in the age of algorithms. Yes, the more we have the more we need automation. But we also increasingly want informed and idiosyncratic selections. Humans are back.
This is why, despite having the world’s most powerful book recommendation engine, Amazon bought Goodreads, a website based around personal book reviews. It is why sites such as Canopy.co thrive atop Amazon. Canopy knows many of Amazon’s best items are hidden in the mediocre morass. Canopy’s founders, all designers, trawl through thousands of entries a day to highlight exceptional products.
It’s why publishers keep producing new imprints, to allow for more diverse and personal lists, and why bookshops are once again flourishing, even though we can find any book we want online. We go to browse their tables. In Japan they talk about tsundoku, or the uneasy feeling of having too many books to read. They also have its solution: a bookshop in Tokyo’s Ginza that sells only one book at a time.
This rejuvenated interest in curation isn’t just happening in publishing. On Spotify you can listen to 30m songs, 20% of which have never been streamed once. To help manage this huge catalogue, Spotify spent a reported $100m (£77m) acquiring a company called the Echo Nest, which pioneered a technique known as audio-fingerprinting, which automatically categorises songs. At the same time, however, Spotify has massively expanded its range of playlist makers, musical experts who are rapidly becoming the new DJs.
Netflix has more TV and film than we could ever want. An early pioneer of using data science to manage culture, it even launched a $1m competition for teams of researchers to improve its algorithms – and then, despite the prize money, didn’t implement any changes as they weren’t seen as good enough. Yet Netflix also trained viewers to tag its content exhaustively. They make judgments machines cannot: is the ending wistful? Are moustaches important in the film or not?
Facebook is mired in a series of controversies about the curation of its news feed, from its broadcasting live killings, to editing out an iconic photo of the Vietnam war, to accusations of political bias. It recently tried to smooth the process out by firing its human editors … only to find the news feed degenerated into a mass of fake and controversial news stories.
Apple’s news and music apps make much of their human curation, even hiring famous names from newsrooms and radio. Twitter invested heavily in its Moments product. While not universally liked, it shows Twitter wants to curate better. Samsung’s news app divides into what you want to know and need to know; the former chosen by algorithm, the latter by editors. Big tech is on a hiring spree for old-fashioned experts.
We’ve also got excess stuff. The average western European household owns 10,000 items, more in the US. But in order to cope we turn not to an app but the Kondo method, the wildly popular home organisation technique relying on our personal histories. In retail’s upper tier, a renewed emphasis on expert selection is behind the success of shops as diverse as fashion boutique Opening Ceremony and “supermarket of the future” Eataly. As with our media, we’ve passed from an era of bulk industrial selection to finely honed choice.
Curation can be a clumsy, sometimes maligned word, but with its Latin root curare (to take care of), it captures this irreplaceable human touch. We want to be surprised. We want expertise, distinctive aesthetic judgments, clear expenditure of time and effort. We relish the messy reality of another’s taste and a trusted personal connection. We don’t just want correlations – we want a why, a narrative, which machines can’t provide. Even if we define curation as selecting and arranging, this won’t be left solely to algorithms. Unlike so many sectors experiencing technological disruption, from self-driving cars to automated accountancy, the cultural sphere will always value human choice, the unique perspective.
This is where the arts and humanities strike back in a world of machine learning. Here is a new generation of jobs. Information overload and its technology-driven response are one of the great transformations of our time. But amid today’s saturation (and those teetering piles of new books), knowledge and subjective judgment are more valuable than ever. In the words of one Silicon Valley investor, “software eats the world”. Well, software can’t eat human curation. Contrary to myth, traditional gatekeeping roles are here to stay.
What we will see are hybrids: rich blends of human and machine curation that handle huge datasets while going far beyond narrow confines. We now have so much – whether it’s books, songs, films or artworks (let alone data) – that we can’t manage it all alone. We need an “algorithmic culture”. Yet we also need something more than ever: human taste.
Comments (…)
Sign in or create your Guardian account to join the discussion