The Stanford Encyclopedia of Philosophy may be the most interesting website on the internet. Not because of the content—which includes fascinating entries on everything from ambiguity to zombies—but because of the site itself.
Its creators have solved one of the internet’s fundamental problems: How to provide authoritative, rigorously accurate knowledge, at no cost to readers. It’s something the encyclopedia, or SEP, has managed to do for two decades.
The internet is an information landfill. Somewhere in it—buried under piles of opinion, speculation, and misinformation—is virtually all of human knowledge. But sorting through the trash is difficult work. Even when you have something you think is valuable, it often turns out to be a cheap knock-off.
The story of how the SEP is run, and how it came to be, shows that it is possible to create a less trashy internet—or at least a less trashy corner of it. A place where actual knowledge is sorted into a neat, separate pile instead of being thrown into the landfill. Where the world can go to learn everything that we know to be true. Something that would make humans a lot smarter than the internet we have today.
The impossible trinity of information
The online SEP has humble beginnings. Edward Zalta, a philosopher at Stanford’s Center for the Study of Language and Information, launched it way back in September 1995, with just two entries.
That makes it positively ancient in internet years. Even Wikipedia is only 14. Sites that have been around 20 years mostly belong to brands that predate the internet—like Bloomberg or MTV—or they’re old sites that just happen to still work, like the classic Space Jam.
The SEP is neither pre-internet, nor is it ossified. It now contains nearly 1,500 entries, and changes are made daily. The site gets over a million page views per month—a respectable number, given how many entries there are with titles like Tibetan epistemology and philosophy of language or Peirce’s theory of signs. The American Library Association’s Booklist review called it “comparable in scope, depth and authority” to the biggest philosophy encyclopedias in print, the 10-volume offerings from Routledge and Macmillan—and that was nearly a decade ago.
John Perry, the director of the center, was the one who first suggested a dictionary of philosophical terms. But Zalta had bigger ideas. He and two co-authors later described the challenge in a 2002 paper (pdf, p. 1):
A fundamental problem faced by the general public and the members of an academic discipline in the information age is how to find the most authoritative, comprehensive, and up-to-date information about an important topic.
That paper is so old that it mentions “CD-ROMs” in the second sentence. But for all the years that have passed, the basic problem remains unsolved. The three requirements the authors list—”authoritative, comprehensive, and up-to-date”—are to information what the “impossible trinity” is to economics. You can only ever have one or two at once. It is like having your cake, eating it, and then bringing it to another party.
Yet if the goal is to share with people what is true, it is extremely important for a resource to have all of these things. It must be trusted. It must not leave anything out. And it must reflect the latest state of knowledge. Unfortunately, all of the other current ways of designing an encyclopedia very badly fail to meet at least one of these requirements.
Where other encyclopedias fall short
Book
Authoritative: √ Comprehensive: X Up-to-date: X
Printed books are authoritative: Readers trust articles they know have been written and edited by experts. Books also produce a coherent overview of a subject, as the editors consider how each entry fits into the whole. But they become obsolete whenever new research comes out. Nor can a book (or even a set of volumes) be comprehensive, except perhaps for a very narrow discipline; there’s simply too much to print.
Crowdsourcing
Authoritative: X Comprehensive: X Up-to-date: √
A crowdsourced online encyclopedia has the virtue of timeliness. Thanks to Wikipedia’s vibrant community of non-experts, its entries on breaking-news events are often updated as they happen. But except perhaps in a few areas in which enough well-informed people care for errors to get weeded out, Wikipedia is not authoritative. One math professor reviewed basic mathematics entries and found them to be a “a hot mess of error, arrogance, obscurity, and nonsense.” Nor is it comprehensive: Though it has nearly 5 million articles in the English-language version alone, seemingly in every sphere of knowledge, fewer than 10,000 are “A-class” or better, the status awarded to articles considered “essentially complete.”
Speaking of holes, the SEP has a rather detailed entry on the topic of holes, and it rather nicely illustrates one of Wikipedia’s key shortcomings. Holes present a tricky philosophical problem, the SEP entry explains: A hole is nothing, but we refer to it as if it were something. (Achille Varzi, the author of the holes entry, was called upon in the US presidential election in 2000 to weigh in on the existential status of hanging chads.) If you ask Wikipedia for holes it gives you the young-adult novel Holes and the band Hole.
In other words, holes as philosophical notions are too abstract for a crowdsourced venue that favors clean, factual statements like a novel’s plot or a band’s discography. Wikipedia’s bottom-up model could never produce an entry on holes like the SEP’s.
Crowdsourcing + voting
Authoritative: ? Comprehensive: X Up-to-date: ?
A variation on the wiki model is question-and-answer sites like Quora (general interest) and StackOverflow (computer programming), on which users can pose questions and write answers. These are slightly more authoritative than Wikipedia, because users also vote answers up or down according to how helpful they find them; and because answers are given by single, specific users, who are encouraged to say why they’re qualified (“I’m a UI designer at Google,” say).
But while there are sometimes ways to check people’s accreditation, it’s largely self-reported and unverified. Moreover, these sites are far from comprehensive. Any given answer is only as complete as its writer decides or is able to make it. And the questions asked and answered tend to reflect the interests of the sites’ users, which in both Quora and StackOverflow’s cases skew heavily male, American, and techie.
Moreover, the sites aren’t up-to-date. While they may respond quickly to new events, answers that become outdated aren’t deleted or changed but stay there, burdening the site with a growing mass of stale information.
The Stanford solution
So is the impossible trinity just that—impossible? Not according to Zalta. He imagined a different model for the SEP: the “dynamic reference work.”
Dynamic reference work
Authoritative: √ Comprehensive: √ Up-to-date: √
To achieve authority, several dozen subject editors—responsible for broad areas like “ancient philosophy” or “formal epistemology”—identify topics in need of coverage, and invite qualified philosophers to write entries on them. If the invitation is accepted, the author sends an outline to the relevant subject editors.
“An editor works with the author to get an optimal outline before the author begins to write,” says Susanna Siegel, subject editor for philosophy of mind. “Sometimes there is a lot of back and forth at this stage.” Editors may also reject entries. Zalta and Uri Nodelman, the SEP’s senior editor, say that this almost never happens. In the rare cases when it does, the reason is usually that an entry is overly biased. In short, this is not somebody randomly deciding to answer a question on Quora.
An executive editorial board—Zalta, Nodelman, and Colin Allen—works to make the SEP comprehensive. They steer the encyclopedia away from the ”wiki-hole”—having to open endless Wikipedia pages defining jargon in order to understand the topic at hand. ”We tell our authors to try to write an entry that is self-contained,” Nodelman explains.
Of course, it’s not just single entries that have to be comprehensive, but the encyclopedia as a whole. The board sees to this too, looking for cases where one long entry should be split up, or where one should absorb another. ”We had an entry on brains in a vat, but that was subsumed by ‘skepticism and external content,'” Nodelman adds (in easily the most philosophy-department line I’ve heard since earning my bachelor’s degree). Subject editors help with this as well, by identifying areas that deserve more attention and soliciting writers.
Can something so thorough be up-to-date? The editors have ways to make sure that it is.
A new entry is expected to contain the freshest possible information and research on a topic. As soon as it is published, the clock starts ticking on a new deadline. In exactly four years—or earlier if research has moved on significantly—the author must again hand in the most up-to-date entry on the topic.
In effect, therefore, each entry is on its own publishing schedule. “This is the only rational way to somehow keep track of all of the arcane topics out there,” adds Zalta. ”We are processing updates and changes daily,” says Nodelman. An ever-changing What’s New page shows the SEP revisions and additions for each day.
Updates come from a variety of sources. Quartz spoke to several SEP authors and editors, some of whom said that the encyclopedia is used frequently both as a reference and as a teaching tool. This means that philosophers are some of the SEP’s core readers, and they can alert authors or subject editors to incorrect or insufficient entries. Knowledgable readers are encouraged to do the same, even if they’re not philosophers.
The fact that there is a specific author and editor, and that the SEP has become so important to philosophy, helps make all of this easier. Any errors reflect poorly on the contributors, and someone who spots a slip-up can talk to a real person about it—neither of which is true with Wikipedia. And if an author is slow or unwilling to respond, the editorial board will transfer his or her responsibilities to a brisker philosopher.
The individual can be wiser than the crowd
There are a bunch of other benefits to this approach. Chief among them is giving the encyclopedia what Zalta calls an ”authorial voice.”
After regularly trawling through the internet information trash heap, it’s easy to forget exactly what that means: something written by a professional writer who has deep knowledge of the material at hand and an actual personality.
An exemplary SEP entry in this regard is the one on Socrates, written by Debra Nails, a specialist in ancient Greek philosophy. It contains a section called “Socrates’ strangeness” that captures the man in a way that’s much too elegant and confident to be on Quora or Wikipedia.
The extant sources agree that Socrates was profoundly ugly, resembling a satyr more than a man—and resembling not at all the statues that turned up later in ancient times and now grace Internet sites and the covers of books. He had wide-set, bulging eyes that darted sideways and enabled him, like a crab, to see not only what was straight ahead, but what was beside him as well; a flat, upturned nose with flaring nostrils; and large fleshy lips like an ass. Socrates let his hair grow long, Spartan-style (even while Athens and Sparta were at war), and went about barefoot and unwashed, carrying a stick and looking arrogant. He didn’t change his clothes but efficiently wore in the daytime what he covered himself with at night.
This authorial voice also avoids the tendency of crowdsourcing to be unhelpfully uncontroversial. For a long time, Wikipedia’s introductory line on Immanuel Kant read that he was “a central figure of modern philosophy.” The SEP, on the other hand, confidently calls him “the central figure in modern philosophy.” It’s a difference of only one word, but it explains the consensus of the philosophical community and conveys Kant’s true significance. (While I was writing this, Wikipedia updated that line to read “the central figure,” but quoting and attributing the SEP.)
Another benefit of the SEP’s not being crowdsourced is that minority views get more exposure. A 2012 survey by Wikimedia, Wikipedia’s parent organization, found that about 90% of its volunteers were men. “Its entries on Pokemon and female porn stars are comprehensive, but its pages on female novelists or places in sub-Saharan Africa are sketchy,” said the MIT Technology Review in its article The Decline of Wikipedia, which criticizes its byzantine editing hierarchy. The same goes for an important idea in philosophy: feminism. Wikipedia’s overview of feminist philosophy is hopelessly short. The SEP, on the other hand, is home to dozens of meticulously researched entries on the topic.
So the SEP model works, and it has 1,500 fact-checked, peer-reviewed entries to prove it.
Who pays the philosopher?
You might think all this can only be possible courtesy of a wealthy patron underwriting generous fees for authors and a large staff of editors. Not at all.
To be fair, Stanford does pay most of the operating costs. But the SEP has a paid staff of only three—Zalta, Nodelman, and Allen—plus five other Stanford employees who spend 20% of their time on technical support. Neither the authors, nor the dozens of subject editors, get so much as a dime for their troubles.
And all the authors and editors I spoke to seemed perfectly happy with this arrangement, even though some entries are a long time in the making. Siegel, the philosophy of mind editor, said that most take at least a few months from start to finish. The longest one she has overseen “stretched out for some years.”
There are a few reasons why contributors are willing to put in the time. First, these are already things that they are deeply interested in and enjoy. Peter Adamson, author of the entries on Al-Kindi and the Theology of Aristotle, noted that he had already written books on these topics. Siegel mentioned that being an editor allowed her to “keep up with interesting segments of the field.”
“I am very lucky to be able to do philosophy for a living,” writes Adamson, “and I am interested in doing things that would justify why I should be allowed to make a living in this rather nice way, where I am effectively paid to do something that I would do for free, as a hobby.”
Then there is the fact that the SEP allows academic philosophers to reach a wider audience. This helps them gain recognition and bring ideas they think are important to the world outside universities and conferences.
“I thought writing this entry would be a good way to bring attention to the many interesting and socially relevant debates that were taking place among feminist philosophers and gender and sexuality theorists about sex markets,” said Laurie Shrage, author of Feminist Perspectives on Sex Markets.
Zalta put it more bluntly: “I’m absolutely sure more people have read my entries on Frege than all my other publications combined.”
But perhaps the overriding motivation of SEP contributors is simply to further the enterprise of philosophy by creating a place to better understand it.
“I liked the fact that the SEP was going to be open access, and it was becoming a very important resource for students, instructors, and scholars in related fields,” said Shrage, when I asked why she contributed. Siegel echoed this altruistic motive: “Philosophy is a complicated subject,” she said. “People feel invested in the SEP in part because it helps philosophers at all stages orient themselves to philosophical problems and figures that may be new to them.”
To pay running expenses not covered by Stanford, the team obtained nearly $2 million in grants over the first 15 years. But they wanted something more sustainable, so they hired a business consultant (this is Stanford, after all), Javier Ergueta, and he proposed an idea that now provides around a third of the budget. The SEP asks academic libraries to make a one-time contribution. That doesn’t get them access to the SEP, since it’s already freely accessible, but they enjoy some extra “member benefits,” like the ability to use their own branding on a version of the encyclopedia, and to save the full archives.
Moreover, their money goes into an SEP endowment, managed by the same company that takes care of Stanford University’s endowment of over $20 billion. If the SEP ever shuts down, Stanford promises to give the libraries that contributed to SEP all their money back, with interest. ”It became a no-risk investment for the libraries, and it’s a way for them to invest in open access,” says Zalta.
Libraries were enthusiastic. The SEP was able to raise over $2 million from the long list of contributors, and Stanford added $1 million to the library endowment. The university also provides 60% of SEP’s budget—not much to ask from such a rich institution. The remaining 10% comes from a “friends of the SEP” program, which for $5, $10, or $25 a year lets individual users download nicely formatted PDFs of the articles, good for printing or archiving for personal use.
All this creative business thinking means the SEP can continue to exist long beyond these 20 years. “Our grant application days are over,” says Zalta. “We are practically self-sufficient as long as we don’t try to grow too much or too fast.”
The internet should look more like the SEP
The SEP is a highly rare case of knowledge being separated from the trash heap. The question is, can we make more of the internet like this?
The model cannot apply universally. Wikipedia is still necessary for its uncanny ability to provide basic (if often flawed) introductions to nearly everything. And StackOverflow probably offers the best chance at bringing some order to the ever-changing world of computer programming, where new languages and frameworks rise and fall with the sun.
Indeed, it might seem like philosophy is almost uniquely well-suited to the SEP’s model. It is a slow-moving discipline practiced by, literally, ”lovers of wisdom,” willing to share lots of their time to spread that wisdom around. The SEP method has been tried in other fields, without success. “People have contacted us from linguistics, to Egyptian studies, to a music department that wanted to make an online reference work,” Zalta says. None have been able to make a full dynamic reference happen.
Still, there are two reasons why it could be replicated.
First, even fast-moving, young disciplines like computer science or economics have core concepts that deserve comprehensive and authoritative explanation. StackOverflow is great at providing answers to highly specific programming questions, like how to round a number to two decimal points in Python, but fails to explain abstract or technical things like the theory of algorithms or the fundamentals of cryptography. In economics, there are dozens of excellent blogs, but where do you go to get an in-depth, impartial, picture of the marginal theory of value or comparative advantage?
These core ideas are fundamental. Self-taught programmers are wont to ”solve” problems by copy-and-pasting code straight from StackOverflow and crossing their fingers, with little sense of what the code is doing or why it works. Economics blogs might tell you that Greece’s economy needs to become more competitive, but it’s hard to understand what exactly that means without an intuition for these central concepts.
The second reason an SEP-like model could work more broadly is that the unpaid labor put in by SEP writers and editors isn’t something new to academia. Refereeing papers, editing journals, and other work outside an academic’s core research and teaching are typically unpaid in most fields. ”It hadn’t been done this way for reference works,” Zalta says; but having changed in philosophy, where writing for the SEP has become just another way to spend time working to make the field better, it could change elsewhere.
Nor does it just have to be only academics who contribute to online reference works. In computer science, say, it would make just as much sense for big tech companies to play the role of Stanford, putting money and staff into an authoritative encyclopedia of software development. Given the shortage of well-trained developers for hire, they have an incentive to do so. And large companies are already used to this form of self-interested altruism: They work on open-source code that benefits other programmers as well as their own.
What would it take to make this happen? The SEP’s model contains lots of clever insights on how to create a reference that stands the test of time—the library funding, the archiving for citation, the automatic deadline for article updates. But Zalta’s ultimate prescription requires nothing clever at all. Just old-fashioned resolve.
“What we had was several people single-mindedly focused on making this work,” he said. “I think our model could be reproduced if you get the right people involved.”
Landfill no more
You hear two things about information online. Nobody trusts it, yet everybody is referring to it, even for critical things like Ebola. That is not a recipe for an informed society.
The SEP is likely too rigorous to be the standard against which all information online is compared. But it shows we can create many more places that explain clearly the things humans know to be true. Bewildered Googling and tab-opening, tumbling indefinitely down the wiki-hole (if such a thing can even be said to exist), could be a thing of the past, if we only tried. It would be a lot more like the internet we always wanted.
Image of Rodin’s The Thinker at the Cleveland Museum of Art by Erik Drost on Flickr, used under Creative Commons 2.0 license. Image of Edward Zalta by Aasrubio on Wikipedia, used under Creative Commons 4.0 license.