Public In/Formation

Librarians in formation: Joan Spencer, Mildred Handy, Mollie Huston Lee, Beatrice Hamlin, and Maude Young, at the Richard B. Harrison Library, Raleigh, North Carolina, 1968. Lee founded the library, which was the first in Raleigh to serve African Americans. [

It took less than a year for New Yorkers to lose sidewalk internet privileges. Much of the city cheered last winter when hundreds of sad, squat payphones were replaced with futuristic monoliths offering free phone calls, device charging, and superfast internet. 1 Tourists could check maps, locals could access municipal services, schoolkids could download homework assignments. 2 (Never mind the poster-size ads and data tracking.) But not all the neighbors were thrilled. Soon came the reports of people gathered for hours around these digital campfires, streaming music or watching movies and porn.

“We know that some users have been monopolizing the Link tablets and using them inappropriately,” officials said in September. “The kiosks were never intended for anyone’s extended personal use.” LinkNYC disabled web browsing and promised to work with “the City and community” to find a solution. 3 Two months later, it’s not clear when access will be restored. The mayor himself delivered the eulogy, describing curbside internet as “a good idea that ended up having a real unintended consequence.” 4

Unintended, maybe, but hardly a surprise. An eighth grader could have called it. Which raises the question: How can you roll out digital infrastructure at this scale without anticipating the “tragicomedy of the commons”? 5 Were there no librarians on the team? Librarians have managed internet access and guided patrons through new digital terrain for decades. 6 They have raucously debated how to accommodate all kinds of online behavior, and have developed tools for promoting free speech and open access while discouraging illegal activity and shielding patrons and staff from offensive images. 7 They have tested policies and procedures — time limits, download caps, and content filters — for ensuring that resources are shared fairly. The information commons is messy, and negotiating such issues is part of living in a robust democracy. 8 What works in the public library can work on the street.

LinkNYC station. [Johannes Schmitt-Tegge/AP]

But let’s broaden the scope. The LinkNYC stations are maintained by CityBridge, a consortium of telecom, hardware, and media companies (notably, the Alphabet subsidiary Sidewalk Labs) under contract with the city government. It’s the kind of public-private infrastructure that excites the urban planners and technologists who imagine New York as a data-driven utopia, a so-called “smart city.” I understand the impulse to roll out unlimited internet, and I know that tech companies thrive by moving fast, often ignoring historical precedent. Still, that’s no excuse for hubris and amnesia. The rocky launch is a sign that planners and engineers need more partners at the table when they dream up new forms of urban intelligence. 9 Yes, I’m talking about librarians.

To be clear, the stakes here are higher than convenient internet access. A would-be strongman is headed to the White House, amidst swirling currents of disinformation. 10 He has threatened to jail political enemies and sue newspapers, further destabilizing a media environment that was already reeling. Online and off, we need to create and defend vital spaces of information exchange, and we need to strengthen the local governments and institutions that shape the public use of those spaces. The future of American democracy depends on it. Bigly.

And we cannot depend on tech companies to safeguard those information spaces. Sidewalk Labs wants to turn Link stations into nodes of intelligent infrastructure that may one day collect data on pedestrian traffic and garbage removal, direct drivers to parking spots, route autonomous vehicles through the streets, and push location-specific targeted advertising. 11 The ideology of data solutionism has taken over city halls, planning departments, law enforcement agencies, and countless other domains of public life — a troubling trend when social technocrats were in charge, and now, with the rise of Trumpism, an alarming one.

Trump surrogate and New York City autocrat Rudolph Giuliani. [Evan Vucci/AP]

With rising distrust at the federal and state level, many observers believe that city governments are the new locus of democracy. We must push our civic leaders to bolster their planning teams with experts in the ethical collection, organization, preservation, and dissemination of information resources. 12 Urban data programs should be counseled by professionals who understand the complex issues of equity, privacy, and security.

Librarians on the planning commission! Archivists in the police academy! Why not? Now more than ever, the agencies and corporations that are “instrumenting” our connected, intelligent cities need exposure to democratic, humanist convictions and sensibilities. I don’t mean to romanticize knowledge workers. I know that cultural institutions have their own dark histories and scandals, that their budgets and mandates are already stretched thin, that they, too, are implicated in political structures that can be oppressive and unjust. Nevertheless, they are guardians of a critical, contextual approach to information, which is a public resource every bit as necessary as streets and sewer lines. 13

As Zadie Smith puts it, librarianship embodies a “different kind of social reality… which by its very existence teaches a system of values beyond the fiscal.” 14 Those values include access and accountability, a balance between openness and privacy, a commitment to preservation and security. 15 And because librarians uphold those noble values on shoestring budgets, without the mentorship of angel investors and tech accelerators, they tend to develop a healthy skepticism about technology, and even about their own fundamental ideals. They oppose the ruthlessly efficient, behaviorist, techno-liberal city, which prioritizes innovation-driven obsolescence, exclusive contracts, and monetization of user data. Librarians on the planning commission will be the ones to ask, why should procurement agreements favor platform providers rather than the citizens who contribute data? Archivists will ask about racial imbalances in data harvesting and push for anonymous and secure preservation of public records. Together, they can be stewards of equity, discretion, interoperability, resilience, and respect for the past — real wisdom, rather than proprietary “smarts.”

At a smart cities conference held in 2012 at the Bartlett Centre for Advanced Spatial Analysis, researchers presented a simulation of London riots. [CASA]

At the Siemens Future Forum in 2014, software exec Thomas Hahn gave a presentation that illustrated the conventional thinking about urban data. He described the evolution of big data and its applications in a variety of fields, including urban planning. We know that “smart cities” around the world are building control centers and information hubs that collate data from numerous systems: energy, water, and transit networks; demographic and economic datasets; imagery from satellites, drones, and street cameras; signals from social media, mobile apps, and embedded sensors. Hahn argued that data managers need to integrate multiple intelligences: computer science, math, stats, physics, engineering, economics. 16 He said nothing about law, ethics, or governance — let alone records management and archival science.

But many researchers are thinking more broadly. They argue that urban data programs should attend to data structures and provenance; that is, to the origin, custody, and ownership of information resources. 17 They push for higher standards of security and privacy, and they focus on issues such as data preservation, ethical reuse, and citizen education. 18 Advocates for best practices in Chinese cities have proposed a principle of “data continuity,” i.e. that digital information is “available in a timely manner and opened in readable form, complete with the context and an assured quality.” 19

Yet even the most progressive cities have failed to embrace basic principles of archival science. Geographer Tracey Lauriault has worked for decades on data infrastructure projects, from community mapping to natural resource modelling to scientific data portals, to postdoctoral research at Rob Kitchin’s Programmable City Project. She says that city governments lag far behind other sectors in their data standards and practices. “Most cities don’t archive their digital collections,” she told me. “They may back stuff up, but a backup doesn’t constitute an archive,” as it lacks metadata to ensure that contents can be reliably retrieved, and there is typically no preservation plan to keep the data secure. 20 She noted that urban geospatial data are “rarely ever archived,” and real-time data — such as streams from traffic feeds and air-quality sensors — “even less so.” 21 Kitchin added that data curation practices are often better in national governments (e.g. in agencies devoted to mapping or environmental protection), where the planning “includes a lot of framework data about cities. Indeed, there are very large global initiatives around building spatial data infrastructures and things like associated standards and metadata.” 22

Environmental sensor built by Argonne National Laboratory for the city of Chicago. [Mark Lopez/ANL]

Why are cities managing their data poorly? Partly it’s a lack of experience. Lauriault found that cities tend to hire technical officers with an IT background, rather than drawing talent from the worlds of librarianship and archival science. There’s also disagreement, even among archivists, about what technically constitutes a “record,” which means there’s no consensus about what data should be preserved. 23 New media have pushed archivists to enlarge their definition of a “record” beyond discrete documents, and they are developing new ways of archiving media flows and streams, but very little of that knowledge finds its way into city governments, where it could be put to good use. 24 Lauriault says that real-time urban data are “probably being written over” and wiped out.

She and Kitchin are especially concerned about the inferior condition of cities’ open data repositories, which include datasets as varied as building permits, crime statistics, taxi trips, and tree counts. In recent years, the open data movement has pushed cities to prioritize transparency and access to the information they control. In theory, this gives everyday citizens a window onto urban operations, and it allows developers to leverage the data for nonprofit or entrepreneurial purposes, like smartphone apps that predict transit times. Yet Kitchin says that “most city open data sites are effectively data dumps,” without even a basic archival infrastructure. They lack consistent metadata and structured vocabularies, and they rely on tags that are plagued by errors and ambiguity. The data themselves are often questionably accurate, and consumers do not have enough metadata to judge their quality. 25 The city may provide summary information about provenance — for example, the age of the data and the identity of its source — but rarely anything about how the dataset has been derived and transformed, or about the accuracy of its spatial and temporal markers. 26

Kitchin and Gavin McArdle have proposed to crowdsource the quality control of open data repositories, 27 but Lauriault has a more ambitious idea. She wrote a report for the Irish government recommending that open data be regarded as official records that require care throughout their life cycle, from conceptualization, creation, and receipt through maintenance, use, reuse, and disposal. 28 Archivists and librarians, she says, should be involved at the highest levels of design to ensure that preservation standards are reflected in the structure of metadata. 29 Open data projects that adhere to archival standards could be designated as “trusted digital repositories” that provide “reliable, long-term access to managed digital resources … now and in the future.” 30

Counterterrorism software by Palantir Technologies, the data analytics company co-founded by Peter Thiel, the Silicon Valley mogul and prominent Trump supporter.

But is the fuss worth it? Are subway turnstile counts and urban noise readings so precious that they must be carefully preserved? In cities that are already constrained for resources and layered with bureaucracy, why should data management be considered one of the essential functions of government?

First, historical data reveal patterns of movement, progress, and decline that are not visible in raw or real-time data. As we move toward a future in which urban operations are data-driven or even automated, it is essential that we have access to that longer view. But those data are not useful unless they are thoughtfully curated. Librarians and archivists can educate city staff about the practical and ethical limits of technology. They can remind us all that knowledge-creation, or intelligence-gathering, should involve more than the exhaustive, indiscriminate accumulation of data. The archive is not all-encompassing, and not all data should be preserved. Information professionals can guide difficult decisions about retention and disposition schedules.

Moreover, archived urban data can provide evidence and aid forensic investigation in the case of disputes, accidents, and disasters. Decisions about how evidentiary data are stored and used should not be left to insurance agents or the police department. In August, Bloomberg reported that “Secret Cameras Record Baltimore’s Every Move From Above” using military technology developed in Iraq. 31 You can bet there are no librarians on that team. Archivists can intervene in the “chain of custody” of new forms of civic data and public records, to ensure best practices. Consider, for example, the increasing prevalence of bodycam footage and bystander videos of interactions between police and civilians. In Los Angeles, university archivists and information scientists are collaborating with civil liberties and public watchdog groups, policymakers, law enforcement agencies, and product vendors to develop standards for “audiovisual evidence management.” 32 The name of their project reflects an emerging reality: “On the Record, All the Time.”

Police officer videos a protest in Washington, DC, in 2011. [Andrew Bossi]

This new reality raises technical and legal challenges, including questions about the costs of data storage, privacy, and security; limits on the access and use of data by various audiences; and problems of media obsolescence and changing file formats. Those concerns apply in other contexts, too. 33 In London, the research agency Forensic Architecture conducts spatial analyses for legal forums investigating human rights and environmental justice cases. They evaluate official archives and compile their own collections of crowd-sourced evidence, such as audio snippets from news broadcasts, which can be cross-referenced with witness videos and photos to produce a spatial model of contested events.

Lauriault argues that maintaining a responsibly archived open data repository is simply a matter of “good governance”; it cultivates transparency and accountability and promotes civic engagement. She continues: “Data, like government documents, are a way for government to communicate what it is doing, and how it is doing it … with its citizenry, its public, its public sector, and the international community.” As Lauriault sees it, public information resources are a natural extension of democracy. They are cultural assets, and if we are to fully exploit their value as such, we have to recognize that “librarians and archivists … have been at this [work] for a while,” and they have a lot to offer urbanists and technologists. 34

City engineering department plan vault, Seattle, 1936. [Seattle Municipal Archives]

Public libraries, in particular, can play a critical role in shaping the new urban data landscapes. Not only do they have experience in negotiating access policies, but they also demonstrate a commitment to openness — as in open source, open access, open doors — contra the black-boxed, proprietary infrastructures and algorithms that dominate urban development (and the “extreme” profiling promised by the new federal regime). Governments often have a mandate to provide open data, but they lack professional guidance about the creation of user-centered services. They can partner with public libraries to meet their obligations. Jim A. Jacobs of Free Government Information proposes that libraries “identify, select, and acquire large datasets of invaluable information content without cost or copyright restrictions,” then add user services on top of them. 35

That approach has found favor in Chattanooga (2013-14) and Boston (2015-17), which received grants from the Knight Foundation to place libraries at the center of their open government initiatives. In Boston, officials reported that their data were not easily searchable in a way that would enable users to create relevant associations. “Making something available isn’t the same as making it useful,” they acknowledged. So they tapped librarians to “find value,” “connect the dots,” and “turn data into knowledge.” Starting with an inventory of existing resources, the city wants to produce a “data catalog” with rich metadata and an enhanced user experience. The librarians are also involved in efforts to create programs and curricula that educate the public about the uses and limits of open data. 36

Similar data literacy initiatives are underway in many cities. 37 BetaNYC is teaching community classes on open data, and Canadian libraries are hosting Open Data Book Clubs, with a different theme or dataset discussed each month. Trevor Owens, at the U.S. Institute for Museum and Library Services, calls for libraries to become a “kind of middle ground for civic data initiatives. That is, the libraries should be spaces where anyone can learn about the data that are being collected about them, or about their communities, and also learn how they can use those data themselves and have a voice in how they are collected, managed, and used.” 38

Librarians are equipping themselves with new tools and skills and getting educated about data rights management, intellectual property, and privacy. They have long been proponents of privacy, but evolving norms and technologies of data capture and surveillance present new challenges. 39 The Library Freedom Project, a partnership among librarians, technologists, attorneys, and privacy advocates, offers workshops on surveillance threats and privacy rights, responsibilities, and strategies. Related initiatives include the Data Privacy Project and International Right to Know Day. Cities like Seattle have created privacy advisory groups that serve as formal mechanisms for assessing how urban data are generated, stored, and used 40 — including, potentially, the power to “conduct forensic internal audits.” 41 But if we want our libraries to lead such initiatives, Owens said, we need to give them statutory authority, as well as appropriate staff and funding.

Long Island City, Queens, New York. [Detail from the Sanborn Fire Insurance Maps, 1903]

Map and geospatial information libraries have benefited from especially ambitious data curation and preservation initiatives. 42 And as Wang Tao argues, spatial data are central to the development and management of intelligent cities. 43 John Hessler, a specialist in modern cartography at the Library of Congress, told me that library resources such as real-time GIS, remote sensing, and digital elevation models can have innumerable applications, from modeling the shadows of tall buildings to addressing line-of-sight issues. Yet even at a well-resourced institution like the Library of Congress (which holds the largest map collection in the world), data preservation and accessibility are ongoing concerns. Hessler noted that architects and planners sometimes ask him, “If I use this data and need to go back a few years from now, will [the data] still be here in a form that is usable?” Proprietary structures and tools are an increasing burden. “Without the data in an archival form,” Hessler said, “the design or analysis cannot be reproduced.” 44

As customers get used to personalized search queries and egocentric mapping apps (with “you are here” at center screen), they often expect institutional resources to be more user-friendly than they can be. Jenny Marie Johnson, a university map librarian, observed that students “do not always understand that the items or data that they require may not be available or may not be in a format well-suited to their needs.” 45 Scott Walker, a specialist in digital cartography, said that he encounters architects and planners who want granular data on, say, all business locations in a certain neighborhood between 1930 and 1940, or the location of sewer and water lines throughout a city. 46 Such requests are often difficult, if not impossible, to fulfill. The fact that we cannot simply input a few parameters and whip up a custom map on any topic, in any region, in any historical period, indicates the limits of our spatial knowledge, as well as the imperfection of the tools that register that knowledge.

Yet maps made for a certain purpose often have applications in other contexts, and librarians can help patrons make those connections. Most libraries have paper maps that have remained “usable” for centuries, and that continue to yield useful data. 47 For example, the exquisitely detailed Sanborn Fire Insurance Maps, first produced in 1866 for insurance assessments and now a staple of library collections, provide researchers and designers with information about urban evolution, changes in land use, individual building footprints and lot dimensions, construction types, and other historical and environmental factors. 48 Many institutions have digitized their paper maps, and the New York Public Library has created tools (through the late, great NYPL Labs) to “rectify” historical maps — layering them atop a contemporary base map — and extract data about specific attributes, which can then be made searchable. Old aerial photographs are another useful resource for “creating a chronological view of 20th-century change,” Johnson said.

The Map Warper tool developed at the New York Public Library shows lower Manhattan overlaid with the 1660 Castello Plan of New Amsterdam (left) and 1850s sheet maps (right). [via

Exposure to an array of maps from different regions and historical periods, with their varying cartographic conventions, helps patrons appreciate “how the process of visualizing urban space has changed over time,” Walker said. Map librarians also construct frameworks for meta-analysis that reveal maps as cultural objects and encourage reflection on their embedded politics and epistemologies. At the Library of Congress, Hessler said, “We try to understand the source of the data we are providing, its accuracy, and … why it was compiled and by whom, and what algorithms were used to process its raw form.” That metadata allow patrons to explore what he calls the map’s “fourth dimension,” its relation to the material world: “What are [the mapmakers] trying to show and say, what are they leaving out, how are they abstracting from the real world to produce a design or map?” He wants users to understand that spatial data must be “critically approached … no different than a text.”

Ultimately, critical inquiry of this kind reveals the limitations of the data we collect. When we measure only those things that can be quantified and pushed through an algorithm, we lose a lot of meaningful knowledge about place. Library and archival collections demonstrate that there are forms of urban intelligence that are vital and useful even if they can’t be downloaded as CSV or KML files. Consider the work that many libraries are doing to present oral histories and help individuals and communities to archive their own projects. This type of work happens in the Memory Lab at the DC Public Library, which, Owens said, is “importantly not about collecting or hoovering up information, but about facilitating preservation and memory of communities.” And it happens at independent collaborative projects like Documenting the Now, which arose after the social unrest in Ferguson, Missouri, to support the ethical use and preservation of social media posts that document social movements. 49

A radical library in Bradford, England. [Ian Clark]

Local data, little data, analog data: these too are building blocks of urban intelligences. 50 Working alongside allied research institutes and advocacy groups, local libraries and archives can create critical links between communities and the various forms of intelligence that reflect and shape who they are and the places they live.

But who will preserve the preservers? In September, the New York Public Library announced the closing of its pioneering and celebrated Labs division, makers of the MapWarper and Building Inspector. I had the pleasure of collaborating with lab members on numerous occasions over the past several years. Just a week earlier, the Sunlight Foundation, a prominent advocate for government transparency and open data, had announced that it was discontinuing its “tool building and database maintenance activities.” Now we face the prospect of a Trump administration without Sunlight. 51

Perhaps the “lab” model is inherently fleeting. At least, that’s how mourners on Twitter consoled themselves. We can hope that the legacy of these teams’ work — so consistent with the foundational principles of librarianship and archival science — will become “socialized and embedded,” as Mozilla’s Kaitlin Thaney put it, within their “parent” institutions. We can hope that their work will continue to animate open and egalitarian infrastructures and public spaces of information exchange. We can hope that the people and agencies who fund projects like this will one day be as committed to sustaining these civic links as they are to pursuing bridges, CityBridges, with their hidden economic and political tolls. We can hope. The stakes now are higher than they have ever been.

Author’s Note

I am greatly indebted to Jordan Hale at the University of Toronto, John Hessler at the Geography and Map Division at the Library of Congress, Jenny Marie Johnson at the Map and Geography Library at the University of Illinois at Urbana-Champaign, Nancy Kandouian and Artis Wright at the New York Public Library Lionel Pincus & Princess Firyal Map Division, Rob Kitchin at Maynooth University, Ryan Mattke at the John R. Borchert Map Library at the University of Minnesota, Trevor Owens at the Institute of Museum and Library Services, and Scott Walker at the Harvard Map Collection. I owe a special debt of gratitude to Tracey Lauriault at Carleton University, who patiently answered my questions and reviewed a draft of this article; and to Ozayr Saloojee and Karen Lutsky, who invited me to share my work at the University of Minnesota.

Notes