6 1. INTRODUCTION: A PROGRAMMABLE WEB
bills, requiring a couple million dollars be given to someone for a particular pet project. (The $398 million “Bridge to Nowhere” being the most famous example.)
Both are fantastic sites and are frequently used by observers of American politics, to good effect. But imagine how much better they would be if you put them together—you could search for major campaign contributors who had received large earmarks.
Note that this isn’t the kind of “mashup” that can be achieved with today’s APIs. APIs only let you look at the data in a particular way, typically the way that the hosting site looks at it. So with OpenSecrets’ API you can get a list of the top contributors to a candidate. But this isn’t enough for the kind of question we’re interested in—you’d need to compare each earmark against each donor to see if they match. It requires real access to the data.
Note also that the end result is ultimately in everyone’s best interest. OpenSecrets.org wants people to find out about the problematic influence of money in politics. Taxpayer.net wants to draw attention to this wasteful spending. The public wants to know how money in politics causes wasteful spending and a site that helps them do so would further each organization’s goals. But they can only get there if they’re willing to share their data.
Fortunately for us, the Web was designed with this future in mind. The protocols that underpin it are not designed simply to provide pages for human consumption, but also to easily accommodate the menagerie of spiders, bots, and scripts that explore its fertile soil. And the original developers of the Web, the men and women who invented the tools that made it the life-consuming pastime that it is today, have long since turned their sights towards making the Web safe, even inviting, for applications.
Unfortunately, far too few are aware of this fact, leading many to reinvent—sloppily—the work that they have already done. (It hasn’t helped that the few who are aware have spent their time working on the Semantic Web nonsense that I criticized above.) So we will begin by trying to understand the architecture of the Web—what it got right and, occasionally, what it got wrong, but most importantly why it is the way it is. We will learn how it allows both users and search engines to co-exist peacefully while supporting everything from photo-sharing to financial transactions.
We will continue by considering what it means to build a program on top of the Web—how to write software that both fairly serves its immediate users as well as the developers who want to build on top of it. Too often, an API is bolted on top
