White House petition sigs by state

White House petitions don't have an API yet so I wrote something quick and dirty for a specific petition in reply to a tweet by @PogoWasRight.

This thing scrapes the petition's signatures and produces histograms by state and city as a javascript file called histograms.js. This gist contains a snapshot copy of that file (generated when there were ~32K signatures) This means you can either clone the gist and view index.html, or simply use Mike Bostock's magic gist viewer.

To scrape histograms.js yourself

1. Install dependencies: on Debian it would be sudo apt-get install python3 python3-bs4
2. Change INITIAL_URL at whphist.py if you're interested in some other White House petition.
3. ./whphist.py > histograms.json fetches all signature pages from petition site. This may take a while. If your network connection fails (happens to us in the 3^rd world quite often), do mv histograms.json old_hist.json and then ./whphist.py old_hist.json > histograms.json to continue where you've left.
4. ./json2js.py < histograms.json > histograms.js this tidies up things a bit, and generates a javascript file easier to work with.

To do (call for procrastination)

• whphist.py has zero user interface at the moment (even the URL is hard-wired). Should be friendlier. Command line? Web front-end? You be the judge.
• I've just started learning d3. I'm sure many of you can do much nicer visualizations. Note that histograms.js also contains breakdown to cities, so you can make pie slices explode and what not :)