Frontier Nerds: An ITP Blog

Ever wanted to download Google Street View panoramas programmatically?

I reverse-engineered their URL query scheme using the Live HTTP Headers plug-in for Firefox. From there, I just parsed through the XML returned to download each jpeg tile and then stitch them together in order.

I used a variation of this code in combination with a few other programs to parse a GPS log, download the street view panorama for each position in the log, and then stitch each frame together into a video documenting my trip on the M5 bus.

import processing.core.*;
import processing.net.*;
import processing.xml.*;

// get a hold of this for the xml library
// better way to do this part?
PApplet main = this;

void setup() {
	// we usually end up with 3328 x 1664 pixel images
	// let's undersample considerably for a more sane window size
	size(832, 416);

	// your google maps api key goes here
	// sign up: http://code.google.com/apis/maps/signup.html
	String apiKey = "REPLACE ME WITH YOUR VERY OWN API KEY";

	// pick a location
	// takes an address or a lat / lon
	String location = "721 broadway, new york, ny";

	// generate the panorama
	Panorama pano = new Panorama(location, apiKey);

	// show the image
	image(pano.fullPano, 0, 0, width, height);

	// save the full-res image to the sketch folder
	pano.fullPano.save(location + ".jpg");

	// skip the draw loop
	noLoop();
}

class Panorama {
	float lat;
	float lon;
	String panoId;
	int imageWidth;
	int imageHeight;
	int tileWidth;
	int tileHeight;
	int xTileCount;
	int yTileCount;
	PImage[][] tiles;
	PImage fullPano;
	String apiKey;

	// constructor
	Panorama(String address, String _apiKey) {
		apiKey = _apiKey;
		String[] location = geocode(address);
		lon = float(location[0]);
		lat = float(location[1]);
		fetchInfo(lat, lon);
		buildImage();
	}

	void fetchInfo(float lat, float lon) {
		String url = "http://maps.google.com/cbk?output=xml&ll=" + lat + "," + lon;

		XMLElement response = new XMLElement(main, url);

		XMLElement kid = response.getChild(0);
		imageWidth = kid.getIntAttribute("image_width");
		imageHeight = kid.getIntAttribute("image_height");
		tileWidth = kid.getIntAttribute("tile_width");
		tileHeight = kid.getIntAttribute("tile_height");
		panoId = kid.getStringAttribute("pano_id");

		xTileCount = imageWidth / tileWidth;
		yTileCount = imageHeight / tileHeight;

		println("Panorama ID: " + panoId);
		println("X Tile Count: " + xTileCount);
		println("Y Tile Count: " + yTileCount);
	}

	void buildImage() {
		fullPano = createImage(imageWidth, imageHeight, RGB);
		fullPano.loadPixels();

		for (int xPos = 0; xPos<= xTileCount; xPos++) {
			for (int yPos = 0; yPos<= yTileCount; yPos++) {
				// &.jpg fools processing into handling it
				String imageUrl = "http://cbk0.google.com/cbk?output=tile&panoid=" + panoId + "&zoom=3&x=" + xPos + "&y=" + yPos + "&.jpg";
				fullPano.set(xPos * tileWidth, yPos * tileHeight, loadImage(imageUrl));
				println("Loaded tile " + xPos + "::" + yPos);
			}
		}

		fullPano.updatePixels();
	}

	String[] geocode(String address) {
		String cleanAddress = address.replace(' ', '+');
		String url = "http://maps.google.com/maps/geo?q=" + cleanAddress + "&output=xml&oe=utf8&sensor=false&key=" + apiKey;
		println(url);
		XMLElement location = new XMLElement(main, url);
		XMLElement kid = location.getChild("Response/Placemark/Point/coordinates");
		String rawCoordinates = kid.getContent();
		String[] latlon = shorten(split(rawCoordinates, ',')); // ditch the 0
		println(latlon);
		return latlon;
	}
}

Here comes the electronics lab:

3.2: The voltage regulator worked fine, delivering a steady 5.00 volts. (For the record, my 9 volt AC adapter was also measured a weirdly accurate 9.00.)

3.3: Switch worked fine.

3.4: In series, the voltage dropped evenly from LED to LED. Up to 3 red LEDs would light but a 4th killed it.

3.5: In parallel, the amperage across the circuit (between the last LED and ground) was 14 milliamps.

3.6: The potentiometer worked as expected. (At first, I accidentally plugging both ends of the pot into power — this turned the curve of voltage from the potentiometer into a dip — it was high at either extreme, and lowest (but not 0) at the center. Might be a useful behavior for future applications.)

Though the ending was wincingly melodramatic, the bulk of E.M. Forster’s The Machine Stops was eerily prescient — even more so given the 1909 publication date. (I thought his prediction of “imitation marble” — like the formica surrounding my kitchen sink — was particularly apt.)

Still, I might have dismissed it as too hyperbolic a warning were it not for a recent brush with the possibility of losing my place in the cloud. I let a typo through while entering my Google account credentials, which brought up a screen declaring that my account had been deleted (for some reason, I was served this doomsday message instead of the usual “incorrect password” elbowing). Though I don’t yet depend on them for air, food, or “medical apparatus” — I’m struck by how anxiety-inducing it was to see the words “this account no longer exists” on my phone. To rebuild from zero seems wearisome at best, and impossible at worst. This threat of silence brought real, physical anxiety. Without my GMail archives, saved articles on Google Reader, my contact lists, etc., rebuilding — much less remembering — my “thousands” of acquaintances would be impossible. My social, professional, and personal existence seem perilously leveraged by a single entity. Who even knows where the actual bits that represent this existence reside — Google’s estimated to have about half a million servers, spread across a dozen countries. This can’t be good. (But the very nature of cloud computing makes it so damn easy to ignore.)

Forster’s take on the inversion of the idea of the frontier also held my interest. Before the latter stages of the industrial revolution, nature provided the frontiers to be pioneered / conquered / etc. — the west, agriculture, sanitation, space, etc. But in the information age, it’s became fashionable to cast technology as a kind of frontier (Or so it was in the '90s… now it seems we’re in transition.). Yet, Forster turns this notion around, imagining technology conquered and the physical world turned back into frontier — this is particularly clear from the descriptions of the Earth’s surface: Kuno assures his mother that “I shall take all precautions” in the face of the surface’s perils. (The parallels to current ecological concerns are impossible to miss.)

I spent a rainy Sunday afternoon at / in / around the Tree Museum.

A slideshow is available on Flickr.

I came upon the following trees on my route from 149th street to 170th and back again:

• Tree 9 — Eric Sanderson, author of local history book. Planted as a street tree, but common to the region. Only the isolated trees survive after an epidemic of fungal disease. See The Manahatta project.
• Tree 31 — 32 year Bronx resident. Recalls canopy of trees where you could find some shade. She named tree 31 Ezekiel.
• Tree 32 — Director of the Bronx Museum. Saving a 3-story tall apple tree that would have been in the way of museum expansion.
• Tree 28 — Bronx singer-songwriter plays a song about the honey locust.
• Tree 27 — Music. Overlapped eerily with some live music playing further south in the park.
• Tree 16 — Valerie Capers, wrote a song about things to do in the Bronx.
• Tree 15 — Franz Sigel Park history.
• Tree 12 — Professor, narrates a history of Franz Sigel. Immigrant contributions.
• Tree 10 — Andrea Polli, ecological concern.
• Tree 6 — Resident, can’t recall trees in her childhood. “You don’t have to leave the neighborhood to live in a better one.” Founded the Sustainable South Bronx community organization.

It probably would have been wise to heed the advice from the woman behind the desk at the Bronx Museum. After listing the current exhibitions, she mentioned the Tree Museum, with the weather-conscious caveat: “You’ll probably want to do that some other day.”

It was just a foul and wet and windy day. The Grand Concourse was just about completely pedestrian free — at least along the stretch I explored. Husks of inside-out umbrellas dotted the sidewalks and parks. It seems like the Tree Museum is grounded in sociability — take a walk, hear some history from the locals, maybe meet some people around a tree — but my trip was mostly silent, and the only people I met were pre-recorded on the other end of my cell phone.

What I wanted when I dialed the number was a conversation (and that’s what we’re wired to expect after dialing a cell phone), but while the exchange was purely reactive, not interactive. Also, standing in front of a tree with a phone to your ear leaves you in an antisocial posture, compounded a bit by the novelty of having a conversation (of sorts) with a tree (arboreal schizophrenia?). In some ways, the phone’s technology brings isolation from, rather than immersion in, the neighborhood and community. The whole process might work better in groups, or on days with more foot traffic when someone might ask what’s going on — sharing the experience would improve it.

The trees were sparse enough that the walk felt a bit like a scavenger hunt — and the positioning of the placards meant I spent more time staring at the sidewalk and less time looking up at the trees. When I did find a tree, I ended up almost subconsciously taking its picture — which seems like the facile and obvious thing to do, but also in line with the scavenger hunt sensibility, proof that the treasure was found.

The absence of people also drove home the sense that the trees are a kind of witness to the history and change in the neighborhood, and I was interested to learn from tree 9’s recording that many were planted when the street was created, which must make some of them exactly as old as the concourse itself. (I had previously assumed that the concourse was carved out of existing forest.)

I would have liked more connection between the phone narratives and the immediate environment — the most effective moment of the trip was the overlap between a live band at the south end of Joyce Kilmer Park and tree 27’s musical recording.

I’d like to go back on a brighter day, to see how a busier concourse contributes to the messages from the trees. Otherwise, the emptiness made the experience feel more like a monument to something lost than a museum of something existent.

Will we ever run out of photographs?

As the web accumulates photos (another 6,037 posted to Flickr in the last minute…), and as projects like Photosynth start to infer intermediate perspectives between these images, one could conceive of collisions — multiple instances the same image, pixel for pixel, from two different authors.

Multi-megapixel full-color pixel images weighing in at dozens of megabytes distance us from what, exactly, these images are at the computationally atomic level of bits. Going back to the same concept on a smaller scale, makes the simplicity of the computational representation of an image clear. Black and white makes this exceedingly basic — an image can be as simple as a grid of 0s and 1s:

Ditch the grid, and you’re left with a big binary number:

100000000000001000000000100010101000001000011100100000100010000001111111000001000001000001001111000001001001000001001111000001001001000001001111000001001111000001000001000001000001000001111111

Convert that to base ten, and you have:

3,138,742,632,065,979,126,417,490,138,422,209,858,437,835,786,299,168,264,319

The picture’s just a number. It’s just one of the possible black and white images that could fit in the 12 x 16 pixel grid. In fact, it’s image number 3,​138,​742,​632,​065,​979,​126,​417,​490,​138,​422,​209,​858,​437,​835,​786,​299,​168,​264,​319 out of 6,​277,​101,​735,​386,​680,​763,​835,​789,​423,​207,​666,​416,​102,​355,​444,​464,​034,​512,​896 possible images.

Every single one of the ~6 octodecillion possible 1 bit 12 x 16 images could be generated computationally. We don’t need a camera, a photographer, or an icon artist — a simple program looping through and rendering each possibility would, eventually, stumble upon that exact spray can. And, eventually, it would run out of images to render, and the creative possibilities of a 12 x 16 pixel 1 bit canvas would be exhausted, or, at least, definitively explored.

These numbers are huge — incomprehensible. For all practical intents and purposes, these numbers are as good as infinity, and yet, they’re finite in every objective sense.

Scaling this idea up to the high-res color images typical of the web, the number gets even more ridiculous. You can find the number of possible distinct images for a given bit depth and resolution with the following formula:

Possible Photos = (2^Bit Depth)^(Width * Height)

This brings up a few questions.

First, this exposes an inherent and often unconsidered flaw of digital mediums — finite resolution means finite potential representations, suggesting a degree of determinism in digital systems.

Second, what might one discover by traversing the range of possible images with an algorithm instead of a computer? Would anything be discovered? Might we see glimpses of the future or past? By definition, these photographs would all have to be present in the set — portraits of every human, past and present, confidential texts, etc.

Third, what does this say about photographic creativity? Is creating a photo an act of genesis and original thought, or is the photographer merely colliding with an inevitable, preexisting combination of bits?

Fourth, exploring data sets like this also brings up the question: What’s the threshold of perception? How many pixels do we need to have before something representational emerges? Of course, deciding on any threshold implies compromise, and the representation of different facets of reality can move this threshold radically. For example, a single black pixel could be said to represent a dark room, and therefore could be said to hold some representational qualities. Representing text on a page or the nuances of a tree, however, would require thousands (maybe millions) of pixels.

A few artists explored this idea in the 90s: Jim Campbell’s The End (1996) John Simon’s Every Icon (1997)

It’s also worth considering that this concept is agnostic to the actual way in which in a particular bit string is interpreted. For any digital object of a particular size, this theory of finite possibilities can be applied and explored.

For example, artist Kyle McDonald explored this idea in sound, generating every possible sequence of the chromatic scale’s 12 notes: