So I built another stupid widget because I wanted to try to use the Twitter API - but I ended up being pretty disappointed by the result.
My goals at the outset were:
- A single page widget that, given a specific Twitter user, returns the distribution of users responsible for the posts in their timeline
- Preferably just in HTML if that's possible, because hosting a sinatra application or whatever is beyond the scope of my interest in this project
But I was immediately stifled by the 1.1 API.
- Every request needs to be magically OAuth signed by your application
- (somewhat less surprisingly) the home timeline requires user authentication
So what I had naïvely envisaged as a single page with some requests to the API had blown out into something that required an OAuth library. At this point I should have stopped, taken a long hard look at what I was trying to achieve, chosen a library in a technology I was comfortable with, and built something that could handle the OAuth user handshake robustly.
Instead, I found a PHP library that would let me do it on one page, without worrying about any additional infrastructure. At the time I just wanted data, but retrospectively this was a mistake (mostly because I haven't touched PHP in three years).
The sole benefit was that I could put my twitter application keys in what was otherwise an HTML page and not worry about exposing them to the outside world; the downside is that the code isn't particularly attractive because a) I was just hacking on examples, and b) more than half of the code on the page is boilerplate for handling OAuth authentication. It's hardly something I'd be proud to release the source for (which, really, was most of the point of the exercise),
Even after making all these technological compromises in order to get data, the data I got didn't turn out to be very helpful. The Twitter API only lets you read the first 200 tweets of a timeline at a time, and the rate limit is particularly aggressive (15 calls / 15 minutes, compared to 180 for something like a lookup of a user's tweets)
This snapshot doesn't do a great job of illustrating the stated point of the exercise - prove one of your friends tweets too much - because it can only look at your most recent 200 tweets. When your feed is active, this snapshot can be for a time period of less than an hour; if someone is tweeting more aggressively than the rest, it's going to be pretty obvious.
Anyway, the moral of the story is that sometimes you have a great idea and then it turns out that to actually get anything out of it requires more data than you can get in one go.
I called it "Overbearing".