Tag Archives: APIs

Graphing My Cycling Progress on Strava and Automating With Zapier

Since I moved to a new home that’s closer to work, I’ve been riding my bike to commute at least a few times each week. My commute to work is very easy as it’s almost completely downhill. I pay for it on the way back, though. I’ve been trying to get healthier, so commuting with my bike has definitely become a priority. In addition to having fun, I’d also like to motivate myself to ride more often and push myself to get better at it (especially the uphill part).

I’ve been using Strava to track my rides, and it turns out that my ride home includes a user-generated segment that automatically tracks my performance when I’m going uphill on Liberty. The first time I rode, it took 10:44 for me to complete. My best time so far is 6:58. The cool part is that I’m able to see the progress I’ve made, and I really do feel accomplished when I beat one of my best times. Here’s the view I’m using to see what my times are:

Stava SegmentThis isn’t the easiest table to read, but you can see that the dates sort of correlate with my times, as the fastest one is in September and the ones in August are slower on average. While beating my fastest time is a good motivator, it’s not realistic to try and break my record every time I ride. I’d rather see consistent improvement as a motivator.

To visualize this, I decided to grab all of the times by viewing the full leaderboard (with just my results). I gathered the times and dates and thew them into a Google spreadsheet to visualize my trend. Here are the results so far.

Strava Trend

Success! It looks like I made some really good progress since I started in July, and my times have been inching downward as I get better at cycling uphill. The times also probably vary a bit because there are some stop signs and a traffic signal within the segment, which can slow me down.

The next problem I wanted to tackle was the data entry bit. Since it’s a pain to update the spreadsheet each time I ride,  I wanted to automate the addition of new efforts (the term that Strava uses to describe single instances of activities on a segment). This is where things get interesting.

Because I’m storing the efforts in a Google Doc, and because Strava has an API, I just need to connect the two together. Unfortunately, my go-to choice for this kind of thing, IFTTT, doesn’t have Strava as a channel. Luckily, I can do something similar with Zapier (btw that’s a referral link, so be sure to click it so I can get more tasks), which is sort of like IFTTT but costs money (albeit with a nice free tier) and has more integrations. You can also set up your own integration, which is what I had to do with Strava.

In order to get the list of my efforts on that particular segment, I had to create a few Zapier “triggers.” One to get my Strava user ID (for use in other API endpoints), another to grab my starred segments (so I could specify which segment I wanted to track), and finally a trigger to listen for any new efforts on my commute’s segment (limited to efforts created by me). I also had to post-process the last trigger so that I could get the date in a format that works in Google Spreadsheets. The result looks something like this:

Strava Zapier

Now, whenever I ride home on my bike while recording the segment with Strava, the effort will be automatically logged and graphed on my Google Spreadsheet! Apparently, I can embed the chart, so here it is!


Chart

While the Zapier integration had a somewhat steep learning curve, it’s nice to just set the “zap” and then forget about it. Any new integrations I might need to write will also go much quicker. As always, Runscope was a really useful tool for exploring the Strava API and getting real responses from the API to play around with. Finally, I learned the right way to spell athlete rather painfully (after misspelling athlete_id a billion times and wondering why my request wasn’t filtering correctly)!

I went from seeing some sketchy looking progress in my Strava results table to being able to visualize it on a graph, while also setting the graph up to update whenever I record a new effort! Automation for the win!

If you’re interested in the details of the integration, or if you want to try it out yourself, let me know and I can invite you as a tester. If there’s a lot of interest, I can also go over the creation of the integration, as I ran into a few gotchas while building it.

Inspecting and Debugging API Versions with Runscope

FL-iPhones This past month at FarmLogs, we implemented a versioning scheme for our internal API. API versioning is often a difficult transition to make, especially if you’re using a third party API and dealing with deprecations (Facebook, I’m looking at you!). It’s often a chore to figure out exactly what has changed between API versions unless you have very good documentation.

Coincidentally, Runscope also has a service called API Changelog that can send you notifications when a third party API changes. Luckily for me, any changes to our internal API were subject to review by our team of engineers, so if I had any questions or concerns, I could bring them up in person. Even so, when dealing with a changing API, the truth is in the response, not necessarily the documentation (or Hipchat or Hackpad). For a while, I was updating the iOS client to the new version by copying and pasting the two responses into a text editor and looking at the differences. About halfway through I remembered that this functionality is built into Runscope, so I started using that. I’ll show you how to compare diffs of two responses so you don’t have to waste half of your time like me!

Prerequisites

To follow along with this blog post, you’ll need just two things:

Some versioned API data

I’m not going to post the inner workings of the FarmLogs API (though it’s pretty easy to reverse engineer, like many other APIs) since that’s probably confidential knowledge or something. Instead, I’ll use the Foursquare API, which is one of my favorites to work with and (in my opinion) has some of the best documentation around. Browsing around their developer page, I found this interesting summary of API changes over the years.

There was a pretty huge change on 6/9/12 that included many breaking changes. Sounds like fun! I actually remember getting caught in this change and having to update my iOS auto-check in app to fit the new API. Let’s pick an endpoint and see what the data used to look like. We’ll be able to compare it to the modern version of the API.

For our example, we’ll hit the /venues/explore/ endpoint. This endpoint takes a lat/long value and returns a list of suggested venues to try. If you are authenticated, it can give you personalized info about why those venues were suggested (like if your friends went there and liked it). If you go to this page, you can get a url for that endpoint along with an oauth token added for you automatically.

Hitting the endpoint in Runscope

Next we’ll copy and paste the Foursquare API url into Runscope. You can create this request in a default bucket or make a new one specifically for this test. Just hit the “New Request” button and paste the url in. Hit the “Launch Request” button to have Runscope make the API request on your behalf and save all of the data for you to compare later. If all goes well, you should see the response appear below the “Launch Request” button. Next, we’ll edit the request so that the format matches the API before the big change on 6/9/12. Foursquare uses a pretty awesome convention for API versioning that uses dates for versions. To edit the response, simply hit the pencil icon on the upper right part of the response. You’ll be taken to the response editor. From there, change the “v” parameter to 20120608 (my 29th birthday!). request editor Hit “Launch Request” again and Runscope will fetch the response from Foursquare for a client that’s using a really old version of the API. After a short wait you should see another response appear. You can either hit the “Compare to Original” button or go to the “All Traffic” section to pick two requests to compare.

Comparing two versions of the same request

Finally we’re at the meat and potatoes of this post! Let’s explore the difference that two years can make to an API endpoint. (For all of these screenshots, the older version is on the left) deprecationwarning The first thing to notice is that Foursquare is really polite about their deprecation warnings. I wonder how standard their meta dictionary with an error message is. I suppose it works well if you have humans reading your API (as we are now) but I’m going to assume most clients do not care about the meta error being shown in this deprecated API call. I wonder if there’s an HTTP status code for “OK now but will be deprecated soon.” exploresuggestionsreasonsAnother thing to notice is that there is a new “reasons” dictionary for each venue suggestion that lists friends that have gone to that venue. It looks like this dictionary didn’t exist before the 6/9/12 API update but probably exists in the older version of the API for compatibility reasons. moretipanduserinfo The tips section also seems to have gotten less info about the user giving the tip. I’m not sure if this is for privacy reasons or something else. Another thing I noticed was that the older version included a tip that I created myself while the newer version removed it (probably because the authenticated user isn’t interested in their own tips when exploring). Finally there’s the terrible implementation of icon images for categories that Foursquare provides. The old format had a prefix, file format and an array of possible sizes for the icon. You’d have to piece the three together like the silver monkey statue to get an icon for your app. The newer version did away with the list of sizes (now you just need to choose a size from 32, 44, 64, and 88).

Conclusion

logo-runscope-wordmark-white

As you can see, Runscope makes it really easy to compare the differences between API versions. Hopefully all of your API version migrations will be smooth and come with months of warning, but in case they don’t, Runscope can be a pretty handy tool to get a handle on what’s changed. If you have any questions or comments on this or other tools to ease the pain of API version migration, please let me know by posting a comment!

Hacking Google’s Text To Speech “API”

When I was at my previous job, one task I had was localizing a large set of phrases to multiple languages, both in text and audio files. I did this by using the awesome Google Translate API.

The Google Translate website has features for translating text and playing audio of it in the translated language. There’s no official API for getting audio, though. Luckily, I’ve never let a lack of an official API stop me before.

I had read a few old blog posts about how Google’s undocumented TTS API could be used, albeit with a 100 character limit. Going over 100 characters would result in a truncated audio file. Some of the text I needed to output to audio was longer than that. It turns out that with a little bit of Chrome web inspector, I could replicate the functionality of the Google Translate site.

The first thing to check out is the url scheme of the audio files, which looks like this:

Breaking down the parameters, “ie” is the text’s encoding, “q” is the text to convert to audio, “tl” is the text language, “total” is the total number of chunks (more on that later), “idx” is which chunk we’re on, “textlen” is the length of the text in that chunk and “prev” is not really important.

The Google Translate site itself gets around its own character limit by breaking big blocks of text into “chunks”. It seems to try and break along punctuation, but for super long sentences it will also break in the middle of a sentence, which ends up sounding pretty weird. Using the Gettysburg Address as an example, Google makes a request for the chunk “civil war”.

Gettysburg Address

In order to download audio files for longer chunks of text, I wrote up a python script that broke the text down and made separate requests to Google. The script would write all of the files to one file, and somehow, it worked! Just to be safe, I also set my script up to use Google’s Flash player as the referer (sic) and set the user agent to a version of Firefox.

At the time, I didn’t want to release the code as it was being used for some uber top secret stuff. But since I’m not working on that project anymore, I refactored the original code into a command line Python script. Along the way I had to learn how to use Python’s argparse, which is a pretty neat way of parsing command line arguments.

The project is available on Github right now, so go grab it and try it out. If you’re curious what the output sounds like, here’s a recording of female Abraham Lincoln reciting the Gettysburg Address (yes, she mispronounces some words). One fun thing to try out is outputting clashing input and output languages. Here’s Female Japanese Abraham Lincoln reciting the same speech (she just seems to be spelling words, slacker).

If you enjoyed this hack, let me know and I could post some other ones I’ve been working on. And if you find a way to improve the code (probably not difficult at all) go ahead and submit a pull request on Github. And if you’re from Google, please don’t shut down my Gmail and Adsense accounts.

The Facebook API: Somewhat Disappointing

I had read a few months back or so about Facebook coming up with a developer API. I hadn’t really felt like trying anything with it until recently when I heard about the FQL (like SQL for Facebook) Query language being created.

Anyway, I tried to think of a few ideas for mini-apps using the API, and today I finally got around to messing with it. I hit a roadblock pretty quickly, though. Apparently Facebook’s API will let you find all of the current user’s friends, but it won’t let you see all of the friends of any other user. This seriously limits the capabilities of the API by a whole lot.

This sort of makes sense. If the API let you see more than a logged in user could, that user could theoretically get past privacy setting that are in place. However, the API just blocks all queries that try to find all friends of anyone other than yourself. When logged in, you can see this info, so I don’t see the harm in offering it in the API.

What this means is that there’s no way to really “traverse graphs” with the API. You only get info on the user’s immediate friends. Thus, any application using the API is really only rehashing or using data that is already available on the main site. That’s fine for simple applications that can sync your address book to Facebook or let you rate how hot your friends are.

But Facebook is a social networking site. The interesting applications require some kind of access to the graph data of the site. It’s really disappointing to see that the API limits this so much.

I really would’ve liked to see a bit more capability with the Facebook API. If Facebook really wants developers to come up with interesting and complex applications, I’d suggest opening it up a bit more.