How Ghost Inspector Helped Me Furnish My House

Act 1

Ikea Stockholm. The average college student would scoff at such luxury, but as a first-time homebuyer, I wanted the best. When I saw the Ikea Stockholm TV Unit (in walnut veneer), I knew I would have it. The problem was that it never seemed to be in stock.

Behold the majesty.
Behold the majesty.

I must have waited for months, maybe even a year, to see if the unit was available in my Ikea store, but apparently there were production issues. The Ikea phone support was never useful and just told me to check again later. I was stuck waiting it out while my current Ikea (non-Stockholm) tv stand bowed under the weight of my 60″ television.

On a random visit in December, though, my interest was piqued. The “buy” button was actually there on the page! I tried faking an order and got a preliminary ship date! For whatever stupid reason, I thought this meant that the supply issues had been fixed. I delayed. I wanted to make sure the thing would fit in my living room. I missed out. Later that week, when I checked the site, the unit was gone.

Act 2

I noticed that the website did something weird when I tried buying the tv unit. The “buy” button was still working. The item was added to my cart. But when I tried to enter my address, I would get an error in my cart saying the unit wasn’t available. There must’ve been some issue with their inventory system.

Instead of checking manually every day to see if Ikea had any real stock, I decided to automate the process using a tool called Ghost Inspector.

Ghost Inspector is a service that lets you define actions for a headless (ghostly) web browser to run for you. It’s mostly intended for running tests on parts of your website (like verifying that a user can log in, get some data, not get an error, etc). If you end up changing a part of your website and it breaks another, Ghost Inspector can automate the process of discovering that regression.

As the title of my blog post implies, Ghost Inspector can also tell you when that piece of furniture you want goes back in stock.

To get started, I created a free account on ghostinspector.com. The free tier allows you to run 100 automated tests per month. This was more than enough for me to check Ikea once every day. To create a “test,” you can either download a Chrome extension that assists you in recording a test, or enter steps into the site itself using actions and CSS selectors (you can also use the extension to record a rough version of a test and edit it later, as I did).

This is what my first test run looked like after I recorded the process of adding the item to the cart, entering my address info and trying to check out:

As you can see, the test failed, which is bad, because I was testing to see if the item was out of stock (which I knew was true). The Ghost Inspector recorder picked up that I clicked on a button with a “pressed” state active. Unfortunately, the headless browser doesn’t hover over a button when clicking it, so it didn’t actually see this element and couldn’t click it. I fixed this by selecting the ‘a’ tag with the specific id that I knew would actually add the item to my cart (I also made the browser pause in case something needed to finish loading before hitting the button):

Here’s the screenshot of my successful (empty cart) test:

At the end of the test, I created an assertion that an element with class “#cartTableError” exists, which shouldn’t happen if your cart is full of Stockholm awesomeness and ready to be shipped. With the test running automatically every day, all that was left for me to do was wait until Ghost Inspector emailed me to notify me of the test “failure,” which meant the item was in stock.

Act 3

The fateful day. I got an email from Ghost Inspector that my test failed. I hurriedly typed “ikea.com” into my browser. The product page slowly revealed itself and I girded my loins for the moment I could click “buy.” But the buy button wasn’t there! The test failed because the website now correctly showed that the item was not for sale. Boo!

Test failed, but not the way I wanted
Test failed, but not the way I wanted

I had to write another test. But unfortunately, I couldn’t assert the non-existence of an object. Instead, I faked clicking the buy button (which apparently worked fine) and then tested whether the cart was empty when I clicked to it. The test looked like this:

Apparently this is a stupid test because it is actually still passing. But Ghost Inspector has a feature where it takes a screenshot after every passed test and compares it to the next test. In this case, the screenshot diff test failed and I was notified  a few days ago. And it turns out that the tv stand is actually in stock!

A comparison of screenshots.
A comparison of screenshots.

Epilogue

I’ve ordered the tv stand and it should be on its way by the end of the month. Many thanks to Ghost Inspector for creating a free tier for this service so that furniture aficionados like me can automate their obsessive compulsive shopping. But in future versions, please support:

  1. More specific selection of css selectors when using the Chrome recording tool
  2. Negative assertions that check when an element doesn’t exist. (I can probably do this by evaluating Javascript but tests take so long to run that it’s hard to debug)

If you’re interested in creating automated tests for your website (or in getting notifications for furniture availability), definitely give Ghost Inspector a shot.

Graphing My Cycling Progress on Strava and Automating With Zapier

Since I moved to a new home that’s closer to work, I’ve been riding my bike to commute at least a few times each week. My commute to work is very easy as it’s almost completely downhill. I pay for it on the way back, though. I’ve been trying to get healthier, so commuting with my bike has definitely become a priority. In addition to having fun, I’d also like to motivate myself to ride more often and push myself to get better at it (especially the uphill part).

I’ve been using Strava to track my rides, and it turns out that my ride home includes a user-generated segment that automatically tracks my performance when I’m going uphill on Liberty. The first time I rode, it took 10:44 for me to complete. My best time so far is 6:58. The cool part is that I’m able to see the progress I’ve made, and I really do feel accomplished when I beat one of my best times. Here’s the view I’m using to see what my times are:

Stava SegmentThis isn’t the easiest table to read, but you can see that the dates sort of correlate with my times, as the fastest one is in September and the ones in August are slower on average. While beating my fastest time is a good motivator, it’s not realistic to try and break my record every time I ride. I’d rather see consistent improvement as a motivator.

To visualize this, I decided to grab all of the times by viewing the full leaderboard (with just my results). I gathered the times and dates and thew them into a Google spreadsheet to visualize my trend. Here are the results so far.

Strava Trend

Success! It looks like I made some really good progress since I started in July, and my times have been inching downward as I get better at cycling uphill. The times also probably vary a bit because there are some stop signs and a traffic signal within the segment, which can slow me down.

The next problem I wanted to tackle was the data entry bit. Since it’s a pain to update the spreadsheet each time I ride,  I wanted to automate the addition of new efforts (the term that Strava uses to describe single instances of activities on a segment). This is where things get interesting.

Because I’m storing the efforts in a Google Doc, and because Strava has an API, I just need to connect the two together. Unfortunately, my go-to choice for this kind of thing, IFTTT, doesn’t have Strava as a channel. Luckily, I can do something similar with Zapier (btw that’s a referral link, so be sure to click it so I can get more tasks), which is sort of like IFTTT but costs money (albeit with a nice free tier) and has more integrations. You can also set up your own integration, which is what I had to do with Strava.

In order to get the list of my efforts on that particular segment, I had to create a few Zapier “triggers.” One to get my Strava user ID (for use in other API endpoints), another to grab my starred segments (so I could specify which segment I wanted to track), and finally a trigger to listen for any new efforts on my commute’s segment (limited to efforts created by me). I also had to post-process the last trigger so that I could get the date in a format that works in Google Spreadsheets. The result looks something like this:

Strava Zapier

Now, whenever I ride home on my bike while recording the segment with Strava, the effort will be automatically logged and graphed on my Google Spreadsheet! Apparently, I can embed the chart, so here it is!


Chart

While the Zapier integration had a somewhat steep learning curve, it’s nice to just set the “zap” and then forget about it. Any new integrations I might need to write will also go much quicker. As always, Runscope was a really useful tool for exploring the Strava API and getting real responses from the API to play around with. Finally, I learned the right way to spell athlete rather painfully (after misspelling athlete_id a billion times and wondering why my request wasn’t filtering correctly)!

I went from seeing some sketchy looking progress in my Strava results table to being able to visualize it on a graph, while also setting the graph up to update whenever I record a new effort! Automation for the win!

If you’re interested in the details of the integration, or if you want to try it out yourself, let me know and I can invite you as a tester. If there’s a lot of interest, I can also go over the creation of the integration, as I ran into a few gotchas while building it.

Impressions of Android Wear (LG G Watch) with my Nexus 5

I finally got the LG G watch that I won in a contest about a week ago. I replaced my Pebble smart watch with the Android Wear device along with using my spare Nexus 5 instead of my iPhone. It’s been about four days or so since I made the switch so I thought I’d write up my impressions.

The Watch

LG G

The LG G watch is one of the two currently available Android Wear devices for purchase. My coworker Jason, who also won a watch in the same contest, got the Samsung model. We both would have rather had the Motorola watch, but it isn’t available until later this summer so Google decided to send us these ones.

The LG watch is not particularly fashionable. It looks like a pretty generic black rectangle with rounded corners. It comes with a pretty neat charging base that you can magnetically attach the watch to. I would’ve preferred some kind of wireless charging, but this gets the job done, and the watch actually charges pretty quickly. The quick charge time is good because the watch only last about 24 hours anyway.

The Software

Android Wear Software

The software for the LG watch is basically an enhanced version of the notification dock in Android. This actually mirrors Google Glass a bit, but the execution is less awkward because you’re not wearing a watch on your face. Each notification that lives on your phone will also appear on the watch. If the notification is enhanced, you might be able to take action on it through the watch. For example, you can reply to a text message or Google Hangout by dictating a message. This is probably the killer app of the watch, though I feel like talking to your watch is still going to be considered antisocial behavior.

Aside from the notifications, you can also tell the watch to take notes, send e-mails, and set alarms. The watch still relies pretty heavily on your Android phone, though. Notes get saved to Google Keep (which I am not really sure is an app that will stick around), and GPS navigation simply opens Google maps on your phone. I’m guessing that in the future the watch will get smarter, but for now I think that’s a pretty decent feature set.

I’ve been toying around with the idea of writing an app that creates a persistent notification that just shows me how late the bus is going to be in the morning. Every morning I check the AATA website to see how it my bus is going to be. The information is contextual and time sensitive, and I only need it for that one time during the day, so it would make sense to make this something that shows up on my watch when I need it and goes away when I don’t. I think that Google Now could do this to some extent, but it’s not smart enough to know that I want to leave 5 minutes before my bus comes around and not when it’s “scheduled” to arrive.

Android

In order to fully test out the Android Wear watch, I took my SIM card out of my iPhone 5S and put it into my Nexus 5. So far the transition hasn’t been too difficult as I mostly rely on Google services anyway. There aren’t really any apps on the iPhone that I haven’t been able to use on my Android phone.

I may have said this before in previous blog post, but I really think that the quality of Google’s services is catching up to the polish of the iPhone. Consider how bad Apple’s online services are, and how good Google’s are. I can live without iCloud and Apple’s email service (does anyone use their .me email as a real email address?), but I am locked into Gmail, both personally and professionally.

I am guessing that Google could increase Android’s marketshare sharply by simply withdrawing support for all Google services on iOS. Meanwhile, Apple is relying more and more on third parties to fill out the expertise that it is lacking (I mean, really, who the fuck uses WebObjects?). I can’t really see a situation in the long term where Apple beats Google on services, either directly or by convincing third party developers to do it for them. As for whether I am actually switching to Android as my main device, I’d need to see what Apple has in store for their next generation. (Sadly, this LG G watch is not nearly awesome enough to make me switch on its own)

On top of the awesome built-in services of Android, the material design stuff coming from them from this past I/O is looking pretty sweet. If Android can give iOS a run for its money in terms of look and feel, and ease of creating really nice custom views and animations, then developers might just defect en masse. I guess the only thing Android needs to do now is get away from Java (if Apple can do this with Swift, then I’m sure Google can figure it out), because developing on it really sucks the joy out of being a developer.

Anyway, enough reverse-fanboying. I think it’s time for the…

Conclusion

While I think Android Wear does a lot of things right, and it has a lot of potential, the watch really isn’t all that revolutionary. To put it in terms of the HBO show SIlicon Valley, it’s not disrupting or making the world a better place through compression algorithms. It also isn’t a particularly stylish piece of metal and glass, so I wouldn’t wear it if it didn’t serve a purpose. Having said that, Google stuff is all about the integrated services, so I am feeling bullish on the fact that Google Now is always getting smarter, and developers should be able to hook into it, giving it a bit more life than it has right now. Would I buy it if I hadn’t won it? Maybe if I was an Android user already anyway, but I tend to buy too many gadgets as it is.

Android Wear Design Contest!

Last week I went to a GDG Ann Arbor Android meetup on Android Wear. The presentation was set up by Google and had a bunch of information on the upcoming Android Wear SDK. At the end, it was announced that there would be a few vouchers for Android Wear devices available. A design contest was announced and the deadline was set at Sunday.

Being naturally competitive and really into new technology, I made a design mockup for a home automation app that I thought would be useful to have on a wearable device (the most common form factor happens to be a watch). Here’s my submission that I originally posted to Google Plus:

Home Arrival

The app uses the user’s location as a way to provide feedback only when it’s necessary, and to stay out of the way when it isn’t. In this example, the app can detect when a user is arriving home and can ask if they want to switch the lights on.

Voice Controls

Instead of switching individual lights on and off, the user can create preset groups and activate them with voice commands. Users can set commonly used groups like “living room” and “kitchen.”

Sleepytime Reminder

Night owl users can set reminders to go to bed, and the app can turn off lights if the user decides to stay up late.

Motion Sensor

The app can also use sensors to detect when motion is occurring in the home and alert the user if no one is supposed to be at home. The user can decide whether the motion is a false alarm or if further action should be taken.

There is obviously much more that can be done with a home automation app, but these are just a few scenarios that would work well on an Android Wear device. I’m looking forward to experimenting on Android Wear when it becomes available!

I heard this week that my submission was a winning entry, so I’ll get an Android Wear device as soon as they go on sale some time this Summer. Thanks to Google and the GDG group here in Ann Arbor for setting up the event!

Inspecting and Debugging API Versions with Runscope

FL-iPhones This past month at FarmLogs, we implemented a versioning scheme for our internal API. API versioning is often a difficult transition to make, especially if you’re using a third party API and dealing with deprecations (Facebook, I’m looking at you!). It’s often a chore to figure out exactly what has changed between API versions unless you have very good documentation.

Coincidentally, Runscope also has a service called API Changelog that can send you notifications when a third party API changes. Luckily for me, any changes to our internal API were subject to review by our team of engineers, so if I had any questions or concerns, I could bring them up in person. Even so, when dealing with a changing API, the truth is in the response, not necessarily the documentation (or Hipchat or Hackpad). For a while, I was updating the iOS client to the new version by copying and pasting the two responses into a text editor and looking at the differences. About halfway through I remembered that this functionality is built into Runscope, so I started using that. I’ll show you how to compare diffs of two responses so you don’t have to waste half of your time like me!

Prerequisites

To follow along with this blog post, you’ll need just two things:

Some versioned API data

I’m not going to post the inner workings of the FarmLogs API (though it’s pretty easy to reverse engineer, like many other APIs) since that’s probably confidential knowledge or something. Instead, I’ll use the Foursquare API, which is one of my favorites to work with and (in my opinion) has some of the best documentation around. Browsing around their developer page, I found this interesting summary of API changes over the years.

There was a pretty huge change on 6/9/12 that included many breaking changes. Sounds like fun! I actually remember getting caught in this change and having to update my iOS auto-check in app to fit the new API. Let’s pick an endpoint and see what the data used to look like. We’ll be able to compare it to the modern version of the API.

For our example, we’ll hit the /venues/explore/ endpoint. This endpoint takes a lat/long value and returns a list of suggested venues to try. If you are authenticated, it can give you personalized info about why those venues were suggested (like if your friends went there and liked it). If you go to this page, you can get a url for that endpoint along with an oauth token added for you automatically.

Hitting the endpoint in Runscope

Next we’ll copy and paste the Foursquare API url into Runscope. You can create this request in a default bucket or make a new one specifically for this test. Just hit the “New Request” button and paste the url in. Hit the “Launch Request” button to have Runscope make the API request on your behalf and save all of the data for you to compare later. If all goes well, you should see the response appear below the “Launch Request” button. Next, we’ll edit the request so that the format matches the API before the big change on 6/9/12. Foursquare uses a pretty awesome convention for API versioning that uses dates for versions. To edit the response, simply hit the pencil icon on the upper right part of the response. You’ll be taken to the response editor. From there, change the “v” parameter to 20120608 (my 29th birthday!). request editor Hit “Launch Request” again and Runscope will fetch the response from Foursquare for a client that’s using a really old version of the API. After a short wait you should see another response appear. You can either hit the “Compare to Original” button or go to the “All Traffic” section to pick two requests to compare.

Comparing two versions of the same request

Finally we’re at the meat and potatoes of this post! Let’s explore the difference that two years can make to an API endpoint. (For all of these screenshots, the older version is on the left) deprecationwarning The first thing to notice is that Foursquare is really polite about their deprecation warnings. I wonder how standard their meta dictionary with an error message is. I suppose it works well if you have humans reading your API (as we are now) but I’m going to assume most clients do not care about the meta error being shown in this deprecated API call. I wonder if there’s an HTTP status code for “OK now but will be deprecated soon.” exploresuggestionsreasonsAnother thing to notice is that there is a new “reasons” dictionary for each venue suggestion that lists friends that have gone to that venue. It looks like this dictionary didn’t exist before the 6/9/12 API update but probably exists in the older version of the API for compatibility reasons. moretipanduserinfo The tips section also seems to have gotten less info about the user giving the tip. I’m not sure if this is for privacy reasons or something else. Another thing I noticed was that the older version included a tip that I created myself while the newer version removed it (probably because the authenticated user isn’t interested in their own tips when exploring). Finally there’s the terrible implementation of icon images for categories that Foursquare provides. The old format had a prefix, file format and an array of possible sizes for the icon. You’d have to piece the three together like the silver monkey statue to get an icon for your app. The newer version did away with the list of sizes (now you just need to choose a size from 32, 44, 64, and 88).

Conclusion

logo-runscope-wordmark-white

As you can see, Runscope makes it really easy to compare the differences between API versions. Hopefully all of your API version migrations will be smooth and come with months of warning, but in case they don’t, Runscope can be a pretty handy tool to get a handle on what’s changed. If you have any questions or comments on this or other tools to ease the pain of API version migration, please let me know by posting a comment!