Monthly Archives: February 2011

Porting Mapskrieg to App Engine

Lately I’ve been playing around a lot with Google App Engine. I think it’s finally at a point where it makes sense to develop for it, and it can be fun and profitable to do so. For example, I wrote Instascriber on App Engine, and so far it has cost me a few pennies in CPU cycles (and like $2 for a domain name or something).

Playing around with memcaching and stuff with Instascriber has led me to become a little obsessed with efficiency (not that I wasn’t already). I took a look at Mapskrieg and found that it was performing pretty slowly. My Google Webmaster Tools thing was saying it took on average 6-8 seconds to load! I wanted to decrease that number, and I figured I could make it work on App Engine, so I started working on it last week.

I modified the parser I use to grab craigslist listings to also send them to Mapskrieg. Then I basically rewrote all of the logic that existed in PHP and ported it over to App Engine. This took a little while but it wasn’t too difficult since Mapskrieg is a pretty simple web app. I hardly changed any code in the Maps API implementation, though I’m thinking of moving to v3 (which is apparently supposed to be faster).

I just switched out the hosting from my MediaTemple PHP based host to the Google App Engine based one. So far the results look good. I think Mapskrieg is faster in terms of response time and the caching is definitely smarter than it was before (a memcache that only gets cleared when the content in it changes versus a time based cache expiration).

I’m going to watch closely to make sure nothing got borked in the transition, but so far so good. If I can keep Mapskrieg on App Engine, I might downgrade the MediaTemple server to save some money and maybe eventually move other services to App Engine as well.

One funny thing I noticed was the pricing plan for App Engine. I’m currently pruning listings from the database when they get too old or move past the limit of listings per listing type (because I’ll never show them). This is actually older behavior from when I had listing info in MySQL. Back then, the size of the database would affect the performance of selecting rows. Google’s datastore apparently does not get slower with respect to size. This is actually pretty awesome.

The weird thing is that adding and removing datastore objects costs quite a lot in CPU time. Additional CPU time past 6.5 hours is $.10 an hour. Storage, on the other hand, is .$01 per 2 gigabytes per day. From my usage, I am predicting that the CPU cycles that I would use to trim the datastore listings would actually cost more than just leaving them in the datastore and paying for storage. Is that insane or what? I still have to test some things out, but it kind of surprises me that storage would be that much cheaper than the CPU cycles to remove something.

Greplin Programming Challenge

I had some spare time on Monday and just read about Greplin getting some funding (and I guess they just went out of private beta today), so I did the Greplin Programming Challenge.

Unlike many programming challenges, the problems in this one were pretty easy to solve. I think a lot of people on Hacker News complained that you could basically brute force them into submission. For example, one question is finding a prime Fibonacci number. I’m sure there’s super smart ways to do it but I just iterated through Fibs and then ran a really slow primeness function on each one. It worked eventually!

I finished the challenge and sent an email to the Greplin guys, since the challenge instructed me to do so. After reading through some of their site, the challenge actually started to make more sense to me. From their jobs page:

You should know the fastest way to do something – which sometimes means the least CPU cycles and sometimes means the least programmer cycles.

Basically, you could figure out the most efficient way to do the challenges (from a computational perspective) at a cost to programming time. Or you could recognize that N was trivially large for all of the challenge cases and just code something really simple up and get the answer. I picked the latter. Greplin is probably looking for both types, so I think they did a good job designing the challenge.

In other news, I signed up for their service and tried it out. It seems like a really cool concept and I like it so far. I kind of wish there was a roll your own version though. I dunno if I can trust Greplin with all of my innermost secrets… It’s bad enough that Google has all of your email; Greplin can have that plus your tweets and Facebook stuff too!

Threadless Shirt – Tweet

I saw this new shirt on Threadless and was surprised it hadn’t already done! Birds have probably known about Twitter for much longer than we have (early bird adopters). I like the fact that the Twitter interface has been recreated in a bird speech bubble, but I have a feeling the shirt will be outdated pretty soon (like when Twitter decides to come out with New New Twitter).

Get Tweet before it’s outdated!

iAd Vs. AdMob – Clicks, Fill Rate, Impressions, eCPM and Revenue

About two weeks ago I updated one of my apps to use AdMob mobile ads when iAd fails to deliver (pretty damn often). Instead of swapping out iAd for AdMob, I use the iAd “didFailToReceiveAdWithError” delegate method to request an AdMob ad when iAd fails. This means that AdMob shouldn’t take any income away from iAd, but only supplement it. I thought I’d take a look at the stats and compare the performance of the two ad networks on my app.

(The following numbers I’m referring to are from the time period from 1/25 to 2/7)

For an idea of scale, my app, Anime Nano, gets about 600 iAd requests per day. My app analytics software (Flurry) is saying I’m getting about 600 sessions per day. The thing is that about 1/5 of those sessions are on the iPad (and I haven’t enabled iAd on the iPad yet), so I think that on average, the iPhone apps are making a little more than one iAd request per session.

Despite the fairly healthy rate of requests that I’m getting, iAds are only getting sent about 15% of the time. This percentage is what’s known as the “fill rate.” This is also what most developers complain about being too low. The clickthrough rate is actually pretty good at 0.81%.

When iAd fails to deliver (85% of the time), I fall back to AdMob. The fill rate for AdMob is a high 88%, though the past few days it’s been closer to 99%. The CTR, however, is a fairly low 0.25%, which is less than half of iAd. This is partly due to the fact that AdMob ads refresh more often, so the number of impressions is higher.

I’m going to describe my revenue in terms of percentages here since ad platforms typically don’t like it when publishers publish their income. Not sure if that’s changed lately, but whatever! In the time period from 1/25/2011 to 2/7/2011, I made 2.3x more with iAd than AdMob, even with the super low fill rate (and impressions) of iAd. This is pretty interesting since iAd only shows up 15% of the time. There were actually a few days where AdMob beat iAd, but there were a couple of days where iAd made a killing (relatively), probably from those super awesome interactive ads which I still haven’t experienced firsthand yet.

For now, I’m content with the fill rate of AdMob, though it sort of pains me to see such a low click rate and revenue per click. Something tells me that if AdMob worked harder at increasing ad quality (like iAd), they’d be able to charge more and their advertisers would probably see a better return on investment. Most of the advertisers don’t even bother to use an image ad, which I assume would increase the clickthrough rate.

I noticed something really strange when looking at ads on my app. There’s no way for advertisers to disable ads on devices that already have the app installed. I keep seeing Amazon Kindle ads on both my iPad and my iPhone that already have the application installed. As a developer, I know there’s a way to prevent those ads from showing by detecting whether the app is installed at runtime. I think this strange behavior happens on both iAd and AdMob. If anyone from either organization is reading this, contact me!

As far as the future of ads in mobile apps goes, I hope Apple will continue working on increasing the fill rate, and that AdMob tries to increase relevancy and ad quality. Right now it’s really frustrating to see my “inventory” not live up to its revenue potential.

Instapaper Friendly – WordPress Plugin

While I was testing Instascriber , I noticed that the way that Instapaper was grabbing the content of my blog was a little off. The Instapaper for Publishers page describes how you can give Instapaper a hint on what your actual content is. I looked for a WordPress plugin that does this but didn’t see anything, so I figured I’d do it myself.

Up to this point I had never made a WordPress plugin, so I had go figure that out first. It’s relatively simple and there’s a lot of good documentation out there. I just needed to hook into the “the_content” API hook and alter the content so it had a div surrounding it. The scary part is mostly that WordPress seems to give you a hella lot of control. So you could seriously eff things up with a simple plugin.

With this being my first plugin, I’m not completely sure it’s been written the right way (kinda like when you make an edit on Wikipedia for the first time). Nevertheless, I’m making the plugin available. So let me know if there’s something glaringly wrong with it. It’s fairly simple, so I’m not sure exactly what could go wrong.

I made a special page for the plugin to live at, so get it at Instascriber Friendly – WordPress Plugin. You can also download it through the internal WordPress plugin search (look for ‘instapaper friendly’).