Like a good data nerd, I downloaded the list of publicly available Facebook names, dumped into text files. I was trying to figure out a neat way to use them. It’s always weird when people have the same first and last names, so I wrote a quick regex to find people with names like “John John” and “Laura Laura.” I used /^([a-z]+) \1$/, but maybe there’s a better one. I think mine left out the middle names.

Apparently there are a lot of people with the same first and last name. 14,495 in my list, to be exact. But that’s out of a 100 million or so (I only did unique names). Here’s the list for anyone who might care. I’m trying to figure out a more useful application for this dataset. Any ideas?

Something that is usually on my mind, either in the forefront or the back of it, is data portability. I like “owning” the data that I create, whether it’s from a tweet or a Facebook status message, or even more mundane like the bit of knowledge via a Last.fm scrobble that I listened to Dexter Gordon’s Wave at 3:27am on August 16, 2007 (I’m thinking this is in GMT?). The point is that data that I create is my property. I would go as far as to say that the online interactions I have, like friends’ activity that I comment on and interact with, is also my property. For example, a friend’s status message that I reply to and that ends up being a long thread.

Some services make gathering my data easy. Twitter has a dead simple API and so does Last.fm. I’ve been toying around with gathering location data from Twitter, embedded in photos I take on my phone, and other sources. One website that still confuses me a bit with regards to their policies is Facebook.

About two years ago, there was a semi-large fuss made over Facebook joining a Data Portability organization. Apparently it was sort of a me-too move to copy MySpace. Since then, there’s been a Stream API created and you can actually grab stuff from your “activity stream” from Facebook.

The thing is, the rules are super vague and contradict each other. For example, the Statement of Rights and Responsibilities states that

You own all of the content and information you post on Facebook, and you can control how it is shared through your privacy and application settings.

This makes it seem like I have the right to collect that content and information that I create, since I “own” it and have control over it. Yet the Developer Policy states:

Storing and Using Data You Receive From Us

1. You must not store or cache any data you receive from us for more than 24 hours unless doing so is permitted by the offline exception, or that data is explicitly designated as Storable Data.
2. You must not give data you receive from us to any third party, including ad networks.
3. You must not use user data you receive from us or collect through running an ad, including information you derive from your targeting criteria, for any purpose off of Facebook, without user consent.
4. Unless authorized by us, your ads must not display user data – such as users’ names or profile photos – whether that data was obtained from us or otherwise.
5. You cannot convert user data you receive from us into Independent Data (e.g., by pre-filling user information with data obtained from the API and then asking the user to save the data).
6. Before making use of user data that may be protected by intellectual property rights (e.g., photos, videos), you must obtain permission from those who provided that data to us.
7. You must not give your secret key to another party, unless that party is an agent acting on your behalf as an operator of your application, but you must never give your secret key to an ad network. You are responsible for all activities that occur under your account identifiers.

This basically says that I have to delete any information gathered within 24 hours. Facebook is making the assumption here that users are not developers and vice versa. I’m not interested in gathering other users’ data; I just want my own. And yet here are two conflicting statements.

I’ve already used the Facebook Stream API in the past to collect my data. While the policy states this is not allowed, it’s basically unenforceable. What bothers me a bit is that it is against policy for me gather my own data using Facebook APIs. Twitter allows this, and even goes a step beyond by suggesting that developers cache data to improve performance. To their credit, Google has a “Data Liberation Front” whose purpose is to keep an eye on products and keep data import/export for users as a priority.

I see data portability as a big issue while considering the natural lifecycle of a social networking website. As I use Facebook less and less, I still want to have a connection with those who are on it, and I want to maintain a record of what happened. I hate to think that while I “own” this data, I have no right to access it, especially if I decide to leave the service.

I started writing this post before realizing that Facebook’s annual f8 conference is actually going on today! I guess I can look towards today’s news to see if anything has been announced re: data portability.

EDIT: Well that was fast! I guess they removed the 24 hour limit thing during the keynote today. What this means directly to data portability is still up in the air, though.

Yes, I’m breaking out the “considered harmful” cliche.

I think Facebook is jumping a proverbial shark. In addition to the “viral” quizzes that are really annoying (and occasionally interesting), the increase in random fan pages is really making the site hard to tolerate. Facebook wants to make fan pages like people. That’s fine, if you’re only allowing people to have fan pages. The fan pages seem to have been designed for entities who want to have a presence on the site. Like companies or celebrities, etc.

Lately there have been fan pages for random shit like hugs, rain, flip flops, music, etc. Who is supposed to be the owner of the rain fan page? Who decides what rain says? I think groups were a fine way of indicating that you supported something. But fan pages are supposed to be pseudo-official (at least when celebrities make them, they are required to show proof of identity). This mixture of official and random-shit fan pages just makes Facebook seem unprofessional.

I am all for developing tools and letting people use them for what they may. But maybe there should be some rules about superfluous junk like fan pages for “wacky waving inflatable arm flailing tubeman.”

OR! Maybe I’m just getting old. *sigh*

I’ve known about this for a while, but now that it’s Facebook Official, I can make a post on it. I won a contest held by Facebook! The contest involved presenting one’s programming environment and explaining how the tools and everything were used. The contest deadline was the day after Christmas, which was cool because I had some free time to make an entry.

You can see my entry above. Click on it to see the big version. Here’s the text that accompanied it:

I have a shiny new Macbook Pro running OS X 10.5 for development since it’s got those command line tools I know and love.

(I don’t use spaces normally but it’s a good way to show everything)

Quadrant #1: vim is better than Emacs, let’s just get that out of the way (I hope the Facebook judges agree)! I use vim for most of my coding unless I’m working with something slightly complicated. For local editing textedit works well. I might also go with DashCode or TextMate if I feel like getting fancy.

Quadrant #2: I use XCode because it’s required to do any coding on the iPhone. Just between us, XCode blows. It’s pretty retarded in the autocomplete and syntax highlighting departments.

Quadrant #3: Firefox and the Firebug plugin are the best thing to happen to design/CSS coding since… sliced bread. I really love Firebug. Oh, and I use Cyberduck for FTPing if I’m not feeling like using scp.

Quadrant #4: My wallpaper consists of dinosaurs with lasers. It’s the awesomest picture I’ve found so far. When I find an awesomer one, I might think about replacing it.

As per the explanation of why I won:

Vim + not-windows + xcode blows + dinosaurs with lasers = win

I mean, can you beat dinosaurs and lasers?

Yep, I pretty much won because of the awesome wallpaper I utilize. It really is pretty sweet. You should follow my example and use it as well. Maybe it’ll win you a contest someday!

For my trouble, I get:

• An iPod Touch 16GB
• A RipStik Classic
• A pair of Sennheiser HD-280 Pro headphones
• A limited puzzle T-shirt (I’ll post a picture when I get it)
• A job at Facebook (editor’s note: I wish!)

I also get the privilege of having lunch with some Facebook engineers when I’m in the area. This is probably the neatest prize of all since Facebook is way cool and I’ve never been to their headquarters before. I’ve been to the Googleplex, the Appleplex (I don’t think it’s really called that) and the Microsoft Campus in Redmond, so this is pretty much the last stop in the list of software places I need to see before I die.

Anyway, I’m way grateful to Facebook for liking my entry enough to make it the grand prize winner. I had fun making it. I’ve gotten a raincheck on having lunch with Facebookers, so maybe during Spring Break or over the Summer I’ll report back on how it went.

I was browsing around Facebook today when I thought of checking the network pages for my universities. Network pages are interesting because you can see stuff that’s popular in your particular network. While the Network pages seem to be rarely used, I still think they’re interesting to have around. For example, here’s the statistics page for Michigan as of right now:

I found it really difficult to find the network page today. In fact, I had to go to google and query for “facebook networks” to actually find the networks page. This is because Facebook is apparently discontinuing the network pages.

Now is a good time for me to ask, “WTF!?” Social networks are interesting because they have multiple variables. People know each other through networks, especially by regional and work networks. By removing network pages, Facebook is losing a pretty important source of info. I understand they’re not removing actual networks; just their pages. But still, why bother getting rid of something? They aren’t really being used, but I still like to check them now and then to get a quick overview of what’s happening.

Recently, Facebook seems to be closer and closer to jumping that proverbial shark. Not just because of this change, but because of changes to the developer platform as well. It’s gotten to the point where one cannot casually write applications for Facebook because the functions they use will surely become deprecated within months. Can you imagine that happening on a real OS? Combine the “alpha” hijinks of the platform with the fact that no meaningful income is being made on the apps, and you have a formula for slow, rotting death of applications!

Will the next Facebook please stand up?

If you recall, I wrote a blog post a few months ago about the viral vs. non-viral growth of Facebook applications that I had developed. One of those apps, Musical Instruments, lets you list which musical instruments you play. It’s kind of fun because some people play really whacked-out instruments (I play pianica and soprano trombone). I think playing instruments is typically a pretty social experience, which sort of led me to think about comparing the “sociability” of certain instruments to each other with the data gathered from this app.

Users input their instruments via an autocompleting text field. If an instrument already exists in the database (and at least 3 or so users have claimed it), it will autocomplete. In the above screenshot, I’ve typed “Trumpet” and you can see there’s many different types of trumpet to choose from. A user can also type an instrument that doesn’t yet exist in the database and it’ll be added automatically. This kind of free vocabulary is nice because it doesn’t require an administrator to continuously accept new instruments.

The data that the application has access to are:

• The user’s FBID (Their unique Facebook user ID in the form of a number)
• The instruments that the user claims to play
• The number of friends that the user has

We ended up getting 8603 rows of data (user/instrument pairs). After getting a bunch of free text instruments, we went to work classifying many of them into groups and subgroups. So Piccolo is in the group “Flute” and subgroup “Woodwind.”

We then generated a survey for people to rank 16 instruments in order of sociability. That is, people who play x instrument probably have more friends than y instrument.

The survey results showed that people thought Vocalists had the most friends and that Guitar was pretty popular too.

From the application data and survey results, we formed a hypothesis. We hypothesized that the instruments given high sociability rank would also have statistically higher mean numbers of friends. So people who played Guitar would have more friends than people who played Flute.

First, we did some basic analysis of the data using R, the free stats program that we were using for class assignments and labs.

This figure shows the histogram of frequency of number of friends. Basically, many people have 0-100 friends, less people have 101-200 friends, etc. This probably follows a power law curve, but we didn’t think it would be really important to find the alpha or anything for our purposes.

This is a graph of the mean number of friends, by instrument. This looks like a pretty standard normal distribution and it shows off the central limit theorem that Lada is always talking about in class.

This is just a boxplot of all of the classified instruments and their # of friends. There are some crazy outliers; people who have 1000 friends. From this boxplot, it’s hard to make out whether or not any of the means are actually statistically significant.

Finally, we ran pairwise t-tests on each set of instruments. We could see that there was a significant difference in the mean number of friends for certain instruments. For example, Guitar and Horn, Guitar and Oboe, and Guitar and Saxophone. Looking at the mean number of friends for these instruments, Saxophone players had on average 20 more friends than Guitar players. This is interesting because Saxophone was ranked 10.7 (not very sociable) and Guitar was ranked very sociable.

The scope of this project was pretty small, and given some more time, I think we could’ve come up with some more interesting conclusions. Stuff like “is flute really a girly instrument?” by looking at the average number of female flute players vs male flute players and “do guitar players get more chicks?” by looking at relationship status of guitar players vs. something like trumpet players (personal burn!).

I was glad my Facebook app actually provided some interesting data. I’ve always been sort of skeptical to the ability of Facebook apps to be profitable. I think the data that the apps provide is very valuable in the context of social network research. Anyway, I hope you found this post to be somewhat entertaining. I’ve also uploaded the project report and presentation slides in PDF if you want to check them out.

Many thanks to Jim and Sameer for sharing much of the work in this project. I ended up providing data, formatting it, and presenting the final presentation, so props to my teammates!

Facebook just recently released a fun new thingy called Lexicon. It shows you a graph of the occurrences of certain words or phrases over time on all Facebook walls. Since I do quite a lot of research using Facebook, I thought I’d take a look at it.

Facebook has some suggested phrase pairs that you can use such as “party tonight” and “hungover.” And as expected, the phrases are cyclical and you tend to see a spike in “hungover” the day after “party tonight.” But it’s worth noting that you can’t prove causation by just correlation.

What’s a little strange is that there doesn’t seem to be any growth going on in Facebook. Maybe they’ve normalized the data so you can spot trends, but you’d kind of assume that word counts would trend towards going up since Facebook is still supposedly growing. Or is Lexicon telling us something Facebook doesn’t want us to know about their growth?

Anyway, at least one question of the ages has been put to rest. According to Lexicon, shampoo really is better than conditioner. STOP LOOKING AT ME, SWAN!

Bonus Analysis:

It looks like the word “happy” is used more on holidays like New Year’s Day and Valentine’s Day. But the word “merry” is kind of exclusively used on Christmas. See that strangely unhappy day near the beginning of March? Since it’s a leap year, Feb 29th was included in this graph. And since less people statistically have birthdays on that leap day, the word “happy” was recorded less. Pretty neat stuff, huh? I’m sure there’s other fun stuff to dig out of Lexicon, if you were really inclined to.

One nice thing about constantly iterating your code is that you get new code out quickly to your users. The bad part is you probably don’t do enough testing and your users get code before it’s ready.

Facebook gives you a choice to mark notifications that you get as spam. This is for when your friends won’t stop sending you notifications that you’ve been bitten by werewolf them. Apparently, you can also mark any notification as spam. Even, say, Facebook’s own wall feature. Facebook’s. Wall. Feature.

You know what would be cool? If Facebook developers weren’t so zealous that they skip sanity checks in order to get their pushes out sooner. Unless Facebook wants users to be able to mark their walls as spam. Which is actually understandable, now that I think of it…

The algorithm is probably pretty simple. Facebook probably just grabs people who are not your friends but share a large number of friends with you. I imagine they aren’t doing any kind of interesting (mathwise) stuff like weighing friends with low degree higher and friends with high degree lower. That would mean that friends who are friend whores count less toward your “People you might know” score than friends who have very few friends.

And that’s why Facebook needs employees well versed in network analysis and graph exploration!

*HINT HINT*

Also, I’d really like to see a “DO NOT WANT” button for the douchebags who keep on showing up in my list of people I may know. Just because I may know them doesn’t mean I want to be “friends” with them, FB.

The growth of Notecentric isn’t what I had hoped it would be. Not too many people use it, probably due to network effects of Facebook promoting a competing app (note to Facebook: if you want to promote a level playing field, don’t play favorites!) and other general performance issues (the application is pretty barebones and the RFacebook library I used to write it is pretty damn slow. It times out a lot!).

Anyway, I got some neat network data from it, which made the whole thing worthwhile. You can check out the original paper I wrote last semester here.

I’m going to be presenting my analysis during the School of Information’s annual expoSItion. It’s like a science fair except without the exploding volcanoes. During my Winter break, I developed another app, mainly for fun. It’s called Musical Instruments. Basically it lets you list which instruments you play and see which instruments your friends play. I had somewhat higher hopes for this application as a data gathering tool, and sure enough, it seems to be doing some cool stuff on first analysis. For expoSItion I figured I’d grab some data from this new app and compare the two.

I ran some initial analysis on the Musical Instruments app. I won’t go over a lot of the original metrics I used (number of peers with app installed, percentage of peers with app installed, etc) and I’ll just skip to the pictures.

This is an initial view of the Notecentric network:

It’s pretty interesting because the application has two large strongly connected components. One includes me and the many others I influenced to add the application and the other is a bunch of people from the Philippines (I think) who decided to add and invite their friends to add the application. Pretty neat stuff. But the majority of the users lie outside of the two large components.

The singleton users make up about 60% of the total users, more or less, during the evolution of the application. This signals to me that Notecentric was never really that “viral” in nature. People didn’t think the application was cool enough to invite their friends and people who saw their friends add the app didn’t feel compelled to add it themselves. Fair enough; it’s a pretty nerdy app. I mean, who wants to take notes on Facebook!?

Here’s a picture of the Musical Instruments application network:

It’s pretty small, so click on the image for a bigger one. This one is pretty neat. Instead of just two large strongly connected components, there’s a whole bunch! One is obviously bigger than the others, though there are quite a few clusters of users. It’s a little unrealistic to assume that this network would form into just one strongly connected component since Facebook users are generally decentralized and musicians can come from anywhere. If I did more analysis, I’d imagine that these components correspond to physical regions. Sure, there’s that “join this group if you’ve played music with someone in it” group, but I’m sure that’s not even a strongly connected graph!

The growth of the Musical Instruments application is interesting. It follows the same sort of domination of large components (possibly due to me inviting all of my friends) but instead of letting smaller components take over, the large (size 5 or bigger) components eventually stabilize at somewhere like 55% or so. This indicates to me that the Musical Instruments application is actually better at viral growth. People seem to invite their musical friends and others want to add the app if they see it on a friend’s profile. This is also the impression I’ve gotten from the add statistics available from Facebook.

Overall, it’s fun to see how the differences in application usage contribute to the kind of growth that each application sees. Notecentric has been stuck at around 670 active installs for a few months now. Compare that to Musical Instruments, which has been growing by around 20 users a day, and has around 2,700 so far. The growth of Musical Instruments is by no means monumental, but it is seeing some success due to its sharable nature.

I’m planning on showing the images that I’ve posted and perhaps some fun statistics that show the differences in the growth of the two apps. I think I’ll also load up GUESS on my laptop so people can check out the two networks in a more interactive setting. There’s prizes for the expoSItion of $750, $500 and $200 for first, second and third place, respectively. There’s also a $500 and $250 prize from Microsoft for entries specifically in Social Computing (which my project should qualify for). So hopefully I’ll make a few Benjamins for my work!