Facebook Face Extractor

A few days ago I was messing around with some face detection libraries when an idea hit me, what better place to look for faces than Facebook! After a few hours and a copious amounts of coffee, I started piling one idea after other until I ended with a rather interesting combination of technologies and data sources.

Before I explain any further, please step back for a second and appreciate the insanity of information stored along with pictures uploaded to Facebook. Not only are the images stored on Facebook’s servers, but they are also tagged with who ever is in the image, including their name and the xy coordinates of that person’s face on the image.

To make things more interesting, the Facebook API allows very easy access to this crazy database, including the images. For example, Photos.Get can be used to get a list of all the images that are tagged with a certain person, then for each image, you can use Photos.GetTags to get the X and Y coordinates to locate where their face is (FBQL would also work). With this information, you can crop and build a collection of smaller images that only include their face. A simple face detection library such as fdlib can be used to verify that the smaller cropped face images are indeed faces since some people tag incorrect locations.

Now you have a collection of smaller images with just the face. Other than the creepy factor, what other uses can this have? There are face recognition algorithms out there that can build a detailed facial characteristics model for a person provided several images of their face, which in turn can be used in automated face recognition systems to identify people. You see where I am going with this…?

Let me give an example of how scary or interesting an implementation of this can be (depending on who you are). Imagine if someone runs these steps for everyone at an University, they would end up a database of facial models for every person that had an image tagged (and they don’t even need to be a member of Facebook since people tag everyone anyways). Now for example, build an iPhone app which can take a picture of anyone in that University, and the system would compare the picture to the facial model database and return who that person is. Or a camera system that identify everyone who walks past it (Remember: Minority Report subway scene with targeted ads). Other uses can be to tag even more images automatically, even on other image databases such as Flickr or Picasa.

Implementations of this technology combination is limitless, yet scary and very disturbing, not to mention the privacy concerns. What’s even more interesting is how simple this whole thing is, my implementation was less than 200 lines of php/c/cgi including the facebook API stuff and did everything including verification with fdlib but not the facial recognition (and no I will not post the source for obvious reasons).

I am torn between considering if this is a good or a horrible use of technology: I can see some very interesting and cool uses, but at the same time it can be used in some very unfavorable ways. Just be careful of what information you put online, specially when they are large databases like Facebook and tagging pictures…