Big data is a critical cornerstone of most social media businesses and Pinterest is no exception. The underlying algorithms that make Pinterest successful and fun — the ones that suggest new pins you might like based on things you’ve liked before, for example — are an example of big data at its finest.
And now, Pinterest is taking big data to another level. The social network has just announced a new visual search feature: a search tool that will allow users to select just a portion of an image, and then look for other similar images within the site.
In collaboration with members of the Berkeley Vision and Learning Center, Pinterest uses deep machine learning to learn image features based on their richly annotated dataset of billions of Pins. Those features are then used to create a similarity score between any two images.
The result is that if you see a lamp you love in a pin of a living room, you can select the lamp, and search for other similar lamps — as well as where to buy them.
Here you can see how it works.
In a separate announcement, Facebook has added a new feature to its Messenger app that will look at your phone’s camera roll for any photos you may have snapped of your Facebook friends, and then prompt you to share them with those friends.
Facebook says it is solving a problem of the digital age: that you may have dozens of photos of friends on your phone that you never get around to sharing.
Facebook’s powerful facial recognition algorithm hopes to make that a problem of the past by recognizing your friends and prompting you to share the photos.
Users can opt out of facial recognition, and users must opt in to the new Photo Magic feature to get notifications about images they may want to share. But Facebook isn’t the only one putting the new algorithms to work. A recent update to the Photos app included with the Mac OS offers a smart album called “selfies” that — you guessed it — picks out all the photos it believes you’ve taken of yourself. In both cases, the technology represents yet another step forward in treating photos as quantifiable data.
Why is this a big deal?
Show a set of photos of lamps to any three-year-old child, and she can pick out the ones that are similar. (Like that old Sesame Street bit: “One of these things is not like the others…”)
But for a computer, that’s a relatively new accomplishment.
Only with the advent of machine learning algorithms has this kind of analysis been possible.
Algorithms are becoming increasingly intelligent and able to help us understand what or who (in Facebook’s case) is in a photo or video.
This is an accomplishment because photos and videos, which were once considered ‘completely unstructured’ data. In any given photo, for example, the computer doesn’t have a reference for who is in it, where it was taken, what time of day, etc.
Until now. Now, photos can be analyzed by ‘robot algorithms’ to give them structure — what’s in it, what color is it, where was it taken, who is in it, are the people pulling a happy or sad face, etc.
This opens up an entirely new realm of data to mine for insights. And social media is at the very forefront of applying the technology.
