Pinterest lets users search for products with their smartphone cameras – addition to benefit retailers and advertisers


Pinterest has launched a new feature that lets users search and find products using their smartphone cameras.

Pinterest adds automatic object detection feature

Automatic Object Detection, multiple objects - GIF


It’s a pretty simple process: users who spot something they make like to buy or want to check out further, launch their Pinterest app and tap the visual search button whilst pointing their camera at the item. This initiates a visual search to find the item or recommend similar products.

In addition, Pinterest has also incorporated similar item recommendations when users open up an image.

The tool gives the company a real edge and Pinterest hasn’t been shy to speak about it. In a blog post, the company says:

Visual search is one of the many fields transformed in recent years by the advances in deep learning. Convolutional neural networks represent images and videos as feature vectors which preserve both semantic concepts and visual information, and allows for fast retrieval when using optimized nearest neighbor techniques. We leveraged this idea, along with our richly annotated image dataset, last November when we released a visual search product that makes searching inside a Pin’s image as simple as dragging a cropper. For our initial launch, we extracted the fully-connected-6 layer of a fine tuned VGG model over a billion Pinterest images and indexed them into a distributed service, as described in our KDD paper.

Essentially, the latest addition to its visual engine combines the consumer’s discovery journey with mCommerce, allowing a better window of opportunity for brands to reach consumers at a critical moment in time – when they’re making a decision.

The visual side of things just adds to the overall experience and puts it into a more engaging context.

But the addition means more than just improving the user experience. Ultimately, it improves Pinterest’s advertising offers and allows marketers to reach a user whilst they are at a certain moment in the purchasing process. Pinterest says it has the technology to access these touch points and can help advertisers to utilise them.

The addition of a shopping cart to Pinterest will also ease the eCommerce process and is already in the works.

Another potential feature rolling out soon are branded pages, which can be set up to work like store displays for retailers. If consumers see a product they like, they can view similar ones on the brand’s page.

Given that 55% of people now say that Pinterest is a place for shopping, the company is really onto something here.