6 April, 2018

AR IN Facebook

How the Social Network Implements AR

Yaroslav Kaplan
In 2015, Snapchat bought Looksery, a start-up from Odessa, for $ 150 million, which is a record for the Ukrainian market. This allowed the American company to expand the functionality of its app by adding filters with AR.

Six months later, Facebook absorbed the Belarusian company MSQRD in order to catch up and outrun Snapchat on the AR camera market.

In this article we will discuss the current functionalities of the Facebook Camera, its development in the short term as well as practical applications besides dog ears in selfies.


At a developers conference in April 2017, Facebook unveiled the Camera Effects platform, which allows for the creation of photo frames and face recognition for application of AR effects. Both functionalities practically copy Snapchat with its Filters and Lenses.

Later on, AR masks appeared in other products of the company such as Instagram and Messenger Kids (currently available only in the USA). However, unlike Snapchat which cashes in on the development of AR masks, Facebook offers this for free and actively promotes developers to create content on its own platform.

In December of last year, a new app came out (currently only for Mac) called AR Studio, in which you can create and publish AR masks. On the developers' page there are instructions and tutorials and there is even a group for developers to promote communication and exchange experiences.

In order to try on Facebook’s AR masks, open the app on your smartphone and launch the camera. In the bottom left corner, you will see a pictogram of a magic wand which launches various effects. The user can choose tens of possible top effects, as determined by a version of Facebook algorithms.

Unfortunately, searching for and adding effects into the library is not yet supported, therefore the only way you could try on other AR masks is by adding them manually. Enthusiasts are collecting links to AR effects in the Face Fuse community as well as on lenslist.co, which has a collection of masks for both Snapchat and Facebook

In addition to the most popular masks á la rabbit ears, you can stumble upon effects from major brands such as Discovery, Lego and Nike.


Facebook, of course, is not fully satisfied with what it has achieved thus far. At that exact same developers conference, they announced a 3D mapping function or deep exploration of surroundings. With the camera, you can create art objects and play games in AR.

Even though officially such as function is not yet available, select developers have already received test access. Below we provide a video projection on a horizontal surface of the flagship Boeing — Dreamliner. In order to try out this effect for yourself, click on this link and switch over to your smartphone’s rear camera.

One additional expected function of Facebook’s camera is marker support for AR experience activation. If Snapchat uses branded yellow QR codes, then Facebook, most likely, will use any image as a marker.

Recently, some information surfaced that the poster for the movie Ready Player One is the first such attempt. After hovering the Facebook camera, a 3D AR object appears.
In addition to this, we know that Facebook is not only working on face tracking, but also the entire body. The camera will learn to recognize poses and differentiate a person from the background surroundings. This will be the first significant difference from Snapchat.

By the way, it is very likely that an analogous solution is already underway at Snap Inc.

Got an idea of immersive experience? Share with us!


The success of Snapchat in the AR sphere is due in large part to its young audience, which enthusiastically uses masks and experiments with virtual objects in space just for fun. For a more mature audience, Facebook will need to provide experiences with a deeper sense.

There is no need to download additional apps to experience AR. Pretty much everyone has a Facebook account.
The eternal problem developers had, which was how to force the user to download the app, is no longer the bane of their existence. Nowadays, it is enough to incorporate an AR effect inside the app, which the target audience uses everyday anyway.

Facebook’s camera can become a great tool in totally different spheres:

Beauty industry
 — with AR, ladies can evaluate makeup before they buy it.
Sport — soccer fans no longer need to paint their faces. All they have to do is point the camera and take a selfie.
Art — objects of art which are not visible to the naked eye, but appear in AR via geo-tagging in certain places.
Gaming — the dinner table could transform into an actual battlefield with tens of personalities.
Marketing — loyalty program gamification elements in retail. Pokémon Go in a supermarket near you
Events — concert announcements and movie posters could come to life, thus demonstrating AR content. On the promo stands of Ready Player One, a QR code was noticed, which leads to the Museum of 80's Pop Culture.

AR is becoming more and more popular and in demand. In addition to Apple and Google, which actively develop their technologies ARKit and ARCore, Facebook has actively entered the "arms race." They have two feathers in their cap: 2 billion users and the support of multiple devices on various platforms.

At the beginning of May, in San Jose there will be a new conference for developers, F8. According to Andrew Bosworth, Director of Facebook’s VR and AR Direction, the company will announce "sensational news in the AR/VR sphere." Possibly, we will see wearable AR devices and a patent request placed last year certainly hints towards this.
We eagerly await.

Did you like this post?


Tags: Facebook, Snapchat, AR, Camera