101 - AR Face Filters

Developed by Michelle Brown

Acknowledgement

We acknowledge Aboriginal and Torres Strait Islander peoples and their continuing connection to land and as custodians of stories for millennia. We respectfully acknowledge the land on which we all meet today, and pay our respects to elders past, present and emerging.

Summary

In this workshop you will design and create your very own Instagram and Facebook filter! Use it as a creative and engaging promotional tool or just for fun for you and your friends.

Skills Introduced

Design thinking

Basic graphic editing skills

An understanding of Augmented Reality as a technology

Introduction to the computer program Spark AR

Tools

Computer

Spark AR program

Smart phone

Facebook/Instagram account

Possibly a smart phone connection cable

Introduction to AR

What is Augmented Reality?

Let's take a look at a basic rundown of the technology and use cases plus equipment that can be used.

Augmented Reality

Introduction to AR Face Filters

Social filters and lenses have become very popular in recent years, with Snapchat, Instagram and Facebook all including the option to augment yourself and environment through videos and photos.

It has become not only a terrific way for digital (and traditional) artists to not only compliment their practice, but also make a great income from creating filters. Because many of the other augmented reality options can be quite costly for hosting and creating apps or using third-party apps, and AR glasses are still in early days and expensive, using Spark AR (Facebook) or Lens Studio (Snapchat) which are free and have lots of community creators and tutorials.

Because we always want to highlight the good and the bad with any new technology, it is important to note that there has been some research into the problems of social media and beauty standards and body dysmorphia in young people. So we feel it's important to note that the workshops we run will not usually cover any of the body/face manipulation aspects.

Let's Get Started

Computer account set up

Let's make sure our computers are turned on and logged in.

Navigate to a web browser (Chrome/Friefox/Safari, doesn't matter) and we'll open this Wiki page so everyone can follow along with the steps in case you get lost, or you can refer back to this page at a later date as a refresher.

The program we are using today is Spark AR, let's make sure we can open that program on our computers, navigate to the Windows menu if you're on one of The Edge computers, and type Spark and it should come up for you to click on and open. If it asks you to login and connect Instagram/Facebook accounts then do that now, otherwise we will do that at a later stage.

If you are following along with this workshop from home, visit here to download - https://sparkar.facebook.com/ar-studio/download/

Once downloaded install the app and follow prompts, including logging into the Facebook and linked Instagram account that you would like the filter to be available through.

Designing our filter

Let's have a think about the theme we would like to use, do you want the face filter to cover your entire face? Or stickers on the cheeks? For this workshop session, we'll show you how to use a pineapple as cheek stickers along with the half skull image for the full face, but as you will see as we go through step by step, you can use the same method for any 2D face attachments.

2D image sourcing

During today's workshop we won't have much time to draw and design our own 2D image, so we'll be using images sourced from the State Library Collection or online that allows for Creative Commons use (https://creativecommons.org/

SLQ Collection items

Let's open the One Search catalogue to search the SLQ collection - http://onesearch.slq.qld.gov.au/primo-explore/search?vid=SLQ

https://www.flickr.com/photos/statelibraryqueensland

Free online resources

There are a few websites that you can visit to download free images for use, even though we aren't selling or making money from our face filters, we still need to be mindful of what images we use and copyright.

Here are a couple of popular sites we use regularly;

Pixabay - https://pixabay.com/vectors/

Provides options for - Free for commercial use, No attribution required

Archive.org - https://archive.org/

One of the best resources available, a non-profit library of millions of free books, movies, software, music, websites, and more.

Editing your image

If we grab a free PNG image (with a transparent background) then we shouldn't need to do any editing before importing it into the Spark AR program to use, but if you choose a more complex image that you want to just grab one part of, you will need to use software to edit. Take a look at some of our online Photoshop intro workshops.

Once you have downloaded the image you want to use, we will choose an image editing software to get the face placement right for use in Spark AR.

Photoshop

Need an account.

GIMP

Free and Open Source.

Example Assets

https://pixabay.com/vectors/pineapple-fruit-food-tropical-5985740/

Right click images to download;

workshops:prototypes:2022-23delivery-lasercutcovers:halfskullfacel.png

workshops:prototypes:2022-23delivery-lasercutcovers:halfskullface.png

Using the Face Mesh template

Before we get to building our filter in Spark AR, we need to download some additional resources to help us with the placement of images on our face.

Face Reference Assets

The Face Reference Assets are a collection of textures and 3D objects that we use when creating face effects in Spark AR Studio.

A folder of the assets are available for download - Download Face Reference Assets

If you download the folder and unzip it (extract compressed on Windows), you will see the folders Mesh and Textures. Mesh is mainly for 3D assets and filters, Textures are what we use for building 2D.

If you are using Photoshop then there is also a handy Photoshop file that has the different textures as layers and one I like to use to set up my texture, you can also import this into Gimp and this is the file we will use today - download that here

For further reference we can check out Spark AR's guide to using these here - https://sparkar.facebook.com/ar-studio/learn/articles/people-tracking/face-reference-assets#whats-included-in-the-face-reference-assets

Bring in your image

In Photoshop go to File>Place embedded and choose your pineapple/skull face/own design so it import in on its own layer.

You can see that the blue tiger face image is still visible, so let's turn that off by clicking on the eye icon on the layer. We should then only see the 'face feminine' layer as visible along with our pineapple.

Place your image

We then want to scale and move our pineapple into the cheek position (or elsewhere on the face you would like it to show) and we can duplicate and arrange till we like the look of our placement. You can switch back and forth between the feminie and masculine templates to make sure the placement works for both.

Export your image

Now that we know our face stickers are in a good position, we can turn off the template layers (click the eye icon) and export as a PNG file with the background transparency.

Making our filter

Now that we have our image we would like to use for our filter, we'll open Spark AR and learn how to use a Face Mesh and attach our image.

Spark AR

We already opened Spark AR and logged in during earlier in the session, but if you don't have the app open, find and open the program and login (or ask the facilitator to login if you don't have a Facebook account).

Once opened, you should see a side bar of options including;

Create New

Learn

Recent projects

Open

Create new

We are going to choose the Create New and the Blank Project option. Once you choose that, your screen should look similar to the below.

Set up your preview camera

There are 2 options for viewing your filter in progress with the preview window (called User in Spark AR). You can choose a webcam attached to your computer or the Real-Time simulation option. If you are using our DML computers, we don't have webcams so we're going to use the simulation option and just test our filters on our phones.

Note - It's always good to check your filter on the different face models they provide, as skin colour can alter the look of your image and we want to make sure it looks good for everyone.

Import our image

Once our camera preview is set up let's move to the next step of importing our image. In the assets window of Spark AR, click on the + sign at the bottom of the window and navigate to where you saved your PNG export, downloads folder or desktop.

When that is imported in should show up under Textures in the Assets window.

Adding a face tracker

The face tracker detects the position and orientation of the face of the person using your effect. It's the basis of any effect that responds to someone's face. To add a face tracker we can navigate to the Scene window in SparkAR and click on the + sign, search Face Tracker if it doesn't show up in the list and choose insert.

Once you have added that you should see it in your Scene window under the Camera>Focal Distance.

Adding a face mesh

The face mesh is a 3D model that responds to facial expressions detected by the face tracker. You'll apply materials to the face mesh to create the mask effect.

We need to add the face mesh as a 'child' of the Face Tracker - this means it will sit under the face tracker and any changes we make to it will effect the face mesh.

Just like how you added the face tracker, navigate to the plus sign and search face mesh and insert, this should show up as a checkerboard (black and white) in the camera view. It should like like this in the Scene window.

Creating a Material

While we have our faceMesh0 selected, we can see on the right hand side of the program the Properties window. In here there is an option to add Materials. Click on the + sign to create a new material.

This will create a new blank material that will show up as grey, like the below.

Renaming objects and assets as they are created will help you to stay organized as you build your project. Simply right-click on the object in the Assets window and select Rename, let's call ours faceMaterial.

Shader Type

When we add a material in SparkAR we can also choose what type of shader it should display as, there are a few options including;

Flat

Standard

Face paint

Blended

For this workshop we will choose Flat, but you can come back to this and try the different options once we've added our texture.

Add our texture

Now we have the base for our material settings, we can add our imported texture. While you still have the faceMaterial selected, navigate to the Texture dropdown in the right hand side properties window and you should see the name of your texture in the list, choose it and you will see your face filter come to life!

MAKE SURE YOU SAVE YOUR PROJECT!

Testing our filter

Our filter might look great on the example simualtions, but we want to see how it will look on our phones and using our faces. This is where there are a couple of options available.

Navigate to the bottom left side of Spark AR where you can see a few different icons, if you hover over them you will see the options, we want to click on the phone icon 'Test on device'.

If an 'experience' hasn't been chosen, we will need to select one, click on Select Experience and then + Add Experience, select Sharing Experience from the 3 options.

Now when we go back to the Test on Device option, we can see the Preview in App option which allows us to send our test to Instagram or Facebook if we have our account connected.

References

https://sparkar.facebook.com/ar-studio/

Downloads

Spark AR

Spark AR Player

GIMP

iTunes

Textures;

Pineapple

Right click images to download;

workshops:prototypes:2022-23delivery-lasercutcovers:halfskullfacel.png

workshops:prototypes:2022-23delivery-lasercutcovers:halfskullface.png