Augmented Reality

Augmented Reality (AR) is a technology that overlays the real world with digital information and media, such as 3D models, animated stickers and videos, overlaying in the camera view of your smartphone, tablet, PC or connected glasses.

Augmented Reality turns the environment around you into a digital interface by placing virtual objects in the real world, in real-time. Examples you may have already been using include Instagram and Snapchat filters and stickers.

It is different to Virtual Reality because VR includes only Virtual environments, not the real world.

How it works A basic breakdown of how AR works in this current age includes;

• Location based AR; uses a GPS, digital compass, velocity meter, or accelerometer which is embedded in the device to provide data based on your location. A big part of markerless augmented reality technology is the wide availability of smartphones and location detection features they provide. An example is the game Pokemon Go which uses GPS and location based areas to show up augmented imagery (Pokemon!). Also can be called markerless AR.

• Marker based AR; markers can be paper-based or physical objects that exist in the real world. Scanning the marker will augment the image or object, or sometimes just another way to access digital content (like a QR code). Also called Recognition based AR.

• Projection based AR; works by projecting artificial light onto real world surfaces. Projection based AR allows for human interaction by sending light onto a real world surface and then sensing the human interaction (i.e. touch) of that projected light.

• Superimposition Based AR; partially or fully replaces the original view of an object with a newly augmented view of that same object, object recognition plays a vital role because the application cannot replace the original view with an augmented one if it cannot determine what the object is. An example of this in use is Ikea’s app and catalogue

Augmented Reality can be used with a variety of screens and connected devices:

Via mobile devices like smartphones and tablets, where the user can see images, animations, holograms and 3D models overlayed in the real world. There are hundreds of Augmented Reality apps are available on iPhone, iPad, and Android.

On head mounted displays, glasses, and lenses, Augmented Reality becomes a part of your entire field of view, making for more life-like Augmented Reality experiences. This is costly tech, such as the Magic Leap ($3K) and Microsoft Hololens, mainly only available for developers, however cheaper glasses like Nreal are starting to become viable options.

Hololens

Magic Leap

Nreal glasses

On PC and connected TV players, Augmented Reality works through a webcam and relayed through the screen. This is the not as common a practice and the least used out of current AR applications but growing quickly.

In the last few years Augmented Reality has been progressing at a fast pace to be used in a variety of ways. It is also becoming more hands on for individuals to create their own AR projects, using existing free or paid apps.

It has also become a lot easier for anyone wanting to create a mobile app to showcase their own content or projects however there are still a few barriers in that area (getting the app on iTunes and Android stores etc).

Basically it’s as simple as loading up an image or location so the devices camera can recognise what it is looking at and execute a set of commands based on that visual input (similar to QR Codes). Content can be as simple as a 2D image of a koala (assigned as a target) and when that ‘target’ is scanned using a mobile device, animations or further content pops up, such as the name for koala in the Indigenous language of the area and facts about the koalas (why they eat eucalyptus).

There are endless opportunities and ideas, particularly in a learning environment and library space. However, things are developing at a rapid pace in these areas which also means in a few years’ time some of this stuff will be redundant, so just something to keep in mind on how much time and energy you may want to spend on things.

With traditional books (example is colouring apps for kids etc) - https://www.youtube.com/watch?v=EIRjmbEMtX4

The AR app Quiver has been used by kuril dhagun at State Library to teach Indigenous language names for certain animals

Learn more about a place through pop up AR info/animations.

Example video for art using Artivive - https://www.youtube.com/watch?v=Dzblxg8Udtk

Example is Ikea Place; lets you virtually 'place' furnishings in your space. Products are in 3D and true to scale so you can make sure it's just the right size and design.

Pokemon Go is an example of this.

Image from Niantic

Where to publish augmented reality;

• To an existing mobile app via an augmented reality SDK
• To an existing AR app
• To a standalone white label AR app
• To a browser with Web AR

When to use an AR SDK

Augmented Reality SDKs let you integrate AR into your own apps.

• You already have your own mobile app
• You have an engaged user base on your app and they are ready to use the AR
• You want to integrate the entire experience with your own branding
• You have long-term usage intentions for AR

Using existing AR apps

You don’t need your own app to publish augmented reality – you can publish it to an existing AR app, some examples listed below. Just download the app and follow the instructions.

When to use a white label AR app

• You don’t have your own app
• You do have your own app, but the AR would feel misplaced within it
• You want to integrate the entire experience with your own branding

When to use Web AR

• You don’t want the customer to have to download an app
• You don’t have physical touchpoints to activate

3D modelling programs (create your own models)

Open Source; Blender - https://www.blender.org/ Open Space 3D - http://www.openspace3d.com/

3D model hosting & resources (use existing) Poly is a website created by Google for users to browse, distribute, and download 3D objects. It is intended to allow creators to easily share and access 3D objects. It features a free library containing thousands of 3D objects for use in virtual reality and augmented reality applications. We’ve created some models and uploaded them to use in this session (Australian native animals). https://poly.google.com/

Other free model hosting resources Sketchfab is a platform to publish, share, discover, buy and sell 3D, VR and AR content. It provides a viewer based on the WebGL and WebVR technology that allows users to display 3D models on the web, to be viewed on any mobile browser, desktop browser or Virtual Reality headset.

https://all3dp.com/1/free-3d-models-download-best-sites-3d-archive-3d/

Issues with Apps; There are issues with using apps developed by others and also developing your own, the main issue is keeping them up to date with IOS updates etc. and managing the backend. You might get comfortable with an AR app you are using and then an update breaks the whole thing and it’s no longer useful. So keep in mind when looking at using one of the many available apps, or developing your own.

https://sparkar.facebook.com/ar-studio/ Face filters and other overlays on Instagram, Facebook, Snapchat etc are very popular. For making Instagram and Facebook ones you can use Spark AR and it's fairly straightforward. There's tutorials and a great online community for support.

Snapchat - https://lensstudio.snapchat.com/

Adobe Aero - https://www.adobe.com/au/products/aero.html

Vuforia - https://developer.vuforia.com/ Cross between a studio/creator app and distribution

Quiver – https://quivervision.com/ In app purchases, used by Kuril Dhagun, adding their own language terms to the printed out materials. Colouring in AR with pre-existing models and print outs (targets).

Artivive - https://artivive.com/ Limited projects for free accounts (5) and more for artists, you can add images and then video/animation to play when art is scanned

Just a Line – https://justaline.withgoogle.com/ Free basic draw on image AR

Eyejack – https://eyejackapp.com/ Available on Mac, can upload 2d assets/animations

RoAR - https://theroar.io/

EchoAR - https://www.echoar.xyz/

CatchAR is a great resource as it has a fairly up-to-date list of apps and AR technology updates https://catchar.io/

For WebAR there are a few options, for example using Aframe and ARjs - https://aframe.io/blog/arjs/