The Last Three Northern White Rhinos Brought to Life in Augmented Reality

As an Augmented Reality company, INDE creates products and experiences that inspire, entertain, inform and educate — next-generation platforms that change the way people interact with digital content. Since 2011 we have launched large-screen and mobile AR systems in over 40 countries, for the world’s leading entertainment and education brands.

INDE’s unique form of “digital resurrection” using our MobileAR platform has long been used in the conservation of animals through schools, museums, zoos and more. One of our latest projects is related to the The Last Three, the world’s biggest rhino sculpture created by Australian artists Gillie and Marc, expected to capture the attention of millions from across the globe.

To help raise awareness of the alarming issue of the Northern White Rhinos (and many other species) permanently disappearing from earth, INDE has decided to join The Last Three initiative and develop a standalone mobile Augmented Reality application for iOS devices, with a dedicated channel that will bring to life the last three Northern White Rhinos in photo-real 3D.

With the app launch fast approaching, here’s a sneak peek at the AR content and the techniques INDE has used in order to take MobileAR to an incredibly realistic level.

Challenges

While mobile 3D technology is improving constantly, MobileAR faces some very definite challenges. Besides creating realistic 3D content, real-world lighting conditions have to be matched and mimicked on the content in real time. 

Animation

Lifelike, realistic animation is quite hard to achieve, especially for quadrupeds like a rhino. The animation also needs to be "infinite", in a way that the 3D character’s motion is interactively random and somewhat unpredictable just like a real-world rhino's motion would be. INDE is using sophisticated randomized motion takes combined with super-realistic traditional animation techniques to achieve the desired realism.

Motion tracking

The app takes advantage of the exceptional environment mapping capabilities of Apple's ARKit in order to achieve the desired AR effect.Models stick to real-life surfaces as never before. By utilising the CPU as well as the GPU, this approach has distinct advantages compared to previous solutions. You can, for example, easily place 3D objects on rough and uneven surfaces... such as a 3D rhino on a real-world couch.

Texturing and shading

Besides the 4096*4096 pixel textures applied to the realistic 3D model, Unity3D's PBR shaders have been utilised in order to achieve realistic specularity. The resulting materials and shaders respond dynamically to the environment’s lighting conditions.

Lighting and rendering

Dynamic lighting is partially taken care of with the integration of ARKit. This enables the software to match the exposure and the color temperature of the real-time camera feed to the 3D content. To composite the 3D content on the camera background, it's simply not enough to utilise a usual 3-point lighting setup with soft shadows.

For a realistic 2D-3D matching, the VFX and film industry have been using Global Illumination techniques for quite some time now. Unfortunately, for such effects to be applied, companies typically use render farms... clearly not an option for real-time mobile AR.

INDE has opted for using Ambient Occlusion, a pseudo-GI approach well known for its quality results in film and video. The smooth contact shadows that are achievable this way can't be matched by other lighting methods. The approach is usually considered hard to achieve on mobile devices, it pushes the CPU and GPU very hard towards their limits. However, team at INDE have been able to find an efficient solution which delivers a steady 40 FPS on an iPhone 6s. 

The INDE MobileAR app, including The Last Three channel will be available in the App Store once the sculpture is unveiled in March 2018.