T-Rex Revealed: The story from a different perspective
The Beginning
When we were tasked to create an “invisible” exhibition themed around recent scientific findings on dinosaurs and T-Rexes in particular, we were excited to work on an educational application that teaches kids about these fascinating creatures in a visual way. We were prepared to work on a large scale but fundamentally simple Augmented Reality application that uses target images to position the content and track the space to keep the content in place.
Images used for triggering AR content
We focused on the quality of animation, since we strongly believe that is what makes or breaks an AR application. We really didn’t want just another AR viewer application that uses AR just for the sake of technology. The goal was to really inform and educate kids and adults about dinosaurs, by putting them in front of the users, so they can experience the sheer size of a T-Rex in relation to the real location. We wanted to give the viewer the feeling that a huge dinosaur is standing or fighting right in front of them.
Challenges
We anticipated challenges with the project, and they didn’t disappoint: they came in every shape and form possible.
The Set
The AR application is built around a fantastic set, designed and executed by our client in Asia. The set creates a platform for the AR content, and wonderfully draws people in to try and experience the application. And while the design and the usefulness of the set is without question amazing, it introduced a few obstacles we had to overcome.
The main AR exhibition is played out on a “stage”, the dinosaurs appearing on a large area that’s surrounded by 4 panels on each side that work as trigger images. Users are able to use two tablets at each panel to see the animations on stage. From each side of the stage, users are able to see a different story, from different angles.
Because the content is played on the stage, way behind the panels with the triggering target images, we had to make sure that the content appears behind its panel and that the closer elements of the stage visually block the content in a believable way.
The set design with a large center cube surrounded by trigger images
The Content
It being an educational application, the dinosaurs appearing in it had to be accurate. Our amazing 3D team did a wonderful job following instructions from our dedicated paleontologist expert behind the project, and although we had to make some compromises to be able to run the content on a mobile device (particularly with the feather that covered T-Rex), we were able to get the look right. And we learned a lot about dinosaurs in the process.
Timing
The biggest challenge of all was that we had no chance of testing the application on the final set until the very last minute. Almost literally.
All in all, almost all of the challenges and the way we solved them could go into a separate article.
AR Tracking, Application
Initially we developed using two different tracking systems in parallel. The set has a huge cube frame structure in the middle to grab attention, and our initial plan was to use the cube as a tracking anchor. We actually built a wooden cube frame that we placed in our yard to see how we can track it accurately. The initial tests were very promising, but because of changing lighting conditions that we had absolutely no control over, we had to eventually ditch the idea and go with the more conventional and reliable way, and stick to Vuforia.
As I mentioned in the beginning, we wanted to give the viewer the feeling that a huge dinosaur is standing or fighting right in front of them. Vuforia allowed us to let the user “lose the marker”, move the camera off the target image in front of them, and follow the action. I’ve personally seen the animations hundreds of times, but I still got the chills when I could look up and see a 5 meter tall T-Rex walk by in front of me.
The “test” cube in our backyard in Budapest
The end result
It was a long road... but we made it and we are all very proud of what we have achieved. T-Rex Revealed is a dinosaur exhibition with a difference: there is no skin, there is no bone, there are no skeletons, it uses solely Augmented Reality developed by INDE. It drew 150.000 people on the opening day in Shanghai on Oct 1, and will continue to wow visitors as it travels Asia.... and this is what it looks like:
The main feature of the event is the custom MobileAR platform we’ve just developed, however, it is combined with Back to the Jurassic, our AR installation for large screens. BroadcastAR (the new version of which is also based on Unity by the way) is another great crowd-puller experience that allows a group of people to interact with true scale dinosaurs and have their snapshots with the amazing giants printed on site or share them via social media.
We consider Augmented Reality an incredibly valuable tool, a new way to educate people about history, about the natural world, about everything they ever dreamt of experiencing but never could. It is about recreating situations and environments that train, educate, inform and entertain by experiencing them first-hand.
We love the fact that the dino experience we have put so much effort into makes it possible for huge crowds to learn about these amazing creatures in such an entertaining way, therefore we decided to make the dinosaurs from this project available to education, entertainment and retail spaces not only in Asia, but anywhere in the world, helping host venues become really popular destinations.
Lesson Learned
One piece of advice: always have a backup plan when you rely on such an unknown factor as location. When dealing with AR, or any kind of computer vision, lighting, environment and the quality of camera in the mobile device all have a huge impact on the final experience. You test a target image for accuracy in different lighting conditions with a couple of Android and iOS devices, find them working perfectly, then you go to the location, and find that a different Android device with a different camera works a bit different.
Why? Because at the location, you have a bright light behind the target image effectively blinding the camera. One camera would even out the lighting differences between the background and the foreground, the other just darkens the whole picture, so if you have a darkish target image, it will be seen as a black smear on camera. Oh, the joys of developing for Android…
Final Thoughts
Speaking of Android and iOS: a huge advantage of using Unity for an AR project is that you write your code once, and save for a few exceptions, you can build the application for both platforms. We wanted to take full advantage of it, and keep our “non-AR” content inside Unity so we didn’t have to deal with maintaining content in native format, in three languages.
We have sections in the application that provide information about dinosaurs, and the exhibition in a text format. It rarely changes, but we wanted to give ourselves the flexibility of modifying the content easily and without potentially updating the whole application, so we store all the data in xml format, and load it dynamically at runtime. It has its disadvantages, but for our purposes, it worked wonderfully. And it provides an additional educational layer to the AR content.
T-Rex Revealed may not be the first AR application that shows dinosaurs (heck, it’s not even our first), but it is the first of our upcoming educational AR applications that utilizes Unity and Vuforia.