iOS 12: All you need to know about ARKit 2

ARKit 2 is Apple’s third major release of its Augmented Reality reality platform enabling developers to deliver enhanced AR apps alongside iOS 12. We look at the features of ARKit to give you an idea of what’s coming in September.

So what does ARKit 2 bring to the party?

Persistent and multiuser experiences - AR experiences can now be shared across time and fixed to real-world locations, so you can create a piece of AR art that can be viewed later in the same place by someone else.

3D object recognition - ARKit 2 recognizes objects and how your device is oriented to them and can use that information to incorporate objects into AR experiences.

Face tracking - Face tracking now sees where you’re looking and can detect whether you’re sticking out your tongue or winking. OK, while fun for Animoji's for all of 30 seconds, does this mean that depth sensing is coming to the rear cameras of our iPhone's and iPad's in the future?

Scene reflection - Enabling virtual objects to reflect the real-world scene in your camera.

USDZ file format - A new format that helps 3D creators to create content and animations optimized for mobile devices.

AR Quick Look - View AR experiences in apps, including Files, Mail, Messages, and Safari.

Measure app - Use your iPhone or iPad camera to measure objects and surfaces both vertically and horizontally as well as automatically detect dimensions of rectangular objects.

First up, we'll start with the elements of ARKit that we've been able to experience today with those devices running iOS 12 beta here in at GSMArena tower.

Quick Look and the usdz file format

With the close links between Apple and Pixar it comes as no surprise that the 'usdz' file format is based on Pixar's open source USD (Universal Scene Description) file format facilitating easy distribution and sharing. When an embedded image is linked within a web page, a 3D box icon is shown in the top right corner informing users that this is an AR object. There are many use cases for this technology, but you can imagine shopping sites benefiting straight off the bat. Due to the nature of the 'usdz' files, they're extremely easy to share with others for viewing.

At the time of writing, the 'usdz' file format viewing is only supported by Apple's upcoming iOS 12 and macOS Mojave, with Adobe stating they'll support 'usdz' for content creation. As with any new standard, for it to have longevity, the format has to be supported for end users outside of the Apple-verse on other platforms otherwise it will wither and die.

AR Quick Look samples

If you're running the Developer or Public Beta of iOS 12 you can tap any of the 3D models below to view the object and recreate it in AR. If you're not running iOS 12 clicking on a model will download the usdz file. If you're not running the iOS 12 beta, and why should you be, we've added some videos following the samples.

The first example is a model that we created here at GSMArena and converted to the new usdz format using the converter tool that ships with XCode 10.

Our created model

Models courtesy of Apple

AR Video samples

The following two videos demonstrate the new Quick Look AR functionality within iOS 12 viewing usdz 3D objects.

In our second video below things get a little shakey. A result of the our logo model being larger - a problem we'll all experience with a larger AR playing field.

Measure app

Measure is a new app in iOS 12, utilizing AR to measure the size of real-world objects.

Measure demonstrates ARKit's ability to recognize objects in 3D and automatically provide the dimensions of the object, specifically rectangular objects such as pictures frames and posters. It shows diagonal measurements, surface area and allows you to take a photo and share it.

Rectangular objects automatically recognised • Select an object to see dimensions • Ruler test

Multiuser experiences supported by 3D object recognition and persistent objects

ARKit 2's multiusers' chops were shown off on stage during Apple's WWDC '18. Lego Director of Innovation Martin Sanders demo recognized the 3D Lego model which proceeded to bring the world to life in AR. The enjoyment doubled when he was joined by his colleague with them both viewing and interacting with the same model.

You can watch the demo below, don't worry, we've highlighted the appropriate clip, so you don't have to view the whole of the WWDC again!

With ARKit 2, when you place a 3D object into an AR world, the object will remain there. You and others will be able to return to it later and interact with it, reinforcing the multi-user and immersive nature of future AR games and experiences.

These shared experiences and persistent enhancements mark a huge step change for AR. Often an isolating and single user experience, we can't wait for developers to release their multi-user AR apps.

Wrapping up

Apple’s CEO Tim Cook is passionate about AR, preferring it over VR

I see AR as being profound. AR can amplify human performance instead of isolating humans. So I am a huge, huge believer in AR. We put a lot of energy on AR. We're moving very fast.

ARKit 2's features are indeed a step in the right direction but Apple, along with the likes of Google, and other major players need to create and agree on a set of standards to ensure the longevity and the universal acceptance of AR.