This startup’s AR headset can hold its own against the HoloLens

 

Avegant, a Silicon Valley startup that sells a pair of headphones equipped with a VR-like portable screen, is breaking into augmented reality. The company today announced that it’s developed a new type of headset technology powered by a so-called light field display.

The research prototype, which Avegant eventually plans on turning into a consumer product, is based on the company’s previous work with its Glyph projector. That device was a visor of sorts that floats a virtual movie screen in front of your eyes, and developing it gave Avegant insight into how to build an AR headset of its own.

Like Microsoft’s HoloLens and the supposed prototype from secretive AR startup Magic Leap, Avegant’s new headset creates virtual images that blend in and interact with the real-world environment. In a demo shown exclusively to The Verge last month, the company’s wired prototype proved to be superior in key ways to the developer version of the HoloLens. Avegant attributes this not to the power of its tethered PC, but to the device’s light field display — a technology Magic Leap also claims to have developed, yet has never been shown off to the public.

The demo I experienced featured a tour of a virtual Solar System, an immersion within an ocean environment, and a conversation with a virtual life-sized human being standing in the same room. To be fair, Avegant was using a tethered and bulky headset that wasn’t all that comfortable, while the HoloLens developer version is a refined wireless device. Yet with that said, Avegant’s prototype managed to expand the field of view, so you’re looking through a window more the size of a Moleskine notebook instead of a pack of playing cards. The images it produced also felt sharper, richer, and more realistic.

In the Solar System demo, I was able to observe a satellite orbiting an Earth no larger than a bocce ball and identify the Big Red Spot on Jupiter. Avegant constructed its demo to show off how these objects could exist at different focal lengths in a fixed environment — in this case a converted conference room at the company’s Belmont, California office. So I was able to stand behind the Sun and squint until the star went out of focus in one corner of my vision and a virtual Saturn and its rings became crystal clear in the distance.

 

In the ocean floor demo, I stood still while a sea turtle coasted around the room. In another, I was able to crouch down and peer into a standard-sized fish tank, with small and sharp aquatic creatures I could reach out and stick my hand through. Avegant says it took all of these images and graphics from the public asset library of game engine Unity. The demo also used off-the-shelf cameras to let the headset do positional tracking and understand where objects like tables and chairs were in the room. But Avegant says it ultimately wants to build a headset similar to Microsoft’s that can do inside-out tracking with cameras and sensors built into the device.

Because Avegant’s prototype didn’t have the same robust tracking system as the HoloLens, I wasn’t able to do any interacting with my environment. This is a big promise of AR — that we’ll be able to not only blend virtual objects with the real world, but also manipulate a mixed version of reality. With the HoloLens’ Minecraft demo, for instance, you can break through the surface of your physical coffee table with the tap of a finger, revealing a virtual sea of lava underneath. Avegant’s product, while superior in image quality and focus at a distance, doesn’t come close to producing these levels of tracking and interaction.

 

But Avegant seems confident that those advances will come in time. What it’s more excited about is its light field display. Light field technology, with regards to AR headsets, works by replicating how the human eye perceives light and then structuring virtual images that can be placed at different, fixed distances in an environment. The human eye can also look at them in the same way as physical objects.

This means you can close one eye and still perceive the full image and even change which object in a scene you’re focusing on by squinting at a far-off object. Because these objects are not made of traditional pixels, and instead constructed out of light and essentially blasted onto your retinas, they can behave more organically than 2D VR images being warped to fit a 3D screen, as is the case with the Oculus Rift.

This is similar to the tech used in experimental Lytro cameras like the Illum, which allow photographs to be refocused after they’ve been shot. The Illum, and other light field devices, understands both the intensity and the direction of the light rays in a scene, which allows them to then be manipulated after the fact. In Avegant’s case, the company is using the science behind light fields not to capture the way light hits an object, but to create its very own artificial objects out of light.

Avegant says it developed this tech while it was working on perfecting the Glyph. The $500 headset-projector combo was Avegant’s first commercial endeavor, produced following a successful 2014 Kickstarter campaign. The end product proved to be clunky, somewhat uncomfortable, and very strange to use in public — the projector technology lived inside two eye-shaped lenses inside the device’s headband, so you looked like you were wearing a Star Trek VISOR when you flipped it down.

 

Avegant says its work on miniaturizing what was essentially the basis of DLP film projection technology gave it insights into how best to tackle the hurdles of AR. By investing in retinal imaging and mirror-based projection, the company discovered a more ideal way to work with human sight.

“We’ve had transparent displays for a long time,” Avegant co-founder Edward Tang told me. Yet these displays had either poor image quality — often by trading quality improvements for focus ones — or were prohibitively expensive to make and sell. These types of devices are more commonly found in military applications, like the $400,000 F-35 fighter pilot helmet. “We’ve come up with a way to generate a light field that allows us to create multiple focal lengths. And it’s high quality and affordable [to produce],” Tang added.

Avegant would not tell me exactly how its prototype differs from the HoloLens, though Tang seemed to suggest that Microsoft relies more on the standard stereoscopic 3D tech favored by virtual reality companies. (Microsoft has never fully disclosed how it’s “light engine” generates virtual images, so it’s unclear if the company uses a light field tech of its own.) Magic Leap CEO Rony Abovitz has also in the past criticized how he thinks the HoloLens produces its AR images, claiming his company’s technology is superior and more fitting for how the human eye perceives light. Yet we don’t know for sure, as Magic Leap has yet to show off a device in a public demonstration.

Fully immersive AR is still a ways away clearly. It’s also unfair to say that what Avegant has created here is further along than the HoloLens, which has achieved a similar result with a wireless headset you can actually purchase. But what Avegant and its light field tech prove is that the market for mixed reality is no longer just for the world’s biggest corporations or its most well-funded (and secretive) startups.

Avegant has neither the wealth of expertise and massive head count of Microsoft Research, nor the more than $1.3 billion in funding of Magic Leap. Yet it has nonetheless produced a remarkably capable AR prototype all the same. So while the company and its light field tech are certainly worth watching as they develop, Avegant likely won’t be the last startup to start catching up and even surpassing the biggest names in AR.