In an unusual exercise, Meta (the parent company of Facebook, Instagram and WhatsApp) recently opened the doors to its R&D labs. In a video conference hosted by Mark Zuckerberg himself with several of the group’s engineers, the social networking giant presented a series of virtual reality headset prototypes – each more futuristic than the last.
The exercise comes as investors worry about fortunes sunk by Menlo Park group to develop metaverse technologies, a hyper-realistic digital universe on which Meta is betting for its future growth. The group’s Reality Labs division lost $10 billion last year.
And in the first quarter, the group blew an additional $3 billion, for barely $700 million in revenues… At a time when the group’s growth is slowing down sharply, Meta must reassure itself about the credibility of its bet.
“Virtual Turing Test”
It is extremely ambitious. The goal of the R&D teams is to create display systems on the same level as human vision. Internally, engineers refer to this as the “virtual Turing test” – a reference to the famous test devised in 1950 to assess a computer program’s ability to impersonate a human being. Similarly, Meta wants users of its headset to be unable to tell whether they are seeing a screen or the reality of the outside world.
“It’s much more complex than displaying a realistic image on a computer screen or television,” Mark Zuckerberg says. The human vision integrates many parameters that must be reproduced if we want to trap the eye and the brain of the user. Meta’s teams have therefore developed several prototypes – each one trying to solve one of the multiple challenges of the “virtual Turing test”.
Meta has developed multiple prototypes of VR headsets over the past decade.Meta
Resolution and variable focus
The first device that Mark Zuckerberg is taking out of his bag is codenamed “Butterscotch”. It aims to drastically increase the number of pixels of the display. Ideally, it would require 8K resolution, across the entire field of view of the VR headset – much wider than a TV screen.
Meta’s teams have managed to more than double the resolution of the Quest 2 (the headset currently on sale), to approach that of the human retina. “We can see the 20/20 line of an optometric chart [ceux utilisés par les ophtalmologues, NDLR] “, rejoices Mark Zuckerberg. But the engineers had to sacrifice half of the field of view, and the device remains heavy and bulky.
Butterscotch helmet approaches retinal resolution.Meta
The second prototype is called “Half Dome” and tackles the issue of depth of field. This one is fixed in the helmets currently marketed. But “in the real world, the lens of our eyes acts like a lens and constantly changes shape to focus on what we are looking at,” explains Mark Zuckerberg. After experimenting as early as 2017 with mechanical systems of variable-focus screens, Meta has moved on to a simpler and more robust electronic version. The headset is thus able to follow the gaze and adapt the depth of field depending on where it is worn – a bit like the autofocus of a camera.
The Half Dome headset series allows you to adjust the depth of field depending on where the user is looking.Meta
A lens hologram
A third model, Starburst, wants to offer brightness and contrast as good as natural light – which means multiplying them by a hundred compared to what current helmets do. The prototype is so heavy that it has handles.
The Starburst headset is the first model with HDR dynamic range equivalent to the best TV screens.Meta
To solve the major concerns of size and weight, Meta has a latest toy: Holocake, a helmet more like a simple ski mask, in which the light does not pass through a lens, but the hologram of a lens. The only problem with this holographic system is that it requires a reliable and especially cheap laser to power it.
Thanks to a holographic system, the Holocake helmet is much thinner and lighter than current models.Meta
Eventually, it will be necessary to integrate all these solutions into a single device. Meta has modeled this future headset, called “Mirror Lake”. “Nothing in physics prevents us from getting there,” boasts Mark Zuckerberg – even if he is careful not to give a date. Not to mention that in addition to the display, it will be necessary to solve a lot of other problems in terms of sensors, onboard computing power, battery …
The first devices taking advantage of the innovations of Meta’s labs will be, according to Mark Zuckerberg, probably too expensive for the general public and reserved for professionals. This will already be the case of a helmet called “Project Cambria”, expected later this year, which is aimed at companies and will be the first to include a system of eye and face tracking to allow avatars to make eye contact and communicate with facial expressions. A small step for the metaverse, a big step for Facebook engineers.
