Executive Offers Peek at Facebook’s Plans for Augmented Reality Glasses

Company unveiled project this week but officially has provided very few details


File photo: A visitor tries out Lenovo augmented reality during the Mobile World Congress in Barcelona, Spain, February 26, 2018.
REUTERS/Yves Herman

When Facebook confirmed on Wednesday that it is developing augmented reality glasses that should be ready in a few years, the social media giant provided few details. However, earlier in the week Ofer Shacham, Facebook’s director of silicon (semiconductors) spoke at a Tel Aviv conference about Facebook’s ideas for the glasses.

AR glasses incorporate a screen on which various digital elements are projected and merge with the real world. Companies like Microsoft, Google and Magic Leap have already unveiled products and Apple is reportedly planning to do the same.

Shacham, a graduate of Tel Aviv and Stanford universities, worked at Google in image processing and machine learning for mobile devices, developing the hardware for the company’s Pixel smartphones. He joined Facebook in July 2018.

Speaking at a meeting of the Silicon Club, which brings together the CEOs of Israeli semiconductor companies, Shacham said that Mark Zuckerberg’s vision is to develop glasses that people can wear all the time. For that to happen, these all-day wearable specs have to be designed in a way that’s socially acceptable, i.e., look like ordinary glasses.

He gave some examples of how the Facebook product, reportedly being developed under the codename “Orion,”could be used, stressing that not all will necessarily come to fruition. But he offered a hint of Facebook’s plans.

The most interesting one – which right now isn’t available in existing AR offerings -- is what Shacham called a “super eye” for the user.

“Imagine you’re walking down a dark street. Wouldn’t it be nice if using computer vision and photography you could ‘turn on the light’ just for you, so the glasses give you a kind of superpower to see in the dark? And what if you could also increase the size of objects or make them look clearer and sharper?” he said in his lecture.

Another possible function would be to present a VR price tag over the faces of people you see in the street or in other public places.

“This is my favorite option: I met a lot of people today, and the prospect of my remembering their names is slim. But imagine that I am wearing the glasses, I shake someone’s hand and the glasses tell me, ‘Hey, that’s Shlomo. You met him a few months ago for coffee in Kfar Sava.’”

A capability like that, drawing from image processing and face identification, is likely to be controversial, raising questions about privacy far deeper than those already being debated.

Shacham sketched out some other functions applicable for vacation or work-related travel.

“I live in California and my family is in Israel. Wouldn’t be nice if I could go to the Grand Canyon and a virtual friend or family member could sit next to me and see what I’m seeing and share the experience with me? Or I’m driving on a highway [in a foreign country] and my glasses are translating the road signs?

“On the job, it would be great if, for example, I could sit on a plane and do my work and the only thing I see is a virtual workplace around me, including my screen and keyboard. Suddenly the flight is my workspace and I can take wherever I want,” he said. Shacham said AR glasses could provide the kind of added value for people he believed would bring people to put them on when they get up in the morning and wear them throughout the day, whether at work, shopping, or with friends and family. “The form we want to achieve is ordinary glasses -- something everyone would want to put on. People don’t want a huge helmet on their heads,” he said. Media reports say Facebook is working with Italian eyewear maker Luxottica and aiming for its glasses to reach the market between 2023 and 2025.