International Defense Security & Technology Your trusted Source for News, Research and Analysis
Related Articles
Synaps Labs displayed a billboard in Moscow changed the ad on its display based on the brand of car passing by. The billboard was created by, which used high-speed cameras placed 180 meters in front of the billboard to take pictures of cars. Machine-learning software determined each car’s make and model. The purpose was to show ads for Jaguar’s expensive new SUV to drivers who already owned expensive cars. AI and in particular machine learning has become quite advanced to extract all kind of information from photographs whether a person in a photo is young or old, male or female and glean many other facts useful for ad targeting. The company has since developed its technology further and installed billboards all over Russia and the United States.
Others, such as the MIT Media Lab, have also pursued the concept of multi-view displays using different techniques in the past. More recently, a startup called MirraViz drummed up attention at the CES technology trade show this year with a system that uses multiple projectors to display different content to multiple people on the same screen. Engadget called it “one of the wildest” things at the show, while noting that it was limited by the number of projectors that could fit around the screen.
Now “parallel reality” a breakthrough display technology has been developed that allows many different people see completely different content on the same screen, simultaneously. When combined with location technology and sensors, this content can be targeted in real time from public displays to specific locations, people and objects, essentially following them in three-dimensional space as they move through the world.
The display technology is based on a “multi-view” pixel. Unlike traditional pixels, each of which emit one color of light in all directions, Misapplied Sciences says its pixel can send different colors of light in tens of thousands or even millions of directions.. They call it a “magic pixel.”
This allows people to observe information only relevant to them for example their flight information in Airports, stats for your favourite players in a stadium and traffic signals on the road. These are examples of the long-term potential for “parallel reality” display technology to personalize the world, as envisioned by Misapplied Sciences Inc., a Redmond, Wash.-based startup founded by a small team of Microsoft and Walt Disney Imagineering veterans.
It works with the naked eye, no headset or high-tech goggles required.
“Multiple people can be looking at the same pixel at the same time, and yet perceive a completely different color,” said Albert Ng, the company’s CEO and co-founder. “That’s each individual pixel. Then, we can create displays by having arrays of these multi-view pixels, and we can control the colors of light that each pixel sends. After coordinating all those light rays together, we can form images at different locations.”
There’s no such limitation with the Misapplied Sciences technology, given the way its multi-view pixel works. Misapplied Sciences has applied for 18 patents related to its technology, three of which have been granted, and the founders say they have more in the pipeline. Its approved patents are for a multi-view architectural lighting system, a computational pipeline and architecture for multi-view display, and multi-view traffic signage, displaying customized content to different vehicles.
“I couldn’t stop thinking about different ways it could be implemented and made useful — if it could be scalable enough and made affordable for different venues in different populations,” said Tremblay, who is also a professor at the University of Washington in the Department of Speech & Hearing Services.
Implications of ‘parallel reality’
That change could be both positive and negative. In a world already struggling with digital tracking and “alternative facts,” the ethical and privacy implications of “parallel reality” could be significant. How will we agree on a shared reality or common set of facts when we can look at the same screen and see something different?
Misapplied Sciences says it’s thinking about safeguards, including ways to extend digital privacy to the physical world of parallel reality. Advertising could be opt-in, for example, and the person being tracked through the airport could be an anonymous blob to the system, identified by flight number after an initial registration. In some situations, they point out, targeting content to an individual person or location could actually make it more private.
“In general, parallel reality is a new medium that connects content with people, and like other powerful media such as internet news or social networks, there are many different ways in which personal information can be provided and used so content can be relevant to the viewer,” said Ng, the company’s CEO. “The current conversations about privacy are taking place on many fronts for many different technologies. The same debates will apply to some applications of parallel reality, as well. And of course we recognize and we share these concerns.”
“Our goal with parallel reality is to provide people with content that’s beneficial, interesting, entertaining while being sensitive to their privacy,” he said. “Ultimately parallel reality will revolutionize how we think about things like accessibility, safety, traffic management, wayfinding, entertainment and many other things. We believe parallel reality will bring tremendous benefit, and will help a lot of people.”
In addition to the multi-view pixel technology, the company says it has developed a custom processor that allows the displays to run efficiently, along with software that allows content creators to easily target content to different points in space.
Applications and challenges
An early investor in the company is Ginger Alford, a teacher, museum director and expert in computer graphics, who is also a leader of the SIGGRAPH graphics and interactive technology conference. Alford called the concept of personalizing the environment in public settings “an astounding idea,” citing potential applications including the ability to tailor classroom lessons to individual students by language, learning styles and cultural touchstones.
“A key aspect is that, while it is individualized, it is not isolating. It is very social,” Alford said. Asked for her take on the biggest challenge Misapplied Sciences will face, Alford said the capabilities “are so far beyond how people currently interact with the world I think it might be hard for people to get their heads around it.”
Daniel Wigdor, a human computer interaction expert who worked with Dietz at Microsoft and Mitsubishi Electric Research Labs, said “People walk around now with their faces pointed at private little screens they carry in their pockets, necessary to get a private, personalized experience, but detaching them from the shared experience of the world,” Wigdor said. “Misapplied Sciences promises a sort of parallel reality, in which people can coexist in a shared world, but still receive private and personalized content.”
Don Dorsey, a legendary former Disney audio engineer and experience designer known for Disneyland’s Electrical Parade and Epcot’s IllumiNations, has also seen the Misapplied Sciences technology. He called it “mind-boggling, in the same way good magic tricks and grand illusions are.” In addition to delivering personalized information, Dorsey said, the technology could be used for an entirely new form of entertainment, giving different views and experiences to different people in an audience.
“This is a type of storytelling we have only begun to explore through multi-player games and virtual reality. Sharing this type of experience with others in the real world is the leap forward here,” Dorsey said.
It’s not without challenges, he said. “On a more practical level, each independent view has to be designed, scripted, created and implemented. This can mean a potentially massive increase in workload and cost on the part of the experience provider. Consideration must also be given to the possibility that the ‘trick’ could supersede the story experience.”
“The breakthroughs that we were creating in the past few years, most of them were to make it affordable and practical and manufacturable,” Ng said. “We are on the cusp of bringing this out as product.
