Virtual Reality (VR) is the use of computer technology to create a simulated environment that can be similar to or completely different from the real world. Unlike traditional user interfaces like viewing a screen in front of them, users are immersed and able to interact with 3D worlds. Applications of virtual reality can include entertainment (3D movies and video games) and educational purposes (i.e. medical or military training).
The focus of virtual reality is on simulating the vision. The user needs to put VR headset screen in front of his/her eyes. Therefore, eliminating any interaction with the real world. Virtual reality training is conducted using head mounted displays (HMD) with an inbuilt tracking system and data gloves to enable interaction within the virtual environment.
Convincing Virtual Reality applications require more than just graphics. Both hearing and vision are central to a person’s sense of space. In fact, human beings react more quickly to audio cues than to visual cues. In order to create truly immersive Virtual Reality experiences, accurate environmental sounds and spatial characteristics are a must. These lend a powerful sense of presence to a virtual world. By simulating as many senses as possible, such as vision, hearing, touch, even smell, the computer is transformed into a gatekeeper to this artificial world. The only limits to near-real VR experiences are the availability of content and cheap computing power.
Other, distinct types of VR style technology include augmented reality and mixed reality. AR augments the real-world scene whereas VR creates completely immersive virtual environments. AR is 25% virtual and 75% real while VR is 75% virtual and 25% real. VR can be used to simulate working in dangerous environments or with expensive, easily damaged tools and equipment, without any of the risks.
AR, on the other hand, can be used to relay essential information directly to the user about whatever happens to be in front of them – reducing the time spent by engineers, technicians, or maintenance staff referring to manuals and looking up information online while on the job. AR uses computer vision, mapping as well as depth tracking in order to show appropriate content to the user.
The potential uses for these technologies in healthcare are obvious, and Virtual reality has already been adopted in therapy, where it is used to treat patients with phobias and anxiety disorders. Combined with biosensors that monitor physiological reactions like heart rate and perspiration, therapists can get a better understanding of how patients react to stressful situations in a safe, virtual environment. VR is also used to help people with autism develop social and communication skills, as well as to diagnose patients with visual or cognitive impairments, by tracking their eye movement.
The adoption of AR in healthcare is forecast to grow even more quickly – with the value of the market increasing by 38% annually until 2025. AR can be used by surgeons – both in the theater and in training – to alert them to risks or hazards while they are working. One app which has been developed uses AR to guide users towards defibrillator devices, should they need one when they are out in public. Another one helps nurses to find patients’ veins and avoid accidentally sticking needles where they aren’t wanted.
Educational experiences in VR and AR will continue to become increasingly common throughout 2020. The immersive nature of VR means that pupils can engage with learning in fun new ways, and AR brings new flexibility to on-the-job training.
Virtual reality has been adopted by the military – this includes all three services (army, navy and air force) – where it is used for training purposes. This is particularly useful for training soldiers for combat situations or other dangerous settings where they have to learn how to react in an appropriate manner. The VR can be used for any type of training need the military has from hanging out of planes, sweeping for mines or practicing shooting dummies.
“The goal here is to put soldiers in a very intense environment, so they feel like they’re actually there,” said Joel Dillon, Vice President of Booz Allen Hamilton. With this advanced training the hope is soldiers are going to go into combat more prepared than ever. “They ultimately are going to be able to save lives because it makes our soldiers better on the battlefield, makes them able to make rapid decisions and feel like they have been in this environment numerous times in the past,” said Dillon.
What is apparent is that virtual environments are ideal set ups for military training in that they enable the participants, i.e. soldiers, to experience a particular situation within a controlled area. For example, a battlefield scenario in which they can interact with events but without any personal danger to themselves. Another use is combat visualisation in which soldiers and other related personnel are given virtual reality glasses to wear which create a 3D depth of illusion.
A virtual reality simulation enables them to do so but without the risk of death or a serious injury. They can re-enact a particular scenario, for example engagement with an enemy in an environment in which they experience this but without the real world risks. This has proven to be safer and less costly than traditional training methods.
Military training is prohibitively expensive especially airborne training so it is more cost-effective to use flight simulators than actual aircraft. Plus it is possible to introduce an element of danger into these scenarios but without causing actual physical harm to the trainees. Flight simulators are a popular theme in military VR training but there are others which include: medical training (battlefield), combat training, vehicle training and ‘boot camp’.
XR & AR technology
VR technology has evolved significantly over the past five years, with improvements on both the hardware and software side. However, issues such as latency, nausea, high prices, and underdeveloped ecosystems have been obstacles to widespread adoption. Today, VR spans three different hardware categories — PC, All-in-one, and mobile — each with their own strengths. Desktop-based virtual reality involves displaying a 3D virtual world on a regular desktop display without use of any specialized VR positional tracking equipment. Mobile VR
Lightweight and low-cost, these headsets use the screen of your smartphone to deliver a quick and affordable entry point to VR.
A head-mounted display (HMD) more fully immerses the user in a virtual world. A virtual reality headset typically includes two small high resolution OLED or LCD monitors which provide separate images for each eye for stereoscopic graphics rendering a 3D virtual world, a binaural audio system, positional and rotational real-time head tracking for six degrees of movement. Options include motion controls with haptic feedback for physically interacting within the virtual world in an intuitive way with little to no abstraction and an omnidirectional treadmill for more freedom of physical movement allowing the user to perform locomotive motion in any direction.
Modern virtual reality headset displays are based on technology developed for smartphones including: gyroscopes and motion sensors for tracking head, body, and hand positions; small HD screens for stereoscopic displays; and small, lightweight and fast computer processors.
Special input devices are required for interaction with the virtual world. These include the 3D mouse, the wired glove, motion controllers, and optical tracking sensors. Controllers typically use optical tracking systems (primarily infrared cameras) for location and navigation, so that the user can move freely without wiring. Some input devices provide the user with force feedback to the hands or other parts of the body, so that the human being can orientate himself in the three-dimensional world through haptics and sensor technology as a further sensory sensation and carry out realistic simulations. This allows for the viewer to have a sense of direction in the artificial landscape. Additional haptic feedback can be obtained from omnidirectional treadmills (with which walking in virtual space is controlled by real walking movements) and vibration gloves and suits.
Hardware: Nvidia and AMD lead the VR/AR chip market, but Qualcomm is a formidable competitor. The company’s Snapdragon XR1 chip, launched in October 2018, is the industry’s first VR/AR-specific processor, offering improved battery support, lower temperatures, and a better audio-video experience. Intel and ARM are designing custom chips for VR devices, while programmable chips from Xilinx and Intel and memory-biased chips from Micron and Samsung are also gaining traction. The use of XR1 in Google Glass Enterprise Edition 2.0 has raised expectations that Google’s Daydream VR, presently operating on the Snapdragon 835 smartphone processor, will also get the new chip.
3D audio: Virtual reality display technologies have improved over the years but audio has generally been overlooked. Digital signal processors (DSPs), which improve audio quality and also make devices more powerful at the edge by reducing dependency on network connections, are becoming standard requirements in the VR market. Established players in the 3D audio business, such as Dolby Labs, Panasonic, and 3D Sound Labs, are exploring new growth opportunities in the VR market. Goertek and AAC Technologies already have dedicated teams involved in the development of audio DSP algorithms for VR headsets.
Headsets: One of the biggest limiting factors with current XR technology is the need for encumbering headsets and display units. This is more of a problem with VR, where the powerful processing hardware needed to generate the graphics is usually contained within the headset. However, hardware devices have started to trend towards being “untethered” – For example, Facebook’s Oculus headset initially needed to be connected to a powerful PC, but this year became available as the self-contained Oculus Quest version.
While early VR headsets primarily required PC or smartphone tethering, new devices such as the Oculus Go, HTC Vive Focus, and Lenovo Mirage Solo have popularised untethered VR headsets. These devices are equipped with built-in processors, GPUs, sensors, batteries, memory chips, and displays, allowing them to operate independently. The devices offer a more powerful VR experience than smartphone-based headsets, but still lag in central processing unit (CPU) and GPU power and functionality. Facebook, Google, and HTC are increasingly bundling more resources into their devices to improve user experience. The growing demand for untethered headsets, along with the inherent limitations of human eyes, have necessitated the synchronisation of visuals with 3D audio for a truly immersive experience. Digital signal processors (DSPs), which improve audio quality and also make devices more powerful at the edge, are becoming standard requirements in the VR market.
AI: The growing pervasiveness of AI techniques – particularly machine learning, context-aware computing, and natural language processing (NLP) – across the expanding breadth of VR applications is enhancing the intelligence of virtual characters and delivering a rich immersive environment. For example, Google’s machine learning tools add 6DoF controller-tracking capabilities to any standalone headset with a pair of cameras, while Facebook’s DeepFocus framework uses AI to create focus effects in VR that mirror real life. LG is using AI to minimise motion sickness for VR users. In January 2019, Microsoft opened an AI and VR incubator in China, and Nvidia launched a graphic design tool to render 3D virtual worlds. Chinese tech leaders Baidu and Tencent are working on integrating AI and VR into mobile solutions and video games.
Cloud technologies promise scalability to VR vendors. As VR-generated data increases in volume, cloud services will store apps, data, and memory in virtual servers and stream them on-demand. This will drive innovation in VR and service updates in an uninterrupted cycle. Low latency, frictionless cloud-native apps will improve customer satisfaction with fresh VR updates. Amazon, Microsoft, Google, and IBM are expected to benefit from the integration of cloud and VR in the coming years, as VR companies will increasingly subscribe to their cloud services. In China, Alibaba, Huawei, Tencent, and Baidu will grow because of this association.
5G promises low latency, high density, and improved reliability, all of which will benefit the VR industry. Non-VR gaming today requires a minimum latency of 50 milliseconds, while VR requires less than 20 milliseconds. Thus, 5G’s expected latency of just one millisecond should deliver exceptional experiences on VR devices. 5G’s potential to support one million devices within a single square kilometre, without the risk of streaming attenuation, would help the VR market to flourish. 5G will also enable the integration of cloud solutions with VR technologies.
Super-fast mobile networks will further boost the potential of XR to strengthen its presence in entertainment and make further inroads into industry during 2020. The potential for data transfer speeds of up to 3 gigabits per second – by comparison, the average home broadband delivers well under 100 megabits per second – means 5G should be fast enough to stream VR and AR data from the cloud. Rather than needing to be wired up to powerful PCs, or encumbered by on-board hardware, viewing devices will upload tracking data to data centers where the heavy processing will be done. The rendered images can be delivered back to the user in real-time thanks to the speed of 5G and other advanced networks.
Conversational platforms: Despite being one of the hottest themes in consumer electronics, conversational platforms have been largely ignored in the VR space. Facebook introduced Oculus Voice in 2017, but the service was basic, with just four commands. The limited functionality meant that Oculus Voice suffered in comparisons with popular virtual assistants. In April 2019, Facebook announced plans for a new voice assistant on Oculus devices but did not give a release date for this feature. Google has yet to integrate Google Assistant into Daydream VR and Lenovo’s Mirage Solo, the first self-contained Daydream headset.
Streaming VR has been possible in a limited way for a few years now – Facebook lets you do it with your phone, but the experience is limited due to data transfer speeds and low on-device processing power. Combining it with the cloud and 5G technology means designers of VR and AR tools will be unencumbered by the need to deliver their experiences into a low-bandwidth, low-powered environment. The result will be cheaper headsets and viewing devices and more realistic VR simulations.
Telecom companies (including AT&T, Ericsson, Verizon, China Mobile, SK Telecom, Orange, and Vodafone) could benefit from the association of 5G and VR, with their pace of penetration, tariffs, and partnerships with VR vendors determining their success. Telecom players are already investing in VR/AR, either through partnerships or in-house initiatives. For example, AT&T partnered with HTC, Nvidia, Arvizio, and PlayGiga to develop its 5G network suitable for VR games.
VR apps are proving increasingly popular. In 2019, VRChat boasted more than 10,000 daily peak concurrent users, while at the time of writing Astro Bot Rescue Mission had over 58,000 watch hours on Steam. Beat Saber became the first VR game to record sales of over a million copies, while Google Earth and BigScreen VR became popular non-game VR apps. Facebook accounts for the most premium apps on VR at the time of writing, while HTC is the only VR company with an app store subscription service. Facebook offers more than 1,000 apps on Oculus Go, and HTC Viveport Infinity has more than 600 VR apps and games. In contrast, Google has approximately 250 Daydream apps.
AR and VR developers have started implementing machine learning and artificial intelligence (AI) into many of their apps. This can be highlighted with basic functions such as Instagram’s filter function, although it’s begun being implemented in a variety of larger capacities across different industries. This has resulted in what’s known as computer vision, which allows devices to see and understand everything within a camera’s range. This has been somewhat basic over the past few years, although it’s expected that these will become increasingly sophisticated over the coming years. This can be seen with Google’s machine learning-enabled microscope, which is becoming increasingly capable of identifying cancer cells in tissue samples. Coupled with VR and AR, this could make medical technologies increasingly more accurate and allow for more complex treatment and diagnostic tools.
New Device Simulates Feel Of Walls, Solid Objects In Virtual Reality
Today’s virtual reality systems can create immersive visual experiences, but seldom do they enable users to feel anything — particularly walls, appliances and furniture. A new device developed at Carnegie Mellon University, however, uses multiple strings attached to the hand and fingers to simulate the feel of obstacles and heavy objects. By locking the strings when the user’s hand is near a virtual wall, for instance, the device simulates the sense of touching the wall. Similarly, the string mechanism enables people to feel the contours of a virtual sculpture, sense resistance when they push on a piece of furniture or even give a high five to a virtual character.
The shoulder-mounted device takes advantage of spring-loaded strings to reduce weight, consume less battery power and keep costs low. “Elements such as walls, furniture and virtual characters are key to building immersive virtual worlds, and yet contemporary VR systems do little more than vibrate hand controllers,” said Chris Harrison, assistant professor in CMU’s Human-Computer Interaction Institute (HCII). User evaluation of the multistring device, as reported by co-authors Harrison, Fang, Robotics Institute engineer Matthew Dworman and HCII doctoral student Yang Zhang, found it was more realistic than other haptic techniques.
“I think the experience creates surprises, such as when you interact with a railing and can wrap your fingers around it,” Fang said. “It’s also fun to explore the feel of irregular objects, such as a statue.”Other researchers have used strings to create haptic feedback in virtual worlds, but typically they use motors to control the strings. Motors wouldn’t work for the CMU researchers, who envisioned a system both light enough to be worn by the user and affordable for consumers.“The downside to motors is they consume a lot of power,” Fang said. “They also are heavy.”
Instead of motors, the team used spring-loaded retractors, similar to those seen in key chains or ID badges. They added a ratchet mechanism that can be rapidly locked with an electrically controlled latch. The springs, not motors, keep the strings taut. Only a small amount of electrical power is needed to engage the latch, so the system is energy efficient and can be operated on battery power. The researchers experimented with a number of different strings and string placements, eventually concluding that attaching one string to each fingertip, one to the palm and one to the wrist provided the best experience. A Leap Motion sensor, which tracks hand and finger motions, is attached to the VR headset. When it senses that a user’s hand is in proximity to a virtual wall or other obstacle, the ratchets are engaged in a sequence suited to those virtual objects. The latches disengage when the person withdraws their hand.
The entire device weighs less than 10 ounces. The researchers estimate that a mass-produced version would cost less than $50. Fang said the system would be suitable for VR games and experiences that involve interacting with physical obstacles and objects, such a maze. It might also be used for visits to virtual museums. And, in a time when physically visiting retail stores is not always possible, “you might also use it to shop in a furniture store,” she added.
Virtual Reality market
Virtual Reality and Augmented Reality (VR/AR) will be the most disruptive technology for the next decade. Worldwide spending on VR/AR is forecast to accelerate out of the COVID-19 pandemic, growing from $12 billion in 2020 to $75 billion in 2024 to $1.5 trillion in 2030. The largest initial VR/AR applications will be in training & education, remote collaborations including field service, and marketing & sales.
The increasing demand for AR and VR technology is one of the major factors propelling the market growth. However, factors such as the high development costs associated with AR and VR apps might hamper growth.
Virtual reality or VR is advancing at a fast pace. Virtual reality is gradually paving its way into the automotive sector. Besides integration of voice assistance, vehicles nowadays come integrated with several advanced featurues to offer improved driver safety. Spurred by these factors, the global virtual reality market share is expected to increase considerably in the near future.
Based on the technology, the market saw maximum growth in the augmented reality segment in 2019. Factors such as new product (hardware and software) launches, growing adoption of AR in different application areas, and rising funding and investments in AR technology are fueling the growth of the segment. Market growth in this segment will be significant over the forecast period.
Companies such as Facebook, Google, Microsoft, HTC, Autodesk, Leap Motion, 3D, Sixense Enterprise, Dassault Systemes, and Eon reality is responsible for the surge in the virtual reality market revenue. Moreover, over 70% of these players have their dominance in North America. Recently Dassault Systemes offered help to the students of the University of Switzerland by making them aware of the latest technologies and industrial process. With the help of virtual reality, the company helped more than 700 engineering students and made them aware of ENOVIA for collaboration, CATIA for multi-disciplinary systems design and documentation, 3DEXCITE for high-end 3D visualization, and DELMIA for manufacturing simulation and robotics. Companies are increasingly investing in virtual reality technology to stay ahead of competition. Some of the recent industry developments are mentioned below:
As of the end of 2018, the three best selling Virtual Reality headsets were Sony’s PlayStation VR (PSVR), Facebook’s Oculus Rift and the HTC Vive. This was not a surprise, seeing as the same three HMDs had also been best sellers in 2017. 2019 sees the VR landscape broadening with Google, HP, Lenovo, and others looking to grab a piece of the still-burgeoning market.
The demand for virtual reality applications is increasing in healthcare, gaming & entertainment media, automotive, manufacturing, and other industries. Of these, gaming and entertainment media covers a 40.5% of the global virtual reality market share. The adoption of virtual reality technology is likely to increase in the education and healthcare industry, thus boosting the virtual reality market size. The technology can be used for providing training, monitoring patients, and practicing surgeries. As per research studies, virtual reality is expected to be the future of online learning. For instance, an education technology start-up called 3rdFlix raised around US$ 5 million from Exfinity Vnetures to create a live learning experience. With the rising awareness about virtual reality, the adoption of concept leaning among students is increasing.
North America covers 40% of the share in the global virtual reality market and is leading the market. US giants such as Facebook, Google, and Apple continue to focus on research and development (R&D) activities for software and hardware to further stimulate the virtual reality market share. The growth in this region is mainly attributable to the increased funding in start-ups, making the market dynamic.
The market in Europe is projected to hold a quarter share in the forecast years. Countries such as France, Germany, and France are the major contributors to the market. Apart from North America and Europe, the market is expected to rise in other regions such as Asia Pacific, Latin America, and the Middle East & Africa.
APAC dominated the market with a 37% share in 2019. Factors such as the presence of key vendors, the growing gaming industry in Asia, especially in Japan, China, and India, and rising investments in AR and VR technologies in different fields are driving the growth of the AR and VR market in APAC. China and Japan are the key markets for the augmented reality and virtual reality market in APAC.