Revolutionizing Military Training: The US Army’s Synthetic Training Environment (STE)

In the ever-evolving landscape of modern warfare, the ability to adapt, strategize, and execute missions effectively is paramount. To meet the challenges of the 21st century battlefield, the US Army is embracing cutting-edge technology through its Synthetic Training Environment (STE) initiative. Designed to revolutionize military training, STE provides soldiers with realistic, immersive simulations to conduct multi-echelon, multidomain battles, and mission command training like never before.

Meeting the Challenges of Modern Warfare:

The modern battlefield is characterized by its complexity, with threats emanating from various state and non-state actors across different domains. To address these challenges, the US Army recognizes the need for a comprehensive training portfolio that spans from individual soldiers to collective units. The STE serves as the cornerstone of this training paradigm, providing a versatile platform for honing skills and mastering mission-critical tasks.

In today’s rapidly evolving global landscape, military readiness is more crucial than ever. With the rise of complex threats and a dynamic operational environment, the US Department of Defense (DoD) faces the challenge of preparing its forces for a diverse array of missions. Enter the Synthetic Training Environment (STE) – a groundbreaking initiative designed to revolutionize military training and enhance operational readiness.

Embracing Innovation:

As technology continues to advance at a rapid pace, traditional training methods alone are no longer sufficient to prepare soldiers for the complexities of modern warfare. Recognizing this need for innovation, the US Army has embarked on a transformative journey with STE, leveraging the power of simulation and virtual reality to create dynamic training environments that mirror real-world scenarios.

Traditional training methods, such as the Multiple Integrated Laser Engagement System (MILES), have long been used to support force-on-force training. However, these systems have limitations, particularly in simulating realistic combat scenarios.

MILES cannot replicate the ballistic trajectory of munitions, simulate a munition’s effect on impact or engage targets using indirect fire, such as artillery or mortars. As a result, only half of the small arms and munitions assigned to a light infantry platoon can be represented accurately in live force-on-force training. The same is true for 40 percent of brigade combat team weapons effects. “We’ve been dealing with deployable systems for some time, but with improved computational power and new COTS technologies, we can provide high fidelity software in a much smaller footprint and reduced cost,” says Lenny Genna, president of the military training sector at L-3 Harris Technologies in Arlington, Texas. “The technology provides the capability to do a lot, but in some cases, you want tactile feel that isn’t fully there yet.”

The STE aims to overcome these shortcomings by leveraging advanced simulation technology to create immersive training environments that accurately replicate the complexities of modern warfare. By integrating live, virtual, constructive, and gaming elements, the STE offers a holistic approach to military training.

Realistic Simulations:

Imagine a training environment that seamlessly blends live, virtual, constructive, and gaming simulations. That’s the power of the STE. It creates a single, interconnected system where soldiers at all levels – from squad to brigade – can train in the most appropriate domain for the specific scenario.

At the heart of STE lies its ability to create highly realistic simulations of diverse operational environments, from urban settings to rugged terrain. Through advanced graphics, artificial intelligence, and scenario-based training exercises, soldiers can immerse themselves in lifelike scenarios that closely resemble the challenges they may face on the battlefield. This level of realism enables soldiers to hone their decision-making skills, practice mission command, and execute tactics with precision.

“As technology progresses, we are noticing an increase in realism and interaction formats that allow users to experience training almost identically to how they would experience it in real life, with the advantage of being able to stop, pause and reset the training experience,” Reddick says, adding that the possibility to simulate and reset on a constant basis, for a nearly unlimited user base and low cost, is where much of the appetite for VR and AR comes from across both the private and public sectors. “The ability to interact with a training workflow, from start to finish, with the ability to reset that workflow instantly for the next user, or have a group of people play out the experience at the same time and track all their results, is where the value of computer training comes in”, he adds. “We can simulate any military experience and, with our analytics and unique eye tracking technology, even determine how it is effecting the users mentally and physically.”

Key Components of the STE:

At its core, the STE represents a paradigm shift in military training, offering a single, interconnected system that supports training across all levels of command. Whether it’s individual soldiers or entire Army Service Component Commands (ASCC), the STE provides a common synthetic environment for conducting realistic training exercises. By combining live, virtual, constructive, and gaming elements, the STE enables soldiers to engage in multi-echelon, multidomain operations, thereby increasing proficiency through repetition and practice.

The STE comprises several key components, each tailored to specific training needs. The Common Synthetic Environment (CSE) serves as the foundation of the STE, providing a comprehensive training and mission rehearsal capability. Other components include the Reconfigurable Virtual Collective Trainer (RVCT) and the Soldier/Squad Virtual Trainer, which offer immersive training experiences for soldiers at all levels.

• Common Synthetic Environment (CSE): This unified simulation environment provides a realistic digital representation of the battlefield, allowing soldiers to train collectively across echelons.
• One World Terrain (OWT): Imagine a 3D replica of the entire Earth! OWT allows soldiers to train in virtual environments that mirror real-world locations.
• Reconfigurable Virtual Collective Trainer (RVCT): This mobile system enables soldiers and aviators to train together in a virtual space.
• Squad Immersive Virtual Trainer (SiVT): Utilizing mixed reality, SiVT allows squads to collectively rehearse battle drills using their organic weapons.
• Virtual Training Interfaces: These interfaces range from basic keyboard-and-mouse setups to fully immersive virtual reality experiences, catering to different training needs.

Global Terrain Capabilities include:

 A digital global with all terrain available to include full 2D, 3D and parametric information on all the buildings/structures, to include interiors and subterranean features, on the planet.
 Soldier-level fidelity of terrain available on a global scale.
 Training without boundaries that allows seamless integration of physical training areas into global scale wrap around exercises in the virtual and constructive training domains.
 Reuse and integration of a variety of data sources, from the reuse of existing training simulation terrain, such as Synthetic Environment – Core (SE-CORE) home station databases; to the importation of the Army’s Standard Shareable Geospatial Foundation (SSGF), the use open source data, to the collection and processing of organic terrain collection data, such as dronecaptured photogrammetry.
 The ability to export 3D mesh-based terrain to 2D vector- and raster-based terrain systems. The Global Terrain Capability concept delivers a geographical representation of the entire 3D world in a geo-referenced ellipsoid representation of the Earth. The goal for data fidelity is to provide subcentimeter resolution and accuracy in terrain, to support full live-synthetic entity interaction in a ‘fairfight’ environment. OWT will need to provide the best available terrain representation, from geo-typical to geo-specific, based on authoritative data, while making use of innovative approaches in procedural terrain generation and sensor fusion to constantly improve the quality of the available global terrain.

The STE’s applications extend beyond preparing soldiers for combat. It can be used for:

• Mission Command Training: Leaders can hone their decision-making skills in realistic simulated scenarios, ensuring they are prepared to lead effectively in any situation.
• Equipment Familiarization: Soldiers can learn to operate new weapons systems and equipment in a safe, virtual environment before deploying them in the field.
• Interoperability Training: The STE can facilitate joint training exercises with allied forces, ensuring seamless collaboration across different militaries.

The Army Futures Command Synthetic Training Environment Cross-Functional Team (STE CFT) stands at the forefront of military innovation, serving as a nexus where industry expertise, academic knowledge, government collaboration, and frontline soldier insights converge. Designed as part of the Army Futures Command’s modernization strategy, the STE CFT plays a pivotal role in accelerating the development and deployment of cutting-edge training capabilities to enhance military readiness.

Virtual Immersive User Interface and Hardware

Virtual Immersive trainers will seek a higher level of ‘form, fit, and function’ for the training audience than the semi-immersive systems. These interfaces into the CSE replace the immersive Combined Arms Tactical Trainers (CATT) found in the Army inventory. However, unlike the large overhead of current CATT trainers, the STE will need low overhead, reconfigurable, and transportable trainers to facilitate training anytime, anywhere. To accomplish this, the STE will require the use of innovative Mixed Reality and Natural User Interface technologies to deliver the following capabilities:

 Software-centric implementation
 Capitalization on rapid advancements in commercial mixed-reality technologies
 Low sustainment and concurrency costs
 Scalable interfaces to support training, without disruption, at the PoN
 Rapid concurrency updates driven through software rather than hardware changes
 Immersive collective training experiences that support suspension of trainee disbelief
 Accurate visual and haptic system representation (e.g., sensors, weapons, survivability capabilities, communications) to prevent negative training transfer or habit formation
 Natural fields of view
 The breadth of tactical trainers supporting Ground and Air Simulation

Ground: This reconfigurable and transportable trainer enables ground platform crew/team through Battalion Task Force to interact with the CSE and a digital representation of the MCIS interfaces and platforms for all WfFs. The immersive trainer provides a motion tracking capability and select highfidelity physical platform controls for crew members. The interface will stimulate sight, sound and touch modalities. Sight provides the Soldiers a natural field-of-view and allows the Soldiers to see the CSE from first person perspectives, sound allows the Soldier to hear and provide voice input into the CSE, and touch allows the Soldier to use physical and tactile controls of systems, subsystems, components, and  mission command information system interfaces to interact with the CSE.

Key considerations for ground immersive training include:
 Vehicle Commander: Weapon system control and sensor controls.
 Driver capabilities: Steer vehicle, change gear (e.g., forward, reverse), accelerate vehicle, brake
vehicle, and control/view dashboard.
 Gunner (combat vehicle): Weapon system control and sensor controls.
 Loader: Loader’s periscope, loading main weapons systems, loader’s weapons systems, radios.
 Gunner/Air Guard (wheeled vehicle): Grip, aim, fire, and reload weapon.

Air: This reconfigurable and transportable trainer enables aviation crew/team through Battalion Task Force to interact with a CSE, and a digital representation of the MCIS interfaces and platforms for all WfF. The immersive trainer provides a motion tracking capability and select high-fidelity physical platform controls for pilot, co-pilot, and non-rated crew members. The interface will stimulate sight, sound and touch modalities. An accurate representation of crew sensory inputs and feedback are critical. The relatively increased danger from crew error in aviation platforms necessitates an expectation of higher fidelity in Air immersive trainers.

Flight, weapon controls, and non-crewmember controls must provide highly accurate tactile control and switch options relative to the aircraft’s digital operational flight program (OFP) capabilities and be in the correct location relative to where the crew member is standing or sitting (e.g., collective is always on the left side, cyclic between the legs), to prevent negative training and habit transfer.

Pilot/Co-Pilot capabilities include dual flight controls to allow the pilot or co-pilot/gunner to fly the aircraft safely (Cyclic, Collective, Pedals). It also includes unique weapon systems interfaces (i.e., Target Acquisition and Display Sight (TADS) Electronic Display and Control (TEDAC) for Attack Helicopter [AH]). The TEDAC for the AH-64 is only for the co-pilot/gunner position. Non-Rated Crewmembers capabilities include unique weapon interfaces (i.e. door gun) for the Utility

Helicopter (UH) and Cargo Helicopter (CH); unique Intercommunications System (ICS) Switch and handheld push to talk capability (UH, CH); and unique hoist controls (UH and CH). Unique cargo hook view space (CH) hoist operations must provide a minimum level of tactile and visual feedback to ensure awareness of proper operations.

Unmanned Aircraft System (UAS) capabilities will include the realistic representation of unmanned systems, to include all kinetic and non-kinetic battlefield effects, as well as the appropriate affordances for user/operator interactions, in order to facilitate collective training.

Central to the STE’s effectiveness is its Information System, a virtual training suite that leverages state-of-the-art technology to create immersive, multi-domain operational environments. Powered by 3D imagery and high-resolution graphics, this system provides soldiers with a realistic training experience that replicates the complexities of modern warfare. Brig. Gen. William R. Glaser, director of the STE CFT, emphasizes the importance of rapidly delivering terrain and operational data to the warfighter, enabling leaders to conduct reconnaissance, war games, and rehearsals with unprecedented realism. Additionally, artificial intelligence and machine learning processes are integrated into STE programs to accurately simulate warfighting elements, collect data on soldier reactions, and inform training and operational planning. Through seamless integration with existing Army systems and live training exercises, the STE CFT ensures that soldiers have access to the best training programs, regardless of their location or operational environment.

Multi-Echelon, Multidomain Battles:

One of the key strengths of STE is its capability to simulate multi-echelon, multidomain battles, where soldiers can engage in complex, large-scale operations across land, air, sea, space, and cyberspace domains. Whether conducting combined arms maneuvers or coordinating joint operations with allied forces, STE provides soldiers with the opportunity to train collaboratively in a virtual environment, enhancing interoperability and readiness for real-world missions.

Mission Command Training:

Effective mission command is essential for military success, requiring leaders to make informed decisions under pressure while effectively communicating and coordinating with their units. STE enables leaders at all levels to practice mission command in realistic scenarios, where they must assess situational awareness, issue orders, and adapt to changing battlefield conditions. By honing their leadership skills in a simulated environment, commanders can better prepare for the complexities of modern warfare.

Benefits of STE:

The benefits of STE extend far beyond traditional training methods. By leveraging simulation technology, the US Army can conduct cost-effective, scalable training exercises that are accessible to soldiers worldwide. Additionally, STE allows for continuous training and evaluation, enabling soldiers to refine their skills and tactics over time.

The STE offers several key advantages over traditional training methods:

• Unprecedented Realism: The STE can replicate diverse environments, from scorching deserts to dense urban jungles. Soldiers can experience the sights, sounds, and even the feel of real-world combat without the risks associated with live-fire exercises.
• Multi-Domain Training: Modern warfare extends beyond the traditional battlefield. The STE can incorporate cyber, space, and electronic warfare elements, preparing soldiers for the full spectrum of threats they might encounter.
• Repetition and Refinement: The STE allows for on-demand training scenarios, enabling soldiers to practice specific skills and tactics repeatedly until they are mastered. This consistent training translates to greater proficiency and confidence on the battlefield.
• Cost-Effectiveness: The STE reduces reliance on expensive live-fire exercises, offering a more cost-efficient approach to training large numbers of personnel.

Furthermore, STE facilitates the integration of emerging technologies, such as artificial intelligence and virtual reality, into military training, ensuring that soldiers remain at the forefront of technological innovation.

Industry Initiatives

Virtual Sand Table: A New Tool for Military Training

Modern technology has revolutionized military training with the introduction of virtual and mixed-reality (XR) environments. MVRsimulation’s Sand Table, unveiled at the 2023 I/ITSEC event, exemplifies this progress.

This collaborative visualization tool allows military personnel to plan and review missions in a 3D virtual world. Users can interact with a geospecific terrain model, rendered by MVRsimulation’s Virtual Reality Scene Generator (VRSG), using Varjo mixed-reality headsets. VRSG’s extensive terrain database and high-resolution insets provide a realistic training environment. MVRsimulation emphasizes the importance of this detailed imagery, highlighting its role in mission planning and scenario development.

The Sand Table offers several distinct functionalities. Firstly, it facilitates mission planning and rehearsal. Trainees can collaborate with colleagues and instructors to plan missions, visualize the terrain, and refine their strategies before deploying simulators like the DJFT or PTMT.

Secondly, the Sand Table serves as a valuable classroom training tool. Instructors can use it to guide students through mission planning and decision-making exercises. Students can see the consequences of their decisions in real-time and receive immediate feedback. Additionally, other trainees can observe the scenario through projected views from the headsets, fostering a collaborative learning environment.

Finally, the Sand Table enables After-Action Review (AAR). After completing missions in networked simulators, trainees and instructors can review a playback of the scenario, analyze critical moments, and gain valuable insights. The system also tracks trainee eye-gaze data, allowing instructors to assess whether trainees focused on crucial cues throughout the mission.

Beyond training exercises, the Sand Table functions as a real-world mission planning tool. It allows users to rehearse actual mission plans, giving them a clear understanding of how to execute tasks effectively. The system can also be connected with DIS-based software to evaluate the success of mission objectives.

Key Features of the Sand Table

One of the key strengths of the Sand Table is its integration with Varjo XR headsets. These headsets offer real-world pass-through capabilities, allowing users to interact with physical objects and military equipment without removing the headset. This minimizes disruption and enhances immersion in the virtual environment.

Furthermore, the headsets feature eye-tracking technology. This allows instructors to monitor trainee gaze data and ensure they are focusing on relevant aspects of the scenario. Additionally, eye-tracking data from other participants can be integrated into the AAR process, providing a more comprehensive understanding of everyone’s focus during the training exercise.

Each user controls their view within the Sand Table using a Valve Index controller. They can zoom in and out to examine specific entities or gain a top-down view of the entire battlefield. The controllers also provide a virtual pointer for highlighting elements and directing attention to specific locations.

Demonstration Scenario and Future Potential

At the I/ITSEC launch, the Sand Table was networked with DJFT and PTMT simulators. Participants conducted a mission planning exercise in a highly detailed 2cm Yuma terrain model. They then observed the mission unfold in real-time as JTAC trainees executed it using the DJFT. Finally, an AAR was conducted using VRSG’s recording and playback functionalities.

The demonstration scenario involved a JTAC call-for-fire on enemy vehicles and launchers preparing to launch missiles. This scenario mirrored a recent real-world event, further emphasizing the Sand Table’s ability to provide relevant and adaptable training experiences.

MVRsimulation plans to develop a two-headset containerized Sand Table by Q1 2024, enhancing its deployability for field operations. The company believes the Sand Table’s lightweight design and ability to utilize real-world locations and models make it a powerful tool for preparing military personnel for real-world challenges.

Looking Ahead:

As the US Army continues to invest in the development of the Synthetic Training Environment, the future of military training looks brighter than ever. By embracing innovation and leveraging the power of simulation technology, soldiers will be better equipped to face the challenges of an increasingly complex and dynamic battlefield. With STE, the US Army is shaping the future of military readiness, ensuring that soldiers are prepared to succeed in any mission, anywhere in the world.