Airborne Cockpit Virtualization

The Problem:

Current flight training programs are based on the extensive use of flight simulators to let pilots experience problems that may arise when they are in flight. Emerging Live Virtual Constructive (LVC) technologies can be used to enhance the training experience in distributed mission training scenarios. However, distributed training is usually based on either real-world or simulated world scenarios. The simulated world limits the amount of exposure to dynamic handling cues that are important in determining proper control response in a tight situation. Live training requires a lot of work (and some luck) to coordinate large number of participants that may include dismounted war fighters, seagoing vessels, armored vehicles, artillery, and airborne assists in order to complete a common objective. Interesting training paradigms may arise when a blend of simulated and real-world content is used in the scenario.

Our Solution:

Rockwell Collins and OPL are jointly developing new LVC training paradigms. In-flight virtualization is an emerging concept that embeds simulator and virtual reality technology into the flight deck to provide military pilots with the capability to “fight as they train.” We jointly developed and integrated an initial prototype environment to study how in-flight virtualization can be transitioned from a concept toward a realized product. In our future research, this platform will generate a deeper understanding of methods to measure and document:

· cognitive loading when training with in-flight virtualization,

· safety of flight and weapons safety issues as well as their associated countermeasures, and

· a framework of a ground and flight training concept of operations (CONOPS).

OPL is leveraging its L-29 Fighter Trainer jet (above) to install a virtualized cockpit. This cockpit was installed in the aft crew station such that the front crew station would remain in the original condition for the safety pilot. In addition to leaving the front cockpit intact the safety pilot also operates the aircraft during critical situations such as take-off and landing and any other emergency situations that may arise.

During the LVC scenario, we generate red and blue forces, proxy terrain, and incorporate the real-world asset in the virtualized scenario. Our long term hypothesis is to quantitatively show that this blend of real and simulated scenario will lead to better transfer of training than a purely real-world or a purely simulated training environment.

Airborne Cockpit Vizualization Video

L-29 Flight Test Aircraft, Cognitive Delfin (COD)

The L-29 is used as high dynamics flight research aircraft for development of pilot state characterization. The aircraft is equipped with Head-Up Display (HUD), Head-Down Display (HDD), and with a Hands-on Throttle and Stick (HOTAS). These items are pictured below; click on the picture for an enlarged view.

Providing Virtual Visual and Audio into the Cockpit

The L-29 is able to be federated in distributed mission scenario exercisesthat result in very realistic air-to-air and air-to-ground scenarios and can generate very realistic workload situations. OPL's Mobile Ground Station is used as the mobile command, control, and communications link between simulated federates and the COD. The trailer has a 900 MHz spread-spectrum data link with a 20 ft mast and automatic rotator device. The trailer also holds OPL's flight simulator featuring Advanced Tactical Flight Simulator (ATAS) software from Alion, the Joint Semi Autonomus Forces (JSAF) system, and the Common Distributed Trainer Station (CDMTS). OPL has also the capability of using the Cognitive Avionics Tool Set (CATS) and the Quality of Training Effectiveness Assessment (QTEA) system, the Distributed Mission Training (DMT) solution to monitor pilot performance in research training applications.

Federating the Cockpit

A simplified illustration of the communication paths is shown below:

The architecture is shown in the following diagram:

Post-Mission Presentation Using Simulation Replay

The Rockwell Collins Advanced Technology lab uses a very sophisticated visualization system called the EPX-500 that can render photorealistic facsimiles of the LVC battlespace. Using this environment, it is possible to monitor the exercise from any LVC entity point of view, as well as re-run the simulation, with full graphical visuals of the federated entities, including Herkey 11 (live L-29) and Herkey 12 (simulated ATAS), simulated terrain, adversarial tanks, and helicopters. As anyone who has worked in these virtual environments knows, one of the most useful features is that the post-mission presentation can be replayed with various points of view from a camera angle perspective. This includes viewing the mission as Herkey 12 would see it from out of the cockpit window, including watching Herkey 11 flying its bombing run, or looking up from the ground from the position of the tanks.

Target Area Tactics for Close Air Support

Benefits

· Traditional M&S ROI: whenever a situation can be studied without bearing the full cost of having the equipment and staff, the time to fully exercise physical entities, and perform these experiments without loss of physical objects, there is a significant cost savings.

· Pilot training ROI: For both commercial and military pilots, training in various environments and situations is critical; being able to place a pilot in a blended virtualized environment allows the pilot to experience situations in the air that would be either too risky or cost prohibitive to create at a flight range.

· Training that applies to more than a single pilot: multiple pilots, the ground support team, and command and control personnel can all be trained at the same time using blended virtualized environments.

· Moving training out of the range: given the correct planning, multiple entity training is not restricted to flight ranges.

· Mission run-throughs while transitioning to combat areas: The hours spent while transitioning to a location can be used to virtually run a mission, preparing the all participants for a successful flight.

Future Challenges

The future challenges to our research falls into three general areas: safety, cognitive challenges related to transitioning between actual and virtual environments, and instrumentation to realize the blended virtualized environment within the cockpit.

Acknowledgement

The OPL would like to thank Rockwell Collins for funding this very interesting joint effort.