NextGen Debrief
Mixed reality flight debriefing
2021
The United States Air Force's debriefs are
hindered by restrictive, outdated technology
Mandatory after each flight, current debriefings visualize flight paths using a variety of mediums, including outdated software, handheld models, and whiteboards. Though familiar to Airmen, these tools are ineffective, and their inefficiency is glaringly apparent when contrasted with modern technological advancements.
Project background
As a designer at The Luminosity Lab, I consulted with a team of Luke AFB Airmen to explore a spatial solution to an important issue facing the Air Force nationwide. Their concept aimed to enhance flight debrief practices, similar to how virtual reality has become commonplace in flight training and preparation.
How might we make flight debriefs more effective and visual with the use of mixed reality?
Key objectives:
Accelerate debriefing completion time
Enhance communication with powerful immersive tools
Enable flexible hybrid-remote debriefings
Storyboarding
I began by observing an existing debrief meeting to understand current practices. To clearly communicate the vision, I then created a storyboard that captured the client's key thoughts and goals.
Constraints & adoption concerns
The concept thoughtfully preserves legacy elements- direct eye-contact, F-16 names, color schemes, untethered movement, and whiteboard annotations like domes and dashed arrows- while introducing a new dimension to the debriefing process.
Technology stack
Our solution leveraged the Microsoft HoloLens 2 for immersive mixed reality, powered by Unity 3D and its Mixed Reality Toolkit (MRTK). I utilized Blender for custom 3D model creation and Bing Maps API for real-world data integration.
Prototyping
I iteratively developed and tested F-16 animations to ensure they accurately reflected provided flight data, including longitude, latitude, altitude, roll, pitch, and yaw. I also held multiple demo days with the clients, utilizing multiple HoloLens 2 headsets and spatial anchors to create a shared 3D space for precise, collaborative debriefings within a unified virtual environment.
Final Pitch
Our presentation to leadership was a resounding success; many were not only intrigued by the potential of this technology but also shared that it was their first time truly realizing mixed reality's practical future. We're excited to have landed as finalists in the SparkTank competition, securing second place. We have high hopes that this concept will be added to the development pipeline as the technology continues to advance.
You can watch the official pitch video on Youtube: