Human-machine research teaches cadets design, human factors, culture

Jenn Kuster, U.S. Air Force Academy Dean of Faculty research publicist, Oct. 31, 2017

U.S. AIR FORCE ACADEMY, Colo. — The idea of an endearing, interactive and witty robot companion may elicit nostalgic flashbacks of Rosie and the Jetsons, but this is not 2062 and the star of the show is not a two-dimensional robot maid.

The year is 2017 and autonomous systems are penetrating societies around the world: from self-driving vehicles and robot waiters in restaurants to robot wingmen flying alongside human pilots. Indeed, the United States Air Force’s vision is for humans and robots to partner as teammates for maximum success on the battlefield and enhanced safety for warfighters.

Cadets studying in the Warfighter Effectiveness Research Center (WERC) are helping to make this vision a reality. Their goal is to work with faculty mentors to transform human-computer interactions such that intelligent machines are seamlessly integrated with humans. In order to achieve this goal, cadets majoring in Behavioral Sciences, Human Factors, Systems Engineering, Computer Sciences, Philosophy and other disciplines are working together in design projects that will ultimately enhance real-world warfighter capabilities. Cadets benefit from a “learning by doing” model in projects that apply what they’ve learned in the classroom. Warfighters benefit by leveraging the results of these studies in current and future systems.

To further understand the interaction between humans and machines and also gain insight into the research of international universities, cadets travelled to South Korea and Japan for a two week visit funded by the Air Force Academy’s International Programs office and the Air Force Office of Scientific Research. The knowledge cadets obtained and brought back to the Academy, was used to kick start eight capstone projects that align with Department of Defense Research and Development activities.

Cadets and faculty saw firsthand the difference between how American society perceives robots with high potential for evil based on how they are portrayed in movies versus how Japanese society views robots as superheroes.

While visiting premier universities in South Korea and Japan to see the latest in human-machine teaming, cadets visited cultural landmarks, experienced how both cultures integrated robots in daily life and discovered how to study the human side of machine interaction from different perspectives. By exploring other cultures, the research can expand its ability to measure subjective factors such as facial expressions to determine emotions. Pairing subjective factors with objective elements, such as eye movement and heart rate, provides for more holistic data for human-machine teaming research analysis.

Cadet Wesley Jackman said he “was blown away by all the research taking place in the world.” “I got excited that I might one day be involved in it! The biggest stand out was the robot face that reflected its emotions using colors and digital facial expressions. Robots that can feel, or at least pretend to feel based off of the humans around them will make them so much more sympathetic as teammates” he said.

Within the WERC, cadets have executed human-machine teaming capstone research projects, demonstrating a deeper understanding of content within their disciplines into their projects.

Thus far, the WERC has focused on the effectiveness of the communication between humans and machines, assessing situational awareness, trust and overall mission performance. Additionally, they try to answer questions including; how does the interface and interaction change when machines have their own intelligence? What is necessary to enhance the trust between humans and autonomous systems in high stress situations, such as flying a fighter plane?

For instance, current human-machine teaming research is developing ways to measure the trustworthiness of robots. Design projects are focusing on developing systems for robots to detect human emotion by recognizing facial expressions and tone of voice. Robots, then sympathize with humans based on perceived emotion and respond appropriately depending on the context. Leveraging these sensing technologies for the design of next-generation human-robot systems helps the Air Force and the Academy advance research in cutting-edge technological systems.

For example, the loyal wingman project, an Air Force Research Laboratory initiative, put autonomous F-16s in the air with piloted F-35 aircraft

Cadets Tyler Murray, Timothy Tyson and Kevin Yalung, all system engineering majors, are formulating effective design standards ensuring the communication between these agents are effective and the workload of F-35 pilots are at safe levels across a wide range of combat tasks. The research requires cadets to spend many hours on the flight simulator. This experience has proven essential in furthering their Air Force career.

“I am investigating things now that I will be studying after graduation in my career,” says Murray. The data cadets collect now fuels future Air Force research as well.

Cadets learn how human interactions with machines change based on cultural differences. By exploring other cultures, the research expands its ability to measure subjective factors such as facial expressions to determine emotions. Pairing subjective factors with objective elements, such as eye movement and heart rate, provides for more holistic data for human-machine teaming research analysis.

Maj. Chad Tossell, Director, WERC, says “Human-machine teaming provides the Air Force an advantage over adversaries and the potential to maximize mission performance in complex and contested environments.”

To this end, human-machine teaming research provides cadets unique opportunities to work with advanced machines and visit leading research institutions as undergraduate students, helping develop world-class leaders who can solve real-life problems and advance Air Force initiatives to continue striving towards air power excellence.

We may have 48 years before we can live in the futuristic era laid out by the Jetsons of flying cars and robot maids but the Air Force Academy and cadets are finding new ways to capture human emotions in robots and gaining more than a maid, a friend. Cadets are advancing research and developing cutting-edge technology we never expected to see in 2017.