There must be human input and oversight when it comes to autonomous systems meant to make decisions on when, where and on whom to pull the trigger, the former deputy commander of U.S. Cyber Command said during a panel at a conference focused entirely on the so-called third offset strategy.
“What’s happening is that the time between when I make a decision, when I press a button to have an effect and when that effect occurs is near simultaneous,” retired Marine Corps Lt. Gen. Robert Schmidle said at the conference, Assessing the Third Offest Strategy, hosted by the Center for Strategic and International Studies on Oct. 28.
“So what that means is that if I’m not thinking ahead of what that effect is going to be, my reaction to that is just going to be that — a reaction, not an action,” he said.
In many regards, certain operations, effects and capabilities happen too quickly for humans to respond. This phenomena is sometimes referred to as cyber or machine speed, as event events occur at the nanosecond level in cyberspace.
Cyberspace is one domain of battle in which top officials have publicly asked for autonomous and automated systems that can react faster than humans, almost exclusively for defensive purposes.
Schmidle explained that in order to get ahead of cyber intrusions, automation is necessary; though he added that decision-makers must be involved in the development of automated algorithms and code.
Rear Adm. Mike Manazir, deputy chief of Naval Operations for Warfare Systems, also believes that while humans will always need to be in the loop, the responsibilities of humans and machines should be altered. During the same panel discussion, he provided an example using the classic OODA loop paradigm — observe, orient, decide and act — with the popular traffic mobile application Waze, which surveys traffic patterns based upon various data sets and user feedback to provide the best, though sometimes unconventional, route to beat traffic.
“You want to be able to decide and act with speed faster than your adversary does. So we spend a lot of time observing and orienting,” Manazir said. With Waze, the machine issues recommendations to take various turns, taking the driver places they may never have thought of taking,” Manazir said, noting that the user ultimately decides to act upon those recommendations.
“A very simplistic example and maybe a silly example to say that my thinking about man-machine interfaces in this discussion about where does the human capitalize and where do you bring the machine,” he said. “If the machine with machine speed and machine algorithms could observe the environment you’re in and orient the human to that environment, then quite simply and maybe too simply the human is [decide, act, decide, act, decide, act, decide, act] because you have [situational awareness] faster.”
Artificial intelligence such as Waze is allowing humans to decide and act, he said. He postulated that maybe the adversary has a similar technology, but if they are placing it at the end of the OODA loop, then they are constantly reorienting and never getting to act.
Within this decision loop framework, he said the human would make the decisions, be it rules of engagement or how to create an effect such as pulling a trigger or putting an effect somewhere — kinetic or non-kinetic — but it is important that the human be able to trust the data and that the data is truthful, accurate and presents a view of the universe humans cannot themselves sense.
Similarly, Schmidle doubled down on his point that decision-makers must have input on how these algorithms are crafted. If an application such as Waze is saying turn right instead of left, there is an algorithm issuing these recommendations, and if the user doesn’t know how the algorithm works — what it is privileging, what it is prioritizing, what the metric is by which it is making that recommendation, what it is using for input — the human might not take it.
How many times has Waze recommended a route that has caused you to question why it is saying to take a particular route, Schmidle asked rhetorically. He questioned whether saving 10 seconds on a trip is valid?
Schmidle noted that it might not be clear what the application is using to generate its recommendations. Is it using the overhead imagery of a traffic jam? Is it getting information from all the Google map users — where they are and how fast they’re going? “All those questions, I would suggest, are part of a process up front we would ask,” he said.
On the flip side, while these intelligent, autonomous systems that would be a boon for operators, what if an adversary were to take away this capability?
“Imagine being so dependent on Waze, that you have an adversary that will look to take it away from you,” Maj. Gen. Walt Piatt, deputy director for operations with the Army’s Rapid Capabilities Office, noted during the same panel. “This is what we saw in Ukraine. For all those that thought we’d never see land war in Europe again and who said all these things we were modernizing were irrelevant, this is exactly what happened to a very good army — the Ukrainian army.”
Describing what the Russians and Russian-backed separatists were able to do in their incursion into Ukraine, Piatt described how they were able to deny intelligence to the Ukrainian army, enabling them to maneuver right under Ukraine’s nose. “It’s checkmate even before the fight gets started,” he said.
“This is what our adversaries have presented us — they’ve seen … the over dependency on technology, and they can take that away in a second and you’ve got to fight without it,” he continued. “We’ve got to be able to fight through that electromagnetic spectrum when it’s contested, when the air domain is contested [and] you no longer have air superiority.”
Though the U.S. military has not been to battle without air superiority in a long time, he noted, this dependency can be a critical vulnerability if taken away. “I never did a patrol in Baghdad or Iraq or Afghanistan unless I had an ISR soak. Now take that away. Our adversaries can take that away. We’ve got to be able to maneuver to be able to destroy their assets … it’s a multi-domain battle,” he said.
Piatt was sure to express that all this is not to say the U.S. is solely dependent on a lot of this technology because some of this is easy to fight through. But this is what happened in Ukraine; it’s not that Ukraine wasn’t up to the challenge, but they simply lost their intelligence, he said.
