{"id":18934,"date":"2019-12-04T19:38:41","date_gmt":"2019-12-04T19:38:41","guid":{"rendered":"https:\/\/one.sightlinemg.com\/c4isrnet\/uncategorized\/2019\/12\/04\/op-ed-stop-the-rise-of-autonomous-weapons\/"},"modified":"2026-08-08T19:18:10","modified_gmt":"2026-08-08T19:18:10","slug":"op-ed-stop-the-rise-of-autonomous-weapons","status":"publish","type":"post","link":"https:\/\/one.sightlinemg.com\/c4isrnet\/opinion\/2019\/12\/04\/op-ed-stop-the-rise-of-autonomous-weapons\/","title":{"rendered":"Op-ed: Stop the rise of autonomous weapons"},"content":{"rendered":"\n

Robotics is rapidly being transformed by advances in artificial intelligence.<\/p>\n\n\n\n

And the benefits are widespread: We are seeing safer vehicles with the ability to automatically brake in an emergency<\/a>, robotic arms transforming factory lines that were once off-shored<\/a> and new robots<\/a> that can do everything from shop for groceries to deliver prescription drugs<\/a> to people who have trouble doing it themselves.<\/p>\n\n\n\n

But our ever-growing appetite for intelligent, autonomous machines poses a host of ethical challenges.<\/p>\n\n\n\n

\"\"
“I am a mechanical man built by human hands,” a robot at the 1936 Texas Centennial Exposition in Dallas. (Library of Congress)<\/figcaption><\/figure>\n\n\n\n

Rapid advances have led to ethical dilemmas<\/b><\/i><\/p>\n\n\n\n

These ideas and more were swirling as my colleagues and I<\/a> met in early November at one of the world\u2019s largest autonomous robotics-focused research conferences \u2013 the IEEE International Conference on Intelligent Robots and Systems<\/a>. There, academics, corporate researchers, and government scientists presented developments in algorithms that allow robots to make their own decisions.<\/p>\n\n\n\n

As with all technology, the range of future uses for our research is difficult to imagine. It\u2019s even more challenging to forecast given how quickly this field is changing.<\/p>\n\n\n\n

Take, for example, the ability for a computer to identify objects in an image: In 2010, the state of the art was successful only about half of the time<\/a>, and it was stuck there for years.<\/p>\n\n\n\n

Today, though, the best algorithms as shown in published papers are now at 86 percent accuracy<\/a>. That advance alone allows autonomous robots to understand what they are seeing through the camera lenses. It also shows the rapid pace of progress over the past decade due to developments in AI.<\/p>\n\n\n\n

This kind of improvement is a true milestone from a technical perspective. Whereas in the past manually reviewing troves of video footage would require an incredible number of hours, now such data can be rapidly and accurately parsed by a computer program.<\/p>\n\n\n\n

But it also gives rise to an ethical dilemma. In removing humans from the process, the assumptions that underpin the decisions related to privacy and security have been fundamentally altered<\/a>.<\/p>\n\n\n\n

For example, the use of cameras in public streets may have raised privacy concerns 15 or 20 years ago, but adding accurate facial recognition technology dramatically alters those privacy implications.<\/p>\n\n\n\n

\"\"
The DARPA Autonomous Robotic Manipulation robot \u201cRobbie\u201d and the NASA\/DARPA Robonaut 1A on display in the Moving Beyond Earth gallery at the National Air and Space Museum. (Mark Avino\/ Smithsonian Institution)<\/figcaption><\/figure>\n\n\n\n

Easy to modify systems<\/b><\/i><\/p>\n\n\n\n

When developing machines that can make own decisions \u2013 typically called autonomous systems \u2013 the ethical questions that arise are arguably more concerning than those in object recognition.<\/p>\n\n\n\n

AI-enhanced autonomy is developing so rapidly that capabilities which were once limited to highly engineered systems are now available to anyone with a household toolbox and some computer experience.<\/p>\n\n\n\n

People with no background in computer science can learn some of the most state-of-the-art artificial intelligence tools<\/a>, and robots are more than willing to let you run your newly acquired machine learning techniques<\/a> on them. There are online forums filled with people eager to help anyone learn how to do this<\/a>.<\/p>\n\n\n\n

With earlier tools, it was already easy enough to program your minimally modified drone to identify a red bag and follow it<\/a>.<\/p>\n\n\n\n

More recent object detection technology<\/a> unlocks the ability to track a range of things that resemble more than 9,000 different object types.<\/p>\n\n\n\n

Combined with newer, more maneuverable drones<\/a>, it\u2019s not hard to imagine how easily they could be equipped with weapons.<\/p>\n\n\n\n

What\u2019s to stop someone from strapping an explosive or another weapon to a drone equipped with this technology?<\/p>\n\n\n\n

Using a variety of techniques, autonomous drones are already a threat. They have been caught dropping explosives on U.S. troops<\/a>, shutting down airports<\/a> and being used in an assassination attempt on Venezuelan leader Nicolas Maduro<\/a>.<\/p>\n\n\n\n

The autonomous systems that are being developed right now could make staging such attacks easier and more devastating.<\/p>\n\n\n\n

\"\"
A captured German V-1 bomb displayed at the Navy Exhibit 6th War Loan Campaign, Chicago, from 11\/18\/1944 – 12\/3\/1944. (National Archives)<\/figcaption><\/figure>\n\n\n\n

Regulation or review boards?<\/b><\/i><\/p>\n\n\n\n

About a year ago, a group of researchers in artificial intelligence and autonomous robotics put forward a pledge<\/a> to refrain from developing lethal autonomous weapons.<\/p>\n\n\n\n

They defined lethal autonomous weapons as platforms that are capable of \u201cselecting and engaging targets without human intervention.\u201d<\/p>\n\n\n\n

As a robotics researcher who isn\u2019t interested in developing autonomous targeting techniques, I felt that the pledge missed the crux of the danger<\/a>. It glossed over important ethical questions that need to be addressed, especially those at the broad intersection of drone applications that could be either benign or violent.<\/p>\n\n\n\n

For one, the researchers, companies and developers who wrote the papers and built the software and devices generally aren\u2019t doing it to create weapons. However, they might inadvertently enable others, with minimal expertise, to create such weapons.<\/p>\n\n\n\n

What can we do to address this risk?<\/p>\n\n\n\n

Regulation is one option, and one already used by banning aerial drones near airports or around national parks.<\/p>\n\n\n\n

Those are helpful, but they don\u2019t prevent the creation of weaponized drones.<\/p>\n\n\n\n

Traditional weapons regulations are not a sufficient template, either. They generally tighten controls on the source material or the manufacturing process.<\/p>\n\n\n\n

That would be nearly impossible with autonomous systems, where the source materials are widely shared computer code and the manufacturing process can take place at home using off-the-shelf components.<\/p>\n\n\n\n

Another option would be to follow in the footsteps of biologists.<\/p>\n\n\n\n

In 1975, they held a conference on the potential hazards of recombinant DNA<\/a> at Asilomar in California. There, experts agreed to voluntary guidelines that would direct the course of future work.<\/p>\n\n\n\n

For autonomous systems, such an outcome seems unlikely at this point. Many research projects that could be used in the development of weapons also have peaceful and incredibly useful outcomes.<\/p>\n\n\n\n

A third choice would be to establish self-governance bodies at the organization level, such as the institutional review boards<\/a> that currently oversee studies on human subjects at companies, universities and government labs.<\/p>\n\n\n\n

These boards consider the benefits to the populations involved in the research and craft ways to mitigate potential harms. But they can regulate only research done within their institutions, which limits their scope.<\/p>\n\n\n\n

Still, a large number of researchers would fall under these boards\u2019 purview \u2013 within the autonomous robotics research community, nearly every presenter at technical conferences are members of an institution. Research review boards would be a first step toward self-regulation and could flag projects that could be weaponized.<\/p>\n\n\n\n

\"\"
A view of a six-legged walker robot at the Naval Surface Weapons Center, March 1, 1983. (National Archives)<\/figcaption><\/figure>\n\n\n\n

Living with the peril and promise<\/b><\/i><\/p>\n\n\n\n

Many of my colleagues and I are excited to develop the next generation of autonomous systems. I feel that the potential for good is too promising to ignore. But I am also concerned about the risks that new technologies pose, especially if they are exploited by malicious people.<\/p>\n\n\n\n

Yet with some careful organization and informed conversations today, I believe we can work toward achieving those benefits while limiting the potential for harm.<\/p>\n\n\n\t