Artificially intelligent systems must be able to explain themselves to operators if they are to be trusted, according to an expert from the Defense Advanced Research Agency, who voiced concern that methods used by current AI systems are often masked by mysterious algorithms. <\/p>\n\n\n\n
\u201cA lot of the machine learning algorithms we\u2019re using today, I would tell you \u2018good luck,\u201d Fred Kennedy, the director of DARPA\u2019s Tactical Technology Office during a panel at Navy League\u2019s Sea-Air-Space on April 10. \u201cWe have no idea why they know the difference between a cat and a baboon.\u201d<\/p>\n\n\n\n
\u201cIf you start diving down into the neural net that\u2019s controlling it,\u201d Kennedy continued, \u201cyou quickly discover that the features these algorithms are picking out have very little to do with how humans identify things.\u201d<\/p>\n\n\n\n
Kennedy\u2019s comments were in response to Deputy Assistant Secretary of the Navy for Unmanned Systems Frank Kelley, who described the leap of faith operators must make when dealing with artificially intelligent systems. <\/p>\n\n\n\n
\u201cYou\u2019re throwing a master switch on and just praying to God that [Naval Research Laboratory] and John\u2019s Hopkins knew what the hell that they were doing,\u201d Kelley said of the process. <\/p>\n\n\n\n
The key to building trust, according to Kennedy, lies with the machines. <\/p>\n\n\n\n
\u201cThe system has to tell us what it\u2019s thinking,\u201d Dr. Kennedy said. \u201cThat\u2019s where the trust gets built. That\u2019s how we start to use and understand them.\u201d<\/p>\n\n\n\n
DARPA\u2019s Explainable Artificial Intelligence program seeks to teach AI how to do just that. The program envisions systems that will have the ability to explain the rationale behind their decisions, characterize their strengths and weaknesses, and describe how they will behave in the future. Such capabilities are designed to improve teamwork between man and machine by encouraging warfighters to trust artificially intelligent systems. <\/p>\n\n\n\n
\u201cIt\u2019s always going to be about human-unmanned teaming,\u201d said Kennedy. \u201cThere is no doubt about that.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"
A DARPA official described the the inexplicable algorithms powering artificial intelligence as a strain on human-machine relationships.<\/p>\n","protected":false},"author":7,"featured_media":27726,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_yoast_wpseo_meta-robots-noindex":"","_yoast_wpseo_meta-robots-nofollow":"","_yoast_wpseo_canonical":"","_acf":"","_yoast_wpseo_primary_category":29,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","_smg_distribution_targets":{"10":{"categories":["navy-league","digital-show-dailies","digital-show-daily","newsletters"],"primary_category":"navy-league"}}},"categories":[106,29,32,21],"tags":[],"coauthors":[1103],"class_list":["post-10105","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-daily-brief","category-home","category-newsletters","category-unmanned"],"acf":{"subheadline":"","legacy_arc_id":"GDC7DGTDUNHSLOV35YLDLVHARE","arc_canonical_url":"\/home\/2018\/04\/10\/darpa-official-to-build-trust-in-ai-machines-must-explain-themselves\/","remove_feature_photo":false,"is_sponsored":false,"subtype":"","redirect_url":"","disable_inline_ads":false,"native_logo_pretext":"Presented By:"},"yoast_head":"\n