For example, the rise of hybrid and multi-cloud architectures present data integration, access and management challenges for many as agencies seek to leverage data assets that span across both on-premise and hosted solutions.

This may be why the GSA Data Center and Cloud Optimization Initiative Program Management Office recently released a Multi-Cloud and Hybrid Cloud Guide to help agencies make better decisions about cloud architecture. Further complicating matters is that the DoD is facing a groundbreaking December award of the Joint Warfighting Cloud Capability procurement.

Pentagon Chief Information Officer John Sherman describes the JWCC as a “multi-cloud effort that will provide enterprise cloud capabilities for the Defense Department at all three security classifications: unclassified, secret, and top secret all the way from the continental United States out to the tactical edge.” At the conclusion of this possible five-year procurement, the DOD plans to launch a full and open competition for a future multi-cloud acquisitions. Until then, DoD data scientists may be forced to work in silos because connecting to live data may not always be possible.

How Technology Can Help

Many government agencies are overcoming these challenges using technologies such as data virtualization to implement a logical data fabric approach capable of ensuring trusted data access and sharing. Data virtualization is a modern data integration technique that integrates data in real-time without having to physically replicate it.

Data virtualization can seamlessly combine views of data from different sources and feed AI/ML engines with data from a common data services layer. Using data virtualization, AI teams can work more efficiently and collaborate more effectively, because the technology provides views of data rather than replicating it.

This not only saves access and storage costs because it provides a unified data-access layer, it also enables stakeholders to implement governance controls from a single point across the department. Creating this “single source of truth” is one of the most valuable characteristics of unifying enterprise data through use the of data virtualization.

This combination of enterprise AI, multi-cloud architecture, and data virtualization is being leveraged by many government organizations to leverage data more effectively and take advantage of the cost savings of the cloud. Together, these technologies underscore the fact that digital transformation is not just about technology, but about using technology in the most intelligent way and providing enhanced value to data science teams and internal and external data consumers.

Also, with the competition for market share between the major Cloud Service Providers (CSP) promising both better value for government and access to a vast array of AL/ML tools to drive better, mission-specific results, there may be conflicting narratives on which CSP — and which AI/ML technology — is best for a given mission.

In this environment, a logical data fabric is rapidly emerging as the technologically elegant solution to the chaos of multi-cloud computing as it simultaneously makes the best features of each CSP available to users.

Defined by Gartner as a design concept that serves as an integrated layer (fabric) of data and connecting processes, a data fabric utilizes continuous analytics over existing, discoverable and inferenced metadata assets to support the design, deployment and utilization of integrated and reusable data across all environments, including hybrid and multi-cloud platforms. This pure play data fabric enables the best of all commercial CSP offerings without vendor lock-in and, in many instances, is proving to be the government’s best answer to overcome these challenges.

Addressing the DoD’s pervasive siloed data, standardizing and improving its quality and access, should be a precondition to having the data necessary to train algorithms for many defense uses. A logical data fabric approach that incorporates data virtualization promises to be a means for quickly collecting, processing, and using information from the DoD’s disparate data sources. It also ensures that a developed AI/ML model in a silo is still relevant to live data and can accelerate better data flow and data access across the entire AI operationalization cycle.

The more data AI/ML models receive, the more they learn creating better and more accurate predictions the DoD requires for mission critical decision-making. However, extracting data from multiple sources and then replicating it to a central repository is an old and inefficient way of getting data access. The process is still prevalent across the federal government and often results in the majority of the project time being spent on data acquisition and preparation tasks.

With the DoD touting their growing capabilities in artificial intelligence and machine learning technologies, integrating data to nourish disparate AI/ML models with their expanding data science teams is still a significant undertaking. Enterprise AI, leveraging a logical data fabric layer, overcomes these challenges. It can act as a central hub for data science teams between different AI/ML systems, reducing the need for data duplication enabling highly sophisticated AI/ML initiatives with enhanced operationalization and accelerated timetables for quicker time-to-production.

Bill Sullivan is Vice President and General Manager, US Federal at data integration and management company Denodo.

Have an opinion?

This article is an Op-Ed and as such, the opinions expressed are those of the authors. If you would like to respond, or have an editorial of your own you would like to submit, please email C4ISRNET Senior Managing Editor Cary O’Reilly.

Want more perspectives like this sent straight to you? Subscribe to get our Commentary & Opinion newsletter once a week.

In a letter to the chairmen and ranking members of the House and Senate Armed Services Committees originally reported by NextGov, the IT Alliance for Public Sector called on Congress to make reports on the Joint Enterprise Defense Infrastructure (JEDI) contract publicly available and questioned whether the DoD’s single-source intent was the best approach for the contract.

“Deployment of a single cloud conflicts with established best practices and industry trends in the commercial marketplace, as well as current law and regulation, which calls for the award of multiple task or delivery order contracts to the maximum extent practicable,” ITAPS Senior Vice President Trey Hodgkins wrote in the letter.

The Federal Acquisition Regulations system, however, gives a great deal of power to contracting officers in determining when a contract would be better suited to a sole-source award. In particular, DoD may rely on allowances in the FAR for when “projected task orders are so integrally related that only a single contractor can reasonably perform the work” and “multiple awards would not be in the best interests of the government” to justify a sole-source approach.

However, the FAR also requires contracting officers to consider “ability to maintain competition among the awardees,” in contract decision-making. Industry groups have voiced the opinion that the current formulation of the JEDI contract actively prevents competition.

The Coalition for Government Procurement said in November 2017 that the widespread needs of the JEDI contract, coupled with the intention to award the contract to a single offeror, favors a select group of companies, such as Amazon and Google, while keeping smaller providers out of the market.

According to the ITAPS letter, limiting the potential offerors to such a small group of companies will eliminate the cost-saving benefits provided by more widespread marketplace competition.

DoD officials have argued that a single award for the JEDI contract is the most effective way of meeting the department’s needs, adding that the JEDI contract only represents about 20 percent of DoD cloud needs overall.

Jay Gibson, chief management officer at the Pentagon, speaking to Federal Times’ sister publication Defense News in his first one-on-one interview since taking office, said that he hadn’t seen the ITAPS letter specifically, but the JEDI contract would be a fair and open competition.

“As you can imagine, we know this is sensitive, so we’ve been hyper sensitive as we’ve looked at it from every angle, including the legal execution for the contract and how we structured it. So we feel good that what we’re doing is the right way to do it,” said Gibson.

The last opportunity for industry feedback on the JEDI contract closed April 30, 2018, with a final solicitation expected to drop sometime in May.

Yet even if the solicitation and eventual award proceed on schedule, it is very likely that, should the award go to an industry heavy-hitter like Amazon, there will be protests from other companies in the market.

Aaron Mehta of Defense News contributed to this report from Washington.

The Pentagon issued the first draft request for proposals on March 7, 2018, after a public industry day where DoD officials emphasized the widespread changes and improvements they hope to bring about through the JEDI cloud.

The updated version of the RFP includes revised requirements and a statement of objectives for the contract, as well as the answers to industry feedback on the first draft.

The Q&A document includes 81 pages of industry questions and government responses, addressing 1,030 items ranging from issues of identity management to cybersecurity.

According to the updated statement of objectives, the JEDI cloud could potentially be used by all military departments, all DoD components, the defense intelligence community, the Office of the Secretary of Defense, the U.S. Coast Guard, the intelligence community, international allies and partners, and contractors supporting defense business and mission operations.

Industry now has a second opportunity to comment on the proposal, based on the provided government responses to past questions and the modifications to JEDI requirements. Commenters have until April 30, 2018, at 11 a.m. EST to submit their feedback.

The final solicitation for the contract is scheduled for early May 2018, with an award anticipated in September.

The question, then, does not become one of technological solutions — some of which use analytics to monitor certain cyber activity — but rather how to get “left of an event” by identifying the individual and providing a venue to act out. These venues, he said, could be as simple as an employee assistance program, an interview with someone in the security department or a peer consultation.

There are highly capable tools to track keyboard strokes and data, but it will not identify an individual that was passed up for a promotion or the individual going through a divorce or financial difficulties, Evanina said. “There is no technological monitoring that can detect that.”

He said there are three categories that are key to understanding and identifying the insider threat: narcissism; Machiavellianism (the ability or a want to manipulate others); and a callous, cold personality.

The key to success for curbing insider threats will be to marry these three categories by understanding the individual’s mindset and have robust monitoring on the individual’s systems and data, Evanina said.

The Department of Defense has previously examined and applied certain technological and behavioral analyticsto monitor threats.

INSA released a white paper outlining behavioral models that can improve the monitoring of insider threats. “Both goals — improving early warning of vulnerability and understanding individual complexity — entail not only defining psychological models but also seeking methodologies and tools that can assist in swift, continuous identification and assessment,” the white paper reads. “Most efforts to data have focused on characterizing individuals at a specific point in time — during an initial or periodic investigation — but employers now recognize the importance of leveraging innovative technology and data sources to monitor and evaluate individuals on a continuous basis.”

With the boom in social media, the report notes that leveraging certain tools can help identify certain individuals and personalities at risk for insider threats. These include personality mapping (psycholinguistics), life-event detection (text analytics) and emotion detection (sentiment analysis).

One of the main lines of effort under JIE include JRSS, which DISA’s Cyber Development Directorate director John Hickey said Jan. 12 is “still our number one cyber initiative.”

“JIE is a concept – this document supports the concept of what we want in a joint information environment. I think we’ll have to change the [term] because it’s too confusing – it’s not a program, it’s not an it. It doesn’t deliver; it’s a concept,” he told reporters in August regarding the DoD “Information Technology Environment Way Forward to Tomorrow’s Strategic Landscape,” a new plan for getting ahead in information technology. “This [document] is absolutely 100 percent of getting to a concept of a JIE, but what it’s trying to do today is lay out the discrete elements of getting to a pathway toward JIE concepts. Its 100 percent aligned with the vision of a joint information environment, but I think what we did was not explain enough that JIE was a vision and what we put out initially for JIE, the end game is the same – but the way of getting their has changed.”

Notably, Halvorsen has also pushed for the elimination of common access cards. “We are embarking on a two-year plan to eliminate [common access cards] from our information systems,” he said in June. “Frankly, CAC cards are not agile enough…It’s really hard to get you a CAC card when people are dropping mortar shells on you and you need to get into your system. That doesn’t work.”

The way forward, instead, will be hybrid authentication or a “true multi-factor” combining attributes such as biometrics, behavioral analytics and passwords.

“If I structure it right, I could build the behavior pattern of that person’s identity. We can like it or not, but one of the best ways for me to check security is to see if their behavior pattern has deviated,” he said. “So some of the things we are thinking about is some combination of behavioral, probably biometric and maybe some personal data information that is set for individuals. There are other thoughts like iris scans. All of those are doable today.”

On the Jan. 11 call Halvorsen admitted some of his other plans, such as closing hundreds of DoD data centers and upgrading military computers to Windows 10 by the end of January 2017, have not fared as well, with both efforts behind schedule.

“We’re not where I would have liked to have been but showing good progress,” he said.

Before serving as DoD CIO, Halvorsen was acting DoD CIO and is the former Department of Navy CIO.

Amber Corrin contributed to this report.

The Data.mil initiative was the brainchild of Maj. Aaron Capizzi, and was just a 20 percent project, meaning the project was done on the side, Lazzeri said. This type of Silicon Valley culture is exactly what Secretary of Defense Ash Carter has tried to foster with DDS, allowing folks from the tech sector to come into the Pentagon for a “tour of duty” to work on complex problems.

Data.mil was also stood up with a minuscule budget and went online in just three weeks. What’s so innovative about this approach, said Ian Greenleigh, head of brand at Data.world, is that typically when government agencies release open data, they stop at accessibility, meaning they don’t put the data sets into a format that can easily be analyzed. With Data.mil, the data is presented in tabular format, as opposed to KML files such as Google Earth.

DoD is also making sure “to provide an environment that adds context so that the data can easily be understood, which is often lacking with open data programs around the world where it’s just here’s a file, no additional information; it doesn’t tell you where it’s from or how it’s been analyzed or used in the past,” he added. “What I really admire about this particular effort with THOR is that all these issues we see with open data generally have been taken care of with a single effort with almost no budget. That’s pretty incredible.”

Greenleigh noted how the data in the THOR project present a unique way to tell stories. Data scientists affiliated with the effort began looking at historical events in the theaters of war that could be associated with the data. For example, within the Vietnam data set is Sen. John McCain’s downed flight that led to his capture as a prisoner of war.

“That is extraordinary,” Greenleigh said. “I’ve never had this experience with data before, to know a story and look for it in a massive data set and be able to see all the aspects represented in front of you.”

In terms of actionable information and usefulness from the Pentagon’s perspective, Lazzeri said some of the next steps are to release more strategic data such as acquisition, logistics, procurement, budget and manufacturing data, to name a few.

“There’s a number of data sets that will be more useful throughout the military to be able to take a look at the raw data in certain areas, and that will help them solve some of those complex problems,” she said. The data could also be used from an education and planning perspective when viewing how targeting evolved from World War II to through precision munitions and to today; the Pentagon has tried to take more extraordinary measure to protect civilians, even amending its Law of War Manual to afford greater protection.

From a non-military perspective, Greenleigh said researchers could study weather patterns dating back to World War I, given that robust records of weather are necessary when flying.

“That was something that was personally interesting for me was looking at all the weather events across these theaters of war,” he said.

This initial data set was really experimental, Lazzeri said, to see if DDS could spark interest in open data.

“We don’t have a budget. DDS is not looking to launch a robust open data program. We’re hoping there are other areas where we can feed this info into parts of DoD and the services to help them put their data online,” she said. “We put it out there without a plan for how it will be used with the notion that researchers can take it where ever it can go.”

Stringent requirements are also in place to secure the database from potential intrusions. Data can only be changed by contributors, and there are strict standards even for that, Greenleigh said. Greenleigh also expressed doubt regarding the possibility of the data being manipulated or used for malicious purposes. The data links back directly to the source and lives on Data.mil as well as Data.world, he said. That means if data was ever to be changed and rereleased, it can be compared to the original source.

DoD said DDS will work with various components within the department and external government agencies in a consultative role as to advise the execution of future bug bounty programs.

“The department’s choice of cyber and IT capabilities lay the foundation for success – from the battlefield, to business, and beyond,” the document notes. “Today the number of organization-specific networks and computing systems used to execute missions, and the incremental manner in which IT is acquired, has resulted in a sub-optimal situation. The unnecessary complexity of this network and computing environment limits visibility and impedes the capability to securely share information and globally execute joint operations.”

About that ‘joint’ thing

Until recently, almost any and all talk related to DoD-wide IT hinged on or stemmed from the Joint Information Environment, the Pentagon’s long-continuing, sweeping effort to standardize IT across the military and create a shared operational network picture.

The new way forward includes JIE, but in a divergence from the path of recent years, it does not center on JIE. Goal No. 1 calls to “Execute Joint Information Environment Capability Initiatives,” and an introductory explanation of the document promises that “Implementing capabilities for the JIE vision, including cloud computing, data center consolidation and improving trusted information sharing, are vital.”

But notably, JIE is not front and center. JIE never had a seven-page, publicly released document outlining its core goals and way forward. And, perhaps not coincidentally, JIE is facing heat from Congress due to questions about funding and the fact that it’s not a program of record, among other troubles.

Halvorsen admitted JIE is presenting some hurdles as he looks to move the ball forward in rapidly evolving areas of technology and in instituting change across DoD. He also took some of the blame for that as the top official shepherding the effort for the past two years.

“JIE is a concept – this document supports the concept of what we want in a joint information environment. I think we’ll have to change the [term] because it’s too confusing – it’s not a program, it’s not an it. It doesn’t deliver; it’s a concept,” he said. “This [document] is absolutely 100 percent of getting to a concept of a JIE, but what it’s trying to do today is lay out the discrete elements of getting to a pathway toward JIE concepts. Its 100 percent aligned with the vision of a joint information environment, but I think what we did was not explain enough that JIE was a vision and what we put out initially for JIE, the end game is the same – but the way of getting their has changed.”

DISA will oversee the secure connections between commercial cloud service providers and DoD cloud consumers and the DoD network, manage security standards for DoD cloud use and integrate cloud services with the DoD Information Network (DoDIN).

“DISA is going to be moving away from participation as cloud broker and in cloud services, with more focus on providing security guidelines to include security reference models, the basis in determining costs and the types of applications that are candidates for cloud services,” said Alfred Rivera, DISA acting director of strategic planning and information. “I think we’re going to continue to play very big role from the cloud broker perspective in that respect as cloud server provider, [and] also be a vehicle for network access to cloud service providers [that is] available, secure and reliable. Those two elements are still going to be germane to DISA’s responsibility.”

DISA’s role in DoD cloud services continues to evolve as agency officials await the final release of a new directive from the DoD CIO office that will formally outline new cloud policies for the military. DoD CIO Terry Halvorsen last month said the new memo, expected to be released Oct. 31, will grant the military departments greater authority to buy commercial cloud services in a bid to speed adoption within DoD.

“We’re still maintaining security requirements, provisioning authorizations [and] designing the architecture for cloud access points. Those key roles, Mr. Halvorsen still expects us to support,” said Dave Mihelcic, DISA chief technology officer. “If you look at the changes, it’s just leveraging the military departments’ capabilities to do some of the acquisition work so that together we can accelerate the pace of adoption of commercial cloud.”

The new memo still requires the departments to conduct thorough business case analyses that consider DISA cloud services, and DISA’s internal services, such as DoD’s internal milCloud, still will need to be used for sensitive or classified data that cannot be housed in the commercial or public cloud space.