A substantial amount of money and human and technical resources are being applied to the collection and analysis of data these days. If you think about all the digital data created, collected, stored, analyzed and used in a single year, you can begin to see the challenge of intelligence collection and analysis these days. Stop for a moment and consider the huge number of sources of intelligence that exist worldwide. Think of all the images and the geospatial information that is out there these days. Now consider how many of those sources are digital and public. Once you do that you get a feel for just how large the data universe actually is today.<\/p>\n\n\n\n
Some industry analysts predict data will grow to will grow to 40 trillion gigabytes in 2020, more than 5,200 gigabytes for every man, woman, and child on the face of the earth in 2020. Another study suggests that the data will reach the zettabyte level in 2020. If you are not familiar with the term zettabyte, you are not alone. It’s 1021. That is a lot of zeroes.<\/p>\n\n\n\n
This is such a vast amount of data that some believe it is essential to divide it up. On the other hand, though, others ask how dividing the data would enable a full big-picture view. Breaking up the data and analysis, they argue, increases the likelihood something will fall thru the cracks. Still others are quick to point out that using that technique could create significant issues give the global networks that exist today that are of national security interest.<\/p>\n\n\n\n
Stop and think of all the data that will be added as we begin receiving data from the estimated 200 billion Internet of Things devices that have already started to connect. Will the tools and techniques we use today \u2014 and even those now being built for the near future \u2014 fall short of what is needed? Our current database approach will not effectively or efficiently scale to the level we need in the next five to seven year. This is yet another area of concern that must be addressed \u2013 now!<\/p>\n","protected":false},"excerpt":{"rendered":"
Debate heats up over how to handle massive data.<\/p>\n","protected":false},"author":7,"featured_media":25506,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_yoast_wpseo_meta-robots-noindex":"","_yoast_wpseo_meta-robots-nofollow":"","_yoast_wpseo_canonical":"","_acf":"","_yoast_wpseo_primary_category":130,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","_smg_distribution_targets":[]},"categories":[16,130,24,58],"tags":[],"coauthors":[858,888],"class_list":["post-21451","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-it-networks","category-net-defense-blogs","category-opinion","category-the-compass"],"acf":{"subheadline":"","legacy_arc_id":"HEEEXAU3NZGLPOCQMAYSVOK2AA","arc_canonical_url":"\/opinion\/the-compass\/net-defense-blogs\/2016\/04\/11\/drowning-in-data\/","remove_feature_photo":false,"is_sponsored":false,"subtype":"","redirect_url":"","disable_inline_ads":false,"native_logo_pretext":"Presented By:"},"yoast_head":"\n