The intelligence community continues to have an open-source intelligence (OSINT) problem. While capturing unclassified and openly available information is easier than ever, turning it into actionable intelligence remains one of the largest challenges the community faces. Just imagine what information is on Twitter — not people posting photos of their cats, but live recordings of events that can be impactful to policymakers. In fact, the recent video recording of Afghans trying to hold onto a U.S. military transport plane as it tried to take off is a good example of this. There is an immense amount of open-source data, but the intelligence community still holds onto its “tried and true” classified data sets.
Moreover, building trust in OSINT cannot be a sideshow. The intelligence community must effectively harness open-source information to maintain relevance in a rapidly evolving national security environment, which includes risks from great-power competition to global pandemics.
The key going forward is to reduce the time spent on discovery and increase the focus on interpreting the “so what” of the information collected. But despite recommendations for using commercial off-the-shelf tools and supporting tradecraft methodology for processing open-source data, the U.S. Intelligence Community has continued to expand efforts and policies directed toward more expensive, and at times riskier, intelligence sources.
This over-reliance on classified intelligence collection not only increases costs, it also creates information blind spots and is harder to share.
Understanding that most classified repositories are necessarily stove-piped, always costly, and inevitably limited to their specific collection capability and tasking, decreases the speed to discovery, impedes information-sharing, and hinders development of the important common understanding of emerging challenges. From social media to the rise in commercial satellite imagery, OSINT offers fast, global and shareable insights across a broad range of intelligence activities that are of interest to national security organizations.
Notably, concerns about trusting OSINT ignore the fact that the criteria used to determine veracity of information remain the same whether collected from classified or unclassified sources. Timeliness, relevancy, accuracy and currency of sources and the intelligence derived from them is how organizations measure their trust, as well as the operational utility of both classified and unclassified information. At the very least, OSINT can and should serve as the forward reconnaissance of analysis, whereby emerging new information can drive collection from a variety of classified sensors, leading to a more complete, all-source analysis tradecraft.
The community continues to generally ignore data easily obtained from trusted and verifiable open-source providers. This is partially because of the increase in advanced technological means of gathering information, mistaking this as open-source intelligence, and the inevitable errors this creates downstream. But more troubling, this is also because of a lack of understanding or protocols to know how to recognize and trust what is reliable open-source information.
Commercial and off-the-shelf information solutions continue to expand, while simultaneously increasing fidelity, and new sources of data and dissemination outlets are created daily. This is creating a challenge to classified collection systems and their ability to keep pace. While trust in sourcing historically has been placed on classified assets, the need for trusted OSINT continues to rise in prominence given its unexploited potential today to serve, if for nothing else, as forward reconnaissance for analytic tipping and cueing, as well as global indications and warning.
One of the most frequently quoted reasons for not engaging with open-source information is the understandable concerns about the “quality” of the expanding volume of highly diverse, publicly available content that is changing rapidly (volume, variety and velocity of OSINF) and, of course, the veracity of what is collected. Intelligence agencies, militaries and security organizations are experiencing an overload of data collected across systems and sensors — mostly unverified and in a variety of formats, structured and unstructured, and not easily integrated, shared and discovered. The result is that critical information remains unprocessed and siloed with a lack of ability to integrate related data sets within one desired system, creating gaps in knowledge generation and time lags to operational actualization.
Open-source intelligence is arguably at the apex of transforming how intelligence analysis is conducted within national security organizations. Yet it remains both underutilized and suspect as a credible source of information. While championed and touted as a resource for real-time indications and warnings, cost effective information gathering, and being more timely and readily available, analysts continue to shy away from exploiting this platform to its fullest extent. While many senior national security leaders agree analysts should be using more open-source information, trusting this resource remains a critical point of apprehension.
This apprehension may well be valid amid disinformation campaigns and fake news concerns. Misinformation, disinformation and automated bots pumping out contradictory information across social media can pollute the data pool from which open-source information is gathered. However, the use of artificial intelligence and rapid data analytics can mitigate these risks by tipping expert analysts on changes in key information, enabling the rapid identification of apparent “outliers” and pattern anomalies. Such human-machine teaming exploits the strengths of both and offers a path to understanding and even protocols for how trusted open-source intelligence can be created by employing traditional tradecraft of verifying and validating sourcing prior to making the intelligence insights available for broad consumption.
When OSINT is collected from trusted and verifiable open-source providers, analysts can spend more time focusing on analyzing data versus hunting for it. Classified resources then can be used to cross-check the connections tipped and cued from OSINT, further reducing the time from first touch of information to developing the “so what” insights of the intelligence picture.
Harry Kemsley, OBE, is president of national security and government at Janes. Prior to joining Janes, he spent 25 years in the Royal Air Force, where he took command of No. 5 (Army Cooperation) Squadron, with over 350 men and women from all three services, and led the squadron’s inaugural operational deployment into Afghanistan.