Hiding in Plain Sight

Matthew Redhead discusses tapping into the power of Open-Source Intelligence 

We tend to assume that ‘intelligence’, by definition, is something covert, or secret. Ken Robertson, an intelligence historian, has defined intelligence work as “the secret collection of other people’s secrets.” This traditional definition presupposes that if something is not hidden, then it cannot be intelligence.  

But although the clandestine remains the core of what most intelligence agencies do, the modern information age has radically widened the potential scope of where intelligence might emerge. With rapidly escalating amounts of data available through the Internet and other open sources, valuable insights are often hiding in plain sight. The key is knowing where, when and how to look. 

The use of this kind of ‘Open-Source Intelligence’ (or ‘OSINT’) has evolved rapidly in the last two decades, as technology has developed more efficient ways to store, analyse and access vast amounts of disparate material, especially through the use of networked data storage, or ‘Cloud computing.’ As a result, OSINT is increasingly helping organisations to understand and manage the risks they face. 

In the commercial world, multinational companies and financial institutions already use OSINT to make strategic decisions about market presence, or inform client relationships. The variety of open-source material now available has also encouraged the development of ‘citizen journalism’, with websites such as Bellingcat using OSINT to conduct detailed investigations of events such as the downing of the Malaysian Airlines plane MH17 in July 2014. 

Government agencies have exploited OSINT to identify and track trends in economies and societies too, including the emergence and spread of public health threats such as the COVID-19 virus. Open-source material has also made an entrance into the more secretive worlds of the military, law enforcement and intelligence. Since 9/11, for example, the US intelligence community has emphasised the need to use OSINT alongside human and signals intelligence. Not only does open-source material have value in its own right, but when combined with secret data can dramatically enhance insight.  

But if OSINT is being more widely deployed in the secret world, challenges to its full acceptance remain. Much of this opposition is cultural and psychological; there is a natural tendency to privilege hard to obtain information, especially in organisations specifically designed to collect it. 

Yet the problems with OSINT have not all been ‘in the mind’ of traditionalist intelligence professionals. Agencies have faced genuine problems with the practical deployment of open-source material in the intelligence cycle. Due to security protocols, agencies have been cautious about allowing wide access to open systems. Even where this has been granted, moreover, it has proved difficult for even experienced intelligence officers to navigate the sprawling but often siloed world of the Internet, leading to well-grounded concerns about the potential for information overload, missed leads or carefully planned deception. 

But in reality, none of these problems is insurmountable, especially when OSINT specialists who know both the intelligence world as well as the potential of technology are involved. In these circumstances, the delivery of open-source material – often a cumbersome process in the past – can be reshaped to more effectively meet agencies’ needs. Whereas the term ‘OSINT collection’ once implied an undirected search via an Internet search engine, dedicated platforms can now tap directly into the richest material available, aligned to an agency’s priorities, and present it in an investigator-friendly and secure environment. In other words – no more ‘drinking from the fire hydrant,’ or indeed – drinking from someone else’s hydrant. 

The best modern OSINT platforms carefully curate their data sources to filter out not only the unnecessary, but also the questionable. Such tools are shaped around the realities of investigation too, using visualisation to present material with impact, and case management functionality to provide a structure for analysis and later audit. First and foremost, they are specifically designed to help intelligence professionals – not just data scientists – use OSINT as a mature stream of intelligence in its own right, but one that can be fully integrated with other more traditional sources.  

Intelligence, law enforcement and other investigative agencies thus have the opportunity now to leverage the value of OSINT – a value much promised but less delivered in the past – using reliable data sources and the best in modern technology. But agencies need to approach potential solutions with care. Some platforms offer impressive Social Network Analysis technology but without appropriately curated data, while major data providers can provide good material in less user-friendly ways. The key therefore in implementing an optimal solution is not only choosing the right platform and the right data, but working with a partner that has an acute understanding howto tap into the power of OSINT. 

By Blackdot Solutions