The Future of OSINT

Over the course of my career, I have watched the world of open source intelligence (OSINT) grow. Techniques have developed immensely: investigators who previously spent hours trawling online forums can now search across surface, deep and dark web in a matter of seconds. With the market value of OSINT technology having passed $5 billion in 2022, it is clear that investigators and analysts are relying on this methodology more and more for a range of investigation types: from due diligence, to anti-financial crime, to serious and organised crime.

At Blackdot’s recent ‘Future of OSINT’ event, I had the opportunity to talk about the current state of the OSINT industry and how our upcoming flagship solution, Videris 5, will meet the challenges that we are likely to face.

The following article is a summary of my presentation. You can view the full recording of the talk, as well as an expert panel discussion led by our Head of Community, Charles Brown, here.

Developments in OSINT

Integration of OSINT with other data sources – such as internal and curated data – is one of the biggest trends I’ve seen in recent years. Combining data types adds much-needed context to every investigation, helping investigators to get more value from their work. This is being recognised more and more in government and law enforcement, the financial services sector and beyond. 

In fact, the UK government has recently developed a new initiative, INDEX, specifically to centralise Open Source Intelligence assessments across government. 

With this expansion in use and growing recognition of its value, we are likely to see further developments across the OSINT industry:

• Automation, AI, and machine learning are going to become even more prevalent. Every day, investigators and analysts are performing time-consuming manual tasks (such as sanctions list searching) that could be done to a higher standard. Businesses are realising that these tasks can be done far more efficiently and accurately through automated software. For example, when a sanctioned entity is identified, a playbook of tasks can be applied instantly. This could include automatically identifying connected corporate entities and searching live internet data for associated risks. The analyst can then simply review this data, putting their skills to better use and improving the overall quality of a check that may otherwise have been limited due to time constraints.
• Monitoring and alerting are going to be used in new and different ways as data volumes continue to grow. Alerting is already commonplace in some types of investigations, such as anti-financial crime, but the speed at which data is changing will mean that investigators from all sectors need to be updated whenever something changes. For example, if an investigator is alerted to a new tweet pertaining to a subject of investigation, the entire course of their investigation could change instantly. Without monitoring and alerting, this key information could be missed or only spotted once the investigation is almost complete. 
• More regulations around encryption and data privacy will be put in place. Growing concerns around cybercrime and demands for increased privacy from the general public are likely to incentivise governments and regulatory bodies to take action. OSINT investigators must therefore make sure that investigations capture value without being unlawfully intrusive. As we embrace more automation, we must ensure that it remains explainable and that data collection is proportionate, in order to ensure that no ethical boundaries are breached.
• Advanced analytics and visualisation will be needed to cope with the increased demand to make sense of open source data. By its nature, OSINT makes sense of unstructured and networked data, and technologies that support the decoding of this data will continue to develop. Investigators will be able to uncover more insights than ever before using network analytics.

Challenges of OSINT

OSINT investigators face a number of challenges when conducting investigations, mostly concerned with the sheer volume of data available. As OSINT becomes even more popular, technology needs to keep up with – and help overcome – these challenges. 

Reliance on search

Like much of the general public in everyday life, OSINT investigators are heavily reliant on manual internet searches. However, in the case of the investigator, searching is often not the most effective way of answering a question. Not only does a search sometimes take a long time to set up (considering which sources to interrogate, selecting search terms and employing boolean operators can be a fiddly process), the resultant search can return vast amounts of data that lack relevance to the entity being investigated. 

Investigators are also challenged with having to make sure that the sources they are drawing from are reliable, in order to guarantee high-quality results and accurate decision-making.

Suggested reading: You can learn more about how reliable OSD is in this article.

Time

As demand for OSINT increases, OSINT investigators are being pulled in every direction. They need solutions that allow them to work more efficiently and improve accuracy, all without sacrificing quality. 

Furthermore, this increased demand puts additional pressure on investigations teams to learn and implement more processes. As OSINT technologies and processes develop, teams are continually required to take time out of their work weeks to adapt to these changes. Ultimately, this results in less time to do actual investigation.

Complex problems need simpler solutions

As new investigative challenges emerge, so do new solutions. However, this isn’t always especially helpful: are investigators’ lives truly enhanced by using a separate solution for every step of the investigative process? Not only do organisations have to consider the cost of maintenance for multiple solutions, they also have to be certain that the data each provides is accurate and up-to-date. Ultimately, solutions intended to simplify the investigative process can often end up adding complexity.  

Overcoming challenges with Videris 5

Since 2016, Blackdot has worked with investigations teams to help them overcome common challenges through technology. We are pleased to announce the next step in this journey: the evolution of our OSINT tool, Videris.

Videris 5 offers new features and a new UI, built through customer and industry feedback, providing a simple and seamless user experience, and even better ROI to organisations seeking efficiency gains. Below are just a few of its features and benefits.

A new, advanced risk screening tool

We have created a new search solution aimed at removing the friction from searching processes that should be simple, such as screening, 

Videris Risk Search enables investigators to overcome the challenges we’ve just discussed:

• Search across more data, more accurately, without investing more time
  Each search includes live internet data from the surface and deep webs, as well as PEP and sanctions lists. AI-driven analysis allows Videris to select key results for human review, avoiding the complications and time-wasting usually associated with reviewing high volumes of data.

• Make search faster and more relevant with improved contextual searching
  Tailor searches with parameters (such as age, nationality, etc) and in multiple languages so that you can be sure that the information returned is relevant to the entity that you are investigating. 

• Automated risk scoring and analysis
  Videris Risk Search automatically creates a risk score based on the data it collects. The entity’s score includes a full analysis of potential risks – preconfigured according to an organisation’s preferences – so that each case can be accurately triaged in seconds, without the risk of missing key data.

• Explainable AI and automation
  We have partnered with Oxide AI to offer a fully explainable AI-driven solution. Investigators and analysts can provide step-by-step reasoning behind their decisions, whilst benefiting from the significant speed and accuracy gains AI provides.

By removing the most manual aspects of search, screening and investigations teams are better able to dedicate the appropriate amount of time to investigation and analysis – all based on real risks, not false positives.

A single pane of glass

The Videris 5 interface was developed alongside real investigators to ensure that it accurately reflects their workflows and meets their needs. Crucially, because it’s so easy to use, Videris 5 will help every investigator to work faster and more effectively.

Videris 5 is a single platform for every part of an investigator’s workflow. For the first time, it’s available through a web browser, further promoting ease of access. This means that an entire investigations team can access all the OSINT tools they need from one place without having to manually switch programmes. With everyone using the same software, consistency and diligence are promoted without any compromise on quality.

By using a single platform, organisations reduce the need to maintain multiple tools. Organisations wishing to integrate Videris with case management systems or similar can do so using an API, without the need to replace existing technology. In other words, they can seamlessly integrate Videris into their existing workflows.

Rapid escalation

No matter how unique or complex an entity may be, Videris is designed to make connections and information easily consumable.

Videris 5 elevates the entire investigative process by allowing investigators to create more complex cases directly from simple screening searches. This avoids duplication of effort when cases are passed between teams.

For due diligence and compliance processes to be valid, it is essential that data and procedures are clearly presented and explainable to regulators and governmental authorities. Videris 5 is designed to simplify and speed up this reporting, retaining sourcing at every stage of an investigation and making each step easy to explain and present.

Key takeaways

OSINT is only going to grow in importance as it’s included in more types of investigations, across more sectors. This, combined with the ever larger volumes of data available online, is likely to create new trends:

• Monitoring and alerting across more types of investigations
• Increased data privacy regulations.
• A need for better tools to search and organise the vast amounts of OSD available in an efficient manner.

Videris 5 is set to take on these challenges by centralising all crucial OSINT tools, so that every OSINT investigation can happen in less time. It is Blackdot’s mission to help break down the complexities of OSINT and bring OSD into focus so that risk can be identified and acted upon.

Get in contact today to see how Videris can elevate your investigations.

Stuart is an internationally respected cybersecurity expert who is responsible for the product and security strategy at Blackdot Solutions. Stuart also has consultancy experience across the areas of digital forensic, cybersecurity and eDiscovery, and has advised the United Nations’ peak cybersecurity body ITU providing cybersecurity training for over 60 national CERTs. You can follow him on LinkedIn here.