5 Advanced Open Source Investigation Techniques

The proliferation of open source data is transforming the world of investigations. However, generating meaningful insights from large volumes of publicly available data is a huge challenge for today’s investigators. 

What is OSINT? 

Open source intelligence is the analysis of open source data (OSD) to produce meaningful insight. With advanced technologies and new techniques, investigators can create intelligence from data. This article will explain what these techniques are and how the right OSINT tools help you to utilise them.

Suggested reading: There are a number of best practices you need to consider to achieve effective investigations using OSINT. Read our deep dive into open source investigation best practices to find out more.

Technique # 1: Integrate internal and external databases

Internal data is information created within a business. External data can come from numerous other sources, including news media, search engines, social media platforms and corporate records data. 

It’s important to make the most of both types of data to ensure that no potential intelligence is being missed. Investigations tools that integrate – and find links between – external and internal data are key in achieving this. The insights derived from combining these information sources enable organisations to resolve complex problems, ranging from financial crime, to due diligence, to fraud and more.

Fundamentally, it’s vital to centralise investigation processes. Multiple data sources and intelligence platforms create inefficiency and increase the risks of data loss or exposures that reduce effectiveness. Investigators need the ability to gather and analyse data across a comprehensive landscape in order to advance decision-making and actions.

Technique #2: Use automation and AI to improve decision-making

Undertaking open source investigations manually is unrealistic for many organisations — there is simply too much data. Technology is key in solving this problem.

However, fully-automated or AI-based decision-making cannot match the expertise and nuance that experienced investigators bring. Automating decisions as impactful as those that occur as a result of OSINT investigations can raise ethical concerns.

It’s important to balance the efficiency gains automation brings with the need for accuracy. Using technology to automate repetitive tasks to improve and accelerate human decision-making is one way of achieving this. For example, technology can provide:

• Intelligent automation of manual processes, such as the gathering and mapping of data
• Automated risk flagging
• Automated flagging of similarities across data types
• Easy to use visualisation capabilities
• Audit tools such as logging and sourcing

Platforms like Videris make it possible to automate time-consuming processes while still leaving experienced investigators in charge, augmenting, rather than replacing, human decision-making. 

Technique #3: Safeguard investigation security

OSINT, like most forms of intelligence, is typically used to identify and manage risk. But compromises to the security of an investigation can hinder these objectives. Investigators should be especially wary of revealing their identities or tipping off their subject, which can undermine an investigation. Visiting unknown websites – especially dark web content – also opens investigators to the risk of malware.

Examples of this could be:

• A bank is investigating a customer’s source of funds. It is vital that the investigator doesn’t reveal that this investigation is taking place to avoid reputational damage.
• A government investigator is carrying out an organised crime investigation. They must take extra precautions to ensure that none of the suspects at the centre of the investigation are tipped off. 

Compliance with GDPR is also a strong concern with OSINT: poor data handling can lead to regulatory repercussions. For example, hoarding the open source data you’ve collected for OSINT when not all of it will be relevant to your investigation may breach GDPR regulations. What’s more, the data you have collected that is relevant must be stored securely to adhere to GDPR.

Building an effective OSINT ecosystem

Investigative integrity is at significant risk if investigations aren’t deployed through an OSINT platform that prioritises security. Implementation of centralised data repositories and IP address security measures are amongst the features organisations should look for when choosing a platform.

Technique # 4: Align your investigations to the Intelligence Cycle

The Intelligence Cycle is a process designed to help investigators focus on transforming data into actionable insights. The four stages of the Intelligence Cycle play a key part in open source investigation best practices by answering the following fundamental questions:

1. Direction: What is the problem and why is intelligence necessary?
2. Collection: What information is needed and where should it be sourced from?
3. Processing: How can the information be presented so that it can be analysed easily?
4. Analysis: What does that information/intelligence mean?
5. Dissemination: How can that information be presented to the people who can put it into action?

When dealing with vast amounts of publicly available data, deriving insights can be challenging. You can quickly become lost in the volumes of data available and irrelevant data collection is almost inevitable. By truly understanding the objectives of an investigation before actually handling the data, investigators can ensure more relevant outcomes.  

Technique #5: Always investigate ethically

Mass surveillance and indiscriminate data hoarding can place your organisation at risk of non-compliance and reputational damage. OSD might be publicly available information, but it still covers personal data that is subject to data privacy regulations in varying degrees.

To ensure ethical OSD usage, organisations must target their data handling to only that which may yield valuable insights. Effective OSINT solutions will offer a simple means of saving sourcing information and collated data. This allows for easy referencing and verification that your investigations are ethical.

By simplifying and streamlining access to open source data, OSINT solutions provide an opportunity to gain insights that might otherwise only be available through more expansive and intrusive forms of intelligence collection.

As excessive automation with no human involvement can lead to incorrect and unethical decisions, it’s crucial to prioritise an approach that leaves humans in charge of OSINT decision-making. Such an approach minimises the risk of poor and potentially unethical decisions. 

The following factors have an impact on the extent to which investigations can be considered ethical: 

• Targeted searches: collecting only the information that is relevant to your investigation.
• Legitimate use cases: collecting data to prevent crime, protect national security or meet regulatory requirements is considered more ethical than collection for marketing or sales purposes.
• Only public data: using data that is freely available online is easily explainable.

Deploying the power of an OSINT platform

The growth of open source data has the potential to improve the effectiveness of investigations in both the public and private sphere. Yet it also leaves companies at risk of drowning in data they can’t make sense of. Advanced investigation techniques, implemented with the help of the right third party platform, are the best way through. 

At Blackdot, we developed Videris to address this problem by providing the intelligent automation necessary to streamline OSINT practices, while always guaranteeing human-led decisions that keep investigations firmly between the lines of an ethical and targeted approach.

With extensive experience in government applications as well as in the private sector, we’ve developed Videris to bring advanced open source investigation techniques to all users, regardless of their level of familiarity with OSINT. Videris does this through a range of capabilities, including:

• Targeted search functions: The ability to highlight search terms or search keywords across multiple public data sets at the same time with Videris Search ensures that investigations, and the data used within them, are always targeted towards outcomes.
• Automatic analysis: Intelligent automation, AI and natural language processing make it easier to sieve through expansive data sets, bringing any valuable findings back to the analysts who can turn them into action.
• Visualised risk: Visualisations make it easier to draw connections across illicit networks and understand all risks, including those that may inform further investigations.
• Combined findings: Videris acts as a single pane of glass, allowing you to combine all your findings in one easily accessible, secure platform to prevent exposure or data breaches, as well as simplifying the overall investigation process.
• Secure searches: The ability to easily anonymise investigations provides security that ensures the integrity and effectiveness of your investigations.
• Data and source capture: By capturing data sourcing as you work, Videris ensures that you’re able to export sources directly into the reports that drive decisions based on your findings. This allows organisations to keep an immutable record of their findings, regardless of whether the evidence is taken down after the fact. 

These capabilities let investigators work at an advanced level to achieve better outcomes. 

Ready to utilise these advanced techniques and take your investigations to the next level? Book a demo to find out exactly how we can help you.

FAQs

What is OSINT? 

OSINT (Open source intelligence) is the application of intelligence gathering techniques and technology to investigations that make use of open source data (OSD).

How can OSINT be used in investigations?

OSINT can be used in investigations to identify and manage risks. Advanced OSINT investigation solutions can be used to find key risks and map out connections across vast amounts of online data . 

What are open source investigation techniques?

Open source investigation is the process of using information from open source data (OSD) data sources to support research, analysis and decision-making. Advanced techniques for open source investigation include:

1. Integrating internal and external databases
2. Harnessing intelligent automation and AI
3. Safeguarding the security of your investigations
4. Using the Intelligence Cycle
5. Targeting investigations to ensure an ethical approach

What sectors benefit from using OSINT?

OSINT is widely used in various investigative fields such as financial services, large corporations, government and law enforcement, and consultancies.

How can Blackdot Videris be used in open source investigations?

Blackdot’s Videris is a comprehensive solution that streamlines the OSINT investigations process. It provides:

• Targeted searches across surface, deep and dark web
• Automatic analysis
• Visualised risk
• Secure live internet search
• Data and source capture