5 Advanced Open Source Investigation Techniques

The proliferation of open source data – found across social media posts, self-published blogs, and other open source repositories – is radically transforming the world of investigations. However, generating meaningful insights from these volumes of publicly available data is the largest challenge facing effective investigations today. 

What is OSINT? 

Open source intelligence is the application of intelligence gathering techniques and technology to investigations that make use of open source data (OSD). When done well, it is the key to managing OSD and using it to drive tangible investigatory outcomes. Advanced technologies and investigative techniques help to differentiate intelligence from data, and draw meaningful connections across investigative landscapes. 

Here, we’re going to look at the advanced techniques that have driven our technology development and can be used to improve outcomes in any context. This article will help you to understand what these techniques are, and how the right OSINT source tools help you to utilise them.

Suggested reading: There are a number of best practices you need to consider before you can start performing effective investigations using OSINT. Read our deep dive into open source investigation best practices to find out more.

Technique # 1: Integrate internal and external databases

Internal data is information created within a business. External data can come from numerous other sources, including news media, search engines, social media platforms and corporate records data. 

It’s important to make the most of both types of data to ensure that no potential intelligence is being missed. Investigations tools that integrate external and internal databases are key in achieving this, as they allow investigators to recognise hidden connections by highlighting suspicious behaviours across disparate data sets.

As more people choose to share their lives through social media, Social Media intelligence (SOCMINT) becomes increasingly important. SOCMINT involves intelligence across two data categories, which are —

1. Original content: Facebook updates/uploaded images/video uploads.
2. Metadata associated with original content: Multimedia/geo-location/date/time.

When integrated with other types of publicly available information, or OSD, the insights derived from these information sources enable organisations to resolve complex problems, ranging from financial crime, to due diligence, to fraud and more. For example, the use of external intelligence such as SOCMINT can allow organisations to identify networks of actors wanting to harm, steal or defraud the organisation. 

Fundamentally, it’s vital to centralise your investigation processes. Multiple data sources and intelligence platforms create inefficiency and increase the risks of data loss or exposures that reduce effectiveness. Investigators need the ability to gather and analyse data across a comprehensive landscape in order to advance decision-making and actions.

Technique #2: Harness intelligent automation to improve decision-making

It’s often not realistic to undertake open source investigations manually — there is simply too much data. 

However, fully-automated and AI-based decision-making cannot match the expertise and nuance that experienced investigators bring to the table. 

Moreover, there are legitimate ethical concerns when it comes to automating decisions as impactful as those that occur within the course of most OSINT investigations.

The approach we’ve embraced at Blackdot Solutions is Intelligent Automation, which automates repetitive tasks in order to improve and accelerate human decision-making. 

This is done through the use of: 

• Intelligent automation of manual processes, such as the gathering and mapping of data
• Automated red-flagging
• Cross-matching technology
• Easy to use visualisation capabilities
• Prioritised human oversight and reporting

An intelligent automation platform makes it possible to automate time-consuming processes while still leaving experienced investigators in charge. augmenting, rather than replacing, human decision-making. 

Technique #3: Safeguarding the security of your investigations 

Compliance with GDPR is a strong concern with OSINT. OSINT, like most forms of intelligence, is typically used to identify and manage risks. But if there are any compromises to the security of an investigation, it could prevent these objectives from being realised and even create more threat entry-points than they can address.

The poor handling of OSINT can also lead to your company facing regulatory repercussions. For example, hoarding the open source data you’ve collected for OSINT when not all of it will be relevant to your investigation breaches GDPR regulations. What’s more, the data you have collected that is relevant needs to be safely and securely stored to adhere to GDPR.

Another security threat that investigative professionals should be wary of is revealing investigators’ identities or tipping off the subject of an investigation, which will be counter-productive to the development of valuable insights.

Examples of this could be:

• A bank is investigating a customer’s source of funds, it is vital that the investigator doesn’t reveal that this investigation is taking place. 
• If a government investigator is carrying out an organised crime investigation, they must take extra precautions to ensure that none of the suspects at the centre of the investigation are tipped off. 

If any of the above do happen, these investigators are at risk of compromising the investigation by causing the subject to further conceal their assets – or of cost to safety, reputational damage amongst high-value clients, and resulting loss of business.

Effective OSINT investigation ecosystems and efficient threat identification should be part of all investigations. However, investigative integrity is still at significant risk if these processes aren’t deployed through an OSINT platform that prioritises security. This can be realised through the implementation of centralised data repositories, IP address security, direct exports, and flexible deployments either on-premises or in the cloud among other priorities.

Technique # 4: Align your investigations to the Intelligence Cycle

The Intelligence Cycle is a process designed to help investigators focus on transforming data into actionable insights. The four stages of the Intelligence Cycle play a key part in open source investigation best practices by answering the following fundamental questions:

1. Direction: What is the problem and why is intelligence necessary?
2. Collection: What information is needed and where should it be sourced from?
3. Analysis: What does that information/intelligence mean?
4. Reporting and dissemination: How can that information be presented to the people who can put it into action?

When dealing with vast amounts of publicly available data, deriving insights can be challenging. You can quickly become lost in the volumes of data available, and irrelevant data collection is inevitable if context isn’t considered throughout the investigation. By accounting for the objectives, utilisation, and outcomes of an investigation before actually handling the data, investigators can ensure the relevance of their outcomes and the final intelligence product. 

When paired with an OSINT tool that facilitates filtering and focus, the direction and control of the Intelligence Cycle makes it possible to ensure intelligent outcomes. 

Technique #5: Always investigate ethically

Mass surveillance and indiscriminate data hoarding can place your organisation at risk of non-compliance and reputational damage. OSD might be publicly available information, but it still covers personal data that is subject to data privacy regulations in varying degrees. You need to consider legal, financial and PR components, along with their associated risks. 

Ethical OSD usage depends on an organisation’s ability, and desire, to target their data handling towards specified objectives that limit the flow of information to only valuable insights. What’s more, effective solutions will offer a simple means of saving sourcing information and collated data — allowing for easy referencing, cross-referencing, and verification that your investigations are being performed ethically.

By simplifying and streamlining access to open source data, OSINT solutions provide an opportunity to gain insights that might otherwise only be available through more expansive and intrusive forms of intelligence collection. Instead of relying solely on intrusive methods, we can step toward the ethical collation and dissemination of publicly available data for a more targeted and sophisticated approach to investigations. 

As excessive automation with no human involvement can lead to incorrect and unethical decisions, it’s crucial to prioritise an IA approach that leaves humans in charge of OSINT decision-making. Such an approach minimises the risk of poor and potentially unethical decisions. 

The following factors also have an impact on the extent to which investigations can be considered ethical: 

• Targeted searches: collecting only the information that is relevant to your investigation.
• Legitimate use cases: collecting data to prevent crime, protect national security or meet regulatory requirements is considered more ethical than collection for marketing or sales purposes.
• Only public data: using data that is freely available online is easily explainable; using privileged data is more likely to be viewed negatively.

Deploying the power of a third party platform

The growth of open source data has the potential to improve the effectiveness of investigations in both the public and private sphere, but it has also left companies at risk of drowning in data they can’t make sense of. Advanced investigation techniques, implemented with the help of the right third party platform, are the best way through. 

At Blackdot, we developed Videris to address this problem by providing the intelligent automation necessary to streamline OSINT practices, while always guaranteeing human-led decisions that keep investigations firmly between the lines of an ethical and targeted approach.

With extensive experience in government applications as well as in the private sector, we’ve developed Videris to bring advanced open source investigation techniques to all users, regardless of their level of familiarity with OSINT. Videris does this through a range of capabilities, including:

• Targeted search functions: The ability to highlight search terms or search keywords across multiple public data sets at the same time with Videris Search ensures that investigations, and the data used within them, are always targeted towards outcomes.
• Automatic analysis: Intelligent automation, robotic process automation and natural language processing make it far easier to sieve through expansive data sets, bringing any valuable findings back to the analysts who can turn them into action.
• Visualised risk: Visualisations, including maps and charts, make it easier to draw connections across illicit networks and understand all risks, including those that may inform further investigations, or drive an investigation in a direction that a manual approach would never have recognised.
• Combined findings: Videris acts as a single pane of glass, allowing you to combine all your findings in one easily accessible, secure platform to prevent exposure or data breaches, as well as simplifying the overall investigation process.
• Secure searches: The ability to easily anonymise investigations provides security that ensures the integrity and effectiveness of your investigations.
• Data and source capture: By capturing full data sourcing as you work, Videris ensures that you’re able to export sources directly into the reports that drive decisions based on your findings. This allows organisations to keep an immutable record of their findings, regardless of whether the evidence is taken down after the fact. 

These capabilities let investigators work at an advanced level to achieve better outcomes. 

Ready to utilise these advanced techniques and take your investigations to the next level? Book a demo to find out exactly how we can help you.

FAQs

What is OSINT? 

OSINT (Open source intelligence) is the application of intelligence gathering techniques and technology to investigations that make use of open source data (OSD).

How can OSINT be used in investigations?

OSINT can be used in investigations to identify and manage risks. Advanced OSINT investigation solutions can be used to find key risks and map out connections across vast amounts of online data . 

What are open source investigation techniques?

Open source investigation is the process of using information from open source data (OSD) data sources to support research, analysis, and decision-making. Advanced techniques for open source investigation include:

1. Integrating internal and external databases
2. Harnessing intelligent automation
3. Safeguarding the security of your investigations
4. Using the Intelligence Cycle
5. Targeting investigations to ensure an ethical approach

What sectors benefit from using OSINT?

OSINT is widely used in various investigative fields such as financial services, large corporations, government and law enforcement, and consultancies.

How can Blackdot Videris be used in open source investigations?

Blackdot’s Videris is a comprehensive solution that streamlines the OSINT investigations process. It provides:

• Targeted searches across surface, deep and dark web
• Automatic analysis
• Visualised risk
• Secure live internet search
• Data and source capture