OSINT Investigations: The Challenge of Dark Web Data Searches and Mining

The dark web has gained widespread notoriety in recent times, with increased scrutiny on surface web activities appearing to drive an influx of new users, including sellers of illicit goods. It now functions as a segment of the internet where criminal networks can operate more openly, and terrorist networks can keep their communications secure.

Despite efforts to conceal information through anonymous networks and encryption, much of the information contained on the dark web is open source data (OSD), and as a result, can be utilised as open source intelligence (OSINT). That’s why the dark web is now playing an important role in the fight against financial crime and other illicit activities.

This article will explore the challenges and opportunities associated with utilising dark web data in OSINT investigations. Let’s get started.

Suggested reading: If you’re interested in learning about open source investigation best practices, check out our free eBook — The OSINT Handbook

What is the dark web?

The dark web originated from a 1990s US Naval Research Laboratory funded attempt to protect intelligence communications by routing traffic via a network of multiple relays known as The Onion Routing (Tor) Project. Since then, Tor has been backed by numerous organisations and institutions, including Human Rights Watch, Facebook, and Google. 

Despite this, the term ‘dark web’ still implies some kind of illicit activity for many, and some of the content you can find on the dark web certainly helps it live up to this reputation:

• Around 50,000 terrorist networks communicate on the dark web, including ISIS, who spread propaganda on .onion forums ahead of the 2015 Paris terrorist attacks.
• Approximately 1% of all dark web addresses are dedicated to financial crime. Credentials for 133,927 C-level executives from Fortune 1000 companies were found on dark web marketplaces in 2020. Personal information and sensitive data, including social security numbers, are sold for as little as $1, and debit and credit card numbers for around $5 to $10.
• Drugs dominate dark web markets, comprising around 48% of listings.

Dark web security

It’s a common misconception that accessing content via a dark web browser like Tor provides instant security and anonymity. The Tor Browser is robust, but as the platform has developed, so too have opportunities to exploit it. That’s why it’s crucial to take appropriate precautions when browsing web pages on the dark web. This can include:

• Using a VPN: The first threat to dark web browsing lies at the entry node. Employing a Virtual Private Network (VPN) ensures that traffic heading from the computer to the entry node is encrypted upon arrival. This prevents the entry node from seeing the connecting device’s IP address.
• Staying up to date: Tor is constantly updated, and older versions are easier to track. If you are using Tor for OSINT purposes, ensure it’s up-to-date before each session.

The dark web in OSINT investigations

Given the increasing use of dark web websites in recent years, it is no surprise that investigators are increasingly looking to search and mine the data it stores. However, security considerations aside, investigators need to weigh up the pros and cons of using the dark web at the outset of any investigation.

Dark web opportunities in OSINT investigations

In recent years the dark web has become one of the richest sources of insight into illicit networks and criminal activity for OSINT investigations. Let’s take a look at some opportunities the dark web provides within investigations:

• Monitoring illicit activities: Dark web forums, marketplaces, and messaging services are easily accessible, providing a straightforward means to monitor users and discussions. Investigators can therefore utilise dark web sites to uncover a rich account of contemporary trends in drug dealing, financial crime, firearms sales and even human and wildlife trafficking.
• Evaluating existing leads: It’s possible to evaluate leads using dark web whistleblowing resources, including GlobaLeaks, Independent Media Center and other services operated by the Associated Whistleblowing Press. This also provides an opportunity to corroborate or disprove information found on the surface web.
• Combating insider threats: In the event of a corporate data breach, leaks or inside hacking, dark web marketplaces and forums can provide evidence that data has been sold on the dark web and by whom. In addition, insider threats may leak identifiable or other incriminating information about themselves.
• Identifying individuals: The implied safety of the dark web offers a seemingly safe haven for criminals to operate, but poor user habits can lead to accidental self-identification. For example, a dark web forum user might use the same username published on social networks, or their language, terminology, or profile picture might match surface web profiles.

Dark web challenges in OSINT investigations

Despite the potential for utilising the dark web for optimised threat intelligence and investigations, it’s crucial to first understand the challenges that dark web data searches and mining present to investigators:

• Poorly structured and hard to search: The dark web is not indexed by surface web search engines like Google, but is explorable via specialist dark web search engines, though these are generally slow and cumbersome. Dark web addresses are often changed to make them challenging to trace.
• Browser fingerprinting: Browser fingerprinting is a tracking technique that subverts onion routing security. While it might be nearly impossible to track users via their traffic, browser fingerprinting aims to identify users by the unique properties of their browser and machine. Tor is designed to protect against fingerprinting by blocking scripts, utilising the same default fallback fonts in every browser and blocking WebGL and the Canvas API, thus making it tricky to differentiate one browser from another. Nevertheless, The Tor Project admits that opportunities to identify users via their Tor browser fingerprint are inevitable.
• Avoiding human error: Researchers are prone to human error in the same way that dark web users might reveal their own identities. While OSINT investigators will be able to apply their existing skills to dark web searches, it’s essential to be careful about what evidence is left behind. This is particularly true when it comes to exhaustive OSINT operations, which may leave researchers weary and more prone to error. 
• Exposure to illegal or traumatic material: Web pages on the dark web host vast quantities of illegal and potentially traumatic material, most of which is purposefully uncensored or unmoderated. This introduces a legal and ethical dilemma — researchers need to develop strategies that moderate or triage certain content so they can access it at arm’s length.

Make the most of the dark web with Videris

Information on the dark web can be crucial to ensuring successful investigative outcomes. Unfortunately, the challenges of using the dark web in investigations are evident, numerous, and difficult to overcome. That’s where leading OSINT tools come in.

Our OSINT platform, Videris, provides secure, seamless access across the internet. Rather than working through Tor, Videris pulls data from the surface, deep and dark web into one secure ecosystem. This accelerates the OSINT gathering process and provides the detail required for modern investigations. 

Videris comes with a wide range of functionality that can help produce desirable outcomes, including:

• Videris Search, which provides exceptionally detailed results that span the surface, deep and dark web, ensures instantaneous results and insights displayed in intuitive formats. 
• Interactive charts and corporate network mapping that create an effective visual representation of organisational structures and hierarchies. 
• Cross-matching tools form links between seemingly disparate sources, empowering researchers with incisive decision-making.

Furthermore, Videris facilitates ethical investigations by blocking the distressing images present throughout the dark web. It can, however, help identify dark web pages with images that contain metadata usually stripped from surface web content. Investigators can then decide whether to examine this media using forensic tools to locate criminal activity.

Hosted securely in the cloud or a standalone corporate network, Videris integrates seamlessly into your current infrastructure. Book a demo with Blackdot today, and explore how Videris can help you overcome the challenges of using dark web data in investigations.

---

¹Terrorist Migration to the Dark Web

²ID Agent – Are Your Passwords Safe?

³Statista – Darknet Markets