Everything you need to know about the darknet.
This week, OWL Cybersecurity officially became DarkOwl, a name change that reflects our ever-growing focus on the darknet and hidden services. To refresh our readers on what that means, and what the darknet actually is, we put together this Darknet 101-style overview to help clarify what is commonly known as the murkiest and most elusive area of the internet.
Before we dive into the darknet, it may be helpful to take a step back and look at the big picture of the internet, which is comprised of several main components: the Surface Web, the Deep Web, and the Darknet.
The Internet: Surface Web, Deep Web, Darknet
The term internet is short for internetwork, a system created by connecting a number of computer networks together. An internet allows for communication between devices that are a part of that internetwork.
The internet, which until recently was denoted by a capital “I”, is the most well-known example of an internetwork. This is the internet that we find indispensable to our daily lives, and it links billions of devices across the world through a network of networks using standardized procedures or protocol.
Browsing websites on the web is not the only way in which information is shared via the internet. Email, instant messaging, and FTP are other ways to share information like emails, messages, and files.
To clarify, the web is not synonymous with the internet and should not be confused with it. The web is simply a way of accessing webpages over the medium of the internet.
the surface web
The websites we browse each day make up only a small percentage of the internet. These sites, collectively known as the surface web, are visible and accessible to common search engines such as Google and Yahoo. While estimates vary, many experts agree that the surface web comprises roughly 4% of all online content.
below the surface: the Deep Web
Beyond the surface web, 96% of online content is found in the deep web and the darknet.
The deep web consists of content that cannot be found or directly accessed via surface web search engines such as Google and Yahoo. Examples of deep web sites include websites that require credentials (registration and login), unlinked sites that require a direct link to access, sites that are purposefully designed to keep search crawlers out, and databases - the majority of content in the deep web.
Deep web databases commonly have their own search functionality which allows users to access the data contained within them. Government databases, patient medical records, and library catalogs are just a few examples of deep web databases. While these databases do not have to require login credentials, many of them do.
Beyond the deep web is the darknet. The darknet is a network, built on top of the internet, that is purposefully hidden, meaning it has been designed specifically for anonymity. Unlike the dark web, the darknet is only accessible with special tools and software - browsers and other protocol beyond direct links or credentials. You cannot access the darknet by simply typing a dark web address into your web browser.
Above we mention that the internet we refer to and use daily is the most well-known example of an internet. Similarly, below are several examples of darknets (each links to more information):
- Tor, or The Onion Router, is a group of volunteer-operated servers that allows people to improve their privacy and security on the Internet. Users connect through a series of virtual tunnels rather than making a direct connection.
- I2P, or the Invisible Internet Project, is an anonymous overlay network - a network within a network - intended to protect communication from surveillance and monitoring.
- Freenet is free software which allows users to anonymously share files, browse and publish "freesites" (web sites accessible only through Freenet) and chat on forums. Communications by Freenet nodes are encrypted and are routed through other nodes to make it extremely difficult to determine who is requesting the information and what its content is.
- DN42 is an example of a darknet, a routing protocol, that is not necessarily meant to be secret - its aim is to explore internet routing technologies.
We'll use Tor, perhaps the most well-known and most-used, to better explain the darknet and dark web. Tor, short for The Onion Router (the project's original name), routes traffic to dark web sites through layers of encryption to allow for anonymity. The term dark web refers to websites on a darknet. In Tor's case, these dark web addresses all end in .onion.
Onion routing is implemented by encryption, nested like the layers of an onion. Tor encrypts the data, including the destination, multiple times and sends it through a circuit of randomly selected Tor relays. Each relay decrypts a layer of encryption to reveal only the next relay in order to pass the remaining encrypted data on. The final Tor relay decrypts the innermost layer of encryption and sends the original data to its destination without revealing, or even knowing, the source address.
The other darknets mentioned above employ similar methods of data transmission, all with the end goal of keeping users, usage, and information anonymous.
Curious about something you've read on our blog? Want to learn more? Please reach out. We're more than happy to have a conversation.
Because of their fluid and often unpredictable nature, accurately characterizing and capturing a snapshot of the various darknets can prove rather challenging. Our analysts are currently engaged in an ongoing research project to provide a solution to this, and have successfully begun to map the darknet using quantitative analytics and machine learning techniques.
Today's blog post details more about our darknet analysis project and offers a first glimpse at what we're doing to understand the current state of the darknet.
The challenge and our approach
Hidden services on Tor frequently come and go, as criminals often change their onion addresses to avoid apprehension and many servers are operated out of personal residences with uptime fluctuating with their daily schedules. Quantitative analysis helps to provide us with an indication of whether we are successfully collecting data that is relevant to our customers, improves the greater understanding of the network, and offers opportunity to fine tune collection methodologies.
The majority of darknet sites can be categorized into a couple of dozen subjects, ranging from X-rated content to drugs and social media sites used for communication.
The engine that powers our DarkOwl Vision platform is constantly and intelligently scraping the darknet for new content, using machine learning to categorize the services captured in an effort to understand the shape and feel its current state.
The following chart shows what the breakdown of what roughly 60,000 sites across Tor and I2P look like, with data analyzed up through the August 8, 2017.
While the chart above provides a snapshot of what we've crawled up until now, we also monitor the “breath” of the darknet by tagging sites who have recently updated their content. This metric provides insight into where current events and developments are taking place.
The following chart shows a breakdown of the most recently updated or modified segments of the darknet, only looking at sites which have uploaded new content over the last five (5) days. The total number of onions and eepsites in this summary contains roughly 15,000 unique addresses.
- Interestingly, most of the current activity on the darknet occurs in the Hackers community, with ~26% of the darknet containing hacking-related content or materials.
- Social Media & Chatrooms and File-Sharing services doubled in proportion, relative to the entire address space.
- Wiki-related sites, such as The Hidden Wiki, have a large footprint on the darknet but have little to no activity over the last week, supporting claims that many Wikimirror sites are scams as opposed to links to legitimate darknet onions.
Curious about something you've read on our blog? Want to learn more? Please reach out. We're more than happy to have a conversation.
While most of what you've likely heard or read about the darknet and dark web sites involves illegal or nefarious activity, there are many legal uses for the darknet.
Facing more sophisticated threats than ever before, information security policies have been increasingly focused on protecting sensitive data. However, despite best efforts to enhance such defenses, the effectiveness of evolving threats has taught us that it is not a matter of if business will be breached, but rather a matter of when they will be breached.
As there is no all-encompassing security solution, businesses must look to a multi-faceted defense that includes not only tools, products and services to protect sensitive data, but also an understanding of the darknet, where the vast majority of their compromised data will likely be found.
So, what is the darknet, and how does DARKINT fit into your cybersecurity defense strategy? Let’s quickly explore the parts of the internet with which we are more familiar for better reference.
The websites we browse each day make up only a small percentage of the internet. These sites, collectively known as the “surface web”, are visible and accessible through common search engines such as Google and Yahoo. According to estimates, the surface web makes up only about 0.03 percent of all content available on the internet, equal to roughly 19 terabytes of data.
Beneath the surface web, one will find the “deep web,” which is commonly mistaken for the darknet, but is actually a different entity entirely (which we’ll get to in a minute). The deep web is typically defined as internet content that cannot be found or directly accessed via conventional search engines.
A common example of the deep web would be a website or database that requires credentials – registration and login – to access. Your paid subscription to an online news site, your protected access to your personal banking information or your home or work server are also examples of the deep web. The deep web comprises a large percentage of all content found on the internet, equaling roughly 7,500 terabytes of data.
Below the deep web is the “darknet.” To access the darknet, one must obtain special tools that a regular internet user wouldn’t normally encounter. This includes a specific browser, network, and skill-set that only the technologically advanced or (rather determined) darknet seeker will be able to ascertain.
The darknet was originally built by the U.S. military to (purposefully) hide the identities of users and thus provide an unprecedented platform focused on prioritizing anonymity for its users. Because of the way the darknet is built, estimating its size is very difficult. Thus, the percentage of the overall internet the darknet comprises as a whole is currently unknown.
While there are valid, legal uses of the darknet (such as a journalist protecting herself and her source through encrypted communication, or political dissidents communicating with each other), anonymity naturally attracts illegal activity. Accessing the darknet is challenging and risky, with obfuscated links, the easy ability to accidentally view illegal or illicit materials and transitory sites and content that come and go frequently – a precaution many illegal site owners take to avoid being caught.
Trade in illegal drugs and weapons, stolen credit cards, credentials, counterfeit documents and intellectual property are a few examples of what is typically found on the darknet. In addition, one can find chatter on planned attacks or breaches and the sharing of viruses, malware and vulnerabilities, as well as a host of other illicit topics.
WHY SHOULD BUSINESSES CARE?
When a business’s proprietary data is found on the darknet, it is time to act. If a business can shorten the timeframe to the detection of its sensitive data on the darknet, it can more quickly detect security gaps and mitigate damage prior to the misuse of that sensitive company data. The cost of mitigating a breach can therefore be lessened, and the potential for reputation damage or other losses can be minimized.
For example, when a financial institution uncovers a trove of stolen credit cards for sale on the darknet, it can notify customers by cancelling those cards and issuing new ones, working to stay ahead of a security incident involving payment card industry (PCI) protected data. The same is true when personally identifiable information (PII) is hacked — early awareness of this allows companies and organizations to mitigate potential damages before criminals can capitalize on the theft of the data. When a breach hits the media, it is disastrous to a company, and according to IBM, breaches like these can cost upwards of $4 million per incident.
While there are no guarantees in cybersecurity, it is important that organizations use all of the tools available to combat potential cyberattacks. Monitoring DARKINT is an important, emerging approach that should supplement your multi-faceted information security defense strategy. Understanding the role that the darknet plays in cybersecurity can help to keep you, your business and your data, safe.