The internet, social media, and mobile devices are the fundamental requirements for conducting business and engaging in society. Whether checking email, catching up on industry news or accessing customer information, most of us use the internet (and the deep web) throughout the day, every day, in a variety of capacities. But, do we understand how it works – technically – even at a basic level? Do we understand the differences between the internet and the deep web or what it means to go even darker into the decentralized anonymous networks like the darknet?
Below is a breakdown of the various layers of the internet, from “regular” search engine-compatible websites to complex hidden networks.
The term internet is short for internetwork, which is a system created by connecting any number of computer networks together. An internet allows for communication between devices that are a part of that internetwork.
The internet is the most well-known example of an internetwork. This is the internet that we find indispensable to our daily lives, and it links billions of devices across the world through a network of networks using standardized procedures or protocol. The traditional server client architecture and HTTP protocol is the backbone of the internet and used extensively in websites and mobile applications.
Browsing websites on the web is not the only way in which information is shared via the internet. Email, instant messaging, and file transfer protocol (FTP) are other ways to share information like emails, messages, and files.
To clarify, the web is not synonymous with the internet and should not be confused with it. The “world wide web” is simply a way of accessing websites over the medium of the internet.
The websites we browse each day make up only a small percentage of the internet.
These sites, collectively known as the surface web (or “clearnet”), are visible and accessible to common search engines such as Google and Yahoo. Youtube videos, blogs, Instagram are all examples of surface web content most interact with every single day. While estimates vary, many experts agree that the surface web comprises roughly 4% of all online content. For more reading on how search engines crawl and index web content, there are several articles that describe systems like Google in detail.
High risk surface web consists of areas of the surface web that have a high degree of hosting criminal or illicit content. Many of the users of the high risk surface web also maintain access to other, darker networks and communities. This includes some “chan”-type imageboards, transient paste sites, and other select non-authenticated forums that mirror dark web sites with surface web top level domains (TLDs).
While .com domains are the most common website domain, DarkOwl regularly tracks various TLDs that are popular with criminals. Our analysts have observed an increase in .top, .ru, and .cc TLDs. Many high risk surface websites popular with Chinese threat actors end in the TLD .cn.
Beyond the surface web, an estimated 96% of online publicly accessible content is hosted in the deep web and the darknet.
The deep web consists of website content that cannot be found or directly accessed via surface web search engines such as Google and DuckDuckGo. Examples of deep web sites include websites that require any authentication credentials, such as registered email address and password, unlinked sites that require the direct URL to access, sites that are purposefully designed to keep search crawlers out, and databases. The majority of content resides in the deep web.
Deep web databases commonly have their own search functionality which allows users to access the data contained within them. Government databases, patient medical records, and library catalogs are just a few examples of deep web databases. While these databases do not always require login credentials, many of them do.
Banking website portals for accessing account holder data and credit card statements are technically in the deep web because most banking websites will not allow access to their sensitive servers without authorization. Most social media is technically deep web content.
A specific example is the Denver Property Taxation and Assessment System website which allows users to search property assessment and tax data by entering a Denver-based address into the system. However, if you enter this same Denver-based address into a Google search (and even include terms such as ‘property assessment’ or ‘tax data’), you will not find any documents or URL results from the Denver Property Taxation and Assessment System website. This database and its search functionality are one example of a deep web database that is hidden from surface web search engines and technically resides in the deep web.
Beyond the deep web is the darknet.
The darknet is any anonymous network, built on top of the internet, that is purposefully hidden, meaning it has been designed specifically for anonymity. Unlike the deep web, the darknet is only accessible with specialized tools and software – browsers and other protocols beyond direct links or credentials. You cannot technically directly access the darknet by simply typing a darknet address into your web browser, even though browsers like Brave offer private tabs with Tor for enhanced privacy.
Most people associate the darknet with Tor, but Tor is one of many darknets available. Let’s explore some of these darknets in more detail:
Navigating these networks can be frustrating and challenging for any OSINT/Darknet investigator and the public often incorrectly uses the terminology associated with these different layers of the internet. Any website that hosts or serves illicit content whether it is in the surface web, deep web, or darknet is technically a segment of the “dark web.” Dark web and darknet are often used interchangeably by us and other information security researchers.
Join us next time when we explore more darknets and darknet adjacent chat platforms like Telegram and Discord. Get on the list so you don’t miss it!