It sometimes amazes me just how far the internet and online activity reaches. With security and privacy concerns aside, browsing the web is something we do on a daily basis without any thought. Unknown to most, there is more to the web than what you see on the surface… lots more.
Did you know that you’re reading this article right now on what’s called the “Visible Web?” There is a hidden part of the internet that is much larger than the parts of the web you use every day. When you are checking your email, shopping online, or Facebooking, you are interacting with what is called the “Visible Web” or the “Surface Web.”
The Visible Web is consisted of webs pages that are indexed by normal search engines, like Google, Bing or Yahoo. It is estimated that there are at least four billion indexed web pages! What you probably don’t realize is that there is much more. There are also places on the web below the Visible Web that are not indexed. This part of the web is called the “Deep Web.”
In a nut shell, the Deep Web is all the data behind firewalls. Think of databases, business intranets, web archives, password protected websites, etc. By some estimates, this part of the internet is around 400-500 times larger than the Visible Web.
Many users think that when they search on Google they’re getting all the web pages, when in reality, it’s a very small fraction of the Deep Web which search engines are bringing to the surface. No one really has a good estimate of how big the Deep Web is. We only know that it is hundreds of times larger than what you typically have access to.
However, the story doesn’t end there. There is an even more secretive part of the web called “Dark Web”. The Dark Web refers to a set of accessible, albeit anonymously-hosted websites that exist within the Deep Web. Because these sites aren’t indexed by normal search engines, you can only access these sites with special software that disguises your IP address as well as other gates you need to pass to gain entry. This is essentially the black market of the web were the anonymity of bitcoin purchases can get you items that are illegal or unobtainable where you live.
There is a secret cat and mouse game being played between secret service field agents, law enforcement and activists and those who wish to leverage the web for less than lawful activities such as terrorists, drug dealers, and the like.
Technology is my business, I spend most of my day using the internet and it is amazing to me that while I am ordering my coffee from amazon prime and going through my daily routines on the web, it has this whole other secret life that most of us will never know about.