The Abyssal Depth of the Deep Web
In the current scenario, people believe that information about everything present in the world can be found through the Google search engine. But there is entirely a massive online world beyond the search of Google or any other similar search engine called the Deep Web. If it is to be believed the scale of the internet world is wide and the number of non – indexed sites is assumed to be 400 to 500 times larger than that of the searchable web. The Deep Web is the dark side of the internet where there are plenty of well-intentioned and law-abiding citizens are found who conduct the online activities below the surface. The area under the searchable web is the part of the Deep Web which is popularly known as the Dark Net, it has become a channel for the illegal and dangerous activities.
The Concept of Deep Web and Dark Net are as old as the World Wide Web and it came into notice when the WWW got popular in the mid-1990s. After the arrest of “Dread Pirate Roberts” also known as Ross William Ulbricht in October 2013, the Deep Web came into notice of the common internet users. Ulbricht gave “Silk Road” a new meaning as the online marketplace where the users can find all manners of illegal drugs. The Deep Web and the Dark Net quickly became fixtures, playing a popular role in the U.S. through the House of Cards series where a reporter hires a hacker to dig into the dirt of the Vice President.
If the Web is assumed to be the iceberg, most of us are interacting with the transparent and easily visible Surface Web. In the darkness beneath the surface web is the invisible part of the Deep Web. This part of the web is not crawled and indexed by the standard search engines. Initial studies have revealed that the Deep Web is a largest expanding reservoir of fresh data. As most of the material over Deep Web is protected the overall quality of the data is better than that from Surface Web.
The Deep Web is growing and multiple technologies such as distributed/cloud computing, sensor networks, mobile computing and ubiquitous computing have contributed to its expansion. Advances in anonymous web hosting services, development of crimeware and cryptocurrencies like bitcoins or dark coins are a further contribution to the growth of the Deep Web. Hackers and even the plenty of legitimate users like journalists, whistleblowers, political leaders and human rights advocates, all use the Deep Web for their researchers which is not possible through surface Web.
A lot has been said about Deep Web and Dark Net but the question that still comes to mind is how we access the same. If anybody is curious to access the Deep Web special tools and techniques are used, some of which may look like that we use for the surface web. The most widely used search engine for accessing it is “TOR” browser. It is specifically designed for browsing the data on Deep Web which can access the different depth or parts of the deep Web. The main challenge faced while using these specially designed search engine is that they only can crawl small part of the Deep Web making users important to visit right hidden online web listing. There are still some public databases that can be considered as the part of the Deep Web because most of the content available in them cannot be searched using the usual search engine. For instance, there may be a few users who are interacting with a part of Deep Web but are not aware of it. For example, the U.S. Library of Congress is one such online database that can also be found on Deep Web. FreeLunch.com, Copyright.gov, Web of Science, WWW Virtual Library, FindLaw's, Directory of Open Access, Census.gov and Wolfram Alpha are also few such sites.
The Deep Web has been increasingly used for the online business trades, information sharing and for file transfers in recent years as keeping anonymity and keeping online activity private is easier for the users. If any user wants to assess any anonymous sites of the Deep Web, TOR browser can be used for the same with “.onion” domain. The Deep Web market has allowed the worldwide access and distribution of products that needed a substantial asset in the supply chain. This disruption created the massive shift in the supply chain especially to the ones that are illegal or are subject to heavy taxation.
To control the illicit activity, many law-abiding groups have accepted the procedures for online criminals and network examination. The FBI has also used these techniques with the help of Metasploit, an app used in “Operation Torpedo” in 2012 and in “Operation Onymous” in 2016 to stop two Deep Web crimes involving Child Pornography and drug dealing. If the legal policymakers want to understand Deep Web, they must think beyond the usual search methods on the Web as Google or Bing. They must understand that underneath the Internet’s visible web is the Deep Web whose activities are of huge concern to the public and government. The Side which deals with Drug Dealing, Hackers, child pornography, human trafficking, money launderers, terrorists and even spies. In a few cases, there are innocent individuals who simply want an added privacy, but this does not mean that the government should turn blind to the enormous Deep Web.
As the Deep Web rises there is a crucial need for the representatives to better understand the Deep Web to develop an improved comprehensive law enforcement and national security response for public safety. The better understanding of the Deep Web market will give the better tailored and more effective National and International level responses and over time better cooperation between the countries worldwide. The Deep Web not only harbors illegal transactions of the traditional goods and hacking tools but is also a battlefield where cyberwar will be waged.
With the emerging of the Deep Web, there are multiple regulatory issues arising at both National and international levels. How well these subjects are resolved will not only disturb the cybersecurity of the countries but also of the Secluded sectors, corporations, and people. Officials must understand the circumstances that might give rise to the Darknet and are germane to the law enforcement and national security.
I believe that Deep Web is nothing but a mirror of the current society. Distorted, mutated, and magnified by the unnatural conditions of the virtual reality online. We have learned so much in last 20 years or so about the enormous new simulated world defined by the Infobahn. As the island of our knowledge expands so does the shore of our ignorance also stretched leading us in overseeing the cybersecurity threat that may happen? It is significant to comprehend that with the powerful search engines like Google and Bing what we have only the minor segment of the surface of the immense data ocean. In the end, the question that stays is that will we ever be able to hit the bottom and understand the vast Deep Web and will the law-abiding policymakers ever be ready for any of the Cyber War that may arrive due to the incomplete information about the same?
Do you like to write about your infosec knowledge, skills, opinions, or exploits?
Publish your original research, tutorials, articles, or other written content on Cybray's blog to be seen by thousands of infosec readers daily!