Cyber Threat Intelligence is a complex topic consisting of a lot of moving parts. It requires a high degree of technical knowledge combined with a healthy dose of paranoia. Staying ahead of the bad guys or simply keeping up with them is a 24/7 job. Threat intel is gathered from a multitude of sources and one place that’s being monitored a lot more now for threat data is the so-called Dark Web. Given this fact, a cottage industry has sprung up in recent years to monitor the Dark Web for signatures related to an organization. The intent is that this information can provide evidence of data breaches or give advance warning of impending attacks. But before examining this service in-depth, we must first have a look at the Dark Web and all that it entails. There’s quite a bit of misunderstanding and confusion around the terms defining this underbelly of the internet, so it’s best to get that cleared up first before venturing further.
The first bit of confusion to clear up is distinguishing between the Deep Web and the Dark Web. Actually, before we do that, let’s take a step back and visualize an iceberg (see this post's feature image). The tip of that chunk of ice with respect to the www is what is known as the Surface Web or Clearnet. It’s the portion of the web that you and I pretty much deal with exclusively. I'm making the assumption that we're the same in this regard.The Surface Web consists of web pages and other files that are indexed by search engines such as Google, Bing, Yahoo!, DuckDuckGo, Yandex, and many others. Of course, Google is the only one that matters </sarcasm>. Indexing begins with the process of web crawling via bots such as Googlebot. These indexed pages are then served up in the search engine results pages (SERPs). Ones on the first page are readily discovered, whereas, ones further in might as well be on the Dark Web. An old Search Engine Optimization (SEO) joke goes something like this: where’s the best place to hide a dead body? On page two of Google. [rim shot] Thank you! I’ll be here all week.
The Deep Web
It’s estimated that 90% of the web is hidden and not discoverable by search engines. This portion is often referred to as the Deep Web -- not to be confused with the Dark Web -- which it often is. Actually, the Dark Web, which will examine in a moment, is estimated to only comprise .01% of the Deep Web portion of the total web. In summary, the Dark Web is a tiny subset of the Dark Web. Much of the confusion comes when "Dark" and "Deep" are used interchangeably. They aren’t the same, not by a long shot!Much of the Deep Web – the other 99.99% - as mentioned previously, consists of files not discoverable by search engine crawlers. This ranges from data behind password-protected sites which includes all intranets, databases and pages that are dynamically rendered via a query and media files that are not part of photo and video-sharing sites such as YouTube. A lot of scientific data used for research purposes is hidden away on the Deep Web, but just because it’s out of view, doesn’t make it nefarious or illegal. The files comprising the CMS (Content Management System) I’m using to create this post (WordPress) is part of the Deep Web. You need a login and password to access it. Privacy certainly has its advantages and though it’s coming under attack right now, it’s a pretty wholesome thing. The Dark Web blows that premise to bits.
A walk on the Dark side
Exploring the Dark Web is not for the faint of heart and is the reason I’ve never ventured in. It’s like turning over a rock to have a look at all the creepy crawlies underneath, or more accurately, peeling back the layers of an onion. Actually, the onion analogy is pretty apt since the technology that drives the Dark Web is based on the Tor network. Tor stands for “The Onion Router.” It was developed during the mid-1990s by several employees of the Naval Research Laboratory in Washington, DC to protect U.S. intelligence online communications and eventually evolved into the Tor Project.Tor obscures both the source and destination IP addresses of data packets by wrapping them in encrypted layers, hence the onion analogy. Packets are sent through three separate geographical hops to conceal both the sender’s and receiver’s actual IP addresses. Each layer is peeled back and decrypted prior to being forwarded to the next hop. Both meet somewhere in the middle to make the data exchange; neither aware of the other’s actual location or identity. Domains on the Dark Web use the pseudo TLD: *.onion. Browsing this hidden portion of the web requires use of the Tor browser.
Den of thieves
So what kind of stuff is stored on the Dark Web and what kind of business is transacted?The answer as you might expect is not pleasant and I’ll state the most unpleasant detail upfront: the majority of the content on the Dark Web and activity transacted centers around child pornography. Some pretty vile and disgusting stuff. The remainder runs the gamut from the sale of illegal drugs, to the sale of firearms, trading of malware exploit code, the sale of stolen credit card numbers and other PII, and bitcoin tumblers which is essentially virtual money laundering. Botnets are also maintained, controlled, and leased on the Dark Web.A good deal of terrorist activity is also conducted on the Dark Web. A host of markets exist on the Dark Web to facilitate trading is these deviant commodities, so it’s little wonder that both buyers and seller would want to remain anonymous in such an underworld. It should also be pointed out that their is no honor among thieves even in the virtual world. Many of the market places are rife with scams to either steal money or infect unwitting users with malware.Believe it or not, there are also some legal or semi-legal uses for the Dark Web. News media organizations have set up servers on the Dark Web to receive information anonymously from their sources. Along these same lines, the Dark Web provides an anonymous forum for whistle blowers. WikiLeaks uses the Dark Web for receiving purloined information with the most notable instance being the stolen classified documents handed over by Edward Snowden. Even Facebook is now facilitating access to Dark Web users wishing to conceal their activity. This is a common need among users living under regimes that restrict the internet access of their citizens.
A glimmer of hope for law enforcement
There is a glimmer of hope for law enforcement when it comes to disrupting the security of the Dark Web. In early November 2014, a coordinated action by the FBI and Europol dubbed “Operation Onymous” conducted a virtual raid where they seized dozens of hidden services on Tor, snagging several popular illegal drug markets. How this was accomplished remains a mystery as the feds are keeping tight-lipped for obvious reasons. Speculation centers around government-initiated DDoS attacks that forced targeted sites to resort to using Tor relays under their control resulting in the tracing of their IP addresses. Another theory is that they used good old-fashioned gumshoe tactics and turned sys admins into informants.The Dark Web also makes a juicy target for vulnerability scanners due to its relatively small size. It’s infinitely quicker to scan and pentest targets on the Dark Web than it is on the Surface Web. This puniness also gives a decided advantage to law enforcement with the FBI maintaining a 5:1 edge of agents to Dark Websites.And then there’s the bottomless well of human folly. In October 2014, undercover agents purchased a firearm from a vendor on the Agora Dark Web marketplace. Despite the anonymity provided by Tor encryption, the perp’s undoing was the result of sloppiness: his prints were found on the firearm. He’s now facing up to fifteen years in prison. Up next:
Monitoring the Dark Web for threat intel.