urlsnarf Lab

[toggle_content title="Transcript"] Hi Leo Dregier in this lab I want to talk to you about a real cool tool called URL snarf is a tool that basically sniffs URLs from the computer or interface you are monitoring from. So it is a pretty cool sniffing tool so I am going to open up the Kali VM open up a terminal and type in the manual for URL snarf. And basically we will do a quick overview and then use it. URL snarf sniff http requests in common log format. Common log format is basically the type of log format any web server would log its requests to. And you can see the syntax here use the URL snarf with a -n which is do not resolve IP addresses to host names. You can leave this or not leave if you just want to work from the IP addresses versus actual names then you can change that accordingly. You can specify a specific interface or if you have already got the traffic captured like if you are capturing something in wire shark. You can use a pcap file as the input and you have v and then you can actually incorporate TCP patterns and expressions in this as well. To use this it is relatively simple all you basically do is type in URL snarf since I don't have a particular interface to use or I am not reading from a file I can basically just set it right here and it will basically say it is listening on the default port ethernet interface for port 80 or 8080 type traffic. So then I am just basically go browse the internet and capture this traffic. Now just to be a little bit clearer what I am going to do is. I am actually going to dump the output here to a URL snarf.pcap file and capture it there. So not only do I get it on the terminal I also captured through a file. The reason why I am going to do this is because I am actually going to use another tool called stack world wide web statistics they basically analyze this file. So we are going to go generate some traffic here. We are just going to go up to www.leodregier.com just to see a website popup open. www.cybrary.it and let each one of these load www.google.com and www.cnn.com as well now I have done basically is generate its URLs to prove that we are sniffing here. So now that is done I can go ahead and basically stop my sniff less the file and basically I can see some stuff in here. Now the challenge at this point is basically going to get into a universal format. So it is just not real easy to read just to kind of show what just one entry looks like that is all one entry. So it has for the source address - the date and time stamp. The get request which is a component of http get versus post the actual URL that you went through and then you can see it is a WordPress site. It is getting style sheet for this particular entry it is using http version 1.1 and coming from a Mozilla .5 browser. So it thinks anyway which is on a Linux system etc. etc. Specifically the [indiscernible 0:03:55] browser. So not too interesting I certainly could grep this file and start digging through it. Try to find different trends and patterns but it is not enough in a nice format. So what I am going to so is just exit out of there and you will have to install the www stack. So I will just run you through the install - you already have installed but nonetheless I will show you. It is apps get install www stack and basically it is going to say zero upgraded zero newly installed because I have already have the latest and greatest version. All you would have to do is basically run through the install if you type in apt-gt install www stack click yes. It only takes a second for it to install and then you can simply run the command. So just to give you an idea let us look at the manual for www stack and the things that I do here to demonstrate for you. This is basically what I would do if I were trying to learn a tool for the first time. So one thing is to install the tool. Second is to look at the help file and to get a better idea of how the syntax is laid out by looking at the help file or the man file and then trying a couple of different instances of it. So here I will just go in and basically it summarizes www server access statistics and it reads a sequence of httpd common log file formats. Access log files etc. and you can read this if you want I just want to show you. You can specify things like dates and URL and you could do. I will put it in the table format which basically is the request the number of bytes sent received the request. Sometimes the category that gets enumerated you can also set the configuration options up in this particular file names. If you like as well - you want to get crazy with this and then it has got quite a few options in here about resolving, caching. Don't get the files, don't worry about cgi well don't worry about it. The time in which you want these things to get statistics for daily, hourly by its particular domain. If you want to sort or not - you will sort by bytes or the volume of the key heading traffic. Sorts in a particular use function and things like that now we kind of basic idea for this. I just want to do a www stack to the URL pcap file from earlier and just run through this and looks like I have got just enough information here. So www statistics for a title - couple of h refs in here and then we are going to scroll down to basically get to my traffic here and you can see the request the byte the format it is displayed and you can see some of the google traffic that we enumerated. The statistics pulling of the different gravitar images. More Facebook traffic cybrary.it you can see that is a WordPress site and it has got plugins and you can actually enumerate some of the specific plugins that are pulled in it reference just by naming convention. So that is a little bit of a tell there – you can see the leodregier.com that is also WordPress sites and some of the plugins as well. And it gives you a much, much nicer cleaner format of the information as the files go to and from the server. So if you want, you can set this type of tool when sort of sniffing port or spam port or network tap set it to run - capture the interface that gets duplicated to your spam port or something like that and then basically just start logging traffic. Once you capture the traffic in a particular log format then you can twist it, modify it and store it and greo it and really look for all sorts of trends and the statistics and patterns and sort by top IPs and things like that. But it is relatively easier to set it up and use it to generate some basic content like we did here. But as you can imagine in an enterprise situation a tool like this for the attacker could do a lot of harm because you can basically enumerate the surfing habits of the user base or from a forensics point of view. You can also use this tool to basically set up like a log collecting or some sort of URL analysis tool. Because it is pretty powerful and the way it collects but also easier enough in terms of its configuration to really just capture the information that you need. So that is it on URL snarf and www stack which is a statistics analyzer for the web tools. So you get two tools in one in this lab from both the offense and the defensive point of views. So that is going to cover it for a cybrary.it - my name is Leo Dregier. Don't forget to check us out on Facebook LinkedIn YouTube Twitter. [/toggle_content] This Sniffing Traffic simulation lab discusses urlsnarf, another sniffing tool for your pen testing arsenal that sniffs the URL path. In the urlsnarf lab, you’ll learn how to use URL sniffing tools from the interface you’re monitoring from. All web servers have what's called a “Common Log Format.” urlsnarf tool does is sniff the HTTP validation requests that are made to that log.  
Recommended Study Material
Learn on the go.
The app designed for the modern cyber security professional.
Get it on Google Play Get it on the App Store

Our Revolution

We believe Cyber Security training should be free, for everyone, FOREVER. Everyone, everywhere, deserves the OPPORTUNITY to learn, begin and grow a career in this fascinating field. Therefore, Cybrary is a free community where people, companies and training come together to give everyone the ability to collaborate in an open source way that is revolutionizing the cyber security educational experience.

Cybrary On The Go

Get the Cybrary app for Android for online and offline viewing of our lessons.

Get it on Google Play

Support Cybrary

Donate Here to Get This Month's Donor Badge



Skip to toolbar

We recommend always using caution when following any link

Are you sure you want to continue?