Handling NMSG Payload with Python

January 28, 2018 | Views: 1810

Begin Learning Cyber Security for FREE Now!

FREE REGISTRATIONAlready a Member Login Here

In this article, I’m going to show you how to handle NMSG payload using python. so before dive in into the programming, we may want to step back to to introduce you what is NMSG.

NMSG is one of the tools that implement passive dns analysis, with nmsg we can capture dns traffic flow and store in database for further analysis.

the installation is pretty simple you just have to install these couple things:

(source:https://www.farsightsecurity.com/technical/SIE-user-guide/sie-debian/)

Install the core nmsg packages and SIE plugins.

apt-get install nmsgtool nmsg-msg-module-sie

Optionally: Install packages for developing Python applications.

apt-get install python-nmsg python-wdns

and thats it you have the tools and the library.But the downside is that not much of tutorial that is provided to utilize the python nmsg library more. so i will start from the basic

at the beginning i want my tools to take 3 arguments, first is how many payload that i want to take, second is the nmsg payload and the last is the result (note: i’m using python version 2.7 for developing the tools)

import nmsg
import time
import argparse
import wdns

def main():

parser = argparse.ArgumentParser()
parser.add_argument(“count”,help=”how many payload that you want extract from the file”) parser.add_argument(“output”,help=”write file”)
parser.add_argument(“input”,help=”read file”)

args_file = parser.parse_args()

data_nmsg = handle_payload(int(args_file.count),args_file.input,args_file.output)

data_nmsg.parse_data() #open the payload
data_nmsg.process() #convert the payload into json

for developing the tools i’m using class instead of function, i dont have any specific reason why im using object oriented program approach well you can use function if you want. ok move to second part, now that i have create the object, now its time to create the constructor.

class handle_payload:

def __init__(self,payload_count,file_name,out_name):
self.file_name = file_name
self.payload_count = payload_count
self.out_name = out_name

now that we have the constructor for initialization. its time create the method. i start with the parse data, it is method for opening the payload.

def parse_data(self):
self.data = nmsg.input.open_file(self.file_name)

simple right? so before i create the process method, i will create other sub method to support the process method, because apparently if you try to just open and read the input it will something like this:

screenshoot: https://imgur.com/a/XKMJb

well for me i don’t want my output of result going to be like this, so i have to process first before output it, from the output as you can see we are present with json formated ish type, and it have a lot section that is present the dns packet section. so to speed things up i just want to take this following section:

[‘query_ip’, ‘response_ip’, ‘rcode’, ‘response_time_nsec’, ‘proto’, ‘qtype’, ‘response_time_sec’, ‘response_packet’, ‘query_port’, ‘qname’, ‘response’, ‘qclass’, ‘response_port’, ‘dns’, ‘udp_checksum’, ‘type’, ‘id’] #the full list of the content keys

to

[response_time_sec, query_ip(ip source),qname(query that is submitted),response_ip(the dns server),query_port,udp_checksum, dns(the dns content), qtype(ex: NS,A,SOA,TXT ),rcode(ex:noerror,NXDOMAIN)]

def convert_type(self,record):

return wdns.rrtype_to_str(record) #handling the qtype

def convert_record(self,record):

return wdns.rcode_to_str(record) #handling the rcode

def remove_nonascii_qname(self,ascii_str):

return wdns.domain_to_str(ascii_str) #handling the qname

def remove_nonascii_bailwick(self,dns_query):

section = wdns.parse_message(dns_query).sec
question_section,answer_section,authority_section= section[0],section[1],section[2]

return question_section,answer_section,authority_section #handling the dns section which contain dns packet

and most of it you just have to put it and arranged by yourself, so i straight create the process method:

def process(self):
count_record = 0

for x in range(0,self.payload_count+1):
self.data_transit = self.data.read()
if self.data_transit[‘type’] == ‘UDP_UNSOLICITED_RESPONSE’:

time_date = float(self.data_transit[‘response_time_sec’][0])
date_time =  time.strftime(‘%Y-%m-%d %H:%M:%S’,time.localtime(time_date))

question_section,answer_section,authority_section= self.remove_nonascii_bailwick(self.data_transit[‘dns’])

file = open(self.out_name+”.txt”,”a”)

file.write(“n+—————————————————-+n”)
file.write(“time: “+str(date_time)+”n”)
file.write(“ip source: “+self.data_transit[‘query_ip’]+”n”)
file.write(“query name: “+self.remove_nonascii_qname(self.data_transit[‘qname’])+”n”)
file.write(“response ip: “+self.data_transit[‘response_ip’]+”n”)
file.write(“query port: “+str(self.data_transit[‘query_port’])+”n”)
file.write(“checksum: “+self.data_transit[‘udp_checksum’]+”n”)
file.write(“question section:n”+str(question_section)+”n”)
file.write(“answer section:n”+str(answer_section)+”n”)
file.write(“authority section:n”+str(authority_section)+”n”)
file.write(“query type: “+self.convert_type(self.data_transit[‘qtype’])+”n”)
file.write(“record code: “+self.convert_record(self.data_transit[‘rcode’])+”n”)
file.write(“n+—————————————————-+n”)

file.close()

print “record:” + str(count_record)
count_record += 1
print “list:” + str(count_record)

so yeah thats it you just create your own converting tools, if you succeed it will something like:

screenshoot: https://imgur.com/iguhmTi

screenshoot: https://imgur.com/jMyauOY

heres is the full code, if you still confuse with my explanation (note: the following tutorial that i just show you is just how you can utilize the library more, in order to utilize more, according to your need you have to experiment the code by yourself):

#!/usr/bin/python

import nmsg
import time
import argparse
import wdns

class handle_payload:

def __init__(self,payload_count,file_name,out_name):
self.file_name = file_name
self.payload_count = payload_count
self.out_name = out_name

def parse_data(self):
self.data = nmsg.input.open_file(self.file_name)

def convert_type(self,record):

return wdns.rrtype_to_str(record)

def convert_record(self,record):

return wdns.rcode_to_str(record)

def remove_nonascii_qname(self,ascii_str):

return wdns.domain_to_str(ascii_str)

def remove_nonascii_bailwick(self,dns_query):

section = wdns.parse_message(dns_query).sec
question_section,answer_section,authority_section= section[0],section[1],section[2]

return question_section,answer_section,authority_section

def process(self):
count_record = 0

for x in range(0,self.payload_count+1):
self.data_transit = self.data.read()
if self.data_transit[‘type’] == ‘UDP_UNSOLICITED_RESPONSE’:

time_date = float(self.data_transit[‘response_time_sec’][0])
date_time =  time.strftime(‘%Y-%m-%d %H:%M:%S’,time.localtime(time_date))

question_section,answer_section,authority_section= self.remove_nonascii_bailwick(self.data_transit[‘dns’])

file = open(self.out_name+”.txt”,”a”)

file.write(“n+—————————————————-+n”)
file.write(“time: “+str(date_time)+”n”)
file.write(“ip source: “+self.data_transit[‘query_ip’]+”n”)
file.write(“query name: “+self.remove_nonascii_qname(self.data_transit[‘qname’])+”n”)
file.write(“response ip: “+self.data_transit[‘response_ip’]+”n”)
file.write(“query port: “+str(self.data_transit[‘query_port’])+”n”)
file.write(“checksum: “+self.data_transit[‘udp_checksum’]+”n”)
file.write(“question section:n”+str(question_section)+”n”)
file.write(“answer section:n”+str(answer_section)+”n”)
file.write(“authority section:n”+str(authority_section)+”n”)
file.write(“query type: “+self.convert_type(self.data_transit[‘qtype’])+”n”)
file.write(“record code: “+self.convert_record(self.data_transit[‘rcode’])+”n”)
file.write(“n+—————————————————-+n”)

file.close()

print “record:” + str(count_record)
count_record += 1
print “list:” + str(count_record)

def main():

parser = argparse.ArgumentParser()
parser.add_argument(“count”,help=”how many payload that you want extract from the file”)
parser.add_argument(“output”,help=”write file”)
parser.add_argument(“input”,help=”read file”)

args_file = parser.parse_args()

data_nmsg = handle_payload(int(args_file.count),args_file.input,args_file.output)
data_nmsg.parse_data()
data_nmsg.process()

main()

Thats it happy converting 🙂

 

Share with Friends
FacebookTwitterGoogle+LinkedInEmail
Use Cybytes and
Tip the Author!
Join
Share with Friends
FacebookTwitterGoogle+LinkedInEmail
Ready to share your knowledge and expertise?
Comment on This

You must be logged in to post a comment.

Our Revolution

We believe Cyber Security training should be free, for everyone, FOREVER. Everyone, everywhere, deserves the OPPORTUNITY to learn, begin and grow a career in this fascinating field. Therefore, Cybrary is a free community where people, companies and training come together to give everyone the ability to collaborate in an open source way that is revolutionizing the cyber security educational experience.

Cybrary On The Go

Get the Cybrary app for Android for online and offline viewing of our lessons.

Get it on Google Play
 

Support Cybrary

Donate Here to Get This Month's Donor Badge

 
Skip to toolbar

We recommend always using caution when following any link

Are you sure you want to continue?

Continue
Cancel