Home 0P3N Blog Handling NMSG Payload with Python
Ready to Start Your Career?
Create Free Account
By: anomali8888
January 28, 2018

Handling NMSG Payload with Python

By: anomali8888
January 28, 2018
By: anomali8888
January 28, 2018
In this article, I'm going to show you how to handle NMSG payload using python. so before dive in into the programming, we may want to step back to to introduce you what is NMSG.NMSG is one of the tools that implement passive dns analysis, with nmsg we can capture dns traffic flow and store in database for further analysis.the installation is pretty simple you just have to install these couple things:(source:https://www.farsightsecurity.com/technical/SIE-user-guide/sie-debian/)Install the core nmsg packages and SIE plugins.apt-get install nmsgtool nmsg-msg-module-sieOptionally: Install packages for developing Python applications.apt-get install python-nmsg python-wdnsand thats it you have the tools and the library.But the downside is that not much of tutorial that is provided to utilize the python nmsg library more. so i will start from the basicat the beginning i want my tools to take 3 arguments, first is how many payload that i want to take, second is the nmsg payload and the last is the result (note: i'm using python version 2.7 for developing the tools)import nmsgimport timeimport argparseimport wdnsdef main():parser = argparse.ArgumentParser()parser.add_argument("count",help="how many payload that you want extract from the file") parser.add_argument("output",help="write file")parser.add_argument("input",help="read file")args_file = parser.parse_args()data_nmsg = handle_payload(int(args_file.count),args_file.input,args_file.output)data_nmsg.parse_data()#open the payloaddata_nmsg.process()#convert the payload into jsonfor developing the tools i'm using class instead of function, i dont have any specific reason why im using object oriented program approach well you can use function if you want. ok move to second part, now that i have create the object, now its time to create the constructor.class handle_payload:def __init__(self,payload_count,file_name,out_name):self.file_name = file_nameself.payload_count = payload_countself.out_name = out_namenow that we have the constructor for initialization. its time create the method. i start with the parse data, it is method for opening the payload.def parse_data(self):self.data = nmsg.input.open_file(self.file_name)simple right? so before i create the process method, i will create other sub method to support the process method, because apparently if you try to just open and read the input it will something like this:screenshoot: https://imgur.com/a/XKMJbwell for me i don't want my output of result going to be like this, so i have to process first before output it, from the output as you can see we are present with json formated ish type, and it have a lot section that is present the dns packet section. so to speed things up i just want to take this following section:['query_ip', 'response_ip', 'rcode', 'response_time_nsec', 'proto', 'qtype', 'response_time_sec', 'response_packet', 'query_port', 'qname', 'response', 'qclass', 'response_port', 'dns', 'udp_checksum', 'type', 'id']#the full list of the content keysto[response_time_sec, query_ip(ip source),qname(query that is submitted),response_ip(the dns server),query_port,udp_checksum, dns(the dns content), qtype(ex: NS,A,SOA,TXT ),rcode(ex:noerror,NXDOMAIN)]def convert_type(self,record):return wdns.rrtype_to_str(record)#handling the qtypedef convert_record(self,record):return wdns.rcode_to_str(record)#handling the rcodedef remove_nonascii_qname(self,ascii_str):return wdns.domain_to_str(ascii_str)#handling the qnamedef remove_nonascii_bailwick(self,dns_query):section = wdns.parse_message(dns_query).secquestion_section,answer_section,authority_section= section[0],section[1],section[2]return question_section,answer_section,authority_section#handling the dns section which contain dns packetand most of it you just have to put it and arranged by yourself, so i straight create the process method:def process(self):count_record = 0for x in range(0,self.payload_count+1):self.data_transit = self.data.read()if self.data_transit['type'] == 'UDP_UNSOLICITED_RESPONSE':time_date = float(self.data_transit['response_time_sec'][0])date_time =  time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time_date))question_section,answer_section,authority_section= self.remove_nonascii_bailwick(self.data_transit['dns'])file = open(self.out_name+".txt","a")file.write("n+----------------------------------------------------+n")file.write("time: "+str(date_time)+"n")file.write("ip source: "+self.data_transit['query_ip']+"n")file.write("query name: "+self.remove_nonascii_qname(self.data_transit['qname'])+"n")file.write("response ip: "+self.data_transit['response_ip']+"n")file.write("query port: "+str(self.data_transit['query_port'])+"n")file.write("checksum: "+self.data_transit['udp_checksum']+"n")file.write("question section:n"+str(question_section)+"n")file.write("answer section:n"+str(answer_section)+"n")file.write("authority section:n"+str(authority_section)+"n")file.write("query type: "+self.convert_type(self.data_transit['qtype'])+"n")file.write("record code: "+self.convert_record(self.data_transit['rcode'])+"n")file.write("n+----------------------------------------------------+n")file.close()print "record:" + str(count_record)count_record += 1print "list:" + str(count_record)so yeah thats it you just create your own converting tools, if you succeed it will something like:screenshoot: https://imgur.com/iguhmTiscreenshoot: https://imgur.com/jMyauOYheres is the full code, if you still confuse with my explanation (note: the following tutorial that i just show you is just how you can utilize the library more, in order to utilize more, according to your need you have to experiment the code by yourself):#!/usr/bin/pythonimport nmsgimport timeimport argparseimport wdnsclass handle_payload:def __init__(self,payload_count,file_name,out_name):self.file_name = file_nameself.payload_count = payload_countself.out_name = out_namedef parse_data(self):self.data = nmsg.input.open_file(self.file_name)def convert_type(self,record):return wdns.rrtype_to_str(record)def convert_record(self,record):return wdns.rcode_to_str(record)def remove_nonascii_qname(self,ascii_str):return wdns.domain_to_str(ascii_str)def remove_nonascii_bailwick(self,dns_query):section = wdns.parse_message(dns_query).secquestion_section,answer_section,authority_section= section[0],section[1],section[2]return question_section,answer_section,authority_sectiondef process(self):count_record = 0for x in range(0,self.payload_count+1):self.data_transit = self.data.read()if self.data_transit['type'] == 'UDP_UNSOLICITED_RESPONSE':time_date = float(self.data_transit['response_time_sec'][0])date_time =  time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time_date))question_section,answer_section,authority_section= self.remove_nonascii_bailwick(self.data_transit['dns'])file = open(self.out_name+".txt","a")file.write("n+----------------------------------------------------+n")file.write("time: "+str(date_time)+"n")file.write("ip source: "+self.data_transit['query_ip']+"n")file.write("query name: "+self.remove_nonascii_qname(self.data_transit['qname'])+"n")file.write("response ip: "+self.data_transit['response_ip']+"n")file.write("query port: "+str(self.data_transit['query_port'])+"n")file.write("checksum: "+self.data_transit['udp_checksum']+"n")file.write("question section:n"+str(question_section)+"n")file.write("answer section:n"+str(answer_section)+"n")file.write("authority section:n"+str(authority_section)+"n")file.write("query type: "+self.convert_type(self.data_transit['qtype'])+"n")file.write("record code: "+self.convert_record(self.data_transit['rcode'])+"n")file.write("n+----------------------------------------------------+n")file.close()print "record:" + str(count_record)count_record += 1print "list:" + str(count_record)def main():parser = argparse.ArgumentParser()parser.add_argument("count",help="how many payload that you want extract from the file")parser.add_argument("output",help="write file")parser.add_argument("input",help="read file")args_file = parser.parse_args()data_nmsg = handle_payload(int(args_file.count),args_file.input,args_file.output)data_nmsg.parse_data()data_nmsg.process()main()Thats it happy converting :) 
Request Demo

Build your Cybersecurity or IT Career

Accelerate in your role, earn new certifications, and develop cutting-edge skills using the fastest growing catalog in the industry