AI and Automating Incident Response

Artificial Intelligence and Automating Incident ResponseUpdated August 2018Managing the sheer number of alerts that get raised by incident response systems on a monthly basis has reached the point of overwhelm in many organizations. For the most part, it’s reached the point where InfoSec professionals have been forced to dial back the sensitivity of detection systems or simply ignore a large percentage of alerts according to a recent survey cited by DarkReading.com. Attempting to manually review log files and threat intelligence quickly succumbs to the “Mongolian Horde” effect where simply throwing more bodies at the problem doesn’t yield a corresponding reduction in time taken. Automating incident response, though appealing in theory, has proven to be disappointing in practice due to numerous false positives and the lack of maturity of such automated systems. So what is the alternative? With security threats only continuing to increase, the need for automating incident response has become even more urgent.Most networks have a few layers of defense.  There are firewalls and intrusion protection systems (IPSs) deployed on the perimeter. These devices rely on a set of rules to do their work. In the case of IPSs, they go one step further and attempt to proactively prevent intrusions as they occur. But the software that contains the rules and signature lists they operate from must be manually updated on a regular basis similar to anti-virus software. The Holy Grail is to have such systems actually “learn” and in effect, self-update. In an ideal world, such smart systems would be able to react in realtime to zero-day exploits. There would be no need to learn the hard way and hope for better luck next time with a subsequent update. In the case of incident response management, self-learning systems that leverage artificial intelligence (AI) are already on the market with more in the pipeline.Existing tools such as SEIM make the life of security analysts somewhat easier, but they only go so far. Analysts must still respond to alerts (or not) and pour over log files to determine the nature of discovered threats and what action, if any, should be taken. It’s this process that AI seeks to automate by, in effect, creating a virtual security analyst. One such project with this goal in mind is a project at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).The MIT system continuously models threat data using machine learning. It does need to rely on periodic feedback from human analysts to guide it (teach) about the proper direction to take for anomalies it’s unfamiliar with. The system has achieved an impressive 85% success rate when it comes to detecting attacks, but perhaps more importantly; it has reduced the number of false positives. False positives are the bane of InfoSec specialists the world over when it comes to intrusion detection alerts.The MIT system has demonstrated improvements over similar AI approaches to threat analysis. Hand holding in the form of providing feedback to self-learning systems has been a deal breaker in many other systems causing many IT security departments to abandon them. The overhead for providing feedback typically exceeds any benefit derived from the automation it provides. The MIT system greatly reduces the number of abnormal events it presents to analysts thus reducing the load on their time. And after just a few iterations, the unsupervised system has been able to reduce the initial number of events requiring verification from 200 down to 30 or 40, a much more manageable quantity indeed.This is the ultimate goal of AI and what was previously dubbed The Holy Grail: a computer system that is able to learn on its own with minimal human intervention. We’re certainly not at the point of replacing human security analysts with machine learning systems, but hopefully, the day isn’t too far off where their jobs are made a lot easier with the help of AI. The net result will be less harried analysts who can focus on only the most urgent security threats which should in turn make for a higher degree of security within the networks and systems they oversee.