Selective regular expression matching

Abstract

The signature-based intrusion detection is one of the most commonly used techniques implemented in modern intrusion detection systems (IDS). One of the powerful tools that gained wide acceptance in IDS signatures over the past several years is the regular expressions. However, the performance requirements of traditional methods for matching the incoming events against regular expressions are prohibitively high. This limits the use of regular expressions in majority of modern IDS products. In this work, we present an approach for selective matching of regular expressions. Instead of serially matching all regular expressions, we compile a set of shortest patterns most frequently seen in regular expressions that allows to quickly filter out events that do not match any of the IDS signatures. We develop a method to optimize the final set of patterns used for selective matching to reduce the amount of redundancy among patterns while maintaining a complete coverage of the IDS signatures set. Our experimental results on the DARPA data set and a live network traffic show that our method leads on average to 18%-34% improvement over a commonly used finite automata-based matching approach. © 2011 Springer-Verlag.