A Game Theoretic Approach to Efficient Mixed Strategies for Intrusion Detection

Abstract

As information technology evolves, and as more intrusion detection (ID) techniques are developed, security architects face the problem of effectively integrating various detection techniques to improve overall detection performance while maintain a high level of efficiency in network operation. In this paper, we consider the problem of optimal intrusion detection strategy in a network environment where multiple ID techniques are deployed. We first formulate a zero-sum attacker/defender game. The objective of the defender is to decide an optimal mixed strategy (i.e., a distribution over a set of strategies with each corresponding to the use of a particular ID technique) that maximizes his expected detection gain. In contrast, the objective of the attacker is to decide an optimal mixed strategy (i.e., a distribution over a set of strategies with each corresponding to a specific attack type or anomaly pattern) that minimizes his expected detection loss. The minmax theorem guarantees an optimal equilibrium strategy pair, which provides a valuable quantitative measure of the contributions from different ID techniques to the overall detection efficiency. Such information can assist security architects in understanding the effectiveness of these techniques, and in selecting the appropriate intrusion detection techniques according to the expected attacks. We also formulate a non-zero-sum noncooperative attacker/defender game where the payoffs of players are non-strictly competitive. We show that this game achieves at least one Nash equilibrium that leads to a defense strategy for the defender. Examples are presented and discussed both analytically and numerically.