Payoff Based IDS Evaluation

IDS are regularly evaluated by comparing their false positive and false negative rates on ROC curves. However, this mechanism generally ignores both the context within which the IDS operates and the attacker’s own ability to adapt to IDS behavior. In this paper, we propose an alternative strategy for evaluating IDS based around multiple strategies. Each strategy defines how an attacker profits from attacking a target, and describes victory conditions for the attacker and defender. By mapping the results of ROC analysis to these strategies, we produce results which evaluate defensive mechanisms by their capacity to frustrate an attacker.