Abstract :-
Many host-based anomaly detection techniques have been proposed to detect code-injection attacks on servers. The vastmajority, however, are susceptible to mimicry attacks in which the injected code masquerades as the original server software,including returning the correct service responses, while conducting its attack. Behavioral distance, by which two diverse replicasprocessing the same inputs are continually monitored to detect divergence in their low-level (system-call) behaviors and hencepotentially the compromise of one of them, has been proposed for detecting mimicry attacks. In this paper, we present a novelapproach to behavioral distance measurement using a new type of Hidden Markov Model, and present an architecture realizing thisnew approach. We evaluate the detection capability of this approach using synthetic workloads and recorded workloads of productionweb and game servers, and show that it detects intrusions with substantially greater accuracy than a prior proposal on uringbehavioral distance. We also detail the design and implementation of a new architecture, which takes advantage of virtualization tomeasure behavioral distance. We apply our architecture to implement intrusion-tolerant web and game servers, and throughtrace-driven simulations demonstrate that it experiences moderate performance costs even when thresholds are set to detect stealthymimicry attacks.Index Terms”Intrusion detection, replicated system, output voting, system call, behavioral distance.