Statistical Modeling of Web Requests for Anomaly Detection in Web Applications

We present an algorithm for learning a statistical representation of web application communication. The algorithm estimates the average probability of every observed web request. If the estimated probability deviates from recent observations, the web request is classified as anomalous. With every classification result, the statistical model parameters are updated, so the algorithm gains on-line learning capabilities and it self-adjusts to the observed data without prior training. Experiments on log data from a social networking web site indicate high detection rates at acceptable false-alarm rates. Also, the evaluation shows that the degree of abnormality of a web request does not explain the potential danger.