Anomaly Detection Using Agglomerative Hierarchical Clustering Algorithm

Intrusion detection is becoming a hot topic of research for the information security people. There are mainly two classes of intrusion detection techniques available till today namely anomaly detection techniques and signature recognition techniques. Anomaly detection techniques are becoming area of interest for the researchers and new techniques are developing every day. However, no techniques have been found to be absolutely perfect. Clustering is an important data mining techniques used to find patterns and data distribution in the datasets. It is mainly used to identify the dense regions and sparse regions in the datasets. The sparse regions were often considered as outliers. There are several clustering algorithms developed till today for the discovery outliers in the datasets. K-means algorithm. K-medoids algorithm, CLARA, CLARANS, DBSCAN, ROCK, BIRCH, CACTUS etc. are some of the popular algorithms dealing with numeric datasets, categorical datasets, spatial datasets or hybrid datasets. Clustering techniques have been successfully used in detection anomaly in dataset. The techniques were found to be useful in the design of a couple of anomaly based Intrusion Detection Systems (IDS). But most of clustering techniques used for these purpose have taken partitioning approach. In this article, we propose a different clustering algorithm for the anomaly detection on network datasets. Our algorithm is an agglomerative hierarchical clustering algorithm which tries to find clusters on the dataset consisting of both numeric and categorical datasets i.e. hybrid datasets. For this purpose, we define a suitable similarity measure on both numeric and categorical attributes available on any network datasets.