A stochastic optimization approach for parameter tuning of Support Vector Machines

Support Vector Machines (SVMs) are both mathematically well-funded and efficient in a large number of real-world applications. However, the classification results highly depend on the parameters of the model: the scale of the kernel and the regularization parameter. Estimating these parameters is referred to as tuning. Tuning requires to estimate the generalization error and to find its minimum over the parameter space. Classical methods use a local minimization approach. After empirically showing that the tuning of parameters presents local minima, we investigate in this paper the use of global minimization techniques, namely genetic algorithms and simulated annealing. This latter approach is compared to the standard tuning frameworks and provides a more reliable tuning method.