ZeroTune 2.0: Enhanced Meta-parameter Selection and Optuna Integration for Zero-Shot Hyperparameter Optimisation

Hyperparameter optimisation is crucial for maximising machine learning model performance. Still, it is computationally intensive due to the iterative nature of hyperparameter optimisation methods like SMAC, SMBOX, and frameworks like Optuna. We introduce ZeroTune 2.0 (ZT2). This zero-shot hyperparameter optimisation method predicts near-optimal hyperparameters in a single step by leveraging a pretrained model built from a large knowledge base of prior hyperparameter optimisation trials. By integrating with the Optuna framework, incorporating an extensive set dataset meta-parameters, and implementing an automated recursive feature selection, ZT2 improves upon the existing ZeroTune algorithm and significantly reduces computational overhead compared to methods like Bayesian optimisation. Benchmarking shows it outperforms a state-of-the-art iterative optimisation method in the early stages, saving up to five hyperparameter optimisation iterations. These findings demonstrate the effectiveness of ZT2 as an efficient, practical, and adaptable solution for hyperparameter optimisation.