SMART RESOURCE MANAGEMENT FOR BUDGET-CONSTRAINED HYPERPARAMETER TUNING

Abstract

Tuning of hyperparameters is crucial in improving the efficiency of the machine learning algorithms as well as the general applicability of the model. However, common hyperparameter tuning techniques like grid search and random search are time-consuming and require a lot of computational power, especially when working with big data or complex architectures. The objective of this work is to learn on how to solve the hyperparameter tuning problem under the constraint of bounded computation time using Bayesian optimization. Since Bayesian optimization searches for the hyperparameter, it uses probabilistic models and the acquisition function to minimize the costs of the process between exploration and exploitation.

We apply the technique demonstrated above using Python and the scikit-optimize library to deal with the Iris dataset and find the optimal hyperparameters of the SVC model. The comparison of the models’ performance in Figure 4 implies that when selecting hyperparameters, it is possible to achieve high accuracy after a certain number of epochs and spend a minimal amount of computational resources. Some of the best performance’s best hyperparameters to use are C=4. 314 and gamma=0. 1, with the average cross-validation accuracy of about 98 percent. 67%.

The utilization of minimal computational resource, low cost, more time to build models, and more appropriate for environments with limited resources: The study also elaborates on budget-constrained hyperparameter tuning. Additionally, we discuss challenges and potential future work like how to deal with high-dimensional search space, how to generalize this framework to different computing system, and how to incorporate techniques such as transfer learning and meta-learning. It is applied in health care, finance, and environment which aids in creating proper machine learning with inadequate resources.