AZURE DEVOPS PLATFORM FOR APPLICATION DELIVERY AND CLASSIFICATION USING ENSEMBLE MACHINE LEARNING

Abstract

 When it comes to software-intensive systems, system logs play a crucial role in Azures. Logs capture the system's status and major occurrences at crucial times in time. The utility of log entries for data analysis and machine learning is severely limited due to their ad hoc, unstructured, and uncoordinated creation. Particularly in a DevOps setting, activities in data automation pipelines, visualizations, and analytics are prone to frequent disruptions due to uncontrolled change in log data format. Using a comprehensive case study at a top telecommunications firm as a foundation, this article outlines the primary obstacles faced by current methods of creating, storing, and overseeing the development of system logs data for sophisticated, big, software-intensive systems. Secondly, we provide a solution that is designed for machine learning and avoids the aforementioned problems when it comes to creating and controlling the development of log information in a Microsoft Azure DevOps environment. Third, we validate the strategy by conducting expert interviews, which prove that it solves the difficulties and challenges that have been identified. As a result, ML models that are fine-tuned for a certain dataset may rapidly become insufficient. Due to modifications to the characteristics and input data, a model that was once quite accurate may become inaccurate over time. Consequently, it is common to need distributed learning that incorporates dynamic model selection. Such a selection process involves replacing underperforming models with new ones, even if the old ones were fine-tuned for the previous data. It is possible to enhance the general accuracy of a set of ML models by using the famous Ensemble ML (EML) technique. There are a number of drawbacks to EML that should be considered. These include the lengthy model-building process, significant risks of overfitting, extensive training dataset requirements, costly processing resources, and the need for continual training. This research presents a new cloud-based approach to autonomous ML model tweaking and selection that outperforms current approaches in terms of automation of both model construction and selection. Prior to the automatic construction of focused supervised learning models, we employ unsupervised learning to get a deeper understanding of the data space. Specifically, we use a novel autoscaling technique to generate and assess ML algorithm instantiations on the fly, and we build a Cloud DevOps framework for autotuning with selection using container orchestration and communications between containers. Datasets pertaining to cloud network security are used to illustrate the suggested technique and tool.