With the rapid development of cloud computing, more and more small and medium-size enterprises purchase cloud database systems to replace their self-built and help maintain database service systems, in order to save human and material resources. However, most users will only use their bought cloud database systems but can hardly find and solve the problem of database performance degradation, because they lack experience in optimizing the performance of database management system. Therefore, it is necessary for cloud service providers to tune database system parameters for users in time to ensure and maintain better database system performance. For cloud service providers with hundreds of thousands of user instances, it is obviously unrealistic to entirely on database experts to optimize database parameters, so how to use AI technology to solve this problem has become increasingly important and urgent.
Supervised by Prof. Ke Zhou and cooperated with Tencent, Ji Zhang, a Ph.D. candidate from Information Storage and Optical Display Laboratory in Huazhong University of Science and Technology, proposed CDBTune, an end-to-end automatic cloud database tuning system using deep reinforcement learning, for the first time. CDBTune can build optimization model with a small amount of experience training data, and provide online automatic optimization service for cloud database users. The results of performance optimization are greatly improved compared with database experts, which also accelerates the efficiency of database maintenance.
The workflow of CDBTune mainly includes two parts: offline training and online tuning. Offline training utilizes standard workload generators to conduct stress testing on database instances. In this process, CDBTune collects training data and train a preliminary configuration recommendation model at the same time. As shown in Figure 1, when users or system administrators need to optimize the database performance, they can submit online tuning requests through corresponding interactive interface. At this time, the cloud controller sends online tuning requests, then fine-tunes the preliminary model according to the users’ real workload, and finally configures the corresponding recommended parameters in database. CDBTune repeatedly executes the above process until the performance meets the need of users or administrators. As shown in Figure 2, extensive experiments under diverse workloads and different types of databases demonstrate that the performance of CDBTune obviously outperforms existing tuning tools and DBA experts. Besides, CDBTune maintains a good adaptability even if the memory size, disk capacity, and workload (with type unchanged) change.
Figure 1. CDBTune System Interaction
Performance comparison under different workloads Impact of memory size/disk capacity change on the model
Figure 2. CDBTune performance testing results
This work has been accepted as a regular paper in SIGMOD 2019, which will be held in Amsterdam, the Netherlands, on June 30. ACM SIGMOD is class-A conference recommended by China Computer Federation (CCF), and it is also the top international conference in the field of database.
Prof. Ke Zhou’s team and Tencent set up Intelligent Cloud Storage Joint Research Center in 2018, which aims to build a leading intelligent cloud storage technology innovation and talents cultivation platform through strong cooperation. By attracting and bringing together top professionals, Tencent and Huazhong University of Science and Technology have jointly tackled key technical issues in distributed storage technology, high performance storage engine and business load forecasting, to break through technical difficulties of super-large scale cloud storage systems, and further promote the technological innovation and application of intelligent cloud storage technology. This accepted paper is one of the researches of Intelligent Cloud Storage Joint Research Center.
This work is supported by the Innovation Group Project of the National Natural Science Foundation of China (No.61821003) and National Key Research and Development Program of China (No.2016YFB0800402).
