A Futures Company: Enhancing DBPaaS Performance and Availability with IOMesh

Many users plan to migrate database applications to container environments to enhance resource delivery efficiency. However, ensuring that both performance and storage availability meet database demands on container platforms like Kubernetes is not an easy task.

Recently, a futures institution built a Database Platform as a Service (DBPaaS) using IOMesh, our Kubernetes-native storage solution, to provide high performance and high availability for the database platform. With IOMesh, the DBPaaS has significantly increased the efficiency of resource application and delivery while reducing the operational and maintenance burden.

In this blog, we will share detailed user practices, explaining how to build DBPaaS and how to select the right storage solution for the platform.

Background

This industry-leading futures company has gradually transitioned its IT infrastructure from physical servers to virtualization, cloud-native architecture, microservices, and distributed architecture. For the company’s IT infrastructure team, one of their primary responsibilities is to provide timely support to the business team, including quickly provisioning resources and maintaining system stability and reliability.

Initially, although the infrastructure team successfully delivered a significant amount of container resources and assisted the business team in deploying numerous containerized applications, most of these applications were stateless. The infrastructure team found it challenging to efficiently provision resources for stateful database applications. Key challenges include:

  1. Inefficient resource application: Whenever the business team needed to apply for database service resources, their engineers had to manually specify their requirements, leading to a lot of repetitive communication efforts.
  2. Slow resource delivery: The infrastructure team used physical servers to provide data services for database applications such as the Dameng database (1 primary and 1 backup) and MySQL database (1 primary, 1 backup and 1 secondary). Each time resources needed to be delivered, the team had to deploy physical servers, configuring each system with disaster recovery and backup servers. This process involved substantial workloads and significantly extended resource delivery times, often taking several days.
  3. Low resource utilization: Each business application required its own set of independent databases, and the resource utilization of each database cluster varied. This made it impossible to fully utilize the same set of hardware resources, resulting in noticeable waste. Furthermore, when new business applications required data services, resource delivery was further delayed due to the scarcity of hardware resources, creating a negative cycle.

To meet the data service needs of various business applications and improve the efficiency of resource requests and delivery, the futures company decided to build a DBPaaS based on Kubernetes clusters. With this platform, the business team could independently apply for database services through a graphical user interface (GUI). Despite the infrastructure team's extensive experience in the cloud-native field, deploying databases in containers presented significant challenges, especially concerning data storage:

  1. There is a significant difference between deploying databases on VMs and on Kubernetes. How to efficiently deploy production-grade and highly available database clusters on Kubernetes?
  2. How to efficiently manage different storage engines at the same time?
  3. How to improve storage data redundancy to provide double assurance for data security? This includes ensuring that even if the business team creates a single-node database cluster, the storage system can still guarantee data availability in the event of a database node failure.
  4. The storage system must provide high performance to cater to the core applications that will be deployed in the future.
  5. How can databases and storage seamlessly integrate to improve the team's operational and maintenance efficiency?

To address these questions, the infrastructure team decided to evaluate potential storage solutions for the DBPaaS.

Evaluation of Storage Solutions and Performance Tests

Database Platform

The futures company decided to use an RDS database cloud platform (hereinafter referred to as "RDS"). Based on cloud-native technologies, the RDS platform can provide mainstream database services such as Oracle, MySQL, and Dameng.

Storage

Initially, the futures company outlined three potential storage options, each with its own drawbacks:

  • Traditional solution: This involved combining RDS databases with existing distributed storage clusters and centralized storage clusters. However, this option was quickly dismissed in the initial evaluation due to its low performance, high cost, and low O&M efficiency.
  • Emerging programs:
    • Open-source cloud-native storage: This option was deemed unsuitable because it lacked long-term and production-grade use cases to verify its performance and stability. Additionally, its storage core was not independently developed, which raised concerns about its reliability.
    • LocalPV with local storage: While this option could provide high performance for database applications, it had insufficient data redundancy protection and complex O&M requirements.

For the three solutions above, the futures company was satisfied with LocalPV’s performance but concerned about its availability. This created a dilemma between performance and storage data security, prompting the company to seek a storage solution that offered both advantages.

While researching various enterprise-grade cloud-native storage products, the company learned about IOMesh, our Kubernetes-native distributed storage solution. With a strong storage core, IOMesh provides high-performance data storage services for stateful applications, along with production-grade high-availability features and O&M support. After an in-depth evaluation, the company decided to test IOMesh's performance and reliability against LocalPV in supporting RDS.

Performance Comparison: IOMesh Delivers High Performance Comparable to LocalPV and Excels in Availability

Test Environment

Hardware configuration

Software version

Performance Comparison

In the test, we created 32C128G Dameng database clusters (1 primary and 1 standby), MySQL database clusters (1 primary, 1 standby and 1 secondary) and PostgreSQL database clusters (1 master and 2 standbys) respectively, and conducted stress tests via sysbench on each database based on LocalPV and IOMesh. The results showed that in the three database use cases, the performance of IOMesh was very close to that of Local PV, which exceeded the user's expectations.

Reliability Comparison

In addition, to verify the reliability of IOMesh, the user simulated a node failure on the Damon database to verify the efficacy of database primary/backup switchover and master/backup cluster recovery.

  • When the primary database node fails, the primary database HA switchover occurs, and its startup time is less than the database master-slave switch time, so no master-slave switch occurs. At this time, the master-slave repair can be automatically completed in the background. Due to the multi-replica data protection, the repair time is relatively short.
  • When the primary database node fails, the primary database HA switchover occurs, and its startup time is greater than the database master-slave switch time, so the master-slave switch occurs. At this time, the master-slave repair can also be automatically completed in the background. Due to the multi-replica data protection, the repair time is relatively short.

In addition to the multi-replica strategy, IOMesh also provides second-level snapshot of PV, intelligent data recovery, and optimized Pod HA capabilities to enhance the reliability of container environment. For more information, please refer to: IOMesh Pod HA: Enhancing Kubernetes Cluster Availability. 

Constructing DBPaaS with IOMesh and RDS

Since the futures company was highly satisfied with IOMesh's performance and reliability in the tests, they ultimately decided to adopt IOMesh as the storage solution for the RDS platform. Currently, the company has deployed several IOMesh clusters in both production and test environments to provide high-performance and reliable storage support for RDS database services. With IOMesh and RDS, the company has built a DBPaaS that allows the business team to independently deploy database services in just three steps:

  1. Log in to the data management platform.
  2. Select the database service and the HA architecture to be deployed.
  3. Submit for deployment.

With IOMesh's superior performance, reliable data protection strategies and O&M-friendly features, the DBPaaS provides users with the following benefits:

  • Enhanced efficiency of resource application: Transformed the process from manual resource application and cross-departmental communication to automated deployment of database services with just a few clicks, taking only seconds.
  • Improved speed of resource delivery: Reduced the time required for data service delivery from "days" to "minutes" and even to "seconds".
  • Enhanced data storage reliability: Addressed the single-point failure risk and lack of reliable data protection in Local PV. IOMesh provides enhanced protection for storage data with production-grade HA features such as a multi-replica strategy and snapshot capabilities.
  • Improved architectural availability: With IOMesh multi-replica strategy (local priority, I/O locality, and capacity balancing), it supports on-demand switching of database nodes, master/standby switching, and other operations without the need for business team intervention.
  • Improved resource utilization: Before introducing IOMesh, users needed to deploy 22 nodes of physical machines to support 8 Damon database master/standby clusters and 2 MySQL database master/standby/slave clusters. After introducing IOMesh, only 3 IOMesh nodes are needed to support 10 RDS databases, and there are still applicable resources.
  • Resource and infrastructure monitoring: IOMesh provides a unified UI where users can view system I/O performance and the health status of all disks, helping the team to accurately identify O&M issues and optimized solutions.
  • Improved management efficiency: IOMesh and RDS can be seamlessly integrated and managed by the infrastructure team, eliminating the need for cross-departmental communication and reducing O&M burden.

Currently, the user's DBPaaS has been running stably for nearly a year with IOMesh's support. They are also leveraging IOMesh to support KubeVirt, providing unified storage services for both virtual machines and containers. Looking ahead, the company plans to expand the use of IOMesh clusters to support more critical business systems, messaging middleware, and other application scenarios. For more information on using IOMesh for persistent storage in KubeVirt, please refer to: Enhanced Performance and Availability: Using IOMesh for Persistent Storage in KubeVirt.