Technical challenges and innovations

Updated at:

PolarDB is built on a storage-compute separation architecture that resolves two core tensions in cloud database design: how to scale without sacrificing reliability, and how to process transactions efficiently beyond the limits of a single node. This document explains the architectural decisions behind PolarDB and the trade-offs between distributed architectures.

How PolarDB balances scalability and reliability

Cloud services must scale as business grows — computing and storage resources need to expand without disrupting running workloads. At the same time, reliability, measured by recovery point objective (RPO) and recovery time objective (RTO), cannot be compromised.

PolarDB addresses both requirements through its shared storage architecture. The compute and storage layers have distinct responsibilities:

Layer Responsibility Scaling behavior
Compute Receives and processes queries Add a node in under 1 minute, regardless of data volume; no data copy required
Storage Stores each user's data across multiple nodes Expands dynamically without interrupting running workloads

Compute layer details

  • New nodes attach directly to shared data replicas — no backup or copy required.

  • PolarProxy, the built-in load balancer, distributes traffic so that node additions and removals are invisible to applications.

  • Resources are isolated per user at the compute layer where possible, giving each user exclusive compute access.

  • PolarDB also implements multitenancy at the network layer and the storage layer.

Storage layer details

  • Data is distributed across multiple storage nodes to minimize per-node waste and maintain low latency and sufficient I/O bandwidth.

  • If resources on a node become insufficient or some data stored on a node is frequently accessed, PolarDB automatically migrates the data from the node to other nodes.

  • Parallel-Raft, PolarDB's replication technology, maintains three replicas per data record, with at least two replicas flushed to disk per I/O operation.

  • Storage links run in user mode to reduce overhead and improve performance.

Failure resilience

Failure scenario Impact Recovery
A node fails Queries on that node stop A replacement node attaches immediately — no data replay or transfer required
Node is removed Traffic is rerouted PolarProxy and shared storage make the switchover invisible to applications
Write occurs Data is at risk until flushed Parallel-Raft ensures at least two on-disk replicas per write, minimizing data loss exposure (RPO)

This produces two concrete reliability guarantees:

  • Minimal RPO: Parallel-Raft ensures at least two on-disk replicas per write.

  • Minimal RTO: Because data is almost fully synchronized across nodes through shared storage, a failed node can be replaced immediately.

Shared-nothing vs. storage-compute separation

Distributed databases have evolved through two major architectures before reaching today's storage-compute separation approach.

Shared-disk architecture

In a shared-disk architecture, multiple compute nodes share a common storage area network (SAN). Adding IOPS or storage capacity requires expanding the underlying SAN hardware, which is costly and difficult to scale. Shared-disk systems are primarily used for high-availability multi-active scenarios rather than general-purpose scale-out.

Shared-nothing architecture

Shared-nothing architecture dominated early distributed database design, particularly for web-scale workloads. Each node owns its compute and storage resources independently:

  • Data is sharded for parallel access and processing.

  • Local compute and storage maximize I/O bandwidth and minimize latency — a critical advantage when network I/O was the primary bottleneck.

However, shared-nothing architecture has significant limitations:

Limitation Impact
Cross-shard operations are complex Distributed transactions and global indexes require expensive coordination
Compute and storage scale together Expanding a cluster forces simultaneous storage and compute expansion, often wasting resources
Scale-out requires data redistribution Migrating records during expansion consumes significant compute and degrades performance
Data is siloed per node Cross-database and cross-application queries become difficult

Why storage-compute separation became the standard

Starting in 2001, Google File System (GFS) demonstrated that commodity x86 servers and disks could provide large-scale distributed storage. However, GFS still coupled compute and storage on some nodes due to limited network bandwidth at the time.

As data center networks advanced to 25G, 50G, and 100G speeds, the bottleneck shifted from network I/O to CPU utilization. At these speeds, storage-compute separation becomes practical and efficient:

Component Hardware profile Scaling model
Storage nodes High-density, low-power hardware Scale out independently for capacity and throughput
Compute nodes Large-memory, high-clock-speed CPUs Scale out independently for query processing

Additional benefits over shared-nothing:

  • No record migration during scale-out: Expanding storage does not trigger data movement, eliminating transaction integrity and consistency concerns during scaling.

  • Independent scaling: Storage and compute expand separately, on demand, without wasting the other resource.

Public cloud providers now use web-based block storage as the default infrastructure. In this environment, storage-compute separation is the mainstream architecture for both online analytical processing (OLAP) and online transaction processing (OLTP) database systems.

Distributed vs. centralized transaction processing

Transaction processing ensures ACID (atomicity, consistency, isolation, and durability) guarantees. When multiple transactions access the same data concurrently, databases rely on multiversion concurrency control (MVCC) and optimistic concurrency control (OCC) to order execution correctly and return consistent results.

Centralized transaction processing

When all transactions are scheduled on a single node, the commit order defines the execution order. This gives strict external consistency without coordination overhead.

The limitation is scale. A single node bounds transaction processing capacity and the range of data the system can operate on. As workloads grow, the single node becomes a ceiling.

Distributed transaction processing

Distributed transaction processing removes the single-node ceiling by spreading transactions across multiple nodes. Transactions scale horizontally, and the data range is no longer limited by what one node can access.

The trade-off is coordination complexity. Operations that are straightforward in centralized systems — such as transactions spanning multiple shards or maintaining global indexes — require additional coordination steps across nodes.

Choosing between centralized and distributed

Characteristic Centralized Distributed
Consistency model Strict external consistency, no coordination overhead Consistent ordering across nodes requires coordination protocols
Scale ceiling Single node bounds capacity Horizontal scale with no single-node limit
Implementation complexity Simple Higher — cross-shard transactions and global indexes require extra coordination
Best fit Workloads that fit within a single node Workloads that require horizontal scale