Lake cache

Prerequisites

An AnalyticDB for MySQL Enterprise Edition, Basic Edition, or Data Lakehouse Edition cluster is created.

Overview

How it works

lake cache works as follows:

1. The lake cache client forwards read requests for OSS data to the lake cache accelerator. The client then connects to a lake cache master node to request file metadata.

2. The lake cache master node returns the file metadata to the lake cache client.

3. Based on the metadata, the lake cache client sends a request to a lake cache worker to retrieve OSS data:

   • If the target file is in the cache space of the lake cache worker, the file is returned directly to the client.

   • If the target file is not in the cache space, the lake cache accelerator retrieves the file from OSS. The accelerator then returns the file to the client and caches it for future requests.

Billing

The lake cache space is billed on a pay-as-you-go basis. For more information, see Pricing for Enterprise Edition and Basic Edition and Pricing for Data Lakehouse Edition.

Usage notes

• lake cache is available only in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), Singapore, US (Silicon Valley), US (Virginia), and Indonesia (Jakarta).

  Important

  If you want to use the lake cache feature in other regions, submit a ticket.

• If a cache hardware failure occurs, data queries continue to run without interruption or errors, although performance may degrade. Cached data is reloaded from OSS, and query speed is restored after the process is complete.

• When the configured cache space is full, the accelerator replaces less frequently accessed files with more frequently accessed ones based on the cache eviction policy. To prevent files from being evicted, scale up the cache space.

Enable lake cache

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. Find the cluster that you want to manage and click the cluster ID.

2. Go to the Cluster Information page. In the Configuration Information section, click Configure next to Lake Cache.

3. In the Lake Cache dialog box, turn on the Lake Cache switch and configure the Disk Cache Settings.

4. Click OK.

Use lake cache

After you enable lake cache, when you read OSS data, you can accelerate OSS data reads by configuring the spark.adb.lakecache.enabled parameter in your Spark job configuration. The following is an example:

-- This is an example of using lake cache. Modify the code to run your Spark program.
SET spark.adb.lakecache.enabled=true;

-- Add your SQL statements here.
SHOW databases;

{
  "comments": [
    "-- This is an example of using lake cache. Modify the code to run your Spark program."
  ],
  "args": ["oss://testBucketName/data/readme.txt"],
  "name": "spark-oss-test",
  "file": "oss://testBucketName/data/example.py",
  "conf": {
    "spark.adb.lakecache.enabled": "true"
  }
}

Monitor lake cache

After you enable lake cache, you can use the CloudMonitor console to verify that your Spark applications are using the cache and to view metrics such as data read volume. To do so, perform the following steps:

1. Log on to the Cloud Monitor console.

2. In the left-side navigation pane, choose Cloud Resource Monitoring > Cloud Service Monitoring.

3. Hover over the AnalyticDB for MySQL card and click AnalyticDB for MySQL 3.0 - Data Lakehouse Edition.

4. Find the target cluster and click Monitoring Charts in the Actions column.

5. Click the LakeCache Metrics tab to view the details.

   The following table describes the monitoring metrics.

   Metric	Description
   LakeCache Cache Hit Ratio (%)	The percentage of read requests fulfilled by the cache. Formula: (Cache Hits / Total Read Requests) × 100%.
   LakeCache Cache Usage (B)	The amount of used cache space, in bytes.
   Total Data Read from LakeCache (B)	The total amount of data read from the cache space, in bytes.