Cross-region data replication

Use cases

If your services span multiple regions, logs are distributed across those regions. You can use data transformation to centralize these logs into a single location through cross-region replication.

Procedure

1. On the data transformation page, enter your transformation statements and click Preview Data. Create a data transformation task.

   Note

   To replicate data without transformation, leave the statement field empty.

   In the editor, enter your transformation statements, select Quick or Advanced preview mode, and click Preview Data. The Transformation Result tab shows the destination, timestamp, and content of each log. The Execution Result Summary panel displays the total logs processed, successful attempts, and failed attempts.

2. After you confirm that the preview result is correct, click Save as Transformation Job.

3. In the Create Data Transformation Job panel, configure the Destination Region.

   Create a data transformation task describes other parameters. Set the Destination name, verify the auto-generated Endpoint, and select transfer acceleration if needed. Specify the Destination Project, Destination Logstore, and authorization method (default role or custom role), then click Add.

FAQ

• How am I billed?

  Billing is based on compressed data volume. For example, 10 MB of data at a 10:1 compression ratio is billed as 1 MB. Billing of pay-by-feature items.

• How can I improve the data compression ratio?

  For web tracking, process the __topic__, __tag__, and __source__ fields to reduce data transmissions and improve the compression ratio. The compression unit is a LogGroup. Too many __source__, __topic__, and __tag__ combinations generate multiple LogGroups and lower the overall ratio.

  

• What can I do if the network connection is unstable?

  • Cross-region replication uses the public network, which may be unstable, especially for cross-border transfers. If disruptions occur, the data transformation task retries automatically.

  • Enable transfer acceleration to improve reliability.

    

    1. Enable transfer acceleration.

    2. In the Create Data Transformation Job panel, select Transfer Acceleration and click Project Transfer Acceleration Test.