Cross-region data replication

Use cases

When services span multiple regions, logs are distributed accordingly. Use data transformation in Log Service to centralize these logs through cross-region replication.

Procedure

1. On the data transformation page, enter your transformation statements and click Preview Data. Create a data transformation task.

   Note

   To replicate data without transformation, leave the statement field empty.

   In the editor, enter your transformation statements, select Quick or Advanced preview mode, and click Preview Data. The Transformation Result tab shows the destination, timestamp, and content of each log. The Execution Result Summary panel displays the total logs processed, successful attempts, and failed attempts.

2. After you confirm that the preview result is correct, click Save as Transformation Job.

3. In the Create Data Transformation Job panel, configure the Destination Region.

   Create a data transformation task describes other parameters. Set the Destination name, verify the auto-generated Endpoint, and select transfer acceleration if needed. Specify the Destination Project, Destination Logstore, and authorization method (default role or custom role), then click Add.

FAQ

• How am I billed?

  Billing is based on compressed data volume. For example, 10 MB of data at a 10:1 compression ratio is billed as 1 MB. Billing of pay-by-feature items.

• How can I improve the data compression ratio?

  For web tracking, process the __topic__, __tag__, and __source__ fields to reduce data transmissions and improve the compression ratio. The compression unit is a LogGroup. Too many __source__, __topic__, and __tag__ combinations generate multiple LogGroups and lower the overall ratio.

  

• What can I do if the network connection is unstable?

  • Cross-region replication uses the public network. Stability is not guaranteed, especially for cross-border transfers. If the network is unstable, the data transformation task retries automatically.

  • Enable transfer acceleration to improve reliability.

    

    1. Enable transfer acceleration.

    2. In the Create Data Transformation Job panel, select Transfer Acceleration and click Project Transfer Acceleration Test.