Backfill data

Use cases

• An e-commerce company runs promotions across multiple time windows and needs to analyze the results, such as revenue and conversion rates.

  For example, an e-commerce company runs promotions during two periods: August 7, 2023, to August 14, 2023, and August 17, 2023, to August 24, 2023. In task orchestration, you can use the time variables startTime and endTime in a single instance SQL node. To analyze the first period, perform a backfill with the business time set to August 7, 2023. To analyze the second, set the business time to August 17, 2023.

  Note

  startTime is the start time of the promotion, and endTime is the end time of the promotion.

• An e-commerce company finds errors in its report data and needs to correct the data.

  For example, on August 28, 2023, the company discovers that a product's report data from August 17, 2023, is incorrect. You can use backfill to regenerate the data for that day.

Procedure

1. Log in to DMS 5.0.

2. In the top navigation bar, choose Data+AI > Data+AI > Task Orchestration.

   Note

   If you use the DMS console in simple mode, move the pointer over the icon in the upper-left corner and choose All Features > Data+AI > Data Development > Task Orchestration.

3. In the Task flow area at the bottom of the page, click the name of your target task flow to open its editor page.

4. In the upper-right corner of the page, click Go to O&M.

5. On the Published tasks tab, find the target task flow version and click Backfill in the Actions column.

6. Configure the backfill parameters.

   1. Specify the business time for the backfill task, accurate to the second. You can select Run at specified time or Run for a specified time range.

      Note

      • A time-range backfill can create an unlimited number of task flow instances.

      • Tasks in a time range run serially. The task for one business time must succeed before the next one can run.

      • To stop a running backfill, go to the O&M center, find the running task, and click Stop. If a task for any business time fails, the entire task flow stops automatically.

   2. Select the task nodes to backfill.

      In the Backfill panel, set the Type to Run at specified time or Run for a specified time range, select a Business Time, confirm the task nodes, and then click Confirm. Disabling a node also disables all its child nodes. The backfill process generates run records based on the current published version of the task flow. You can view these records on the scheduling trigger tab.

7. Click Confirm.

   To view the run logs for the backfill task, return to the task flow editor and click Go to O&M in the upper-right corner. On the Running history > Scheduling trigger tab, expand the target task and click View Logs.