Partial submission for MaxFrame jobs

Retrieve partial results from a failed job

Once partial submission is enabled, you can retrieve results from successful Fuxi instances even if the job fails.

Set the following parameter at the job level to enable partial submission.

options.sql.settings = {
    "odps.sql.always.commit.result": "true"  # Enable partial submission.
}

• (Recommended) Write final results to a destination table

  The following example writes final results to the destination table data_output. Query this table to retrieve partial results.

  df.mf.flatmap(
      ... # Your business logic here
  ).to_odps_table("data_output").execute()

  You can also view the successful Fuxi instances in the job's Logview.

  WARNING:[0,0]  [MF_UDF_20250507084433360GZQPJR9T002_USER_UDF_xxx($0, $1, $2, $3, $4, $5)]: more optimization opportunities are
      possible if [MF_UDF_20250507084433360GZQPJR9T002_USER_UDF_xxx($0, $1, $2, $3, $4, $5)] is annotated as deterministic.
  resource cost: cpu 23.25 Core * Min, memory 29.06 GB * Min
  inputs:
      llm_mf_bj.llm_mf_bj.test_input: 87 (442569 bytes)
  outputs:
      llm_mf_bj.default.tmp_mf_20250507084433360gzqpjr9t0o2_c0d07f4412d0c87742def840ffddb d5f_0: 870 (43620 bytes, 87 files)
  --------------------------------------------CommitFileStatusInfo--------------------------------------------
  PartialCommitSummary(Because odps.sql.always.commit.result=true, and the job(SQL_0_0_0_job_0) is FAILED, so we commit partial result):
      llm_mf_bj.default.tmp_mf_20250507084433360gzqpjr9t0o2_c0d07f4412d0c87742def840ffddb d5f_0:
      Successfully committed Fuxi instance count: 87
      Total Fuxi instance count: 100
  --------------------------------------------JOB:SQL_0_0_0_job_0--------------------------------------------
  Job run time: 139.214
  Job run mode: fuxi job 2.0
  Job run engine: execution engine
  M1:
      instance count: 87
      rerun worker count: 1
      run time: 177.844
      instance time:
          min: 9.384, max: 25.759, avg: 16.035
      input records:
          TableScan1: 87  (min: 1, max: 1, avg: 1)
      input bytes:
          TableScan1: 442569  (min: 5087, max: 5087, avg: 5087)
      output records:
          TableSink1: 870  (min: 10, max: 10, avg: 10)
      output bytes:
          TableSink1: 43620  (min: 489, max: 512, avg: 501)
      output file count:
          TableSink1: 87  (size min: 489, max: 512, avg: 501)
      metrics_output_count:
          TableFunctionScan1: 870  (min: 0, max: 10, avg: 8)

• Query using a temporary table

  If the job does not write results to a destination table, find the auto-generated temporary table name in Logview and use it to retrieve partial results.

  df.mf.flatmap(
      ...
  ).execute()

  To find the temporary table name, go to the Job Details tab in Logview and select the Progress Chart view. The auto-generated temporary table name appears at the output node of the flowchart, in a format like j.default.tmp_mf_<timestamp>. Use this table name to query the partial results.

Set a timeout for partial results

For jobs prone to long-tail tasks or infinite loops, set a timeout. When the timeout is reached, the system terminates the job but retains results from successful Fuxi instances. You can retrieve these partial results from either a destination table or a temporary table.

Enable the partial submission feature and set a job timeout:

options.sql.settings = {
    "odps.sql.always.commit.result": "true",  # Enable partial submission.
    "odps.sql.job.max.time.hours": "1"        # Set the timeout to 1 hour.
}

Pay-as-you-go

MaxFrame jobs in pay-as-you-go mode are billed based on the actual CU-hours consumed.

You can download detailed usage information from the Usage Details page. For more information, see Analyze MaxCompute billing and usage details.

Subscription

MaxFrame jobs in subscription mode consume resources from the subscription resource group based on the requested amount. No additional fees apply.