This topic describes common scenarios for error code ODPS-0010000 (System internal error) and provides solutions for each.
Error 1: fuxi job failed, caused by: kWorkerOutOfMemory(errCode:256) at Odps/xxx/xxx. Detail error msg: KILL_NAKILL_NA: plannedResource({Memory=xxx, CPU=xxx}) usedResource({Memory=xxx, CPU=xxx})
Error message
ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory(errCode:256) at Odps/xxx/xxx. Detail error msg: KILL_NAKILL_NA: plannedResource({Memory=xxx, CPU=xxx}) usedResource({Memory=xxx, CPU=xxx})
Cause
The job was killed because actual resource usage exceeded the planned allocation. The plannedResource field shows the allocated memory and CPU. The usedResource field shows what the job actually consumed at runtime.
Solution
Read the plannedResource and usedResource values from the error message to determine how much additional memory is needed. The task type is indicated by the letter prefix in the job name: M (mapper), R (reducer), or J (joiner).
Use SET statements to adjust the parameters for the affected stage. For example, for a mapper stage, adjust odps.stage.mapper.mem and odps.stage.mapper.split.size. For the full list of configurable parameters, see SET operations.
Error 2: fuxi job failed, caused by: Merge into statement not support update or delete duplicate row
Error message
ODPS-0010000:System internal error - fuxi job failed, caused by: Merge into statement not support update or delete duplicate row, please check your query, the query text location is from [line m, column n] to [line i, column j]
Cause
MERGE INTO does not support updating or deleting duplicate rows. The join condition between line m, column n and line i, column j matches multiple rows on one or both sides, producing duplicate output rows that cause the UPDATE or DELETE to fail.
Solution
Check whether the join condition between line m, column n and line i, column j matches multiple rows with the same value. Deduplicate those rows before running the MERGE INTO statement.
Error 3: OSSRequestException: req_id: xxxxxx, http status code: 400, error code
Error message
ODPS-0010000:System internal error - build/release64/common/io/oss/oss_client.cpp(99): OSSRequestException: req_id: xxxxxx, http status code: 400, error code: InvalidArgument, message: Authorization header is invalid
Cause
The OSS service experienced a transient interruption that caused momentary unavailability. This is a temporary service instability on the OSS side, not a permissions issue.
Solution
Retry the job. The error resolves once the OSS service recovers.
Error 4: OSSRequestException: Authorization header is invalid
Error message
ODPS-0010000:System internal error - build/release64/common/io/oss/oss_client.cpp(99): OSSRequestException: req_id: xxxxx, http status code: 400, error code: InvalidArgument, message: Authorization header is invalid
Cause
The current user does not have permission to access the OSS external table. Unlike error 3, this is a persistent permissions issue, not a transient service interruption.
Solution
Grant the user permission to access the corresponding OSS external table.
Error 5: Shortcut optimize failed because the script is too complex
Error message
ODPS-0010000:System internal error - Shortcut optimize failed because the script is too complex
Cause
The SQL statement is too complex for shortcut optimization to process.
Solution
Simplify the SQL statement. Common approaches:
Split a large, complex statement into smaller queries.
Reduce deeply nested subqueries.
Minimize the number of JOIN operations.
Reduce the use of aggregate functions or window functions where possible.
Use a WHERE clause to limit the query scope and reduce the amount of data processed.
Query only the necessary partitions by using partitioned tables.
Error 6: alloc too large size
Error message
ODPS-0010000:System internal error - fuxi job failed, caused by: alloc too large size: xxx
Cause
The job attempted to allocate an excessively large block of memory and failed. This can be caused by a MaxCompute internal issue or by abnormal data, such as an ARRAY column containing an unusually large number of elements.
Solution
Check whether any ARRAY columns contain an abnormally large number of elements and reduce the data volume. If the issue persists, submit a ticket to get technical support.
Error 7: Regex compile error
Error message
ODPS-0010000:System internal error - fuxi job failed, caused by: Regex compile error
Cause
The regular expression in the SQL statement is invalid.
Solution
Fix the regular expression syntax in the SQL statement.
Error 8: failed to get error message of java exception because of outofmemoryerror
Error message
ODPS-0010000:System internal error - failed to get error message of java exception because of outofmemoryerror
Cause
The job ran out of memory, preventing the JVM from generating a meaningful error message.
Solution
Increase the mapper memory and CPU allocation:
SET odps.sql.mapper.cpu=400;
SET odps.sql.mapper.memory=8192;
Error 9: codeGenCallBuiltinFunction fail in CopmareBase for function
Error message
ODPS-0010000:System internal error - fuxi job failed, caused by: codeGenCallBuiltinFunction fail in CopmareBase for function: NEQ, xxx
Cause
This error commonly occurs when a not-equal (NEQ) operation is applied to an ARRAY column or expression that does not support this operator. For example, see COLLECT_SET.
Solution
Replace the NEQ operation with a function that supports ARRAY types, such as SIZE(ARRAY)!=0. For details, see SIZE. Alternatively, upgrade MaxCompute to the latest version.
Error 10: process core dumped
Error message
ODPS-0010000:System internal error - fuxi job failed, caused by: process core dumped
Cause
This error commonly occurs when a not-equal (NEQ) operation is applied to an ARRAY column or expression that does not support this operator. For example, see COLLECT_SET.
Solution
Replace the NEQ operation with a function that supports ARRAY types, such as SIZE(ARRAY)!=0. For details, see SIZE. Alternatively, upgrade MaxCompute to the latest version.
Error 11: fuxi job failed, caused by: kWorkerOutOfMemory
Error message
ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory(errCode:256) at Odps/autonavi_location_20230831212 329151gio81xoniqh_SQL_0_1_0_job_0/R2_1@68d01417.na62#4533.
Detail error msg: KILL_NAKILL_NA: plannedResource ({Memory=4096, CPU=100}) usedResource ({Memory=5751, CPU=164}).
Cause
The job path
SQL_0_1_0_job_0/R2_1identifies the failure as an out-of-memory (OOM) event in the R2_1 portion of the Reduce stage.-
Reading the resource fields:
plannedResource({Memory=4096, CPU=100})— the memory (MB) and CPU (%) allocated to this task.usedResource({Memory=5751, CPU=164})— the actual memory and CPU consumed at runtime.
Solution
Set the memory and CPU for the stage where OOM occurred. For the Reduce stage shown in this example, run:
set odps.sql.reducer.memory=6144;— set this to any value greater than theusedResourcememory (5751 MB in this example).set odps.sql.reducer.cpu=200;— set this to any value greater than theusedResourceCPU (164 in this example).