DocsICP FilingConsole
Documentation

CreateJob

Updated at:
Copy as MD
Product
My Favorites

Creates a job to run in a cluster. You can specify the data source configuration, code source configuration, startup command, and compute resource configuration for each node where the job runs.

Operation description

Make sure that you fully understand the billing methods and pricing of PAI-DLC before you call this operation.

Important The total length of the parameters for the CreateJob operation, including system-generated parameters, cannot exceed 65,536 bytes.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

  • Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.

  • API: The API that you can call to perform the action.

  • Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.

  • Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.

    • For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.

    • For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.

  • Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.

  • Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

paidlc:CreateJob

create

*All Resource

*

 None None

Request syntax

POST /api/v1/jobs HTTP/1.1

Request parameters

Parameter

Type

Required

Description

Example
body

object

No

The request body parameters.

DisplayName

string

Yes

The name of the job. The name must meet the following requirements:

  • The name cannot exceed 256 characters in length.

  • The name can contain digits, letters, underscores (_), periods (.), and hyphens (-).

tf-mnist-test
JobType

string

Yes

The type of the job. This parameter is case-sensitive. Valid values:

  • TFJob

  • PyTorchJob

  • MPIJob

  • XGBoostJob

  • OneFlowJob

  • ElasticBatchJob

  • SlurmJob

  • RayJob

  • DataJuicerJob

Valid values:

  • OneFlowJob :

    OneFlow

  • PyTorchJob :

    PyTorch

  • SlurmJob :

    Slurm

  • ElasticBatchJob :

    ElasticBatch

  • XGBoostJob :

    XGBoost

  • DataJuicerJob :

    DataJuicer

  • MPIJob :

    MPIJob

  • TFJob :

    Tensorflow

  • RayJob :

    Ray

TFJob
JobSpecs

array

Yes

JobSpecs describes various configurations for the job at runtime, such as the image address, startup command, node resource declaration, and number of replicas.

A DLC job consists of different types of nodes. Nodes of the same type have identical configurations, which are referred to as a JobSpec. JobSpecs describes the configurations for all types of nodes and is an array of JobSpec objects.

JobSpec

No

The runtime configuration of the job.

UserCommand

string

Yes

The startup command for all nodes of the job.

python /root/code/mnist.py
DataSources

array<object>

No

The list of data sources used for running the job.

object

No

The list of all data sources used by the job. The data sources are mounted to the local directories of the containers where each node resides based on the configurations in the data sources. The local directories are specified by the MountPath parameter in the data source configurations.

The processes in the startup command of the job directly access the distributed file system represented by each data source by using the MountPath.

DataSourceId

string

No

The ID of the data source. For more information about how to view the data source ID, see ListDatasets.

d-cn9dl*******
MountPath

string

No

The path to be mounted for the job. This parameter is optional. By default, the mount path in the data source is used.

/root/data
Uri

string

No

The path of the data source.

oss://bucket.oss-cn-hangzhou-internal.aliyuncs.com/path/
Options

string

No

The mount property of the custom dataset. Currently, only OSS is supported.

{ "fs.oss.download.thread.concurrency": "10", "fs.oss.upload.thread.concurrency": "10", "fs.jindo.args": "-oattr_timeout=3 -oentry_timeout=0 -onegative_timeout=0 -oauto_cache -ono_symlink" }
DataSourceVersion

string

No

The version of the dataset.

v1
MountAccess

string

No

The permissions when the dataset is mounted. Valid values:

  • RO: read-only mount.

  • RW: read-write mount.

RO
EnableCache

boolean

No

Specifies whether to enable the cache service for the data source. This parameter takes effect only when the cluster supports it and a resource group is configured.

AccessPointId

string

No

The ID of the access point. Currently, only access points of CPFS for AI are supported.

RoleChain

string

No

The role chain, which is a JSON-formatted string. Example: [{"roleType":"service","roleArn":"acs:ram::Cloud service resource account UID:role/xxxtodlcrole","assumeRoleFor":"Cloud service resource account UID"},{"roleType":"user","roleArn":"acs:ram::Cloud service account UID:role/roletoassumecustomerrole"},{"roleType":"service","roleArn":"acs:ram::End user UID:role/use-bmcpfs-access-ap-role","assumeRoleFor":"End user UID"}]

[{"roleType":"service","roleArn":"acs:ram::Cloud service resource account UID:role/xxxtodlcrole","assumeRoleFor":"Cloud service resource account UID"},{"roleType":"user","roleArn":"acs:ram::Cloud service account UID:role/roletoassumecustomerrole"},{"roleType":"service","roleArn":"acs:ram::End user UID:role/use-bmcpfs-access-ap-role","assumeRoleFor":"End user UID"}]
RoleArn

string

No

The ARN of the data source role. This parameter is required when OSS and CPFS access point authentication are enabled.

acs:ram::1872xxxxxxxxx:role/account-pai-prod
UserVpc

object

No

The VPC configuration of the user.

VpcId

string

No

The ID of the user VPC.

vpc-abcdef****
SwitchId

string

No

The ID of the user vSwitch. This parameter is optional.

  • If this parameter is left empty, the system automatically selects an appropriate vSwitch based on inventory.

  • You can also specify a vSwitch ID.

vs-abcdef****
SecurityGroupId

string

No

The ID of the user security group.

sg-abcdef****
ExtendedCIDRs

array

No

The extended CIDR blocks.

  • If the vSwitch ID is empty, this parameter is optional and the system automatically obtains all CIDR blocks under the VPC.

  • If the vSwitch ID is not empty, this parameter is required. We recommend that you specify all CIDR blocks under the VPC.

string

No

The extended CIDR block.

192.168.0.1/24
DefaultRoute

string

No

The default routing. Valid values:

eth0
CodeSource

object

No

The code source used by the job. Before the nodes of the job start, DLC automatically downloads the code configured in the code source and mounts it to the local directory of the container.

CodeSourceId

string

No

The ID of the code source. For more information about how to obtain the code source ID, see ListCodeSources.

code-20210111103721-xxxxxxx
Branch

string

No

The branch of the referenced code repository when the job runs. This parameter is optional. By default, the branch configured in the code source is used.

master
Commit

string

No

The commit ID of the code to be downloaded for the job. This parameter is optional. By default, the commit ID configured in the code source is used.

44da109b5******
MountPath

string

No

The path to be mounted for the job. This parameter is optional. By default, the mount path configured in the code source is used.

/root/data
IsSharedMountPath

boolean

No

Specifies whether the mount path configured in access code source is a shared cloud storage path. If this parameter is set to true, code clone optimization is enabled in the background. In multi-node job scenarios, the clone operation is performed on only one node, and other nodes can directly access access code through the shared cloud storage path.

ThirdpartyLibs

array

No

The list of Python third-party libraries to be installed.

string

No

Specifies a third-party Python library and the corresponding version requirement, such as numpy==1.16.1.

numpy==1.16.1
ThirdpartyLibDir

string

No

The name of the folder where the Python third-party library file (requirements.txt) is located. Before each node runs the specified UserCommand, PAI-DLC retrieves the requirements.txt file from the specified folder and runs the pip install -r command to install the libraries.

/root/code/
Envs

object

No

The environment variable configuration.

string

No

An environment variable in the Key=Value format.

Important Environment variables must comply with the POSIX standard.

ENABLE_DEBUG_MODE
JobMaxRunningTimeMinutes

integer

No

The maximum running duration of the job, in minutes.

1024
WorkspaceId

string

No

The ID of the workspace. For more information about how to obtain the workspace ID, see ListWorkspaces.

ws-20210126170216-xxxxxxx
ResourceId

string

No

The ID of the resource group. This parameter is optional.

  • If this parameter is left empty, the job is submitted to the public resource group.

  • If the current workspace is attached to a resource quota, you can specify the corresponding resource quota ID here. For details about how to query the resource quota ID, see Manage resource quotas.

rs-xxx
Priority

integer

No

The priority of the job. This parameter is optional. Default value: 1. Valid values: 1 to 9.

  • 1: the lowest priority.

  • 9: the highest priority.

8
Settings JobSettings

No

The additional parameter settings for the job.

ElasticSpec JobElasticSpec

No

This parameter is not supported. Ignore this parameter.

DebuggerConfigContent

string

No

This parameter is not supported. Ignore this parameter.

“”
Options

string

No

The additional configuration for the job. You can use this parameter to adjust the behavior of mounted data sources. For example, if the job mounts an OSS data source, you can set this parameter to fs.oss.download.thread.concurrency=4,fs.oss.download.queue.size=16 to overwrite the default parameter settings of JindoFS.

key1=value1,key2=value2
SuccessPolicy

string

No

The success policy for distributed multi-node jobs. Currently, only multi-node TensorFlow jobs are supported. Valid values:

  • ChiefWorker: If this value is specified, the entire job is considered successful as long as the chief pod ends successfully.

  • AllWorkers (default): The entire job is considered successful only when all worker pods end successfully.

AllWorkers
CredentialConfig CredentialConfig

No

The access credential configuration.

Accessibility

string

No

The visibility of the job. Valid values:

  • PUBLIC: Visible to all users in the workspace.

  • PRIVATE: Visible only to you and administrators in the workspace.

PRIVATE
TemplateId

string

No

The ID of the job template.

tplabc1234567
TemplateVersion

integer

No

The version of the job template.

1
Description

string

No

The description of the job.

This is a training job
SchedulingStrategy

string

No

The scheduling strategy.

Valid values:

  • Auto
  • Binpack
  • Spread

Auto
CustomEnvs

array<object>

No

The list of custom environment variables.

object

No

Key

string

No

The name of the environment variable.

PUB
Value

string

No

The value of the environment variable.

keep
Visible

string

No

The visibility of the environment variable. Valid values: public and private.

public
DryRun

boolean

No

Specifies whether to enable dry run mode.

Response elements

Element

Type

Description

Example

object

The JSON-formatted string.

JobId

string

The ID of the job created by this call.

dlc7*******
RequestId

string

The ID of the request. It is used for troubleshooting and support.

473469C7-AA6F-4DC5-B3DB-xxxxxxx

Examples

Success response

JSON format

{
  "JobId": "dlc7*******",
  "RequestId": "473469C7-AA6F-4DC5-B3DB-xxxxxxx"
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.