任务循环

OOS提供了Task的特殊属性Loop,其用来支持对单个任务的循环。

API类action和云产品action的任务可支持循环,功能性的action(如Trigger、Sleep等)的任务不支持循环。

任务循环支持并发,不过单步模式执行下循环的任务则自动调整为依次执行无并发。

限制

Loop Items的个数限制当前最大为1000个。

语法

  • YAML语法结构

---
Tasks:
  - Name: TaskName1 # 必填,任务名称,有效字符为[a-zA-Z0-9_-],即英文字母数字下划线中划线,长度限制200个字符。建议使用驼峰式命名,如StartInstance。
    Action: TaskType # 必填,任务类型(也叫动作),请参考动作相关的文档。
    Description: description # 可选,描述,描述当前任务的用途。
    Properties: # 根据所使用的动作不同,具体的属性列表不同,请参考动作相关的文档。
      Property1: Value1 # 动作所依赖的属性和值。
    Outputs:  # 当前任务的输出参数,可以用作后续任务的输入或模板的输出。
      OutputParameterName1:
        Type: TypeName # 可选,输出参数类型,默认为String。
        ValueSelector: "jq selector" # 如Task是一个OpenAPI调用时,把OpenAPI的调用结果当做一个JSON输入,应用jq从该JSON中截取所需要的值,请参考云产品动作和公共模板等具体例子。
    Loop:
      Items: [i-id1,i-id2,i-id3,i-id4] #必填,循环元素,可接受一个包含Item的List,遍历List得到的元素,将作为循环任务每次执行时的参数;还可接受一个可被解析为List的参数名,如{{describeInstance.InstanceIds}}。
      RateControl:
          MaxErrors: 0 # 可选,允许数值或百分比的定义方式,如10或10%,默认为0。
          Mode: "Batch/Concurrency" # 必填,循环执行速率控制的模式,可选项为Concurrency或Batch;Concurrency表示并发执行,Batch表示分批执行。
          Batch:  [1, 2, 3] # 可选,该参数在Mode=Batch时生效,表示分批控制。定义方式参考下文批次的控制。
          ConcurrencyInBatches: [1, 1, 1]  #可选,该参数在Mode=Batch时生效,表示Batch模式下对每批次的并发度控制,默认20,定义方式为[]内填写每批对应的并发数。
          Concurrency: 1 # 可选,该参数在Mode=Concurrency时生效,表示所有执行并发数控制,允许数值或百分比。默认为1。
          BatchPauseOption: "FirstBatchPause/Automatic/EveryBatchPause" # 可选,该参数在Mode=Batch时生效,表示分批模式下,每批执行完成后暂停选项。可选项FirstBatchPause/Automatic/EveryBatchPause分别表示首批暂停,自动执行不暂停,每批暂停。
      Outputs:
        FinalOutputParameterName1: #循环任务经函数处理后输出的参数名称。
          AggregateType: BuiltInFunctionName1 # 选择一种内置函数方式对输出进行聚合如Fn::Sum、Fn::Max、Fn::ListJoin。
          AggregateField: OutputParameterName1 # 循环的元素执行输出的参数名称。
  • JSON语法结构(请参考YAML注释说明)

{
  "Tasks": [
    {
      "Name": "TaskName1",
      "Action": "TaskType",
      "Description": "description",
      "Properties": {
        "Property1": "Value1"
      },
      "Outputs": {
        "OutputParameterName1": {
          "Type": "TypeName",
          "ValueSelector": "jq selector"
        }
      },
      "Loop": {
        "Items": [
          "i-id1",
          "i-id2",
          "i-id3",
          "i-id4"
        ],
        "RateControl": {
          "MaxErrors": 0,
          "Mode": "Concurrency/Batch",
          "Batch": [
            1,
            2,
            3
          ],
          "ConcurrencyInBatches": [
            1,
            1,
            1
          ],
          "Concurrency": 1,
          "BatchPauseOption": "FirstBatchPause/Automatic/EveryBatchPause"
        },
        "Outputs": {
          "FinalOutputParameterName1": {
            "AggregateType": "BuiltInFunctionName1",
            "AggregateField": "OutputParameterName1"
          }
        }
      }
    }
  ]
}

循环列表Items

Items接受一个List,如一个直接的List:[item1, item2, item3]。

Items也可以是一个List类型的参数名,如上一个Task的List类型Outputs。

  • Items为一个List。 

    • YAML

    Loop:
      Items: [item1,item2,item3,item4] # 一个包含Item的List,如[i-id1,i-id2,i-id3,i-id4],Items迭代作为Loop所在Task执行的参数。					
    • JSON(请参考YAML注释说明)

    {
      "Items": [
        "item1",
        "item2",
        "item3",
        "item4"
      ]
    }		
  • Items为一个List类型的参数名。 

    • YAML

    Items: '{{ParameterName1}}' # 一个可被解析为List类型的参数名,如describeInstance.InstanceIds;Items迭代作为Loop所在Task执行的参数。	
    • JSON(请参考YAML注释说明)

    {
      "Items": "{{ParameterName1}}"
    }		

循环执行速率控制

分两种模式:

  1. 并发数控制,保持这个并发数的执行速率,直到所有item执行结束。

  2. 批次控制,把item明确地分成多个批次,然后一个批次运行完再运行下一个批次,若上一个批次未全部完成的情况下,下一个批次不会开始。

并发数的控制(Concurrency)

  • 如Items是10个item,并发是3,且会保持一直有3个item在并发,直到没有更多未执行的item。 

    • YAML

    Concurrency: 3 # 允许数值或百分比表并发控制,如3。
    Mode: Concurrency
    • JSON(请参考YAML注释说明)

    {
      "Concurrency": 3,
      "Mode": "Concurrency"
    }
  • 如Items是10个item,并发是20%,表示20%*10=2的并发,且会保持一直有2个item在并发,直到没有更多未执行的item。

    • YAML

    Concurrency: 20% # 可选,允许数值或百分比的定义方式,如20%。
    Mode: Concurrency
    • JSON(请参考YAML注释说明)

    {
      "Concurrency": "20%",
      "Mode": "Concurrency"
    }

批次的控制(Batch)

  • 如Items是10个item,批次是[3],会将item分为大小为3的多个批次,结果为3,3,3,1,一共4个批次(假设item的执行结果都是成功)。 

    • YAML

    Batch: [1, 2, 3] # 可选,允许List内含数值或百分比的方式定义表批次控制,如[3]。
    Mode: Batch
    • JSON(请参考YAML注释说明)

    {
      "Batch": [
        1,
        2,
        3
      ],
      "Mode": "Batch"
    }
  • 如Items是10个item,批次是[30%],表示第一个批次是3,会将item分为大小为10*30%=3的多个批次,结果为3,3,3,1,一共4个批次(假设item的执行结果都是成功):

    • YAML

    Batch: [30%] # 可选,允许List内含数值或百分比的方式定义表批次控制,如[30%]。
    Mode: Batch
    					
    • JSON(请参考YAML注释说明)

    {
      "Batch": [
        "30%"
      ],
      "Mode": "Batch"
    }
  • 如Items是10个item,批次是[3,10%,30%],表示第一个批次是3,第二个批次是总量的10%,第三个及以后批次为[30%],结果为3,1,3,3一共4个批次(假设item的执行结果都是成功)

    • YAML

    Mode: Batch
    Batch: [3, 10%,30%] # 可选,允许List内含多个数值或百分比的方式定义表批次控制,如[3, 10%,30%]。
    					
    • JSON(请参考YAML注释说明)

    {
      "Mode": "Batch",
      "Batch": [
        3,
        "10%",
        "30%"
      ]
    }

错误控制MaxErrors

在Loop中,您也可以定义MaxErrors,其意义如下:

  1. 允许数值或百分比(表现为Items乘以百分比)的定义方式,如10,或10%。默认为0,即任何错误都会暴露到loop-task级别。

  2. Loop里面的第一个Item,总会被执行。

  3. 是否能够执行到第二个Item或后续的Item取决于实际MaxErrors和ErrorCount的关系。 

    • 当实际ErrorCount>MaxErrors时,后续的Item停止执行,且当前Loop所在任务标记为失败。

    • 当实际的ErrorCount<=MaxErrors时,则持续向下执行。

  4. 根据ErrorCount>MaxErrorsLoop判定Loop所在任务的执行状态,如果大于则是Failed,否则是Success。

  • 数值方式定义。

    • YAML

      MaxErrors: 2 # 可选,允许数值或百分比的定义方式,如2,表示允许最大ErrorCount为2。
    • JSON(请参考YAML注释说明)

      {
       "MaxErrors": 2 
      }
  • 百分比方式定义。 

    • YAML

      MaxErrors:  25% # 可选,允许数值或百分比的定义方式,如25%,共4个Item则表示允许最大ErrorCount为25%*4=1。
    • JSON(请参考YAML注释说明)

      {
       "MaxErrors": "25%"
      }

循环输出Loop Outputs

Task或template的Loop也支持outputs,由于Loop的实际outputs中包含多个Item的outputs,因此Loop最终的outputs需要把多个子任务outputs进行聚合,您可通过定义AggreateType来选择不同内置函数的聚合方式。

定义AggreateType。

  • YAML

AggregateType: Fn::Max # 选择一种内置函数对Loop输出进行聚合,如Fn::Max,表示把所有子任务输出的Numer类型的最大的OutputParameterName1作为Loop的输出值。
  • JSON(请参考YAML注释说明)

{
"AggregateType": "Fn::Max"
}		

示例

并发模式

  • YAML格式

FormatVersion: OOS-2019-06-01
Description: Creates one or more ECS instances.
Parameters:
  regionId:
    Description: The ID of region.
    Type: String
    Default: '{{ ACS::RegionId }}'
  imageId:
    Description: The ID of the image resource that you specify when you create the
      instance.
    Type: String
    AllowedPattern: '[A-Za-z0-9_\-\.]*'
    MinLength: 1
    MaxLength: 100
  instanceType:
    Description: The type of the instance.
    Type: String
    AllowedPattern: ecs\.[A-Za-z0-9\.\-]*
    MaxLength: 30
    MinLength: 1
  securityGroupId:
    Description: The ID of the security group to which the instance belongs.
    Type: String
    AllowedPattern: sg-[A-Za-z0-9]*
    MaxLength: 30
    MinLength: 1
  vSwitchId:
    Description: The ID of the VSwitch.
    Type: String
    AllowedPattern: vsw-[A-Za-z0-9]*
    MaxLength: 30
    MinLength: 1
  amount:
    Description: The specified number of instances you want to create.
    Type: Number
    Default: 1
  internetMaxBandwidthIn:
    Description: The maximum inbound public bandwidth.
    Type: Number
    Default: 200
  internetMaxBandwidthOut:
    Description: The maximum outbound public bandwidth.
    Type: Number
    Default: 0
Tasks:
- Name: runInstances
  Action: ACS::ExecuteAPI
  Description: Creates one or more instances.
  Properties:
    Service: ECS
    API: RunInstances
    Parameters:
      RegionId: '{{ regionId }}'
      Amount: '{{ amount }}'
      ImageId: '{{ imageId }}'
      InstanceType: '{{ instanceType }}'
      SecurityGroupId: '{{ securityGroupId }}'
      VSwitchId: '{{ vSwitchId }}'
      InternetMaxBandwidthIn: '{{ internetMaxBandwidthIn }}'
      InternetMaxBandwidthOut: '{{ internetMaxBandwidthOut }}'
  Outputs:
    instanceIds:
      Type: List
      ValueSelector: InstanceIdSets.InstanceIdSet[]
- Name: untilInstanceReady
  Action: ACS::WaitFor
  Description: Waits for the created instances to be Running.
  Properties:
    Service: ECS
    API: DescribeInstances
    Parameters:
      RegionId: '{{ regionId }}'
      InstanceIds:
      - '{{ ACS::TaskLoopItem }}'
    DesiredValues:
    - Running
    PropertySelector: Instances.Instance[].Status
  Loop:
    RateControl:
      Mode: Concurrency
      MaxErrors: 0
      Concurrency: 1
    Items: '{{ runInstances.instanceIds }}'
Outputs:
  instanceIds:
    Type: List
    Value: '{{ runInstances.instanceIds }}'
			
  • JSON格式

{
  "FormatVersion": "OOS-2019-06-01",
  "Description": "Creates one or more ECS instances.",
  "Parameters": {
    "regionId": {
      "Description": "The ID of region.",
      "Type": "String",
      "Default": "{{ ACS::RegionId }}"
    },
    "imageId": {
      "Description": "The ID of the image resource that you specify when you create the instance.",
      "Type": "String",
      "AllowedPattern": "[A-Za-z0-9_\\-\\.]*",
      "MinLength": 1,
      "MaxLength": 100
    },
    "instanceType": {
      "Description": "The type of the instance.",
      "Type": "String",
      "AllowedPattern": "ecs\\.[A-Za-z0-9\\.\\-]*",
      "MaxLength": 30,
      "MinLength": 1
    },
    "securityGroupId": {
      "Description": "The ID of the security group to which the instance belongs.",
      "Type": "String",
      "AllowedPattern": "sg-[A-Za-z0-9]*",
      "MaxLength": 30,
      "MinLength": 1
    },
    "vSwitchId": {
      "Description": "The ID of the VSwitch.",
      "Type": "String",
      "AllowedPattern": "vsw-[A-Za-z0-9]*",
      "MaxLength": 30,
      "MinLength": 1
    },
    "amount": {
      "Description": "The specified number of instances you want to create.",
      "Type": "Number",
      "Default": 1
    },
    "internetMaxBandwidthIn": {
      "Description": "The maximum inbound public bandwidth.",
      "Type": "Number",
      "Default": 200
    },
    "internetMaxBandwidthOut": {
      "Description": "The maximum outbound public bandwidth.",
      "Type": "Number",
      "Default": 0
    }
  },
  "Tasks": [
    {
      "Name": "runInstances",
      "Action": "ACS::ExecuteAPI",
      "Description": "Creates one or more instances.",
      "Properties": {
        "Service": "ECS",
        "API": "RunInstances",
        "Parameters": {
          "RegionId": "{{ regionId }}",
          "Amount": "{{ amount }}",
          "ImageId": "{{ imageId }}",
          "InstanceType": "{{ instanceType }}",
          "SecurityGroupId": "{{ securityGroupId }}",
          "VSwitchId": "{{ vSwitchId }}",
          "InternetMaxBandwidthIn": "{{ internetMaxBandwidthIn }}",
          "InternetMaxBandwidthOut": "{{ internetMaxBandwidthOut }}"
        }
      },
      "Outputs": {
        "instanceIds": {
          "Type": "List",
          "ValueSelector": "InstanceIdSets.InstanceIdSet[]"
        }
      }
    },
    {
      "Name": "untilInstanceReady",
      "Action": "ACS::WaitFor",
      "Description": "Waits for the created instances to be Running.",
      "Properties": {
        "Service": "ECS",
        "API": "DescribeInstances",
        "Parameters": {
          "RegionId": "{{ regionId }}",
          "InstanceIds": [
            "{{ ACS::TaskLoopItem }}"
          ]
        },
        "DesiredValues": [
          "Running"
        ],
        "PropertySelector": "Instances.Instance[].Status"
      },
      "Loop": {
        "RateControl": {
          "Mode": "Concurrency",
          "MaxErrors": 0,
          "Concurrency": 1
        },
        "Items": "{{ runInstances.instanceIds }}"
      }
    }
  ],
  "Outputs": {
    "instanceIds": {
      "Type": "List",
      "Value": "{{ runInstances.instanceIds }}"
    }
  }
}

分批模式

  • YAML格式

FormatVersion: OOS-2019-06-01
Description: Creates one or more ECS instances.
Parameters:
  regionId:
    Description: The ID of region.
    Type: String
    Default: '{{ ACS::RegionId }}'
  imageId:
    Description: The ID of the image resource that you specify when you create the
      instance.
    Type: String
    AllowedPattern: '[A-Za-z0-9_\-\.]*'
    MinLength: 1
    MaxLength: 100
  instanceType:
    Description: The type of the instance.
    Type: String
    AllowedPattern: ecs\.[A-Za-z0-9\.\-]*
    MaxLength: 30
    MinLength: 1
  securityGroupId:
    Description: The ID of the security group to which the instance belongs.
    Type: String
    AllowedPattern: sg-[A-Za-z0-9]*
    MaxLength: 30
    MinLength: 1
  vSwitchId:
    Description: The ID of the VSwitch.
    Type: String
    AllowedPattern: vsw-[A-Za-z0-9]*
    MaxLength: 30
    MinLength: 1
  amount:
    Description: The specified number of instances you want to create.
    Type: Number
    Default: 1
  internetMaxBandwidthIn:
    Description: The maximum inbound public bandwidth.
    Type: Number
    Default: 200
  internetMaxBandwidthOut:
    Description: The maximum outbound public bandwidth.
    Type: Number
    Default: 0
Tasks:
- Name: runInstances
  Action: ACS::ExecuteAPI
  Description: Creates one or more instances.
  Properties:
    Service: ECS
    API: RunInstances
    Parameters:
      RegionId: '{{ regionId }}'
      Amount: '{{ amount }}'
      ImageId: '{{ imageId }}'
      InstanceType: '{{ instanceType }}'
      SecurityGroupId: '{{ securityGroupId }}'
      VSwitchId: '{{ vSwitchId }}'
      InternetMaxBandwidthIn: '{{ internetMaxBandwidthIn }}'
      InternetMaxBandwidthOut: '{{ internetMaxBandwidthOut }}'
  Outputs:
    instanceIds:
      Type: List
      ValueSelector: InstanceIdSets.InstanceIdSet[]
- Name: untilInstanceReady
  Action: ACS::WaitFor
  Description: Waits for the created instances to be Running.
  Properties:
    Service: ECS
    API: DescribeInstances
    Parameters:
      RegionId: '{{ regionId }}'
      InstanceIds:
      - '{{ ACS::TaskLoopItem }}'
    DesiredValues:
    - Running
    PropertySelector: Instances.Instance[].Status
  Loop:
    RateControl:
      Mode: Concurrency
      MaxErrors: 0
      Concurrency: 1
    Items: '{{ runInstances.instanceIds }}'
Outputs:
  instanceIds:
    Type: List
    Value: '{{ runInstances.instanceIds }}'
			
  • JSON格式

{
  "FormatVersion": "OOS-2019-06-01",
  "Description": "Creates one or more ECS instances.",
  "Parameters": {
    "regionId": {
      "Description": "The ID of region.",
      "Type": "String",
      "Default": "{{ ACS::RegionId }}"
    },
    "imageId": {
      "Description": "The ID of the image resource that you specify when you create the instance.",
      "Type": "String",
      "AllowedPattern": "[A-Za-z0-9_\\-\\.]*",
      "MinLength": 1,
      "MaxLength": 100
    },
    "instanceType": {
      "Description": "The type of the instance.",
      "Type": "String",
      "AllowedPattern": "ecs\\.[A-Za-z0-9\\.\\-]*",
      "MaxLength": 30,
      "MinLength": 1
    },
    "securityGroupId": {
      "Description": "The ID of the security group to which the instance belongs.",
      "Type": "String",
      "AllowedPattern": "sg-[A-Za-z0-9]*",
      "MaxLength": 30,
      "MinLength": 1
    },
    "vSwitchId": {
      "Description": "The ID of the VSwitch.",
      "Type": "String",
      "AllowedPattern": "vsw-[A-Za-z0-9]*",
      "MaxLength": 30,
      "MinLength": 1
    },
    "amount": {
      "Description": "The specified number of instances you want to create.",
      "Type": "Number",
      "Default": 1
    },
    "internetMaxBandwidthIn": {
      "Description": "The maximum inbound public bandwidth.",
      "Type": "Number",
      "Default": 200
    },
    "internetMaxBandwidthOut": {
      "Description": "The maximum outbound public bandwidth.",
      "Type": "Number",
      "Default": 0
    }
  },
  "Tasks": [
    {
      "Name": "runInstances",
      "Action": "ACS::ExecuteAPI",
      "Description": "Creates one or more instances.",
      "Properties": {
        "Service": "ECS",
        "API": "RunInstances",
        "Parameters": {
          "RegionId": "{{ regionId }}",
          "Amount": "{{ amount }}",
          "ImageId": "{{ imageId }}",
          "InstanceType": "{{ instanceType }}",
          "SecurityGroupId": "{{ securityGroupId }}",
          "VSwitchId": "{{ vSwitchId }}",
          "InternetMaxBandwidthIn": "{{ internetMaxBandwidthIn }}",
          "InternetMaxBandwidthOut": "{{ internetMaxBandwidthOut }}"
        }
      },
      "Outputs": {
        "instanceIds": {
          "Type": "List",
          "ValueSelector": "InstanceIdSets.InstanceIdSet[]"
        }
      }
    },
    {
      "Name": "untilInstanceReady",
      "Action": "ACS::WaitFor",
      "Description": "Waits for the created instances to be Running.",
      "Properties": {
        "Service": "ECS",
        "API": "DescribeInstances",
        "Parameters": {
          "RegionId": "{{ regionId }}",
          "InstanceIds": [
            "{{ ACS::TaskLoopItem }}"
          ]
        },
        "DesiredValues": [
          "Running"
        ],
        "PropertySelector": "Instances.Instance[].Status"
      },
      "Loop": {
        "RateControl": {
          "Mode": "Concurrency",
          "MaxErrors": 0,
          "Concurrency": 1
        },
        "Items": "{{ runInstances.instanceIds }}"
      }
    }
  ],
  "Outputs": {
    "instanceIds": {
      "Type": "List",
      "Value": "{{ runInstances.instanceIds }}"
    }
  }
}