ALIYUN::EMR::Cluster类型用于创建一个E-MapReduce集群。

语法

{
  "Type": "ALIYUN::EMR::Cluster",
  "Properties": {
    "SshEnable": Boolean,
    "EasEnable": Boolean,
    "WhiteListType": String,
    "InitCustomHiveMetaDB": Boolean,
    "IoOptimized": Boolean,
    "HostGroup": List,
    "Config": List,
    "KeyPairName": String,
    "VpcId": String,
    "AutoRenew": Boolean,
    "RelatedClusterId": String,
    "BootstrapAction": List,
    "InstanceGeneration": String,
    "VSwitchId": String,
    "NetType": String,
    "UserDefinedEmrEcsRole": String,
    "Name": String,
    "ClusterType": String,
    "ZoneId": String,
    "IsOpenPublicIp": Boolean,
    "OptionSoftWareList": List,
    "Configurations": String,
    "MasterPwd": String,
    "MachineType": String,
    "EmrVer": String,
    "SecurityGroupName": String,
    "DepositType": String,
    "SecurityGroupId": String,
    "LogPath": String,
    "Period": Integer,
    "HighAvailabilityEnable": Boolean,
    "UseCustomHiveMetaDB": Boolean,
    "UserInfo": List,
    "ChargeType": String,
    "AuthorizeContent": String,
    "UseLocalMetaDb": Boolean
  }
}

属性

属性名称 类型 必须 允许更新 描述 约束
SshEnable Boolean 是否开启SSH
EasEnable Boolean 是否高安全集群
WhiteListType String 暂时无需填写
InitCustomHiveMetaDB Boolean 保留字段,无需填写
IoOptimized Boolean 是否开启IO优化 取值范围:true(默认值)、false
HostGroup List 机器组
Config List 自定义配置项
KeyPairName String 密钥对
VpcId String VpcID 当NetType=vpc时,此参数为必填参数。
AutoRenew Boolean 包年包月集群是否自动续费
RelatedClusterId String 当前集群是gateway时,其关联的主集群ID
BootstrapAction List 引导操作
InstanceGeneration String ECS实例分代
VSwitchId String 交换机ID 当NetType=vpc时,此参数为必填参数。
NetType String 网络类型
UserDefinedEmrEcsRole String 用于ECS调用的EMR权限名
Name String 集群名 只允许包含中文、字母、数字、-、_,长度为1~64个字符。
ClusterType String 集群类型 取值范围:HADOOP、KAFKA、DRUID、ZOOKEEPER、DATA_SCIENCE、GATEWAY
ZoneId String 可用区ID
  • 华东1(杭州)支持:cn-hangzhou-b
  • 华北2(北京)支持:cn-beijing-a、cn-beijing-b、cn-beijing-c
IsOpenPublicIp Boolean 是否开启公网IP 若开启,则默认会带8MB的带宽。
OptionSoftWareList List 可选软件列表
Configurations String 无需填写
MasterPwd String Master节点SSH访问密码 需要满足ECS的密码规则:8~30个字符,且同时包含任意三项(大、小写字母,数字和特殊符号)。
MachineType String 机器类型
EmrVer String EMR版本
SecurityGroupName String 需要新建的安全组名称 如果不指定安全组ID,则将使用这个名字创建一个新的安全组。当集群创建完成后,可以在集群详情中看到创建的安全组ID。这个安全组将会带有默认的安全组策略:入只开放22端口,出开放所有端口。
DepositType String 托管类型
SecurityGroupId String 安全组ID,可在ECS中创建后使用。 若使用已有的安全组,会被增加上默认安全组策略:入只开放22端口,出开放所有端口。
LogPath String OSS日志路径
Period Integer 包年包月时间
  • 取值范围:1、2、3、4、5、6、7、8、9、12、24、36,单位:月
  • 当ChargeType=PrePaid时,此参数为必填参数。
HighAvailabilityEnable Boolean 是否开启高可用集群 若开启,则需要2台Master节点。
UseCustomHiveMetaDB Boolean 保留字段,无需填写
UserInfo List 用户信息
ChargeType String 付费类型 取值范围:PostPaid(按量付费)、PrePaid(包年包月)
AuthorizeContent String 暂时无需填写
UseLocalMetaDb Boolean 是否使用本地Hive元数据库

HostGroup 语法

"HostGroup": [
  {
    "Comment": String,
    "SysDiskType": String,
    "DiskCapacity": Integer,
    "NodeCount": Integer,
    "ClusterId": String,
    "DiskCount": Integer,
    "CreateType": String,
    "DiskType": String,
    "AutoRenew": Boolean,
    "HostGroupType": String,
    "SysDiskCapacity": Integer,
    "VSwitchId": String,
    "ChargeType": String,
    "Period": Integer,
    "HostKeyPairName": String,
    "HostPassword": String,
    "HostGroupId": String,
    "InstanceType": String,
    "GpuDriver": String,
    "HostGroupName": String
  }
]

HostGroup 属性

属性名称 类型 必须 允许更新 描述 约束
Comment String 保留字段,无需填写
SysDiskType String 机器组的系统盘类型
DiskCapacity Integer 机器组的数据盘容量
NodeCount Integer 机器组节点数
ClusterId String 保留字段,无需填写
DiskCount Integer 机器组的数据盘数量
CreateType String 保留字段,无需填写
DiskType String 机器组的数据盘类型 取值范围: CLOUD、CLOUD_EFFICIENCY、CLOUD_SSD
AutoRenew Boolean 包年包月集群是否自动续费
HostGroupType String 机器组类型 取值范围:MASTER、CORE、TASK
SysDiskCapacity Integer 机器组的系统盘容量
VSwitchId String 交换机ID 当NetType=vpc时,此参数为必填参数。
ChargeType String 付费类型 取值范围:PostPaid(按量付费)、PrePaid(包年包月)
Period Integer 包年包月时间
  • 取值范围:1、2、3、4、5、6、7、8、9、12、24、36,单位:月
  • 当ChargeType=PrePaid时,此参数为必填参数。
HostKeyPairName String 主机组的密钥对名称 目前只支持网关
HostPassword String 主机的密码 目前只支持网关
HostGroupId String 保留参数
InstanceType String 机器组型号
GpuDriver String GPU驱动
HostGroupName String 机器组名字

Config 语法

"Config": [
  {
    "Encrypt": String,
    "ConfigKey": String,
    "FileName": String,
    "ServiceName": String,
    "Replace": String,
    "ConfigValue": String
  }
]

Config 属性

属性名称 类型 必须 允许更新 描述 约束
Encrypt String 保留字段
ConfigKey String 自定义配置项的key
FileName String 自定义配置项所属文件名
ServiceName String 自定义配置项服务名(大写)
Replace String 保留字段
ConfigValue String 自定义配置项的值

BootstrapAction 语法

"BootstrapAction": [
  {
    "Path": String,
    "Name": String,
    "Arg": String
  }
]

BootstrapAction 属性

属性名称 类型 必须 允许更新 描述 约束
Path String 引导操作脚本路径
Name String 引导操作名字
Arg String 引导操作参数

UserInfo 语法

"UserInfo": [
  {
    "UserName": String,
    "Password": String,
    "UserId": String
  }
]

UserInfo 属性

属性名称 类型 必须 允许更新 描述 约束
UserName String knox账号用户名
Password String 集群密码
UserId String knox账号的aliyun用户ID

返回值

Fn::GetAtt

  • ClusterId:集群ID
  • HostGroups:集群的主机组列表。
  • MasterNodePubIps:集群主节点的公共ip列表。
  • MasterNodeInnerIps:集群主节点的内部ip列表。

示例

{
  "ROSTemplateFormatVersion": "2015-09-01",
  "Resources": {
    "Cluster": {
      "Type": "ALIYUN::EMR::Cluster",
      "Properties": {
        "SshEnable": {
          "Ref": "SshEnable"
        },
        "EasEnable": {
          "Ref": "EasEnable"
        },
        "WhiteListType": {
          "Ref": "WhiteListType"
        },
        "InitCustomHiveMetaDB": {
          "Ref": "InitCustomHiveMetaDB"
        },
        "IoOptimized": {
          "Ref": "IoOptimized"
        },
        "HighAvailabilityEnable": {
          "Ref": "HighAvailabilityEnable"
        },
        "Config": {
          "Ref": "Config"
        },
        "KeyPairName": {
          "Ref": "KeyPairName"
        },
        "VpcId": {
          "Ref": "VpcId"
        },
        "AutoRenew": {
          "Ref": "AutoRenew"
        },
        "RelatedClusterId": {
          "Ref": "RelatedClusterId"
        },
        "BootstrapAction": {
          "Ref": "BootstrapAction"
        },
        "InstanceGeneration": {
          "Ref": "InstanceGeneration"
        },
        "VSwitchId": {
          "Ref": "VSwitchId"
        },
        "NetType": {
          "Ref": "NetType"
        },
        "Name": {
          "Ref": "Name"
        },
        "ClusterType": {
          "Ref": "ClusterType"
        },
        "ZoneId": {
          "Ref": "ZoneId"
        },
        "IsOpenPublicIp": {
          "Ref": "IsOpenPublicIp"
        },
        "OptionSoftWareList": {
          "Ref": "OptionSoftWareList"
        },
        "Configurations": {
          "Ref": "Configurations"
        },
        "MasterPwd": {
          "Ref": "MasterPwd"
        },
        "MachineType": {
          "Ref": "MachineType"
        },
        "EmrVer": {
          "Ref": "EmrVer"
        },
        "SecurityGroupName": {
          "Ref": "SecurityGroupName"
        },
        "DepositType": {
          "Ref": "DepositType"
        },
        "HostGroup": {
          "Ref": "HostGroup"
        },
        "SecurityGroupId": {
          "Ref": "SecurityGroupId"
        },
        "LogPath": {
          "Ref": "LogPath"
        },
        "Period": {
          "Ref": "Period"
        },
        "UserDefinedEmrEcsRole": {
          "Ref": "UserDefinedEmrEcsRole"
        },
        "UseCustomHiveMetaDB": {
          "Ref": "UseCustomHiveMetaDB"
        },
        "UserInfo": {
          "Ref": "UserInfo"
        },
        "ChargeType": {
          "Ref": "ChargeType"
        },
        "AuthorizeContent": {
          "Ref": "AuthorizeContent"
        },
        "UseLocalMetaDb": {
          "Ref": "UseLocalMetaDb"
        }
      }
    }
  },
  "Parameters": {
    "SshEnable": {
      "Type": "Boolean",
      "Description": "Indicates whether SSH is enabled.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "EasEnable": {
      "Type": "Boolean",
      "Description": "Indicates whether the cluster is a high-security cluster.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "WhiteListType": {
      "Type": "String",
      "Description": "Not required."
    },
    "InitCustomHiveMetaDB": {
      "Type": "Boolean",
      "Description": "A reserved parameter. Not required.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "IoOptimized": {
      "Default": true,
      "Type": "Boolean",
      "Description": "Indicates wether I/O optimization is enabled. Default value: true.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "HighAvailabilityEnable": {
      "Type": "Boolean",
      "Description": "Indicates whether the cluster is a high-availability cluster. A value of true indicates\nthat two master nodes are required.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "Config": {
      "Type": "Json",
      "Description": ""
    },
    "KeyPairName": {
      "Type": "String",
      "Description": "The name of the key pair."
    },
    "VpcId": {
      "Type": "String",
      "Description": "The ID of the VPC. A value is required when NetType=vpc."
    },
    "AutoRenew": {
      "Type": "Boolean",
      "Description": "Indicates whether the subscription cluster is auto-renewed.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "RelatedClusterId": {
      "Type": "String",
      "Description": "The ID of the primary cluster (when the cluster that you create is a Gateway cluster)."
    },
    "BootstrapAction": {
      "Type": "Json",
      "Description": ""
    },
    "InstanceGeneration": {
      "Type": "String",
      "Description": "The generation of the ECS instances."
    },
    "VSwitchId": {
      "Type": "String",
      "Description": "The ID of the Vswitch. A value is required when NetType=vpc."
    },
    "NetType": {
      "Type": "String",
      "Description": "The type of the network."
    },
    "Name": {
      "Type": "String",
      "Description": "The name of the cluster. The name can be 1 to 64 characters in length and only contain\nChinese characters, letters, numbers, hyphens (-), and underscores (_)."
    },
    "ClusterType": {
      "Type": "String",
      "Description": "The type of the cluster. Allowd values: HADOOP, KAFKA, DRUID, ZOOKEEPER, DATA_SCIENCE, GATEWAY.",
      "AllowedValues": [
        "HADOOP",
        "KAFKA",
        "DRUID",
        "ZOOKEEPER",
        "DATA_SCIENCE",
        "GATEWAY"
      ]
    },
    "ZoneId": {
      "Type": "String",
      "Description": "The zone ID."
    },
    "IsOpenPublicIp": {
      "Type": "Boolean",
      "Description": "Indicates whether a public IP address is assigned. A value of true indicates that\na bandwidth of 8 MB is set by default.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "OptionSoftWareList": {
      "Type": "Json",
      "Description": "The list of optional services."
    },
    "Configurations": {
      "Type": "String",
      "Description": "Not required."
    },
    "MasterPwd": {
      "MinLength": 8,
      "Type": "String",
      "Description": "The SSH password for the master node. The password must meet the following requirements.\nLength constraints: Minimum length of 8 characters. Maximum length of 30 characters.\nIt must contain three types of characters (uppercase letters, lowercase letters, numbers,\nand special symbols).",
      "MaxLength": 30
    },
    "MachineType": {
      "Type": "String",
      "Description": "The type of the machine."
    },
    "EmrVer": {
      "Type": "String",
      "Description": "The version of EMR."
    },
    "SecurityGroupName": {
      "Type": "String",
      "Description": "The name of the security group to create. If the ID of the security group is not specified,\nthis name is used to create a new security group. After the cluster is created, you\ncan view the ID of the security group on the Cluster Management page. The default\nsecurity group policy is applied to this security group: Only port 22 is open at the\ninbound and all ports are open at the outbound. You need to specify either SecurityGroupId\nor SecurityGroupName."
    },
    "DepositType": {
      "Type": "String",
      "Description": "The hosting type."
    },
    "HostGroup": {
      "Type": "Json",
      "Description": ""
    },
    "SecurityGroupId": {
      "Type": "String",
      "Description": "The ID of the security group. You can create a security group in the ECS console and\nuse it. Note: If you use an existing security group, the default security group policy\nis applied to this security group: Only port 22 is open at the inbound and all ports\nare open at the outbound. You need to specify either SecurityGroupId or SecurityGroupName."
    },
    "LogPath": {
      "Type": "String",
      "Description": "The log path in OSS."
    },
    "Period": {
      "Type": "Number",
      "Description": "The length of the subscription. Unit: months. Valid values: 1, 2, 3, 4, 5, 6, 7, 8,\n9, 12, 24, and 36. A value is required when ChargeType=PrePaid.",
      "AllowedValues": [
        1,
        2,
        3,
        4,
        5,
        6,
        7,
        8,
        9,
        12,
        24,
        36
      ]
    },
    "UserDefinedEmrEcsRole": {
      "Type": "String",
      "Description": "The role that is assigned to EMR for calling ECS resources."
    },
    "UseCustomHiveMetaDB": {
      "Type": "Boolean",
      "Description": "A reserved parameter. Not required.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    },
    "UserInfo": {
      "Type": "Json",
      "Description": ""
    },
    "ChargeType": {
      "Type": "String",
      "Description": "The billing method. Valid values: PostPaid and PrePaid. PostPaid: pay-as-you-go. PrePaid:\nsubscription.",
      "AllowedValues": [
        "PostPaid",
        "PrePaid"
      ]
    },
    "AuthorizeContent": {
      "Type": "String",
      "Description": "Not required."
    },
    "UseLocalMetaDb": {
      "Type": "Boolean",
      "Description": "Indicates whether the local Hive metadatabase is used.",
      "AllowedValues": [
        "True",
        "true",
        "False",
        "false"
      ]
    }
  },
  "Outputs": {
    "ClusterId": {
      "Description": "The ID of the cluster.",
      "Value": {
        "Fn::GetAtt": [
          "Cluster",
          "ClusterId"
        ]
      }
    }
  }
}