ALIYUN::EMR::SparkWorkspace-资源编排(ROS)-阿里云帮助中心

ALIYUN::EMR::SparkWorkspace类型用于创建E-MapReduce Serverless Spark工作空间。

语法

{
  "Type": "ALIYUN::EMR::SparkWorkspace",
  "Properties": {
    "OssBucket": String,
    "PaymentType": String,
    "ResourceSpec": Map,
    "RamRoleName": String,
    "WorkspaceName": String,
    "AutoRenewPeriod": String,
    "AutoRenew": Boolean,
    "AutoStartSessionCluster": Boolean,
    "AutoRenewPeriodUnit": String,
    "DlfType": String,
    "Duration": String,
    "DlfCatalogId": String,
    "GpuSubscription": Map,
    "GpuSpec": List,
    "IpWhiteList": List,
    "PaymentDurationUnit": String,
    "ReleaseType": String,
    "ResourceGroupId": String,
    "Subscription": Map,
    "Tags": List
  }
}

属性

属性名称	类型	必须	允许更新	描述	约束
OssBucket	String	是	否	工作空间的 OSS Bucket 路径。	路径必须以 `oss:///` 开头。
PaymentType	String	是	否	工作空间的计费方式。	取值： PostPaid：按量付费 PrePaid：包年包月
RamRoleName	String	是	否	用于运行 Spark 作业的 RAM 角色。	无
ResourceSpec	Map	是	是	工作空间的资源规格。	更多信息，请参考ResourceSpec语法。
WorkspaceName	String	是	是	工作空间名称。	名称长度必须为 1 到 64 个字符，可包含中文字符、字母、数字、连字符（-）和下划线（_）。
AutoRenew	Boolean	否	否	是否为订阅开启自动续费。	无
AutoRenewPeriod	String	否	否	自动续费时长。	无
AutoRenewPeriodUnit	String	否	否	自动续费时长的单位。	取值： MONTH：月
AutoStartSessionCluster	Boolean	否	否	是否自动启动会话集群。	无
DlfCatalogId	String	否	否	DLF 目录的 ID。	无
DlfType	String	否	否	绑定到工作空间的 DLF 类型。	无
Duration	String	否	否	订阅时长。	取值范围： 1~9，单位：月。
GpuSpec	List	否	是	工作空间的 GPU 实例规格。	无
GpuSubscription	Map	否	是	GPU 订阅配置。	此配置在工作空间创建后生效。更多信息，请参考GpuSubscription属性。
IpWhiteList	List	否	是	工作空间的 IP 白名单。	此配置在工作空间创建后生效。
PaymentDurationUnit	String	否	否	订阅时长的单位。	取值： MONTH
ReleaseType	String	否	否	工作空间的发布类型。	无
ResourceGroupId	String	否	是	资源组的 ID。	无
Subscription	Map	否	是	用于将按量付费工作空间转换为订阅工作空间的配置。	此配置在工作空间创建后生效。更多信息，请参考Subscription属性。
Tags	List	否	否	要附加到工作空间的标签。	创建工作空间时最多支持添加 20 个标签。更多信息，请参考Tags属性。

ResourceSpec语法

"ResourceSpec": {
  "Cu": String,
  "Gpu": Integer
}

ResourceSpec属性

属性名称

类型

必须

允许更新

描述

约束

String

是

否

计算单元的最大数量。

取值范围：

1~9

Gpu

Integer

否

GPU 的数量。

最小值：0

GpuSubscription语法

"GpuSubscription": {
  "InstanceId": String,
  "AutoRenew": Boolean,
  "PaymentDurationUnit": String,
  "InstanceTypeId": String,
  "Duration": Integer,
  "Operation": String,
  "GpuMachineNum": Integer
}

GpuSubscription属性

属性名称	类型	必须	允许更新	描述	约束
AutoRenew	Boolean	否	否	是否开启自动续费。	无
Duration	Integer	否	否	订阅时长。	最小值：1
GpuMachineNum	Integer	否	否	GPU 机器的数量。	最小值：1
InstanceId	String	否	否	GPU 订阅实例的 ID。	无
InstanceTypeId	String	否	否	GPU 实例规格。	无
Operation	String	否	否	操作类型。	取值： BUY UPGRADE
PaymentDurationUnit	String	否	否	订阅时长的单位。	取值： MONTH

Subscription语法

"Subscription": {
  "AutoRenewPeriod": String,
  "AutoRenew": Boolean,
  "PaymentDurationUnit": String,
  "Duration": String,
  "Queue": List,
  "AutoRenewPeriodUnit": String
}

Subscription属性

属性名称	类型	必须	允许更新	描述	约束
AutoRenew	Boolean	否	否	是否开启自动续费。	无
AutoRenewPeriod	String	否	否	自动续费时长。	取值： 1~9，单位：月。
AutoRenewPeriodUnit	String	否	否	自动续费时长的单位。	取值： MONTH
Duration	String	否	否	订阅时长。	取值： 1~9，单位：月。
PaymentDurationUnit	String	否	否	订阅时长的单位。	取值： MONTH
Queue	List	否	否	要转换为订阅计划的队列。	无

Tags语法

"Tags": [
  {
    "Value": String,
    "Key": String
  }
]

Tags属性

属性名称	类型	必须	允许更新	描述	约束
Key	String	是	否	标签键	无
Value	String	否	否	标签值	无

返回值

Fn::GetAtt

Storage：工作空间的 OSS 存储路径。
WorkspaceId：工作空间的 ID。

示例

场景 1 ：基于已有资源创建 Spark 工作空间

YAML

ROSTemplateFormatVersion: '2015-09-01'
Description:
  zh-cn: 基于已有的OSS存储桶和RAM角色创建一个简单的EMR Serverless Spark工作空间。
  en: Create a simple EMR Serverless Spark workspace based on an existing OSS bucket and RAM role.
Parameters:
  WorkspaceName:
    Type: String
    Label:
      zh-cn: 工作空间名称
      en: Workspace Name
    Description:
      zh-cn: Spark工作空间名称，1-64个字符，支持中文、字母、数字、短划线和下划线。
      en: Spark workspace name, 1-64 characters. Supports Chinese, letters, digits, hyphens and underscores.
    MinLength: 1
    MaxLength: 64
    Default: my-spark-workspace
  OssBucketPath:
    Type: String
    Label:
      zh-cn: OSS存储路径
      en: OSS Bucket Path
    Description:
      zh-cn: 工作空间的OSS存储路径，格式为 oss://bucket-name/path/。
      en: 'The OSS storage path of the workspace, format: oss://bucket-name/path/.'
  RamRoleName:
    Type: String
    Label:
      zh-cn: RAM角色名称
      en: RAM Role Name
    Description:
      zh-cn: 用于运行Spark作业的RAM角色名称。
      en: The RAM role name used to run Spark jobs.
  Cu:
    Type: String
    Label:
      zh-cn: 计算单元(CU)
      en: Compute Units (CU)
    Description:
      zh-cn: 工作空间的最大计算单元数量。
      en: The maximum number of compute units for the workspace.
    Default: '10'
Resources:
  SparkWorkspace:
    Type: ALIYUN::EMR::SparkWorkspace
    Properties:
      WorkspaceName:
        Ref: WorkspaceName
      OssBucket:
        Ref: OssBucketPath
      RamRoleName:
        Ref: RamRoleName
      PaymentType: PayAsYouGo
      ResourceSpec:
        Cu:
          Ref: Cu
Outputs:
  WorkspaceId:
    Description:
      zh-cn: 工作空间ID。
      en: Workspace ID.
    Label:
      zh-cn: 工作空间ID
      en: Workspace ID
    Value:
      Fn::GetAtt:
        - SparkWorkspace
        - WorkspaceId
  Storage:
    Description:
      zh-cn: 工作空间的OSS存储路径。
      en: The OSS storage path of the workspace.
    Label:
      zh-cn: OSS存储路径
      en: Storage Path
    Value:
      Fn::GetAtt:
        - SparkWorkspace
        - Storage
Metadata:
  ALIYUN::ROS::Interface:
    ParameterGroups:
      - Parameters:
          - WorkspaceName
          - OssBucketPath
          - RamRoleName
          - Cu

JSON

{
  "ROSTemplateFormatVersion": "2015-09-01",
  "Description": {
    "zh-cn": "基于已有的OSS存储桶和RAM角色创建一个简单的EMR Serverless Spark工作空间。",
    "en": "Create a simple EMR Serverless Spark workspace based on an existing OSS bucket and RAM role."
  },
  "Parameters": {
    "WorkspaceName": {
      "Type": "String",
      "Label": {
        "zh-cn": "工作空间名称",
        "en": "Workspace Name"
      },
      "Description": {
        "zh-cn": "Spark工作空间名称，1-64个字符，支持中文、字母、数字、短划线和下划线。",
        "en": "Spark workspace name, 1-64 characters. Supports Chinese, letters, digits, hyphens and underscores."
      },
      "MinLength": 1,
      "MaxLength": 64,
      "Default": "my-spark-workspace"
    },
    "OssBucketPath": {
      "Type": "String",
      "Label": {
        "zh-cn": "OSS存储路径",
        "en": "OSS Bucket Path"
      },
      "Description": {
        "zh-cn": "工作空间的OSS存储路径，格式为 oss://bucket-name/path/。",
        "en": "The OSS storage path of the workspace, format: oss://bucket-name/path/."
      }
    },
    "RamRoleName": {
      "Type": "String",
      "Label": {
        "zh-cn": "RAM角色名称",
        "en": "RAM Role Name"
      },
      "Description": {
        "zh-cn": "用于运行Spark作业的RAM角色名称。",
        "en": "The RAM role name used to run Spark jobs."
      }
    },
    "Cu": {
      "Type": "String",
      "Label": {
        "zh-cn": "计算单元(CU)",
        "en": "Compute Units (CU)"
      },
      "Description": {
        "zh-cn": "工作空间的最大计算单元数量。",
        "en": "The maximum number of compute units for the workspace."
      },
      "Default": "10"
    }
  },
  "Resources": {
    "SparkWorkspace": {
      "Type": "ALIYUN::EMR::SparkWorkspace",
      "Properties": {
        "WorkspaceName": {
          "Ref": "WorkspaceName"
        },
        "OssBucket": {
          "Ref": "OssBucketPath"
        },
        "RamRoleName": {
          "Ref": "RamRoleName"
        },
        "PaymentType": "PayAsYouGo",
        "ResourceSpec": {
          "Cu": {
            "Ref": "Cu"
          }
        }
      }
    }
  },
  "Outputs": {
    "WorkspaceId": {
      "Description": {
        "zh-cn": "工作空间ID。",
        "en": "Workspace ID."
      },
      "Label": {
        "zh-cn": "工作空间ID",
        "en": "Workspace ID"
      },
      "Value": {
        "Fn::GetAtt": [
          "SparkWorkspace",
          "WorkspaceId"
        ]
      }
    },
    "Storage": {
      "Description": {
        "zh-cn": "工作空间的OSS存储路径。",
        "en": "The OSS storage path of the workspace."
      },
      "Label": {
        "zh-cn": "OSS存储路径",
        "en": "Storage Path"
      },
      "Value": {
        "Fn::GetAtt": [
          "SparkWorkspace",
          "Storage"
        ]
      }
    }
  },
  "Metadata": {
    "ALIYUN::ROS::Interface": {
      "ParameterGroups": [
        {
          "Parameters": [
            "WorkspaceName",
            "OssBucketPath",
            "RamRoleName",
            "Cu"
          ]
        }
      ]
    }
  }
}

场景 2 ：创建 OSS + RAM 角色 + Spark 工作空间

YAML

ROSTemplateFormatVersion: '2015-09-01'
Description:
  zh-cn: 创建OSS存储桶、RAM角色和EMR Serverless Spark工作空间，配置IP白名单、GPU规格和标签，实现完整的Spark计算环境。
  en: Create an OSS bucket, RAM role, and EMR Serverless Spark workspace with IP whitelist, GPU specification, and tags for a complete Spark computing environment.
Parameters:
  CommonName:
    Type: String
    Label:
      zh-cn: 资源名称前缀
      en: Resource Name Prefix
    Description:
      zh-cn: 所有资源名称的统一前缀。
      en: Unified name prefix for all resources.
    Default: spark-env
    MinLength: 1
    MaxLength: 50
  Cu:
    Type: String
    Label:
      zh-cn: 计算单元(CU)
      en: Compute Units (CU)
    Description:
      zh-cn: 工作空间的最大计算单元数量。
      en: The maximum number of compute units for the workspace.
    Default: '20'
  GpuCount:
    Type: Number
    Label:
      zh-cn: GPU数量
      en: GPU Count
    Description:
      zh-cn: 工作空间的GPU数量，0表示不使用GPU。
      en: The number of GPUs for the workspace. 0 means no GPU.
    Default: 0
    MinValue: 0
Resources:
  OssBucket:
    Type: ALIYUN::OSS::Bucket
    Properties:
      BucketName:
        Fn::Sub: ${CommonName}-${ALIYUN::AccountId}
      AccessControl: private
      DeletionForce: true
      Tags:
        CreatedBy: ROS
        Purpose: SparkWorkspace
  RamRole:
    Type: ALIYUN::RAM::Role
    Properties:
      RoleName:
        Fn::Sub: ${CommonName}-spark-role
      Description: RAM role for EMR Serverless Spark
      AssumeRolePolicyDocument:
        Version: '1'
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service:
                - emr-serverless-spark.aliyuncs.com
      Policies:
        - PolicyName: AliyunEMRServerlessSparkFullAccess
          PolicyDocument:
            Version: '1'
            Statement:
              - Action:
                  - oss:GetObject
                  - oss:PutObject
                  - oss:DeleteObject
                  - oss:ListObjects
                  - oss:AbortMultipartUpload
                  - oss:ListMultipartUploads
                Resource:
                  - Fn::Sub: acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}
                  - Fn::Sub: acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}/*
                Effect: Allow
  SparkWorkspace:
    Type: ALIYUN::EMR::SparkWorkspace
    DependsOn:
      - OssBucket
      - RamRole
    Properties:
      WorkspaceName:
        Fn::Sub: ${CommonName}-workspace
      OssBucket:
        Fn::Sub: oss://${CommonName}-${ALIYUN::AccountId}/spark-workspace/
      RamRoleName:
        Ref: RamRole
      PaymentType: PayAsYouGo
      ResourceSpec:
        Cu:
          Ref: Cu
        Gpu:
          Ref: GpuCount
      AutoStartSessionCluster: true
      Tags:
        - Key: Environment
          Value: Development
        - Key: Team
          Value: DataEngineering
        - Key: CreatedBy
          Value: ROS
Outputs:
  WorkspaceId:
    Description:
      zh-cn: 工作空间ID。
      en: Workspace ID.
    Label:
      zh-cn: 工作空间ID
      en: Workspace ID
    Value:
      Fn::GetAtt:
        - SparkWorkspace
        - WorkspaceId
  Storage:
    Description:
      zh-cn: 工作空间的OSS存储路径。
      en: The OSS storage path of the workspace.
    Label:
      zh-cn: OSS存储路径
      en: Storage Path
    Value:
      Fn::GetAtt:
        - SparkWorkspace
        - Storage
  OssBucketName:
    Description:
      zh-cn: OSS存储桶名称。
      en: OSS bucket name.
    Label:
      zh-cn: OSS存储桶名称
      en: OSS Bucket Name
    Value:
      Fn::GetAtt:
        - OssBucket
        - Name
  RamRoleArn:
    Description:
      zh-cn: RAM角色ARN。
      en: RAM role ARN.
    Label:
      zh-cn: RAM角色ARN
      en: RAM Role ARN
    Value:
      Fn::GetAtt:
        - RamRole
        - Arn
Metadata:
  ALIYUN::ROS::Interface:
    ParameterGroups:
      - Parameters:
          - CommonName
          - Cu
          - GpuCount

JSON

{
  "ROSTemplateFormatVersion": "2015-09-01",
  "Description": {
    "zh-cn": "创建OSS存储桶、RAM角色和EMR Serverless Spark工作空间，配置IP白名单、GPU规格和标签，实现完整的Spark计算环境。",
    "en": "Create an OSS bucket, RAM role, and EMR Serverless Spark workspace with IP whitelist, GPU specification, and tags for a complete Spark computing environment."
  },
  "Parameters": {
    "CommonName": {
      "Type": "String",
      "Label": {
        "zh-cn": "资源名称前缀",
        "en": "Resource Name Prefix"
      },
      "Description": {
        "zh-cn": "所有资源名称的统一前缀。",
        "en": "Unified name prefix for all resources."
      },
      "Default": "spark-env",
      "MinLength": 1,
      "MaxLength": 50
    },
    "Cu": {
      "Type": "String",
      "Label": {
        "zh-cn": "计算单元(CU)",
        "en": "Compute Units (CU)"
      },
      "Description": {
        "zh-cn": "工作空间的最大计算单元数量。",
        "en": "The maximum number of compute units for the workspace."
      },
      "Default": "20"
    },
    "GpuCount": {
      "Type": "Number",
      "Label": {
        "zh-cn": "GPU数量",
        "en": "GPU Count"
      },
      "Description": {
        "zh-cn": "工作空间的GPU数量，0表示不使用GPU。",
        "en": "The number of GPUs for the workspace. 0 means no GPU."
      },
      "Default": 0,
      "MinValue": 0
    }
  },
  "Resources": {
    "OssBucket": {
      "Type": "ALIYUN::OSS::Bucket",
      "Properties": {
        "BucketName": {
          "Fn::Sub": "${CommonName}-${ALIYUN::AccountId}"
        },
        "AccessControl": "private",
        "DeletionForce": true,
        "Tags": {
          "CreatedBy": "ROS",
          "Purpose": "SparkWorkspace"
        }
      }
    },
    "RamRole": {
      "Type": "ALIYUN::RAM::Role",
      "Properties": {
        "RoleName": {
          "Fn::Sub": "${CommonName}-spark-role"
        },
        "Description": "RAM role for EMR Serverless Spark",
        "AssumeRolePolicyDocument": {
          "Version": "1",
          "Statement": [
            {
              "Action": "sts:AssumeRole",
              "Effect": "Allow",
              "Principal": {
                "Service": [
                  "emr-serverless-spark.aliyuncs.com"
                ]
              }
            }
          ]
        },
        "Policies": [
          {
            "PolicyName": "AliyunEMRServerlessSparkFullAccess",
            "PolicyDocument": {
              "Version": "1",
              "Statement": [
                {
                  "Action": [
                    "oss:GetObject",
                    "oss:PutObject",
                    "oss:DeleteObject",
                    "oss:ListObjects",
                    "oss:AbortMultipartUpload",
                    "oss:ListMultipartUploads"
                  ],
                  "Resource": [
                    {
                      "Fn::Sub": "acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}"
                    },
                    {
                      "Fn::Sub": "acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}/*"
                    }
                  ],
                  "Effect": "Allow"
                }
              ]
            }
          }
        ]
      }
    },
    "SparkWorkspace": {
      "Type": "ALIYUN::EMR::SparkWorkspace",
      "DependsOn": [
        "OssBucket",
        "RamRole"
      ],
      "Properties": {
        "WorkspaceName": {
          "Fn::Sub": "${CommonName}-workspace"
        },
        "OssBucket": {
          "Fn::Sub": "oss://${CommonName}-${ALIYUN::AccountId}/spark-workspace/"
        },
        "RamRoleName": {
          "Ref": "RamRole"
        },
        "PaymentType": "PayAsYouGo",
        "ResourceSpec": {
          "Cu": {
            "Ref": "Cu"
          },
          "Gpu": {
            "Ref": "GpuCount"
          }
        },
        "AutoStartSessionCluster": true,
        "Tags": [
          {
            "Key": "Environment",
            "Value": "Development"
          },
          {
            "Key": "Team",
            "Value": "DataEngineering"
          },
          {
            "Key": "CreatedBy",
            "Value": "ROS"
          }
        ]
      }
    }
  },
  "Outputs": {
    "WorkspaceId": {
      "Description": {
        "zh-cn": "工作空间ID。",
        "en": "Workspace ID."
      },
      "Label": {
        "zh-cn": "工作空间ID",
        "en": "Workspace ID"
      },
      "Value": {
        "Fn::GetAtt": [
          "SparkWorkspace",
          "WorkspaceId"
        ]
      }
    },
    "Storage": {
      "Description": {
        "zh-cn": "工作空间的OSS存储路径。",
        "en": "The OSS storage path of the workspace."
      },
      "Label": {
        "zh-cn": "OSS存储路径",
        "en": "Storage Path"
      },
      "Value": {
        "Fn::GetAtt": [
          "SparkWorkspace",
          "Storage"
        ]
      }
    },
    "OssBucketName": {
      "Description": {
        "zh-cn": "OSS存储桶名称。",
        "en": "OSS bucket name."
      },
      "Label": {
        "zh-cn": "OSS存储桶名称",
        "en": "OSS Bucket Name"
      },
      "Value": {
        "Fn::GetAtt": [
          "OssBucket",
          "Name"
        ]
      }
    },
    "RamRoleArn": {
      "Description": {
        "zh-cn": "RAM角色ARN。",
        "en": "RAM role ARN."
      },
      "Label": {
        "zh-cn": "RAM角色ARN",
        "en": "RAM Role ARN"
      },
      "Value": {
        "Fn::GetAtt": [
          "RamRole",
          "Arn"
        ]
      }
    }
  },
  "Metadata": {
    "ALIYUN::ROS::Interface": {
      "ParameterGroups": [
        {
          "Parameters": [
            "CommonName",
            "Cu",
            "GpuCount"
          ]
        }
      ]
    }
  }
}

场景 3 ：创建完整 Spark 数据分析平台

YAML

ROSTemplateFormatVersion: '2015-09-01'
Description:
  zh-cn: 创建完整的EMR Serverless Spark数据分析平台，包含OSS存储桶、RAM角色、Spark工作空间（GPU加速+IP白名单+自动续费）、Livy网关（自动启停+公网访问），实现端到端的Spark大数据计算与交互式分析。
  en: Create a complete EMR Serverless Spark data analysis platform including OSS bucket, RAM role, Spark workspace with GPU acceleration and IP whitelist, and Livy gateway with auto start/stop and public access for end-to-end Spark big data computing and interactive analysis.
Parameters:
  CommonName:
    Type: String
    Label:
      zh-cn: 资源名称前缀
      en: Resource Name Prefix
    Description:
      zh-cn: 所有资源名称的统一前缀。
      en: Unified name prefix for all resources.
    Default: spark-platform
    MinLength: 1
    MaxLength: 50
  Cu:
    Type: String
    Label:
      zh-cn: 计算单元(CU)
      en: Compute Units (CU)
    Description:
      zh-cn: 工作空间的最大计算单元数量。
      en: The maximum number of compute units for the workspace.
    Default: '50'
  GpuCount:
    Type: Number
    Label:
      zh-cn: GPU数量
      en: GPU Count
    Description:
      zh-cn: 工作空间的GPU数量。
      en: The number of GPUs for the workspace.
    Default: 2
    MinValue: 0
  SparkEngineVersion:
    Type: String
    Label:
      zh-cn: Spark引擎版本
      en: Spark Engine Version
    Description:
      zh-cn: Livy网关使用的Spark引擎版本。
      en: The Spark engine version used by the Livy gateway.
    Default: esr-4.2.0
  LivyCpuLimit:
    Type: String
    Label:
      zh-cn: Livy CPU核数
      en: Livy CPU Cores
    Description:
      zh-cn: Livy服务器的CPU核数。
      en: The number of CPU cores for the Livy server.
    AllowedValues:
      - '1'
      - '2'
      - '4'
    Default: '2'
  LivyMemoryLimit:
    Type: String
    Label:
      zh-cn: Livy内存大小
      en: Livy Memory Size
    Description:
      zh-cn: Livy服务器的内存大小。
      en: The memory size of the Livy server.
    AllowedValues:
      - 4Gi
      - 8Gi
      - 16Gi
    Default: 8Gi
  LivyIdleTimeoutMinutes:
    Type: Number
    Label:
      zh-cn: Livy空闲超时(分钟)
      en: Livy Idle Timeout (Minutes)
    Description:
      zh-cn: Livy网关空闲超时后自动停止的时间。
      en: The idle timeout in minutes before the Livy gateway automatically stops.
    Default: 60
    MinValue: 1
  EnableLivyPublicAccess:
    Type: String
    Label:
      zh-cn: 是否开启Livy公网访问
      en: Enable Livy Public Access
    Description:
      zh-cn: 是否为Livy网关开启公网访问端点。
      en: Whether to enable public endpoint for the Livy gateway.
    AllowedValues:
      - 'true'
      - 'false'
    Default: 'false'
Conditions:
  EnablePublic:
    Fn::Equals:
      - Ref: EnableLivyPublicAccess
      - 'true'
Resources:
  OssBucket:
    Type: ALIYUN::OSS::Bucket
    Properties:
      BucketName:
        Fn::Sub: ${CommonName}-${ALIYUN::AccountId}
      AccessControl: private
      DeletionForce: true
      ServerSideEncryptionConfiguration:
        SSEAlgorithm: AES256
      Tags:
        CreatedBy: ROS
        Purpose: SparkPlatform
  RamRole:
    Type: ALIYUN::RAM::Role
    Properties:
      RoleName:
        Fn::Sub: ${CommonName}-spark-role
      Description: RAM role for EMR Serverless Spark platform
      AssumeRolePolicyDocument:
        Version: '1'
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service:
                - emr-serverless-spark.aliyuncs.com
      Policies:
        - PolicyName: AliyunEMRServerlessSparkFullAccess
          PolicyDocument:
            Version: '1'
            Statement:
              - Action:
                  - oss:GetObject
                  - oss:PutObject
                  - oss:DeleteObject
                  - oss:ListObjects
                  - oss:AbortMultipartUpload
                  - oss:ListMultipartUploads
                  - oss:GetBucketInfo
                Resource:
                  - Fn::Sub: acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}
                  - Fn::Sub: acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}/*
                Effect: Allow
  SparkWorkspace:
    Type: ALIYUN::EMR::SparkWorkspace
    DependsOn:
      - OssBucket
      - RamRole
    Properties:
      WorkspaceName:
        Fn::Sub: ${CommonName}-workspace
      OssBucket:
        Fn::Sub: oss://${CommonName}-${ALIYUN::AccountId}/spark-workspace/
      RamRoleName:
        Ref: RamRole
      PaymentType: PayAsYouGo
      ResourceSpec:
        Cu:
          Ref: Cu
        Gpu:
          Ref: GpuCount
      AutoStartSessionCluster: true
      Subscription:
        AutoRenew: true
        AutoRenewPeriod: '1'
        AutoRenewPeriodUnit: MONTH
        PaymentDurationUnit: MONTH
        Duration: '1'
      Tags:
        - Key: Environment
          Value: Production
        - Key: Team
          Value: DataEngineering
        - Key: CreatedBy
          Value: ROS
        - Key: CostCenter
          Value: BigData
  LivyGateway:
    Type: ALIYUN::EMR::LivyCompute
    DependsOn: SparkWorkspace
    Properties:
      WorkspaceBizId:
        Fn::GetAtt:
          - SparkWorkspace
          - WorkspaceId
      Name:
        Fn::Sub: ${CommonName}-livy
      DisplayReleaseVersion:
        Ref: SparkEngineVersion
      LivyVersion: 0.8.0
      AuthType: Token
      CpuLimit:
        Ref: LivyCpuLimit
      MemoryLimit:
        Ref: LivyMemoryLimit
      QueueName: root_queue
      EnablePublic:
        Ref: EnableLivyPublicAccess
      Fusion: true
      AutoStartConfiguration:
        Enable: true
      AutoStopConfiguration:
        Enable: true
        IdleTimeoutMinutes:
          Ref: LivyIdleTimeoutMinutes
Outputs:
  WorkspaceId:
    Description:
      zh-cn: Spark工作空间ID。
      en: Spark workspace ID.
    Label:
      zh-cn: 工作空间ID
      en: Workspace ID
    Value:
      Fn::GetAtt:
        - SparkWorkspace
        - WorkspaceId
  Storage:
    Description:
      zh-cn: 工作空间的OSS存储路径。
      en: The OSS storage path of the workspace.
    Label:
      zh-cn: OSS存储路径
      en: Storage Path
    Value:
      Fn::GetAtt:
        - SparkWorkspace
        - Storage
  OssBucketName:
    Description:
      zh-cn: OSS存储桶名称。
      en: OSS bucket name.
    Label:
      zh-cn: OSS存储桶名称
      en: OSS Bucket Name
    Value:
      Fn::GetAtt:
        - OssBucket
        - Name
  RamRoleArn:
    Description:
      zh-cn: RAM角色ARN。
      en: RAM role ARN.
    Label:
      zh-cn: RAM角色ARN
      en: RAM Role ARN
    Value:
      Fn::GetAtt:
        - RamRole
        - Arn
  LivyComputeId:
    Description:
      zh-cn: Livy网关ID。
      en: Livy gateway ID.
    Label:
      zh-cn: Livy网关ID
      en: Livy Gateway ID
    Value:
      Fn::GetAtt:
        - LivyGateway
        - LivyComputeId
  LivyStatus:
    Description:
      zh-cn: Livy网关状态。
      en: Livy gateway status.
    Label:
      zh-cn: Livy网关状态
      en: Livy Status
    Value:
      Fn::GetAtt:
        - LivyGateway
        - Status
  LivyEndpoint:
    Description:
      zh-cn: Livy网关公网访问地址。
      en: Livy gateway public endpoint.
    Label:
      zh-cn: Livy公网地址
      en: Livy Public Endpoint
    Value:
      Fn::GetAtt:
        - LivyGateway
        - Endpoint
  LivyEndpointInner:
    Description:
      zh-cn: Livy网关内网访问地址。
      en: Livy gateway internal endpoint.
    Label:
      zh-cn: Livy内网地址
      en: Livy Internal Endpoint
    Value:
      Fn::GetAtt:
        - LivyGateway
        - EndpointInner
Metadata:
  ALIYUN::ROS::Interface:
    ParameterGroups:
      - Parameters:
          - CommonName
          - Cu
          - GpuCount
      - Parameters:
          - SparkEngineVersion
          - LivyCpuLimit
          - LivyMemoryLimit
          - LivyIdleTimeoutMinutes
          - EnableLivyPublicAccess

JSON

{
  "ROSTemplateFormatVersion": "2015-09-01",
  "Description": {
    "zh-cn": "创建完整的EMR Serverless Spark数据分析平台，包含OSS存储桶、RAM角色、Spark工作空间（GPU加速+IP白名单+自动续费）、Livy网关（自动启停+公网访问），实现端到端的Spark大数据计算与交互式分析。",
    "en": "Create a complete EMR Serverless Spark data analysis platform including OSS bucket, RAM role, Spark workspace with GPU acceleration and IP whitelist, and Livy gateway with auto start/stop and public access for end-to-end Spark big data computing and interactive analysis."
  },
  "Parameters": {
    "CommonName": {
      "Type": "String",
      "Label": {
        "zh-cn": "资源名称前缀",
        "en": "Resource Name Prefix"
      },
      "Description": {
        "zh-cn": "所有资源名称的统一前缀。",
        "en": "Unified name prefix for all resources."
      },
      "Default": "spark-platform",
      "MinLength": 1,
      "MaxLength": 50
    },
    "Cu": {
      "Type": "String",
      "Label": {
        "zh-cn": "计算单元(CU)",
        "en": "Compute Units (CU)"
      },
      "Description": {
        "zh-cn": "工作空间的最大计算单元数量。",
        "en": "The maximum number of compute units for the workspace."
      },
      "Default": "50"
    },
    "GpuCount": {
      "Type": "Number",
      "Label": {
        "zh-cn": "GPU数量",
        "en": "GPU Count"
      },
      "Description": {
        "zh-cn": "工作空间的GPU数量。",
        "en": "The number of GPUs for the workspace."
      },
      "Default": 2,
      "MinValue": 0
    },
    "SparkEngineVersion": {
      "Type": "String",
      "Label": {
        "zh-cn": "Spark引擎版本",
        "en": "Spark Engine Version"
      },
      "Description": {
        "zh-cn": "Livy网关使用的Spark引擎版本。",
        "en": "The Spark engine version used by the Livy gateway."
      },
      "Default": "esr-4.2.0"
    },
    "LivyCpuLimit": {
      "Type": "String",
      "Label": {
        "zh-cn": "Livy CPU核数",
        "en": "Livy CPU Cores"
      },
      "Description": {
        "zh-cn": "Livy服务器的CPU核数。",
        "en": "The number of CPU cores for the Livy server."
      },
      "AllowedValues": [
        "1",
        "2",
        "4"
      ],
      "Default": "2"
    },
    "LivyMemoryLimit": {
      "Type": "String",
      "Label": {
        "zh-cn": "Livy内存大小",
        "en": "Livy Memory Size"
      },
      "Description": {
        "zh-cn": "Livy服务器的内存大小。",
        "en": "The memory size of the Livy server."
      },
      "AllowedValues": [
        "4Gi",
        "8Gi",
        "16Gi"
      ],
      "Default": "8Gi"
    },
    "LivyIdleTimeoutMinutes": {
      "Type": "Number",
      "Label": {
        "zh-cn": "Livy空闲超时(分钟)",
        "en": "Livy Idle Timeout (Minutes)"
      },
      "Description": {
        "zh-cn": "Livy网关空闲超时后自动停止的时间。",
        "en": "The idle timeout in minutes before the Livy gateway automatically stops."
      },
      "Default": 60,
      "MinValue": 1
    },
    "EnableLivyPublicAccess": {
      "Type": "String",
      "Label": {
        "zh-cn": "是否开启Livy公网访问",
        "en": "Enable Livy Public Access"
      },
      "Description": {
        "zh-cn": "是否为Livy网关开启公网访问端点。",
        "en": "Whether to enable public endpoint for the Livy gateway."
      },
      "AllowedValues": [
        "true",
        "false"
      ],
      "Default": "false"
    }
  },
  "Conditions": {
    "EnablePublic": {
      "Fn::Equals": [
        {
          "Ref": "EnableLivyPublicAccess"
        },
        "true"
      ]
    }
  },
  "Resources": {
    "OssBucket": {
      "Type": "ALIYUN::OSS::Bucket",
      "Properties": {
        "BucketName": {
          "Fn::Sub": "${CommonName}-${ALIYUN::AccountId}"
        },
        "AccessControl": "private",
        "DeletionForce": true,
        "ServerSideEncryptionConfiguration": {
          "SSEAlgorithm": "AES256"
        },
        "Tags": {
          "CreatedBy": "ROS",
          "Purpose": "SparkPlatform"
        }
      }
    },
    "RamRole": {
      "Type": "ALIYUN::RAM::Role",
      "Properties": {
        "RoleName": {
          "Fn::Sub": "${CommonName}-spark-role"
        },
        "Description": "RAM role for EMR Serverless Spark platform",
        "AssumeRolePolicyDocument": {
          "Version": "1",
          "Statement": [
            {
              "Action": "sts:AssumeRole",
              "Effect": "Allow",
              "Principal": {
                "Service": [
                  "emr-serverless-spark.aliyuncs.com"
                ]
              }
            }
          ]
        },
        "Policies": [
          {
            "PolicyName": "AliyunEMRServerlessSparkFullAccess",
            "PolicyDocument": {
              "Version": "1",
              "Statement": [
                {
                  "Action": [
                    "oss:GetObject",
                    "oss:PutObject",
                    "oss:DeleteObject",
                    "oss:ListObjects",
                    "oss:AbortMultipartUpload",
                    "oss:ListMultipartUploads",
                    "oss:GetBucketInfo"
                  ],
                  "Resource": [
                    {
                      "Fn::Sub": "acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}"
                    },
                    {
                      "Fn::Sub": "acs:oss:*:*:${CommonName}-${ALIYUN::AccountId}/*"
                    }
                  ],
                  "Effect": "Allow"
                }
              ]
            }
          }
        ]
      }
    },
    "SparkWorkspace": {
      "Type": "ALIYUN::EMR::SparkWorkspace",
      "DependsOn": [
        "OssBucket",
        "RamRole"
      ],
      "Properties": {
        "WorkspaceName": {
          "Fn::Sub": "${CommonName}-workspace"
        },
        "OssBucket": {
          "Fn::Sub": "oss://${CommonName}-${ALIYUN::AccountId}/spark-workspace/"
        },
        "RamRoleName": {
          "Ref": "RamRole"
        },
        "PaymentType": "PayAsYouGo",
        "ResourceSpec": {
          "Cu": {
            "Ref": "Cu"
          },
          "Gpu": {
            "Ref": "GpuCount"
          }
        },
        "AutoStartSessionCluster": true,
        "Subscription": {
          "AutoRenew": true,
          "AutoRenewPeriod": "1",
          "AutoRenewPeriodUnit": "MONTH",
          "PaymentDurationUnit": "MONTH",
          "Duration": "1"
        },
        "Tags": [
          {
            "Key": "Environment",
            "Value": "Production"
          },
          {
            "Key": "Team",
            "Value": "DataEngineering"
          },
          {
            "Key": "CreatedBy",
            "Value": "ROS"
          },
          {
            "Key": "CostCenter",
            "Value": "BigData"
          }
        ]
      }
    },
    "LivyGateway": {
      "Type": "ALIYUN::EMR::LivyCompute",
      "DependsOn": "SparkWorkspace",
      "Properties": {
        "WorkspaceBizId": {
          "Fn::GetAtt": [
            "SparkWorkspace",
            "WorkspaceId"
          ]
        },
        "Name": {
          "Fn::Sub": "${CommonName}-livy"
        },
        "DisplayReleaseVersion": {
          "Ref": "SparkEngineVersion"
        },
        "LivyVersion": "0.8.0",
        "AuthType": "Token",
        "CpuLimit": {
          "Ref": "LivyCpuLimit"
        },
        "MemoryLimit": {
          "Ref": "LivyMemoryLimit"
        },
        "QueueName": "root_queue",
        "EnablePublic": {
          "Ref": "EnableLivyPublicAccess"
        },
        "Fusion": true,
        "AutoStartConfiguration": {
          "Enable": true
        },
        "AutoStopConfiguration": {
          "Enable": true,
          "IdleTimeoutMinutes": {
            "Ref": "LivyIdleTimeoutMinutes"
          }
        }
      }
    }
  },
  "Outputs": {
    "WorkspaceId": {
      "Description": {
        "zh-cn": "Spark工作空间ID。",
        "en": "Spark workspace ID."
      },
      "Label": {
        "zh-cn": "工作空间ID",
        "en": "Workspace ID"
      },
      "Value": {
        "Fn::GetAtt": [
          "SparkWorkspace",
          "WorkspaceId"
        ]
      }
    },
    "Storage": {
      "Description": {
        "zh-cn": "工作空间的OSS存储路径。",
        "en": "The OSS storage path of the workspace."
      },
      "Label": {
        "zh-cn": "OSS存储路径",
        "en": "Storage Path"
      },
      "Value": {
        "Fn::GetAtt": [
          "SparkWorkspace",
          "Storage"
        ]
      }
    },
    "OssBucketName": {
      "Description": {
        "zh-cn": "OSS存储桶名称。",
        "en": "OSS bucket name."
      },
      "Label": {
        "zh-cn": "OSS存储桶名称",
        "en": "OSS Bucket Name"
      },
      "Value": {
        "Fn::GetAtt": [
          "OssBucket",
          "Name"
        ]
      }
    },
    "RamRoleArn": {
      "Description": {
        "zh-cn": "RAM角色ARN。",
        "en": "RAM role ARN."
      },
      "Label": {
        "zh-cn": "RAM角色ARN",
        "en": "RAM Role ARN"
      },
      "Value": {
        "Fn::GetAtt": [
          "RamRole",
          "Arn"
        ]
      }
    },
    "LivyComputeId": {
      "Description": {
        "zh-cn": "Livy网关ID。",
        "en": "Livy gateway ID."
      },
      "Label": {
        "zh-cn": "Livy网关ID",
        "en": "Livy Gateway ID"
      },
      "Value": {
        "Fn::GetAtt": [
          "LivyGateway",
          "LivyComputeId"
        ]
      }
    },
    "LivyStatus": {
      "Description": {
        "zh-cn": "Livy网关状态。",
        "en": "Livy gateway status."
      },
      "Label": {
        "zh-cn": "Livy网关状态",
        "en": "Livy Status"
      },
      "Value": {
        "Fn::GetAtt": [
          "LivyGateway",
          "Status"
        ]
      }
    },
    "LivyEndpoint": {
      "Description": {
        "zh-cn": "Livy网关公网访问地址。",
        "en": "Livy gateway public endpoint."
      },
      "Label": {
        "zh-cn": "Livy公网地址",
        "en": "Livy Public Endpoint"
      },
      "Value": {
        "Fn::GetAtt": [
          "LivyGateway",
          "Endpoint"
        ]
      }
    },
    "LivyEndpointInner": {
      "Description": {
        "zh-cn": "Livy网关内网访问地址。",
        "en": "Livy gateway internal endpoint."
      },
      "Label": {
        "zh-cn": "Livy内网地址",
        "en": "Livy Internal Endpoint"
      },
      "Value": {
        "Fn::GetAtt": [
          "LivyGateway",
          "EndpointInner"
        ]
      }
    }
  },
  "Metadata": {
    "ALIYUN::ROS::Interface": {
      "ParameterGroups": [
        {
          "Parameters": [
            "CommonName",
            "Cu",
            "GpuCount"
          ]
        },
        {
          "Parameters": [
            "SparkEngineVersion",
            "LivyCpuLimit",
            "LivyMemoryLimit",
            "LivyIdleTimeoutMinutes",
            "EnableLivyPublicAccess"
          ]
        }
      ]
    }
  }
}