如何安装Spark单机版-Resource Orchestration Service(ROS)-阿里云帮助中心

ROS lets you install Spark in standalone mode by creating a stack.

Overview

Apache Spark is a general-purpose computing engine for large-scale data processing. It uses Scala as its application framework and supports in-memory distributed datasets for interactive queries and iterative workloads.

The Installs Spark on an ECS instance (existing VPC) sample template creates an ECS instance and binds an elastic IP address (EIP) in an existing Virtual Private Cloud (VPC), vSwitch, and security group. The template uses the following software versions:

Java Development Kit (JDK): 1.8.0
Hadoop (distributed system infrastructure): 2.7.7
Scala (programming language): 2.12.1
Spark (computing engine): 2.1.0

After the stack is created, get the SparkWebSiteURL to access the Spark management console. To access SparkWebSiteURL over the internet, add inbound rules to the security group for ports 8088 and 8080. Add a security group rule.

Step 1: Create a stack

Log on to the ROS console.
In the left-side navigation pane, choose Templates > Public Templates.
Find the Installs Spark on an ECS instance (existing VPC) template.

On the Sample Templates page, search for Spark and click Create stack on the template card.
Click Create Stack.

On the Configure Parameters page, enter a Stack name and configure the following parameters.

Parameter	Description	Example
Existing VPC ID	The ID of the VPC. Create and manage a VPC.	vpc-bp1m6fww66xbntjyc****
VSwitch Zone ID	The zone ID of the vSwitch in the VPC.	Hangzhou Zone K
VSwitch ID	The ID of the vSwitch in the VPC. Create and manage vSwitches.	vsw-bp183p93qs667muql****
Business security group ID	The ID of the security group for the ECS instance. Search for security groups.	sg-bp15ed6xe1yxeycg7o****
Instance specification	The instance type of the ECS instance. Select a valid instance type from the available Instance families.	ecs.c5.large
Image ID	The image ID of the ECS instance. Default: `centos_7`. Images.	centos_7
Instance password	The password of the ECS instance.	Test_12****
Public IP bandwidth	The bandwidth of the public IP address. Valid values: 1 to 100. Unit: Mbit/s.	5
Disk type	Valid values: `cloud_efficiency`: ultra disk. `cloud_ssd`: standard SSD. `cloud_essd`: enterprise SSD (ESSD). `cloud`: basic disk. `ephemeral_ssd`: local SSD. Cloud disk overview.	cloud_efficiency
System disk space	The system disk size, in GB. Valid values: 40 to 500 Unit: GB	40

Click Create.
On the Stack information tab, check the stack status. After creation completes, click the Output tab to obtain the SparkWebSiteURL.
Open the SparkWebSiteURL to log on to the Spark management console.

Step 2: View resources

In the left-side navigation pane, choose Stacks.
On the Stacks page, click the ID of the stack.

Click the Resources tab to view resource information.

The stack creates the following resources.

Resource	Quantity	Description	Specifications
ALIYUN::ECS::Instance	1	Creates an ECS instance to install Spark in standalone mode.	Total quantity: 1 Instance type: ecs.c5.large Disk type: ultra disk System disk space: 40 GB Assign public IP address: Yes

Note

For pricing details, see the pricing documentation for each product.