Environment preparation for user profile analysis-DataWorks(DataWorks)-阿里云帮助中心

Background

To improve business strategies, you need to extract basic user profile data, such as geographic and social attributes, from user website behavior. This enables scheduled profile analysis for the fine-grained management of website traffic.

Before you begin

Before you start, read the experiment introduction to understand the complete workflow of the user profile analysis use case.

Notes

This tutorial provides the required user information dataset and website access test dataset. You can use them directly.
This tutorial provides mock data for hands-on practice in DataWorks.
This tutorial uses Data Studio (New Version) for data processing.

Prepare the OSS environment

For this tutorial, you need to create an Object Storage Service (OSS) bucket. You will later synchronize user information and website access logs to this bucket for data modeling and analysis.

Log on to the OSS console.
In the left-side navigation pane, click Buckets. On the Buckets page, click Create Bucket.

In the Create Bucket dialog box, configure the parameters and click Complete Creation. The following table describes the parameters.

Parameter	Value
Bucket Name	For this tutorial, enter `dw-spark-demo`.
Region	Select China (Shanghai).
OSS-HDFS	Enable OSS-HDFS as prompted on the UI.

On the Buckets page, click the name of your Bucket Name to open its File Management page.

Prepare the EMR Serverless Spark workspace

This tutorial uses EMR Serverless Spark. Ensure you have a Spark workspace. If not, create a workspace. The following table describes the parameters.

Parameter	Value
Region	China (Shanghai)
Payment Type	pay-as-you-go
Workspace Name	Enter a custom name.
DLF for Metadata Storage	Select the Data Lake Formation (DLF) data catalog to associate. If you want to completely isolate metadata between different EMR clusters, select different catalogs. Important You can select a DLF or DLF-Legacy (displayed as DLF 1.0 in the console) version. After you select a version, create Paimon or Hive tables based on your selection.
Workspace Directory	Select an OSS bucket path to store job log files.

Note

Professional Edition: This workspace includes all features of the Basic Edition, as well as advanced features and performance improvements. It is suitable for large-scale ETL jobs.
Basic Edition: This workspace includes all basic features and provides a high-performance compute engine.

Prepare the DataWorks environment

Before you use DataWorks for development, make sure that you have activated DataWorks. For more information, see Activate DataWorks.

Step 1: Create a workspace

If you already have a workspace that uses the new version of Data Studio in the China (Shanghai) region, you can use it and skip this step.

Log on to the DataWorks console. In the top navigation bar, switch the region to China (Shanghai). In the left-side navigation pane, click Work space to go to the workspace list page.
Click Create Workspace to create a Use Data Studio (New Version) workspace that uses Data Studio (New Version) and has Isolate Development and Production Environments.

Note
Starting February 18, 2025, Data Studio (New Version) is enabled by default for new DataWorks workspaces created in the China (Shanghai) region by first-time users.

For more information, see Create a workspace.

Step 2: Create a serverless resource group

Purchase a serverless resource group.

This tutorial requires a DataWorks serverless resource group for data synchronization and scheduling. You must purchase a serverless resource group and complete the initial setup.
Go to the DataWorks - Resource Group List page. In the top navigation bar, switch the region to China (Shanghai). In the left-side navigation pane, click Resource Group to go to the Resource Groups page.
Click Create Resource Group. On the resource group purchase page, set Region and Zone to China (Shanghai) and specify a Resource Group Name. Configure other parameters as prompted and complete the payment. For information about the billing of serverless resource groups, see Billing of serverless resource groups.

Note
If no Virtual Private Cloud (VPC) or vSwitch is available in the current region, click the corresponding console link in the parameter description to create them. For more information about VPCs and vSwitches, see What is a Virtual Private Cloud (VPC)?.
Bind the resource group to a DataWorks workspace.

You must bind a new serverless resource group to a workspace before you can use it.

Go to the DataWorks - Resource Group List page. In the top navigation bar, switch the region to China (Shanghai). Find the serverless resource group that you purchased. In the Operation column, click Associate Workspace, and then click Associate next to the DataWorks workspace that you created.

Configure public internet access for the resource group.

This tutorial retrieves sample data from the internet. By default, resource groups do not have access to the public internet. Therefore, you must configure an Internet NAT Gateway and an Elastic IP Address (EIP) for the VPC that is bound to the resource group to enable internet access.

Log on to the Virtual Private Cloud - Internet NAT Gateway console. In the top navigation bar, switch the region to China (Shanghai).

Click Create Internet NAT Gateway and configure the parameters. The following table lists the key parameters for this tutorial. Use the default values for other parameters.

Parameter	Value
Region	China (Shanghai).
Network and Zone	Select the VPC and vSwitch that are bound to the resource group. You can go to the DataWorks console, switch the region to China (Shanghai), and then click Resource Group in the left-side navigation pane to go to the Resource Groups page. Find the resource group that you created, click Network Settings in the Operation column, and then view the bound VPC and Switch in the Data Scheduling & Data Integration section. For more information about VPCs and vSwitches, see What is a Virtual Private Cloud (VPC)?.
Network Type	Internet NAT Gateway.
EIP	Select Create Elastic IP Address.
Service-linked role creation	If this is the first time you create a NAT gateway, you must create a service-linked role. Click Create Service-Linked Role.

Click Buy Now, agree to the terms of service, and then click Activate to complete the purchase.

After purchasing the Internet NAT Gateway instance, return to the console and create an SNAT entry for it.

Note

The resource group can access the internet through the VPC only after you configure an SNAT entry.

In the Actions column of the newly purchased instance, click Management to go to the management page of the instance. Then, click the SNAT Management tab.

In the SNAT Entry List section, click Create SNAT Entry. The following table describes the key parameters.

Parameter	Value
SNAT entry granularity	Select VPC to ensure that all resource groups in the VPC where the NAT gateway resides can access the internet through the configured Elastic IP Address.
Select an Elastic IP address	Select the Elastic IP Address associated with the current NAT gateway instance.

After configuring the SNAT entry, click OK.

In the SNAT Entry List, when the Status of the newly created SNAT entry changes to Enabled, the VPC bound to the resource group can access the internet.

For more information, see Use a serverless resource group.

3. Associate EMR Serverless Spark

Go to the DataWorks - Workspace List page. In the top navigation bar, switch the region to China (Shanghai). Find the workspace that you created and click its name to go to the Workspace Details page.
In the left-side navigation pane, click Computing Resources.

Click Associate Computing Resources, select a Compute Resource Type, and then configure the parameters.

This tutorial uses EMR Serverless Spark as the computing and storage resource. Set EMR Serverless Spark to EMR Serverless Spark and configure the parameters. The following table describes the key parameters. You can keep the default values for other parameters.

Parameter	Description
EMR Serverless Spark Workspace	Select the EMR Serverless Spark workspace to associate. If you do not have one, click New to go to the EMR Serverless Spark console and create one. Then, return to DataWorks and select the newly created workspace. Note If you selected the option to isolate development and production environments when you created the workspace, you must select separate Spark workspaces for the development and production environments. For more information about how to create a Spark workspace, see Create a Spark workspace.
Default Engine Version	When you create an EMR Spark node in Data Studio, the engine version and resource queue that are configured here are used by default.
Default Resource Queue
Default Access Identity	The identity that is used to access the Spark workspace from the current workspace. Development environment: Only the Executor identity is supported. Production environment: The Alibaba Cloud primary account, Alibaba Cloud RAM sub-account, and Node Owner identities are supported.
Computing Resource Instance Name	A unique name to identify the computing resource. This name is used to select the computing resource when a node runs.

Click Confirm to complete the Serverless Spark computing resource configuration.

For more information about how to associate a computing resource, see Associate a computing resource.

Next steps

You have now prepared the environment. In the next tutorial, you will learn to synchronize user data and website access logs to OSS, and then use a Spark SQL node to create tables and query the synchronized data. For details, see Synchronize data.