Connect a personal development environment to VS Code for local development-DataWorks(DataWorks)-阿里云帮助中心

This solution is for DataWorks users who want to develop more efficiently. It connects a local Visual Studio Code editor to a cloud personal development environment over SSH. You can use familiar local tools together with cloud compute and data resources to code and debug more efficiently.

Process overview

Prepare the local environment: Generate an SSH key on your local computer.
Configure the environment: Configure the SSH public key and access method for the personal development environment in DataWorks.
Install the service: Install and start the SSH service in the personal development environment.
Connect remotely: Connect to the personal development environment from Visual Studio Code.

Step 1: Generate an SSH key pair

Before configuring a connection, generate an SSH key pair on your local development machine (the computer from which you will connect).

macOS

Run the key generation command: Open the Terminal application, enter ssh-keygen, and press Enter.
```
ssh-keygen
```
Use the default settings: When prompted for a save path and passphrase, press Enter three times to accept all defaults so that other tools can connect seamlessly later.
The command creates two files in the default directory:
- id_rsa: Your private key. Store it securely and never disclose it.
- id_rsa.pub: Your public key. You will add it to DataWorks.
Copy the public key: The default directory is /Users/<your_username>/.ssh/. To copy the public key to the clipboard, run the following command in Terminal:
```
pbcopy < ~/.ssh/id_rsa.pub
```

Windows

Run the key generation command: Open PowerShell from the Start menu or use Git Bash, and then enter ssh-keygen, and press Enter.
```
ssh-keygen
```
Use the default settings: When prompted for a save path and passphrase, press Enter three times to accept the default path and leave the passphrase blank, so that tools such as VS Code can connect seamlessly.
The command creates two files in the default directory:
- id_rsa: Your private key. Store it securely and never disclose it.
- id_rsa.pub: Your public key. You will add it to DataWorks.
Copy the public key: The default directory is C:\Users\<your_username>\.ssh\. Open the public key file id_rsa.pub in Notepad or another text editor, and copy its entire contents.

Linux (not recommended)

Important

Visual Studio Code requires a graphical interface, so we recommend using an operating system with a desktop environment.

Run the key generation command: Open the Terminal application, enter ssh-keygen, and press Enter.
```
ssh-keygen
```
Use the default settings: When prompted for a save path and passphrase, press Enter three times to accept all defaults so that other tools can connect seamlessly later.
The command creates two files in the default directory:
- id_rsa: Your private key. Store it securely and never disclose it.
- id_rsa.pub: Your public key. You will add it to DataWorks.
Copy the public key:
- root user keys are stored in /root/.ssh/.
- Regular user keys are stored in /home/<your_username>/.ssh/.
- Run the following command in a terminal and copy all of the public key shown in its output:
```
cat ~/.ssh/id_rsa.pub
```

Step 2: Configure the personal development environment

Go to the Workspaces page in the DataWorks console. In the top navigation bar, select a desired region. Find the desired workspace and choose Shortcuts > Data Studio in the Actions column.
Open your personal development environment.
On the top navigation bar, find Personal Development Environment - Select and click the adjacent icon to open the personal development environment that you created.
In the Personal Development Environment - Instance Configuration dialog box, select a configuration method that meets your needs.
For complete configuration instructions, see Create a personal development environment instance.

Scenario A: Access over the public Internet (flexible, but incurs additional charges)

Use this method to connect from any network with Internet access.

Configure the following settings:
- Enable SSH:Turn on .
- SSH public key: Paste the public key that you copied in Step 1.
- SSH access method: Select both Log on from the VPC and Log on over the public Internet.
- NAT gateway: Select an existing NAT gateway or create one as prompted on the page.
- Elastic IP address: Select an existing EIP or create one as prompted on the page.
  Important
  If you configured a VPC in network settings, configure a NAT gateway for that VPC. Otherwise, configure a NAT gateway for the resource group's default VPC.
Click Submit Changes and wait for the changes to take effect.
Important
Billing reminder: The NAT gateway and and elastic IP address (EIP) used for public Internet access are separate billable cloud products.The NAT gateway and EIP continue to incur charges even when the development environment instance is stopped.If you no longer need them, manually delete them in the Alibaba Cloud console to avoid unnecessary charges.

Scenario B: Access over the VPC internal network (secure, with no additional network charges)

Use this method in either of the following situations:

Your on-premises network is connected to an Alibaba Cloud VPC by VPN or a dedicated connection.
You connect from another ECS instance in the same VPC.

Configure the following settings:
- Enable SSH:Turn on .
- SSH public key: Paste the public key that you copied in Step 1.
- NAT gateway: No configuration is required.
- Elastic IP address: No configuration is required.
Click Submit Changes and wait for the changes to take effect.

Step 3: Install the SSH service

Go to the new Data Development page and open the configured personal development environment.

In the Terminal, run the following commands:

# Update the package list and install the SSH server
sudo apt-get update
sudo apt-get install openssh-server -y

# Start the SSH service
sudo service ssh start

# (Optional) Check SSH service status. "active (running)" indicates success
sudo service ssh status

If you need environment variables from the personal development environment in your local connection, also run the following commands:

sed -i 's/^#\?PermitUserEnvironment.*$/PermitUserEnvironment yes/' /etc/ssh/sshd_config
cat /proc/1/environ | tr '\0' '\n' > ~/.ssh/environment
service ssh restart

Step 4: Connect to the development environment from Visual Studio Code

Get the connection command
On the top navigation bar, find Personal Development Environment - Select and click the adjacent icon, then click Manage Development Environments. For the target instance, view Access Configuration.
- Public access configured: Copy the command under Public Access Method.
- VPC access only: Copy the command under VPC Access Method.
Install the Visual Studio Code extension
Important
This step applies to graphical systems such as Windows and macOS. If you use a headless Linux system, connect from a terminal.
In the Visual Studio Code extension marketplace, search for and install the Remote - SSH extension.
Add and connect to a host
- Click the remote connection icon in the lower-left corner of Visual Studio Code..
- In the menu that opens, select Connect to Host... > Add New SSH Host....
- Paste the complete SSH command copied in Step 1 into the input box, and press Enter.
  Important
  If the private key is not in the default directory, specify its path. Also add ssh -i /your/path/rsa root@xx.xx.xx.xx -p 1024.
- Visual Studio Code prompts you to select an SSH configuration file. Select the default file.
- When a notification appears in the lower-right corner, click Connect.
- A new Visual Studio Code window opens and starts connecting. After the connection succeeds, a new window appears and the connected SSH host address is shown in the lower-left corner.
Open the working directory
After connecting, Visual Studio Code may open in the /root directory. Your code files are usually in /mnt/workspace.
- . In the Explorer pane, click Open Folder.
- Enter /mnt/workspace as the path, and then click OK.
- You can now view and edit in Visual Studio Code all files under Personal Directory > workspace in your DataWorks environment.

Appendix: differences between local development and Data Studio development

EMR Serverless Sparkdevelopment

Compared with Data Studio development, local development in Visual Studio Code requires the additional parameter --project {EMR Serverless Spark cluster ID} --endpoint {EMR Serverless Spark endpoint}.

MaxFrame development

Compared with Data Studio development, local development in Visual Studio Code requires the additional parameters --project {MaxCompute project name} --endpont {MaxCompute endpoint}.