Usage notes of Mars-阿里云帮助中心

This topic describes how to create Mars clusters, use Mars to read data from MaxCompute tables, and obtain the URL of the Mars UI.

Prerequisites

The Mars runtime environment is prepared. For more information, see Preparations.

Mars cluster operations

Create a Mars cluster
Run the following commands to create a Mars cluster. This process takes a while to complete.
```
from odps import options
options.verbose = True  
# If the preceding commands have been configured on the DataWorks PyODPS 3 node, you do not need to run the two commands. 
client = o.create_mars_cluster(5, 4, 16, min_worker_num=3)
```
Parameters:
- 5: the number of worker nodes in the cluster.
- 4: the number of CPU cores for each worker node.
- 16: the memory size of each worker node. Unit: GB.
  Note
  The memory size that you request for each worker node must be greater than 1 GB. The optimal ratio of CPU cores to the memory size is 1:4. For example, configure a worker node with 4 CPU cores and 16 GB of memory.
  You can create a maximum of 30 worker nodes. If the number of worker nodes exceeds the upper limit, the image server may be overloaded. If you require more than 30 worker nodes, search for the DingTalk group (ID: 11782920) to join the MaxCompute developer community to contact technical support.
- min_worker_num: the minimum number of worker nodes that must be started for the system to return a client object. If this parameter is set to 3, the system returns a client object after the three worker nodes are started.
If you set options.verbose to True when you create a Mars cluster, the URLs of Logview and Mars UI of the MaxCompute instance are displayed in the command output. You can use the Mars UI to connect to Mars clusters and query the status of clusters and jobs.
Submit a job
When you create a Mars cluster, the cluster creates a default session that connects to the cluster. You can call the .execute() method to submit a job to the cluster and run the job in the default session.
```
import mars.dataframe as md
import mars.tensor as mt

md.DataFrame(mt.random.rand(10, 3)).execute()  # Call the .execute() method to submit the job to the created cluster. 
```
Stop and release a cluster
A Mars cluster is automatically released three days after it is created. You can also call the client.stop_server() method to release a cluster.
```
client.stop_server()
```

Read and write operations on MaxCompute tables

Mars can directly read data from and write data to MaxCompute tables. This section uses reading from and writing to the test_mars and test_mars_persist tables as examples. The following code shows the table creation statements for test_mars and test_mars_persist.

CREATE TABLE test_mars(col1 INT,col2 INT);
INSERT into test_mars(col1, col2) VALUES (1, 2);
INSERT into test_mars(col1, col2) VALUES (3, 4);
INSERT into test_mars(col1, col2) VALUES (5, 6);

CREATE TABLE test_mars_persist(col1 INT,col2 INT);

Read data from a table
Mars calls the o.to_mars_dataframe method to read data from a MaxCompute table and returns a Mars DataFrame.
```
df = o.to_mars_dataframe('test_mars')
df.head(3).execute()
```
The returned result:
```
  col1 col2
0    1    2
1    3    4
2    5    6
```

Write data to a table

Mars calls the o.persist_mars_dataframe(df, 'table_name') method to save a Mars DataFrame as a MaxCompute table.

df = o.to_mars_dataframe('test_mars')
df2 = df + 1
o.persist_mars_dataframe(df2, 'test_mars_persist')  # Save the Mars DataFrame as a MaxCompute table. 
o.get_table('test_mars_persist').to_df().head(6)  # Call the PyODPS DataFrame API operation to query data.

The returned result:

   col1  col2
0     2     3
1     4     5
2     6     7

Other operations

Use an existing Mars cluster
Recreate a Mars cluster based on the instance ID.
```
client = o.create_mars_cluster(instance_id=**instance-id**)
```
Obtain the URL of the Mars UI
If you set options.verbose to True when you create a Mars cluster, the URL of the Mars UI is automatically displayed in the command output. You can use client.endpoint to obtain the URL of the Mars UI.
```
print(client.endpoint)
```
Obtain the Logview URL of an instance
If you set options.verbose to True when you create a Mars cluster, the Logview URL is automatically displayed in the command output. You can also use client.get_logview_address() to obtain the Logview URL.
```
print(client.get_logview_address())
```