PyODPS is the Python SDK for MaxCompute. It lets you develop MaxCompute tasks, analyze data, and manage resources from Python.
PyODPS
PyODPS provides a DataFrame API and methods for managing MaxCompute objects. It supports Python 2 (version 2.6 and later) and Python 3.
Learn more about PyODPS from these resources:
-
Learn about PyODPS: PyODPS documentation.
-
Install PyODPS: PyODPS installation guide.
-
Develop with PyODPS: PyODPS development guide.
Initialize the client
Before using PyODPS, initialize a MaxCompute client object with your Alibaba Cloud account credentials:
import os
from odps import ODPS
# Make sure that the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET
# environment variables are set to your AccessKey ID and AccessKey Secret.
o = ODPS(
os.getenv('ALIBABA_CLOUD_ACCESS_KEY_ID'),
os.getenv('ALIBABA_CLOUD_ACCESS_KEY_SECRET'),
project='your-default-project',
endpoint='your-end-point',
)Parameters:
-
ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET: Set these environment variables to an AccessKey ID and an AccessKey Secret that have the necessary permission to operate on objects in the target MaxCompute project. You can obtain these credentials from the AccessKey management page.
-
your-default-project: The name of your MaxCompute project. You can find the project name in the MaxCompute console under Workspace > Projects in the left-side navigation pane.
-
your-end-point: The endpoint for the project's region.
Methods
Common PyODPS methods for MaxCompute operations:
|
Object type |
Method |
Description |
|
project |
get_project(project_name) |
Retrieves a MaxCompute project. |
|
exist_project(project_name) |
Checks whether a MaxCompute project exists. |
|
|
table |
list_tables() |
Lists all tables in a MaxCompute project. |
|
exist_table(table_name) |
Checks whether a table exists. |
|
|
get_table(table_name, project=project_name) |
Retrieves a table. Cross-project access is supported. |
|
|
create_table() |
Creates a table. |
|
|
read_table() |
Reads data from a table. |
|
|
write_table() |
Writes data to a table. |
|
|
delete_table() |
Deletes a table. |
|
|
table partition |
exist_partition() |
Checks whether a table partition exists. |
|
get_partition() |
Retrieves a table partition. |
|
|
create_partition() |
Creates a table partition. |
|
|
delete_partition() |
Deletes a table partition. |
|
|
SQL statement |
execute_sql()/run_sql() |
Executes an SQL statement. |
|
open_reader() |
Reads the execution result. |
|
|
instance |
list_instances() |
Lists all instances in a MaxCompute project. |
|
exist_instance() |
Checks whether an instance exists. |
|
|
get_instance() |
Retrieves an instance. |
|
|
stop_instance() |
Stops a running instance. |
|
|
resource |
create_resource() |
Creates a resource. |
|
open_resource() |
Opens a resource for reading. |
|
|
get_resource() |
Retrieves a resource. |
|
|
list_resources() |
Lists all resources in the project. |
|
|
exist_resource() |
Checks whether a resource exists. |
|
|
delete_resource() |
Deletes a resource. |
|
|
function |
create_function() |
Creates a function. |
|
delete_function() |
Deletes a function. |
|
|
data tunnel |
create_upload_session() |
Creates a data upload session. |
|
create_download_session() |
Creates a data download session. |
The create_table(), read_table(), write_table(), and delete_table() methods require parameters. For usage details, see Python SDK examples: Table.