Use the Spark client to submit jobs in MaxCompute-MaxCompute(MaxCompute)-阿里云帮助中心

Download the Spark client and submit jobs with the spark-submit command. Spark 3 or later is recommended.

Spark-submit

The bin directory contains the spark-submit script, which launches applications on a cluster. It supports all Spark cluster managers through a unified interface, so you do not need to configure your application for each manager. Apache Spark documentation.

Spark releases

Version	Download	Description
Spark3.5.2	Spark-3.5.2-odps0.49.0	Concurrent reads Vectorized read/write for all data types Three-tier model project read/write Kubernetes scheduling Shuffle Service
Spark3.4.2	Spark-3.4.2-odps0.48.0	Concurrent reads Vectorized read/write for all data types Three-tier model project read/write Kubernetes scheduling Shuffle Service
Spark3.1.1	Spark-3.1.1-odps0.47.0	Vectorized read/write for all data types Three-tier model project read/write Schema evolution table reads OSS/OTS external table reads Dynamic resource scaling
Spark2.4.5	Spark-2.4.5-odps0.47.0	Vectorized reads for all data types Schema evolution table reads OSS/OTS external table reads Dynamic resource scaling
Spark2.3.0	Spark-2.3.0-odps0.47.0	Vectorized reads for all data types Improved read performance odps-sdk upgraded from 0.30.12 to 0.35.9 StsAccount job submission Overwrite in local mode Removed fastjson dependency Partition column type support Partition columns support STRING, VARCHAR, CHAR, TINYINT, SMALLINT, INT, and BIGINT data types