WebSocket connection guide

Updated at:

Use a WebSocket connection to access the Sambert real-time speech synthesis service. This topic covers the service endpoint, request headers, and interaction workflow.

The DashScope SDK currently supports only Java and Python. For other programming languages, use a WebSocket connection to communicate with the service.

User guide: For model details and recommendations, see Speech synthesis.

Service endpoint

Sambert is available only in the Beijing region.

The WebSocket service endpoint is: wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference

Replace {WorkspaceId} with your actual workspace ID.

ImportantThe URL is fixed and must use the wss:// protocol. Set the Authorization header as described in Request headers.

ImportantAlibaba Cloud Model Studio has released a workspace-specific domain for the China (Beijing) region. The new dedicated domain delivers superior performance and higher stability for inference requests. We recommend migrating from dashscope.aliyuncs.com to {WorkspaceId}.cn-beijing.maas.aliyuncs.com.

Replace {WorkspaceId} with your actual Workspace ID. The existing domain remains fully functional.

Request headers

Include the following headers in the request:

Parameter

Type

Required

Description

Authorization

string

Yes

Authorization token in the format Bearer <your_api_key>. Replace <your_api_key> with your actual API key.

user-agent

string

No

Client identifier for server-side request tracking.

X-DashScope-WorkSpace

string

No

The workspace ID of Alibaba Cloud Model Studio.

X-DashScope-DataInspection

string

No

Controls data inspection. Data inspection is enabled by default when this parameter is not set. Set it to enable to explicitly turn on data inspection. Don't set this parameter unless necessary.

ImportantAuthorization is verified during the WebSocket handshake. If the API key is invalid or missing, the handshake fails with an HTTP 401/403 error.

Interaction workflow

image

For details about client and server events, see Client events and Server-sent events.

The client and server interact in the following sequence:

  1. Establish a connection: The client establishes a WebSocket connection with the server.
  2. Start a task: The client sends a run-task event to start a task. Sambert sends all text to synthesize in a single run-task event.
  3. Wait for confirmation: The client receives a task-started event from the server, which indicates that the task has started.
  4. Receive audio: The client receives an audio stream from the server over the binary channel. The client also receives result-generated events that carry additional information such as timestamps.
  5. Complete the task: The client receives a task-finished event from the server, which indicates that the task is complete.
  6. Close the connection: The client closes the WebSocket connection.

After task-finished, send another run-task with a new task_id on the same connection. Submit the complete text in each run-task.

ImportantSambert supports output-only streaming (out), not bidirectional (duplex) streaming. All text to synthesize must be sent in a single run-task event. The continue-task and finish-task commands aren't supported.

For object pool configuration and a complete Java SDK concurrency example, see Speech synthesis best practices. When using the raw WebSocket protocol, reuse connections as described above.

Code examples

These examples use generic WebSocket libraries to send protocol events directly, with sambert-zhichu-v1. Obtain an API key, set the DASHSCOPE_API_KEY environment variable, and replace {WorkspaceId} with the workspace ID for the region. When switching regions, change both the endpoint and API key.

Go

go mod init tts-example
go get github.com/gorilla/websocket@v1.5.3 github.com/google/uuid@v1.6.0
package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"os"
	"time"

	"github.com/google/uuid"
	"github.com/gorilla/websocket"
)

const (
	wsURL      = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference" // WebSocket server address
	outputFile = "output.mp3"                                        // Output file path
)

func main() {
	// If you have not configured the API Key as an environment variable, replace the next line with: apiKey := "your_api_key". It is not recommended to hard-code the API Key directly in production code to reduce the risk of API Key leakage.
	apiKey := os.Getenv("DASHSCOPE_API_KEY")

	// Check and clear the output file
	if err := clearOutputFile(outputFile); err != nil {
		fmt.Println("Failed to clear output file:", err)
		return
	}

	// Connect to WebSocket service
	conn, err := connectWebSocket(apiKey)
	if err != nil {
		fmt.Println("Failed to connect to WebSocket:", err)
		return
	}
	defer closeConnection(conn)

	// Create a channel to receive task completion notifications
	done := make(chan struct{})

	// Start an asynchronous goroutine to receive messages
	go receiveMessage(conn, done)

	// Send run-task command
	if err := sendRunTaskMsg(conn); err != nil {
		fmt.Println("Failed to send run-task command:", err)
		return
	}

	// Wait for task completion or timeout
	select {
	case <-done:
		fmt.Println("Task ended")
	case <-time.After(5 * time.Minute):
		fmt.Println("Task timed out")
	}
}

// Define message struct
type Message struct {
	Header  Header  `json:"header"`
	Payload Payload `json:"payload"`
}

// Define header information
type Header struct {
	Action       string                 `json:"action,omitempty"`
	TaskID       string                 `json:"task_id"`
	Streaming    string                 `json:"streaming,omitempty"`
	Event        string                 `json:"event,omitempty"`
	ErrorCode    string                 `json:"error_code,omitempty"`
	ErrorMessage string                 `json:"error_message,omitempty"`
	Attributes   map[string]interface{} `json:"attributes"`
}

// Define payload information
type Payload struct {
	Model      string     `json:"model,omitempty"`
	TaskGroup  string     `json:"task_group,omitempty"`
	Task       string     `json:"task,omitempty"`
	Function   string     `json:"function,omitempty"`
	Input      Input      `json:"input,omitempty"`
	Parameters Parameters `json:"parameters,omitempty"`
	Output     Output     `json:"output,omitempty"`
	Usage      Usage      `json:"usage,omitempty"`
}

// Define input information
type Input struct {
	Text string `json:"text"`
}

// Define parameters information
type Parameters struct {
	TextType                string  `json:"text_type"`
	Format                  string  `json:"format"`
	SampleRate              int     `json:"sample_rate"`
	Volume                  int     `json:"volume"`
	Rate                    float64 `json:"rate"`
	Pitch                   float64 `json:"pitch"`
	WordTimestampEnabled    bool    `json:"word_timestamp_enabled"`
	PhonemeTimestampEnabled bool    `json:"phoneme_timestamp_enabled"`
}

// Define output information
type Output struct {
	Sentence Sentence `json:"sentence"`
}

// Define sentence information
type Sentence struct {
	BeginTime int    `json:"begin_time"`
	EndTime   int    `json:"end_time"`
	Words     []Word `json:"words"`
}

// Define word information
type Word struct {
	Text      string    `json:"text"`
	BeginTime int       `json:"begin_time"`
	EndTime   int       `json:"end_time"`
	Phonemes  []Phoneme `json:"phonemes"`
}

// Define phoneme information
type Phoneme struct {
	BeginTime int    `json:"begin_time"`
	EndTime   int    `json:"end_time"`
	Text      string `json:"text"`
	Tone      int    `json:"tone"`
}

// Define usage information
type Usage struct {
	Characters int `json:"characters"`
}

func receiveMessage(conn *websocket.Conn, done chan struct{}) {
	for {
		msgType, message, err := conn.ReadMessage()
		if err != nil {
			fmt.Println("Failed to parse server message:", err)
			close(done)
			break
		}

		if msgType == websocket.BinaryMessage {
			// Process binary audio stream
			if err := writeBinaryDataToFile(message, outputFile); err != nil {
				fmt.Println("Failed to write binary data:", err)
				close(done)
				break
			}
			fmt.Println("Audio segment written to local file")
		} else {
			// Process text message
			var msg Message
			if err := json.Unmarshal(message, &msg); err != nil {
				fmt.Println("Failed to parse event:", err)
				continue
			}
			if handleMessage(conn, msg, done) {
				break
			}
		}
	}
}

func handleMessage(conn *websocket.Conn, msg Message, done chan struct{}) bool {
	switch msg.Header.Event {
	case "task-started":
		fmt.Println("Task started")

	case "result-generated":
	// To retrieve additional messages, add the corresponding code here

	case "task-finished":
		fmt.Println("Task completed")
		close(done)
		return true

	case "task-failed":
		if msg.Header.ErrorMessage != "" {
			fmt.Printf("Task failed: %s\n", msg.Header.ErrorMessage)
		} else {
			fmt.Println("Task failed due to unknown reason")
		}
		close(done)
		return true

	default:
		fmt.Printf("Unexpected event: %v\n", msg)
		close(done)
	}

	return false
}

func sendRunTaskMsg(conn *websocket.Conn) error {
	runTaskMsg, err := generateRunTaskMsg()
	if err != nil {
		return err
	}
	if err := conn.WriteMessage(websocket.TextMessage, []byte(runTaskMsg)); err != nil {
		return err
	}
	return nil
}

func generateRunTaskMsg() (string, error) {
	runTaskMessage := Message{
		Header: Header{
			Action:    "run-task",
			TaskID:    uuid.New().String(),
			Streaming: "out",
		},
		Payload: Payload{
			Model:     "sambert-zhichu-v1",
			TaskGroup: "audio",
			Task:      "tts",
			Function:  "SpeechSynthesizer",
			Input: Input{
				Text: "The white sun descends behind the mountains, the Yellow River flows into the sea. To see a thousand li further, climb yet one more floor.",
			},
			Parameters: Parameters{
				TextType:                "PlainText",
				Format:                  "mp3",
				SampleRate:              16000,
				Volume:                  50,
				Rate:                    1.0,
				Pitch:                   1.0,
				WordTimestampEnabled:    true,
				PhonemeTimestampEnabled: true,
			},
		},
	}

	runTaskMsgJSON, err := json.Marshal(runTaskMessage)
	return string(runTaskMsgJSON), err
}

func connectWebSocket(apiKey string) (*websocket.Conn, error) {
	header := make(http.Header)
	header.Add("X-DashScope-DataInspection", "enable")
	header.Add("Authorization", fmt.Sprintf("bearer %s", apiKey))
	conn, _, err := websocket.DefaultDialer.Dial(wsURL, header)
	if err != nil {
		fmt.Println("Failed to connect to WebSocket:", err)
		return nil, err
	}
	return conn, nil
}

func writeBinaryDataToFile(data []byte, filePath string) error {
	file, err := os.OpenFile(filePath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
	if err != nil {
		return err
	}
	defer file.Close()
	_, err = file.Write(data)
	return err
}

func closeConnection(conn *websocket.Conn) {
	if conn != nil {
		conn.Close()
	}
}

func clearOutputFile(filePath string) error {
	file, err := os.OpenFile(filePath, os.O_TRUNC|os.O_CREATE|os.O_WRONLY, 0644)
	if err != nil {
		return err
	}
	file.Close()
	return nil
}

C#

Sample code:

using System.Net.WebSockets;
using System.Text;
using System.Text.Json;

class Program {
    // If you have not configured the API Key as an environment variable, replace the next line with: private const string ApiKey="your_api_key". It is not recommended to hard-code the API Key directly in production code to reduce the risk of API Key leakage.
    private static readonly string ApiKey = Environment.GetEnvironmentVariable("DASHSCOPE_API_KEY") ?? throw new InvalidOperationException("DASHSCOPE_API_KEY environment variable is not set.");

    private const string WebSocketUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference"; // WebSocket server address
    private const string OutputFilePath = "output.mp3"; // Output file path

    static async Task Main(string[] args) {
        var ws = new ClientWebSocket();
        try {
            // 1. Connect to the WebSocket service with authentication
            await ConnectWithAuth(ws, WebSocketUrl);

            // 2. Start the message receiving thread
            var receiveTask = ReceiveMessages(ws);

            // 3. Send the run-task command
            string textToSynthesize = "The white sun descends behind the mountains, the Yellow River flows into the sea. To see a thousand li further, climb yet one more floor.";
            string taskId = GenerateTaskId();
            await SendRunTaskCommand(ws, textToSynthesize, taskId);

            // 4. Wait for the receiving task to complete
            await receiveTask;
        } catch (Exception ex) {
            Console.WriteLine($"Error: {ex.Message}");
        } finally {
            if (ws.State == WebSocketState.Open) {
                await ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing connection", CancellationToken.None);
            }
        }
    }

    private static async Task ConnectWithAuth(ClientWebSocket ws, string url) {
        var uri = new Uri(url);
        ws.Options.SetRequestHeader("Authorization", $"bearer {ApiKey}");
        ws.Options.SetRequestHeader("X-DashScope-DataInspection", "enable");
        await ws.ConnectAsync(uri, CancellationToken.None);
        Console.WriteLine("Connected to the WebSocket server.");
    }

    private static string GenerateTaskId() {
        return Guid.NewGuid().ToString("N");
    }

    private static async Task SendRunTaskCommand(ClientWebSocket ws, string text, string taskId) {
        var command = CreateRunTaskCommand(text, taskId);
        var buffer = Encoding.UTF8.GetBytes(command);
        await ws.SendAsync(new ArraySegment<byte>(buffer), WebSocketMessageType.Text, true, CancellationToken.None);
        Console.WriteLine("run-task command sent.");
    }

    private static string CreateRunTaskCommand(string text, string taskId) {
        var command = new {
            header = new {
                action = "run-task",
                task_id = taskId,
                streaming = "out"
            },
            payload = new {
                model = "sambert-zhichu-v1",
                task_group = "audio",
                task = "tts",
                function = "SpeechSynthesizer",
                input = new {
                    text = text
                },
                parameters = new {
                    text_type = "PlainText",
                    format = "mp3",
                    sample_rate = 16000,
                    volume = 50,
                    rate = 1,
                    pitch = 1,
                    word_timestamp_enabled = true,
                    phoneme_timestamp_enabled = true
                }
            }
        };
        return JsonSerializer.Serialize(command);
    }

    private static async Task ReceiveMessages(ClientWebSocket ws) {
        var buffer = new byte[1024 * 4];
        var fs = new FileStream(OutputFilePath, FileMode.Create, FileAccess.Write);
        bool taskStarted = false;
        bool taskFinished = false;

        while (ws.State == WebSocketState.Open && !taskFinished) {
            var result = await ws.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);

            switch (result.MessageType) {
                case WebSocketMessageType.Text:
                    var message = Encoding.UTF8.GetString(buffer, 0, result.Count);
                    var jsonMessage = JsonSerializer.Deserialize<JsonElement>(message);

                    ProcessTextMessage(jsonMessage, ref taskStarted, ref taskFinished);
                    break;
                case WebSocketMessageType.Binary:
                    if (taskStarted) {
                        await fs.WriteAsync(buffer, 0, result.Count);
                        Console.WriteLine("Audio data received.");
                    }
                    break;
                case WebSocketMessageType.Close:
                    Console.WriteLine("Server closed the connection.");
                    taskFinished = true;
                    break;
            }
        }
        fs.Close();
    }

    private static void ProcessTextMessage(JsonElement jsonMessage, ref bool taskStarted, ref bool taskFinished) {
        if (jsonMessage.TryGetProperty("header", out JsonElement header) && header.TryGetProperty("event", out JsonElement eventToken)) {
            var eventType = eventToken.GetString();
            switch (eventType) {
                case "task-started":
                    taskStarted = true;
                    Console.WriteLine("Task started.");
                    break;
                case "result-generated":
                    // To retrieve additional messages, add the corresponding code here
                    break;
                case "task-finished":
                    taskFinished = true;
                    Console.WriteLine("Task completed.");
                    break;
                case "task-failed":
                    taskFinished = true;
                    Console.WriteLine("Task failed.");
                    break;
            }
        }
    }
}

PHP

The sample code directory structure:

my-php-project/

├── composer.json

├── vendor/

└── index.php

composer.json contents (adjust dependency versions as needed):

{
    "require": {
        "react/event-loop": "^1.3",
        "react/socket": "^1.11",
        "react/stream": "^1.2",
        "react/http": "^1.1",
        "ratchet/pawl": "^0.4"
    },
    "autoload": {
        "psr-4": {
            "App\\": "src/"
        }
    }
}

index.php contents:

<?php

require 'vendor/autoload.php';

use Ratchet\Client\Connector;
use React\EventLoop\Loop;
use React\Socket\Connector as SocketConnector;

# If you have not configured the API Key as an environment variable, replace the next line with: $api_key="your_api_key". It is not recommended to hardcode the API Key directly in production code to reduce the risk of API Key leakage.
$api_key = getenv("DASHSCOPE_API_KEY");
$websocket_url = 'wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference'; // WebSocket server URL
$output_file = 'output.mp3'; // Output file path

$loop = Loop::get();

if (file_exists($output_file)) {
    // Clear file content
    file_put_contents($output_file, '');
    echo "File cleared\n";
}

// Create a custom connector
$socketConnector = new SocketConnector($loop, [
    'tcp' => [
        'bindto' => '0.0.0.0:0',
    ],
    'tls' => [
        'verify_peer' => true,
        'verify_peer_name' => true,
    ],
]);

$connector = new Connector($loop, $socketConnector);

$headers = [
    'Authorization' => 'bearer ' . $api_key,
    'X-DashScope-DataInspection' => 'enable'
];

// Connect to WebSocket service
$connector($websocket_url, [], $headers)
    ->then(function ($conn) use ($output_file) {
        echo "Connected\n";

        // Receive WebSocket messages asynchronously
        $conn->on('message', function ($msg) use ($conn, $output_file) {
            if ($msg->isBinary()) {
                // Write binary data to local file
                file_put_contents($output_file, $msg->getPayload(), FILE_APPEND);
                echo "Binary data written to file\n";
            } else {
                $data = json_decode($msg, true);
                switch ($data['header']['event']) {
                    case 'task-started':
                        echo "Task started\n";
                        break;
                    case 'result-generated':
                        // Add code here to retrieve additional messages if needed
                        break;
                    case 'task-finished':
                        echo "Task completed\n";
                        $conn->close();
                        break;
                    case 'task-failed':
                        echo "Task failed: " . $data['header']['error_message'] . "\n";
                        $conn->close();
                        break;
                    default:
                        echo "Unknown event: " . $msg . "\n";
                }
            }
        });

        // Listen for connection close
        $conn->on('close', function($code = null, $reason = null) {
            echo "Connection closed\n";
            if ($code !== null) {
                echo "Close code: " . $code . "\n";
            }
            if ($reason !== null) {
                echo "Close reason: " . $reason . "\n";
            }
        });

        // Send run-task command
        $conn->send(json_encode([
            'header' => [
                'action' => 'run-task',
                'task_id' => bin2hex(random_bytes(16)),
                'streaming' => 'out'
            ],
            'payload' => [
                'model' => 'sambert-zhichu-v1',
                'task_group' => 'audio',
                'task' => 'tts',
                'function' => 'SpeechSynthesizer',
                'input' => [
                    'text' => 'Before my bed, moonlight shines bright,I suspect it is frost upon the ground.I raise my eyes to gaze at the bright moon,then bow my head, thinking of home.'
                ],
                'parameters' => [
                    'text_type' => 'PlainText',
                    'format' => 'mp3',
                    'sample_rate' => 16000,
                    'volume' => 50,
                    'rate' => 1,
                    'pitch' => 1,
                    'word_timestamp_enabled' => true,
                    'phoneme_timestamp_enabled' => true
                ]
            ]
        ]));
        echo "run-task command sent\n";
    }, function (Exception $e) {
        echo "Connection failed: {$e->getMessage()}\n";
        file_put_contents('error.log', $e->getMessage() . "\n", FILE_APPEND);
    });

$loop->run();

Node.js

Install the required dependencies:

npm install ws
npm install uuid

Sample code:

const WebSocket = require('ws');
const fs = require('fs');
const { v4: uuidv4 } = require('uuid');

// If you have not configured the API Key as an environment variable, replace the next line with: apiKey = 'your_api_key'. It is not recommended to hardcode the API Key directly in production code to reduce the risk of API Key leakage.
const apiKey = process.env.DASHSCOPE_API_KEY;
const wsUrl = 'wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference'; // WebSocket server URL
const outputFilePath = 'output.mp3'; // Replace with your audio file path

async function main() {
  await checkAndClearOutputFile(outputFilePath);
  createWebSocketConnection();
}

const fileStream = fs.createWriteStream(outputFilePath, { flags: 'a' });
function createWebSocketConnection() {
  const ws = new WebSocket(wsUrl, {
    headers: {
      Authorization: `bearer ${apiKey}`,
      'X-DashScope-DataInspection': 'enable'
    }
  });

  ws.on('open', () => {
    console.log('Connected to the WebSocket server');
    sendRunTaskMessage(ws);
  });

  ws.on('message', (data, isBinary) => handleWebSocketMessage(data, isBinary, ws));
  ws.on('error', (error) => console.error('WebSocket error:', error));
  ws.on('close', () => console.log('WebSocket connection closed'));

  return ws;
}

function sendRunTaskMessage(ws) {
  const taskId = uuidv4();
  const runTaskMessage = {
    header: {
      action: 'run-task',
      task_id: taskId,
      streaming: 'out'
    },
    payload: {
      model: 'sambert-zhichu-v1',
      task_group: 'audio',
      task: 'tts',
      function: 'SpeechSynthesizer',
      input: {
        text: 'The white sun descends behind the mountains, the Yellow River flows into the sea. To see a thousand li further, climb yet one more floor.'
      },
      parameters: {
        text_type: 'PlainText',
        format: 'mp3',
        sample_rate: 16000,
        volume: 50,
        rate: 1,
        pitch: 1,
        word_timestamp_enabled: true,
        phoneme_timestamp_enabled: true
      }
    }
  };
  ws.send(JSON.stringify(runTaskMessage));
  console.log('run-task command sent');
}

function handleWebSocketMessage(data, isBinary, ws) {
  if (isBinary) {
    fileStream.write(data);
  } else {
    const message = JSON.parse(data);
    handleWebSocketEvent(message, ws);
  }
}

function handleWebSocketEvent(message, ws) {
  switch (message.header.event) {
    case 'task-started':
      console.log('Task started');
      break;
    case 'result-generated':
      console.log('Result generated');
      break;
    case 'task-finished':
      console.log('Task completed');
      ws.close();
      fileStream.end(() => {
        console.log('File stream closed');
      });
      break;
    case 'task-failed':
      console.error('Task failed: ', message.header.error_message);
      ws.close();
      fileStream.end(() => {
        console.log('File stream closed');
      });
      break;
    default:
      console.log('Unknown event: ', message.header.event);
  }
}

function checkAndClearOutputFile(filePath) {
  return new Promise((resolve, reject) => {
    fs.access(filePath, fs.F_OK, (err) => {
      if (!err) {
        fs.truncate(filePath, 0, (truncateErr) => {
          if (truncateErr) return reject(truncateErr);
          console.log('File cleared');
          resolve();
        });
      } else {
        fs.open(filePath, 'w', (openErr) => {
          if (openErr) return reject(openErr);
          console.log('File created');
          resolve();
        });
      }
    });
  });
}

main().catch(console.error);