Integrate the basic voice cloning web SDK

Updated at:

Integrate the voice cloning service into your front-end web pages by using the basic web SDK provided by Intelligent Production.

Usage

The following procedure applies to version 0.0.2 of the basic voice cloning web SDK.

The voice cloning process consists of four steps: 1) Environment check: Grant microphone access and ensure a quiet environment. 2) Record 20 high-quality audio clips: Use your phone's built-in microphone, keeping it about 10 cm away, and speaking clearly and expressively. 3) Wait for voice training: The cloning process takes about 30 minutes to complete. 4) Audition and use: Review the results on the Customized Voice page in the Alibaba Cloud console. After agreeing to the Customized Voice Service Agreement, click Start cloning.

Procedure

  1. Import the voice cloning web SDK in the <head> tag of your front-end page.

    <head>
       <script src="https://g.alicdn.com/video-cloud-fe/tts-factory/0.0.2/index.js"></script>
    </head>

    Add a <div> node in the <body> tag to serve as the component's mount point.

    <body>
      <div id="app" ></div> // You can change the height of the container as needed.
    </body>
  2. Initialize the web SDK.

    • API definition:

      • InitConfig

        interface InitConfig {
          container: HTMLElement;
          api: {
            // Create a voiceId.
            createVoiceId: (params: { voiceId?: string, voiceName: string, gender: string, scenario: string }) => Promise<Result<{ voiceId: string }>>;
            // Get the list of recording prompts.
            getRecorderList: (params: { sex: string, scenario: string }) => Promise<Result>;
            // Upload a recorded audio clip.
            uploadRecord: (params: any) => Promise<Result<{fileUrl: string}>>;
            // Validate a recorded audio clip.
            validRecord: (params: any) => Promise<Result<{ reason: string; pass: boolean }>>;
            // Submit the recorded audio clips.
            submitRecord: (params: any) => Promise<Result>;
            // Customize UI text.
            getCopywriting?:(config: any)=>T;
          }
        }
      • AliyunTTSFactory class

        class AliyunTTSFactory {
          destroy(): void
        }
    • Sample code

      Sample code

        const ttsFactory = window.AliyunTTSFactory.init({
            container: container,
            api: {
              createVoiceId: async (params) => {
                const res = await request('CreateCustomizedVoiceJob', {
                  VoiceName: params.voiceName,
                  VoiceId: params.voiceId,
                  Gender: params.gender,
                  Scenario: params.scenario,
                });
                return {
                  success: get(res, 'data.Success', false),
                  data: {
                    voiceId: get(res, 'data.Data.VoiceId', ''),
                  },
                };
              },
              getRecorderList: async (params) => {
                const res = await requestGet('GetDemonstrationForCustomizedVoiceJob', {
                  Scenario: params.scenario,
                });
                return {
                  success: get(res, 'data.Success', false),
                  data: get(res, 'data.Data.DemonstrationList', []).map((item) => {
                    return {
                      audioId: item.AudioId,
                      audioUrl: item.DemoAudio,
                      text: item.Text,
                    };
                  }),
                };
              },
              uploadRecord: async (params) => {
                const fileName = `${params.voiceId}-${
                      params.index
                    }-${Date.now()}.wav`;
                const result = await uploadFile(fileName,params.file);
                console.log('uploadRecord', params,fileName,result);
                return { success: true, data: { fileUrl: result.url } }
              },
              validRecord: async (params) => {
                const res = await request('DetectAudioForCustomizedVoiceJob', {
                  VoiceId: params.voiceId,
                  RecordUrl: params.uploadData.fileUrl,
                  AudioRecordId: params.index + 1,
                });
                return {
                  success:  get(res, 'data.Success', false),
                    data: {
                      pass:  get(res, 'data.Data.Pass', false),
                      reason: get(res, 'data.Data.Reason', ''),
                    },
                };
              },
              submitRecord: async (params) => {
                const res = await request('SubmitCustomizedVoiceJob', {
                  VoiceId: params.voiceId,
                  // DemoAudioMediaURL: '', // The URL of the demo audio generated by voice cloning.
                });
                return {
                  success: get(res, 'data.Success', false),
                };
              },
              getCopywriting:  (config)=>{
                // Example of how to customize UI text.
                console.log('>>config', config);
                config.views.Tutorial.startBtn = 'Start cloning'
                return config;
              }
            },
          });
  3. Call the destroy() method to release the instance. After the instance is destroyed, the container becomes an empty element.

    ttsFactory.destroy();

Sample code

Sample code

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <link rel="icon" type="image/svg+xml" href="/vite.svg" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.21/lodash.min.js"></script>
  <script src="http://gosspublic.alicdn.com/aliyun-oss-sdk-6.19.0.min.js"></script>
  <script src="https://g.alicdn.com/video-cloud-fe/tts-factory/0.0.2/index.js"></script>
  <title>ALI TTS DEMO</title>
</head>
<body>
  <div id="app"></div>
  <script>
    async function request(action, params) {
      const res = await fetch('/openApiPost', {
        method: 'POST', body: JSON.stringify({...params, Action: action}), headers: {
          'Content-Type': 'application/json; charset=utf-8'
        }
      })
      const data = await res.json();
      return {data};
    }
    async function requestGet(action, params) {
      const searchParams = new URLSearchParams();
      searchParams.set('Action', action);
      Object.keys(params).forEach((key) => {
        searchParams.set(key, params[key]);
      });
      const res = await fetch(`/openApi?${ searchParams.toString() }`, {method: 'GET'})
      const data = await res.json();
      return {data};
    }
    const refreshSts = async () => {
      // Create temporary access credentials by using STS to upload files to Object Storage Service (OSS). For more information, see https://www.alibabacloud.com/help/en/oss/developer-reference/authorize-access-6
      const res = await fetch("/createSTS", {method: 'POST'});
      const data = await res.json();
      return data.data;
    }
    const uploadFile = async (fileName,file) => {
      // Upload objects to OSS. For more information, see https://www.alibabacloud.com/help/en/oss/developer-reference/simple-upload-8
      const sts = await refreshSts();
      const client = new OSS({
        // Set yourRegion to the region where your bucket is located. For example, if the bucket is in the China (Hangzhou) region, set yourRegion to oss-cn-hangzhou.
        region: `oss-${ sts.region }`,
        accessKeyId: sts.accessKeyId,
        accessKeySecret: sts.accessKeySecret,
        stsToken: sts.securityToken,
        // Specify the bucket name.
        bucket: sts.bucket,
        // Refresh the temporary access credentials.
        refreshSTSToken: async () => {
          const refreshToken = await refreshSts();
          return {
            accessKeyId: refreshToken.accessKeyId,
            accessKeySecret: refreshToken.accessKeySecret,
            stsToken: refreshToken.securityToken,
          };
        },
      });
      const result = await client.multipartUpload(fileName, file, {
        progress: (p, _cpt, res) => {
          console.log('res', res);
        },
        parallel: 2,
        partsize: 1024 * 512,
      });
      const url = new URL(result.res.requestUrls[0]);
      return {url: `${ url.origin }${ url.pathname }`}
    }
    const get = _.get;
    const container = document.getElementById('app');
    const ttsFactory = window.AliyunTTSFactory.init({
      container: container,
      api: {
        createVoiceId: async (params) => {
          const res = await request('CreateCustomizedVoiceJob', {
            VoiceName: params.voiceName,
            VoiceId: params.voiceId,
            Gender: params.gender,
            Scenario: params.scenario,
          });
          return {
            success: get(res, 'data.Success', false),
            data: {
              voiceId: get(res, 'data.Data.VoiceId', ''),
            },
          };
        },
        getRecorderList: async (params) => {
          const res = await requestGet('GetDemonstrationForCustomizedVoiceJob', {
            Scenario: params.scenario,
          });
          return {
            success: get(res, 'data.Success', false),
            data: get(res, 'data.Data.DemonstrationList', []).map((item) => {
              return {
                audioId: item.AudioId,
                audioUrl: item.DemoAudio,
                text: item.Text,
              };
            }),
          };
        },
        uploadRecord: async (params) => {
          const fileName = `${params.voiceId}-${
                params.index
              }-${Date.now()}.wav`;
          const result = await uploadFile(fileName,params.file);
          console.log('uploadRecord', params,fileName,result);
          return { success: true, data: { fileUrl: result.url } }
        },
        validRecord: async (params) => {
          const res = await request('DetectAudioForCustomizedVoiceJob', {
            VoiceId: params.voiceId,
            RecordUrl: params.uploadData.fileUrl,
            AudioRecordId: params.index + 1,
          });
          return {
            success:  get(res, 'data.Success', false),
              data: {
                pass:  get(res, 'data.Data.Pass', false),
                reason: get(res, 'data.Data.Reason', ''),
              },
          };
        },
        submitRecord: async (params) => {
          const res = await request('SubmitCustomizedVoiceJob', {
            VoiceId: params.voiceId,
            // DemoAudioMediaURL: '', // The URL of the demo audio generated by voice cloning.
          });
          return {
            success: get(res, 'data.Success', false),
          };
        },
        getCopywriting:  (config)=>{
          // Example of how to customize UI text.
          console.log('>>config', config);
          config.views.Tutorial.startBtn = 'Start cloning'
          return config;
        }
      },
    });
  </script>
</body>
</html>

References

To learn more about voice cloning, see Voice cloning.