Configure range origin fetch

更新时间:
复制 MD 格式

Range origin fetch enables a point of presence (POP) to include a Range header in its request to an origin server. When the origin server receives the request from the POP, it returns only the specified portion of the resource. This method improves file delivery efficiency, reduces back-to-origin traffic and the load on your origin server, and speeds up response times.

Background

The Range header in an HTTP request specifies which part of a resource to retrieve. For example, a request with the header Range: bytes=0-100 asks for the first 101 bytes of a file.

After you enable the range origin fetch feature, when a receives a user request, if the resource is not cached on the POP or has expired, the POP performs an origin fetch by using a Range request. This allows the POP to retrieve only the required part of the resource from the origin server in segments and cache it on the POP.

This diagram illustrates how range origin fetch works.

image

Usage notes

Before you enable range origin fetch, consider the following:

  • Before you enable range origin fetch, make sure your origin server supports Range requests. The origin server must be able to process the Range header and respond with a 206 Partial Content status. If your origin server does not support Range requests, enabling this feature may cause caching issues or client request failures.

  • Range origin fetch is an optional feature and is disabled by default in the console.

  • The Multipart Ranges feature is disabled by default. Enabling range origin fetch does not automatically enable Multipart Ranges. To enable this feature, submit a ticket.

  • Enabling range origin fetch increases the QPS of origin fetches, which may trigger rate limiting on your origin server. To work around this issue, call the DescribeL2VipsByDomain operation to retrieve the IP addresses of origin-pull POPs, and add the IP addresses of the origin-pull POPs to the IP address allowlist of your origin server.

Procedure

  1. Log on to the CDN console.

  2. In the left navigation pane, click Domain Names.

  3. On the Domain Names page, find the target domain name and click Manage in the Actions column.

  4. In the domain's navigation pane, click Video.

  5. In the Range Origin Fetch section, click Modify.

  6. Based on the parameter descriptions in the following table, select Do Not Enable Range Origin Fetch, Match Client, or Enable Range Origin Fetch (Recommended for Large File Delivery).

    If you select Match Client or Enable Range Origin Fetch (Recommended for Large File Delivery), you can set the shard size. The default shard size is 512 KB.

    Parameter

    Option

    Description

    Example

    Range Origin Fetch

    Do Not Enable Range Origin Fetch

    By default, Do Not Enable Range Origin Fetch is selected, which means that regardless of whether a client sends a Range request to a CDN POP, the CDN POP requests the entire file during an origin fetch, resulting in low file distribution efficiency for large files.

    For example, if a request from a client to a CDN POP contains Range: bytes=0-100, the CDN POP sends a request to the origin server without the Range parameter. The origin server sends the entire file to the CDN POP. For example, if the file is 10 MB, the origin server sends the 10 MB file to the CDN POP. The CDN POP caches the file that it receives from the origin server and then responds to the client with the content for the Range: bytes=0-100 request.

    Match Client

    After you enable Match Client, if a client sends a Range request to a CDN POP, the CDN POP performs an origin fetch by using a Range request. For the first origin fetch request, the CDN POP requests a data block from your origin server. The size of this block is determined by rounding up the range size from the client's request to the nearest integer multiple of the shard size. All subsequent origin fetch requests use the shard size that you specify.

    For example, when the shard size is 512 KB, if a client sends a request to a CDN POP that contains Range:bytes=0-614399 (which is 600 KB), and the file is not cached on the CDN POP, the first origin fetch retrieves a 1024 KB shard (600 KB rounded up to 1024 KB). For subsequent requests for other uncached shards of the file, the CDN POP accesses the origin server by using a shard size of 512 KB.

    Enable Range Origin Fetch (Recommended for Large File Delivery)

    After Enable Range Origin Fetch (Recommended for Large File Delivery) is enabled, regardless of whether a client sends a Range request to a CDN POP, CDN POPs always use Range requests for origin fetch. All Range requests sent from CDN POPs to the origin server use the shard size that you specify.

    None

    Shard Size

    • 512 KB

    • 1 MB

    • 2 MB

    • 4 MB

    You can set the shard size if you select Match Client or Enable Range Origin Fetch (Recommended for Large File Delivery). The default size is 512 KB.

    1 MB

    Rule Condition

    A rule condition determines whether a configuration applies to a request by evaluating various parameters in the request.

    Important

    When a feature references rule conditions configured in the rules engine, the execution order follows the priority of the associated rule conditions, not the order of the feature configurations.

    • Do not use conditions: Disables conditional rules.

    • You can add or edit conditional rules in Rules engine.

    A rule condition evaluates parameters in a user request to determine whether the configuration applies.

    Do not use

  7. Click OK to save the configuration.

Handling out-of-range HTTP Range requests

When CDN fetches large files from an OSS origin server, issues may occur if the OSS server responds with a cache-control:no-cache policy or if a client request to CDN triggers a Range-based origin fetch. These issues can manifest as abnormally slow downloads or even timeouts (after about 30 seconds), and a dropped connection when an out-of-range slice (such as the last slice of the file) is downloaded.

  • Behavior without a compatibility policy

    If an HTTP Range request is valid, Object Storage Service (OSS) returns a 206 response and includes the Content-Range header. If an HTTP Range request is invalid, or if the specified range is out of bounds, the Range request is ignored. OSS returns a 200 response and transfers the entire object. The following examples show invalid HTTP Range requests and explain the errors.

    Note

    The following examples assume an object size of 1000 bytes, with a valid range of 0 to 999. To prevent the specified range from being out of bounds, send a HeadObject request to get the object size before you read the range.

    • Range: byte=0-499: Invalid format. byte should be bytes.

    • Range: bytes=0-1000: The end byte, 1000, is out of the valid range.

    • Range: bytes=1000-2000: The specified range is out of the valid range.

    • Range: bytes=1000-: The start byte is out of the valid range.

    • Range: bytes=-2000: The specified range is out of the valid range.

    Use the following command to test the validity of the Range parameter:

    curl -r 0-100 http://xxxx.oss-cn-hangzhou.aliyuncs.com/xx.zip -o /tmp/xx1.zip -v
  • Behavior with a compatibility policy

    Add the x-oss-range-behavior:standard header to an HTTP Range request to change how OSS behaves when the specified range is out of bounds. The following examples show how the behavior changes:

    Note

    The following examples assume an object size of 1000 bytes, with a valid range of 0 to 999. If you use an HTTP Range request to get part of a large file and select an invalid range, OSS returns an InvalidRange error code. For more information about how to resolve this issue, see OSS returns a 416 error. The detailed error message is as follows:

    The requested range cannot be satisfied

    • Range: bytes=500-2000: The end byte is out of the valid range. OSS returns the content in the byte range of 500 to 999.

    • Range: bytes=1000-2000: The start byte is out of the valid range. OSS returns a 416 (InvalidRange) error.

    • Range: bytes=1000-: The start byte is out of the valid range. OSS returns a 416 (InvalidRange) error.

    • Range: bytes=-2000: The specified range is out of the valid range. OSS returns the content in the byte range of 0 to 999, which is the complete file.

    This topic provides the following HTTP Range request examples.

    Note

    The following examples assume an object size of 1000 bytes, with a valid range of 0 to 999.

    • Request the content of an object within the byte range of 0 to 499.

      GET /ObjectName
      Range: bytes=0-499
      Host: bucket.oss-cn-hangzhou.aliyuncs.com
      Date: Fri, 18 Oct 2019 02:51:30 GMT
      Authorization: Signature
      
      206 (Partial Content)
      content-length: 500
      content-range: bytes 0-499/1000
      connection: keep-alive
      etag: "CACF99600561A31D494569C979E6FB81"
      x-oss-request-id: 5DA928B227D52731327DE078
      date: Fri, 18 Oct 2019 02:51:30 GMT
      [500 bytes of object data]
    • Request the content of an object from byte 500 to the end of the file.

      GET /ObjectName
      Range: bytes=500-
      Host: bucket.oss-cn-hangzhou.aliyuncs.com
      Date: Fri, 18 Oct 2019 03:24:39 GMT
      Authorization: Signature
      
      206 (Partial Content)
      content-length: 500
      content-range: bytes 500-999/1000
      etag: "CACF99600561A31D494569C979E6FB81"
      x-oss-request-id: 5DA9307750EBE33332E3720A
      date: Fri, 18 Oct 2019 03:24:39 GMT
      [500 bytes of object data]
    • Request the last 500 bytes of an object.

      GET /ObjectName
      Range: bytes=-500
      Host: bucket.oss-cn-hangzhou.aliyuncs.com
      Date: Fri, 18 Oct 2019 03:23:22 GMT
      Authorization: Signature
      
      206 (Partial Content)
      content-length: 500
      content-range: bytes 500-999/1000
      etag: "CACF99600561A31D494569C979E6FB81"
      x-oss-request-id: 5DA9302A6646AC37397F7039
      date: Fri, 18 Oct 2019 03:23:22 GMT
      [500 bytes of object data]
Note
  • For large file distribution where the average file size is more than 20 MB, configure this setting for CDN and origin fetch to OSS.

  • If access authentication is enabled on your Alibaba Cloud OSS origin server and clients are responsible for signing origin fetch requests, clients must include the x-oss-range-behavior:standard request header in the signature calculation. Alibaba Cloud OSS includes all request headers with the x-oss- prefix in its signature calculation. If the client's signature calculation does not include the x-oss-range-behavior request header, a signature mismatch occurs and the request is rejected.