Manage Logstores

更新时间:
复制 MD 格式

A Logstore is a storage unit in Simple Log Service (SLS) used to collect, store, and query log data.

Core concepts

What is a Logstore?

A Logstore is a data container in Simple Log Service. You can create multiple Logstores under a Project to isolate and manage logs from different applications or sources.

Additionally, certain Alibaba Cloud services or SLS features automatically create dedicated Logstores for specific purposes. These Logstores cannot accept other types of data. Examples include the following:

LogStore specification comparison

Simple Log Service offers two Logstore types—Standard and Query—that differ in features and cost.

Type

Cost (Index Traffic Fee Comparison)

Scenarios

Standard

CNY 0.350/GB

Use this type for interactive analytics, real-time monitoring, visualization, or building observability systems.

Query type (Query)

CNY 0.1/GB

This type does not support analytics. Use it for log archival, audit log storage, or troubleshooting scenarios where you only need fast keyword search without analysis. Typical use cases include long-term retention (months or years) with infrequent access.

Scope and permissions

Permission notes (expandable)

  • If you log on using an Alibaba Cloud account, you have full permissions by default and can manage Logstores directly.

  • To log on as a Resource Access Management (RAM) user, you can request the following two Simple Log Service system policies from the Alibaba Cloud account owner as needed.

    • AliyunLogFullAccess: grants full management permissions for Simple Log Service.

    • AliyunLogReadOnlyAccess: grants read-only access to Simple Log Service.

    If system policies do not meet your needs, create a custom policy for fine-grained permission management.

    Operation

    Required permissions

    Manage Logstore

    • log:ListProject

    • log:GetAcceleration

    • log:ListDomains

    • log:GetLogging

    • log:ListTagResources

    • log:GetProject

    • log:ListLogStores

    • log:*LogStore

    • log:*Index

    • log:ListShards

    • log:GetLogStoreHistogram

    • log:GetLogStoreContextLogs

    Query Logstore

    • log:ListProject

    • log:GetAcceleration

    • log:ListDomains

    • log:GetLogging

    • log:ListTagResources

    • log:GetProject

    • log:ListLogStores

    • log:GetLogStore

    • log:GetLogStoreHistogram

    • log:GetIndex

    • log:CreateIndex

    • log:UpdateIndex

    • log:ListShards

    • log:GetLogStoreContextLogs

Create a basic Logstore

Console

  1. Log on to the Simple Log Service console. In the Project list, click the target Project.

  2. On the Log Storage > Logstores tab, click the + icon.

  3. On the Create Logstore page, configure the settings and click OK.

    1. Logstore type: Standard by default.

    2. Billing mode:

      • Pay-by-feature (cannot be changed): You are billed separately for each resource used (such as storage, indexing, and read/write operations) and receive a monthly free quota, which helps control costs in small-scale scenarios.

      • Pay-by-ingested-data: You pay only for the raw ingested data volume. Storage and core features are free for 30 days, offering a simpler and more cost-effective pricing model.

      Quick tip: The closer your retention period is to 30 days and the more fields you index (especially full-text indexing), the more suitable pay-by-ingested-data becomes.
    3. Logstore name: Must be unique within the Project and cannot be changed after creation.

    4. Data retention period: 30 days by default.

    5. Keep all other settings at their default values.

    Full Logstore parameter list (expandable)

    Parameter

    Description

    Logstore type

    SLS Logstores support Standard and Query types. Choose based on your scenario to optimize costs.

    • Standard includes SLS’s full suite of analytics features, ideal for real-time monitoring, interactive analysis, and building observability systems.

    • Query has an index traffic price that is 29% of Standard. For the same budget, Query lets you index more fields at lower cost. However, Query supports only keyword search and does not support statistical analysis.

    Billing mode

    • Pay-by-feature: The original SLS billing model. You pay for actual usage of resources (such as storage and indexing) and features (such as data transformation and data shipping) on a pay-as-you-go basis.

    • Pay-by-ingested-data: A simplified SLS billing model that charges only for the raw ingested data volume. After ingestion, you get 30 days of free storage and free access to features like data transformation and data shipping. This model is simple, predictable, and cost-effective for deep SLS usage.

    Logstore name

    The Logstore name must be globally unique within the Project and cannot be modified after creation.

    WebTracking

    WebTracking enables quick collection of browser, iOS/Android app, and frontend access logs. Disabled by default.

    Data retention period

    Retention duration in days. Valid values: 1–3650. Setting this to 3650 means permanent retention. Logs are deleted once they reach the configured retention period.

    Intelligent tiering

    Automatically tiers data using lifecycle management.

    • Hot storage:

      • Hot storage is a scalable, highly available solution for frequently accessed data.

      • It supports real-time access and delivers high-performance log query and analysis, ideal for frequent query and analysis scenarios.

    • IA storage class

      • IA storage class (formerly cold storage) reduces long-term storage costs while preserving full capabilities for querying, analysis, visualization, alerting, shipping, and transformation.

      • It suits low-frequency query and analysis scenarios, such as issue backtracking.

    • Archive Storage

      • Archive Storage provides a lower-cost, queryable, and analyzable long-term storage option beyond hot and IA storage.

      • It is ideal for long-term audit log retention.

    Number of shards

    Each shard supports 5 MB/s of write throughput and 10 MB/s of read throughput. If data traffic exceeds a shard's capacity, you can split the shard. If data traffic falls below a shard's maximum read and write capacity, you can merge shards to save costs.

    Automatic sharding

    If write traffic exceeds the read/write capacity of existing shards for more than 5 minutes, enabling automatic sharding increases the number of shards based on data volume.

    Maximum shards

    With automatic sharding enabled, up to 256 shards can be created automatically.

    Record public IP

    After receiving logs, the system automatically adds the client’s public IP address and the log arrival time.

API

Create a Logstore

Modify Logstore configuration

You can configure the following parameters during creation. This section uses Logstore modification as an example.

  1. Click image Log Storage. On the Logstores page, hover over the target Logstore and select Modify.

  2. In the Logstore properties, modify the relevant settings based on your scenario.

Delete specific logs or set log retention period

Console

In Basic Information, click Modify, adjust the data retention period, and click Save.

Simple Log Service does not support deleting specific log entries. You can only delete logs by time using the retention period setting or by stopping billing or deleting the Logstore to remove all logs.
  • Specified Days: Valid values: 1–3650. Setting this to 3650 means permanent retention. Logs are deleted when the retention period ends.

  • Permanent Storage: Retains logs in the Logstore indefinitely.

Note

Changes take effect immediately, but deleting expired data may take some time.

API

Adjust the log retention period by updating the ttl value in Update Logstore.

Optimize storage costs using tiering

Console

  1. In Basic Information, click Modify and enable intelligent tiering.

  2. Configure the Storage Policy: The sum of days across all three tiers must equal the total data retention period.

    • Hot storage: minimum 7 days.

    • IA storage class: minimum 30 days.

    • Archive Storage: minimum 60 days.

    Set Data retention period to Limited days, enable Intelligent tiering, and configure tier transitions in Storage policy: hot storage automatically transitions to IA storage after its period ends, IA storage transitions to Archive Storage, and Archive Storage is automatically deleted after its period ends.

  3. Click Save. For details, see Intelligent tiering.

API

Dynamically adjust the retention policy for storage tiers by updating the ttl, hot_ttl, and infrequentAccessTTL values in Update Logstore.

Collect frontend logs

Simple Log Service provides WebTracking to collect logs from mini programs, mobile apps (iOS/Android), and browsers.

You can use this feature in two ways:

  • Use STS authentication for transmission. This method is suitable for production environments and does not require Logstore configuration changes.

  • Use anonymous transmission via OpenAPI. This method is for testing only and requires enabling a setting in the Logstore, as described below.

Console

In Basic Properties, click Modify, enable WebTracking, and click Save.

API

Enable WebTracking by setting the enable_tracking parameter to true in Update Logstore.

Add public IP and log arrival time automatically

When enabled, subsequent log ingestion automatically adds:

  • __tag__:__client_ip__: the public IP address of the log source device.

  • __tag__:__receive_time__: the time the log arrives at the Simple Log Service server, formatted as a UNIX timestamp (seconds since 1970-01-01 00:00:00 UTC).

Console

In Basic Properties, click Modify, enable Record public IP, and click Save.

API

Enable public IP recording by setting the appendMeta parameter in Update Logstore.

Adjust ingestion performance using shards

Each shard supports 5 MB/s or 500 writes per second for ingestion, and 10 MB/s or 100 reads per second for consumption. These are soft limits. The system tries to serve requests beyond these limits but does not guarantee quality of service. If traffic exceeds a shard’s capacity, split the shard to increase capacity.

Console

In Basic Properties, click Modify, enable Automatic sharding, set the shard limit, and click Save.

Simple Log Service supports splitting and merging individual shards.

API

Split shard.

Merge shard.

Stop billing or delete Logstore

Warning

Once deleted, all log data in the Logstore is permanently lost and cannot be recovered. Proceed with caution.

Console

  1. Clean up before deletion.

    1. Delete all associated Logtail configurations before deleting the Logstore.

    2. If LogShipper is enabled for this Logstore, stop writing new data and confirm all existing data has been successfully shipped before deletion.

    3. In the Job Management section, review all tasks for the current Project and delete any tasks associated with this Logstore.

  2. Deletion steps.

    1. On the Log Storage > Logstores tab, hover over the target Logstore and select Delete.

    2. In the Warning dialog box, click Confirm Deletion.

  3. Post-deletion notes.

    1. Storage fees still apply on the day of deletion. No fees accrue from the next day onward. You will not receive a bill for this Logstore starting on the third day after deletion.

    2. After deletion, all export tasks, data transformation jobs, scheduled SQL tasks using this Logstore as a source, and import tasks targeting this Logstore are deleted.

API

Delete Logstore

Configuration examples for real-world scenarios

Real-time monitoring and analysis for high-volume workloads

Your online application generates large volumes of logs in real time. When failures occur, you need to quickly locate error logs and monitor performance metrics (such as QPS and response latency) with real-time alerts.

Recommended configuration: Standard Logstore + pay-by-ingested-data + automatic sharding.

Reason: Standard Logstore supports analytics, real-time monitoring, and visualization. With high ingestion volume and potential need for extensive indexing, pay-by-ingested-data offers better cost efficiency, and automatic sharding ensures consistent ingestion and analysis performance.

Compliance, auditing, and regulatory scenarios

Industry regulations require storing user operation logs and security logs for more than six months for auditing, but daily query and analysis frequency is very low.

Recommended configuration: Query Logstore + intelligent tiering.

Reason: Query Logstore supports search-only workloads at a lower index traffic cost than Standard. Intelligent tiering reduces long-term storage costs by automatically moving older logs to lower-cost storage tiers.

Related references

Logstore comparison under pay-by-feature billing

Query Logstores support only pay-by-feature billing. Under this model, Standard and Query Logstores compare as follows:

Comparison Item

Standard

Query Specification

Cost

Index traffic

CNY 0.350/GB

CNY 0.1/GB

Features

Data ingestion (for application logs only)

Supported

Does not support cloud product log ingestion.

Enable intelligent tiering

Supported

Supported

Search

Supported

Supported

Analytics (SQL statement)

Supported

Not supported

Contextual query

Supported

Supported

LiveTail

Supported

Supported

LogReduce

Supported

Not supported

Reindex

Supported

Supported

Dashboard

Supported

Not supported

Alerting

Supported

Supports only query-based alerts

Scheduled SQL

Supported

Not supported

Data transformation

Supported

Supported

Data shipping

Supported

Supported

Standard consumption

Supported

Supported

Limits

Pay-by-ingested-data supports the full SLS feature set. Value-added features such as query and analysis, data transformation, intelligent alerting, and consumption/shipping incur no extra charges but are subject to quota limits as follows.

Quota limit

Description

Data transformation volume

Maximum 100 TB per Logstore per month.

Scheduled SQL data volume

Maximum 20 TB per Logstore per month.

Shipping data volume

Maximum 100 TB per Logstore per month.

Consumption data volume

Maximum 100 TB per Logstore per month.

Alert job computation volume

Maximum 100 TB per Logstore per month.

Billing overview

Logstore costs depend primarily on the selected billing mode.

  • Pay-by-feature: You are billed separately for actual usage of resources such as storage capacity, index traffic, read/write operations, and shard count.

  • Pay-by-ingested-data: You pay only for the raw ingested data volume, with 30 days of free storage and multiple free features included.

Key pricing details:

  • Standard index traffic: CNY 0.350 /GB.

  • Query index traffic: CNY 0.1 /GB.

Cost optimization tips:

  • If your log retention period is close to or exceeds 30 days, pay-by-ingested-data is usually more cost-effective.

  • For archival and retrieval-only scenarios, use Query Logstores to reduce indexing costs.

  • Use intelligent tiering to move infrequently accessed data to lower-cost storage tiers.

FAQ

Cannot create a Logstore

By default, you can create up to 200 Logstores per Project. Delete unused Logstores or request a quota increase as follows.

  1. Log on to the Simple Log Service console, and click the target Project in the Project list.

  2. On the Project page, go to Overview > Basic Information > Resource Quota and click Manage. In the Resource Quotas panel, adjust the Logstore quota limit and click Save to submit your request. Allow about one hour for the change to take effect.

Are logs missing in Simple Log Service?

  • Project or Logstore missing

    If you manually delete a Project or Logstore, logs cannot be recovered. Use ActionTrail to check for deletion events within the last 90 days.

  • No logs were collected. For more information, see Troubleshooting Guide for LoongCollector Log Collection Issues.

  • If your account remains overdue for more than 7 days, the service is considered abandoned. Projects are reclaimed and data is permanently deleted. For details, see Overdue payments.

How can I reduce log storage costs?