Data Management

As time series data accumulates, unmanaged storage grows unbounded and query performance degrades. LindormTSDB provides write, delete, and retention policy operations — and a downsampling workflow — so you can control what data is stored, for how long, and at what resolution.

LindormTSDB stores time series data as a continuous, append-only stream. Three management tasks arise as data accumulates:

• Writing data: ingest new measurements into a database

• Deleting data: remove specific data points or entire measurements on demand

• Managing retention: automatically expire data outside a configured time window

For high-frequency workloads that accumulate large volumes of raw data, downsampling lets you aggregate historical data at a coarser granularity and discard the raw points — reducing storage cost without losing analytical value.

The following table maps each operation category to the relevant commands:

---

Write data

Writes are always appended to the time series store — there is no in-place update. Each write operation specifies:

• A database name

• A measurement name

• One or more tag key-value pairs (indexed dimensions)

• One or more field key-value pairs (the metric values)

• An optional timestamp

If you omit the timestamp, LindormTSDB assigns the server time at ingestion. For high-throughput workloads, batch multiple points in a single write request to reduce round-trip overhead.

---

Delete data

LindormTSDB distinguishes between targeted deletion (removing rows that match a predicate) and structural deletion (removing an entire object).

Targeted deletion

Use DELETE to remove data points that match a tag filter and time range:

DELETE FROM "cpu_usage"
WHERE "host" = 'web-01' AND time < now() - 7d

This removes all cpu_usage data points for web-01 older than seven days, leaving all other data intact.

Structural deletion

Use DROP commands to remove entire objects:

DROP operations are irreversible. Verify the target before executing.

---

Configure retention policies

A retention policy (RP) defines how long LindormTSDB keeps data in a database. When data falls outside the retention window, LindormTSDB removes it automatically — no manual DELETE is required.

How retention enforcement works

LindormTSDB stores data in shard groups, where each shard group covers a fixed time range. The engine checks shard groups — not individual rows — against the retention policy. A shard group is removed only after its entire time range falls outside the retention window.

This means data may persist for up to one shard group duration beyond your configured retention period. For example, if your retention policy is 7d and your shard group duration is 1d, data from day 7 may remain accessible until day 8.

Create a retention policy

CREATE RETENTION POLICY "one_week" ON "my_database"
  DURATION 7d
  REPLICATION 1
  DEFAULT

Update a retention policy

ALTER RETENTION POLICY "one_week" ON "my_database"
  DURATION 14d

Verify the policy

SHOW RETENTION POLICIES ON "my_database"

Example output:

name       duration  shardGroupDuration  replicaN  default
one_week   336h0m0s  24h0m0s             1         true

The shardGroupDuration field shows the time range each shard group covers — the maximum extra time data may persist after the retention window expires.

---

Downsample and retain data

High-frequency writes produce exact, high-resolution data that is valuable in the short term but expensive to keep long-term. Downsampling resolves this by:

1. Aggregating raw data into coarser time buckets (for example, 30-minute averages instead of per-second samples)

2. Writing the aggregated data to a separate database or measurement with a longer retention period

3. Allowing the raw data to expire under its short retention policy

This workflow combines two retention policies with a continuous query (CQ) to move data from hot (short-term, high-resolution) storage to cold (long-term, low-resolution) storage automatically.

Example: archive hourly averages from 2-hour raw data

Step 1 — Create the short-term retention policy for raw data:

CREATE RETENTION POLICY "two_hours" ON "sensor_data"
  DURATION 2h
  REPLICATION 1
  DEFAULT

Step 2 — Create the long-term retention policy for aggregated data:

CREATE RETENTION POLICY "one_year" ON "sensor_data"
  DURATION 52w
  REPLICATION 1

Step 3 — Create the continuous query:

CREATE CONTINUOUS QUERY "cq_30m" ON "sensor_data" BEGIN
  SELECT mean("temperature") AS "mean_temperature",
         mean("humidity")    AS "mean_humidity"
  INTO "one_year"."downsampled_sensors"
  FROM "raw_sensors"
  GROUP BY time(30m), *
END

This CQ runs every 30 minutes. It reads raw data from raw_sensors (under the two_hours RP) and writes 30-minute averages into downsampled_sensors under the one_year RP.

Step 4 — Verify the continuous query is active:

SHOW CONTINUOUS QUERIES

Example output:

name    query
cq_30m  CREATE CONTINUOUS QUERY cq_30m ON sensor_data BEGIN SELECT mean(temperature) ...

Once the CQ is running, raw data expires automatically after 2 hours. Aggregated data is available for up to one year.

---

Best practices

Align shard group duration with your retention window

LindormTSDB enforces retention at the shard group level. If you set a 24-hour retention policy, use a 1-hour or shorter shard group duration to minimize the gap between your intended and actual expiry time.

A common pairing:

Use retention policies instead of manual DELETE for routine cleanup

DELETE acquires a lock and must scan matching rows. For time-based expiry at scale, a retention policy is more efficient: LindormTSDB drops entire shard groups in one operation with no row-by-row scan.

Use DELETE only when you need to remove data that does not align with a time boundary — for example, data from a specific host that was decommissioned.

Pair downsampling with explicit retention policies

A continuous query without a short-term retention policy on the source data results in both raw and aggregated data accumulating indefinitely. Always configure a short retention policy on the source measurement so raw data expires after the CQ has processed it.

Back up before dropping

DROP DATABASE and DROP MEASUREMENT are immediate and irreversible. Take a snapshot or export the data before executing structural deletes in production.

---

What's next

• Configure retention policies for your databases

• Set up continuous queries for long-term data archiving

• Monitor shard group sizes and retention enforcement in the LindormTSDB console