LogisterDocumentationArchive storage
Data retention / Archive storage

Prepare archive storage before users require archives before deletion.

Users see archive controls in Project > Settings > Data. This page explains what has to be ready before those controls can safely preserve old telemetry outside the normal product views.

What users see

Archive writes only help when there is somewhere to put them.

In the Data tab, Archive retained data means Logister may write gzip JSONL archive files. Require archive before deletion means old matching data should not be removed until its archive export succeeds. Configure storage before enabling that second option on important projects.

Do not treat archives as full backups

Archives contain old telemetry rows. They do not replace PostgreSQL backups for users, projects, memberships, API keys, settings, and workflow history.

Configuration

Use S3-compatible storage for durable archives.

shell
ACTIVE_STORAGE_SERVICE=amazon
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION=us-east-1
AWS_S3_BUCKET=
LOGISTER_ARCHIVE_PREFIX=telemetry
# Optional:
# LOGISTER_ARCHIVE_STORAGE_SERVICE=amazon
# AWS_S3_ENDPOINT=
# AWS_S3_FORCE_PATH_STYLE=true

What an archive contains

Archives are compressed JSONL files.

Example key
telemetry/ingest_events/project=PROJECT_UUID/year=2026/month=06/day=20/20260620T180000Z-1000-1999.jsonl.gz

Each line includes enough metadata to identify the export and the original telemetry row. Archives are useful for later inspection or migration, but they are not meant to be browsed from the normal Logister UI.

Verify

Write and read one archive before requiring archives broadly.

  1. Configure storage on both the web and worker processes.
  2. Restart the processes so both read the same storage settings.
  3. Run one controlled archive command from the jobs page.
  4. Confirm Archive Center shows a successful run in Catalog.
  5. Open the bucket and confirm the expected .jsonl.gz object exists under the configured prefix.

Troubleshooting

Archive failures usually mean storage configuration differs between processes.

SymptomWhat to check
Archive run fails immediatelyCheck bucket, credentials, region, endpoint, and path-style settings.
Web shows settings but worker cannot writeConfirm the worker process has the same archive environment variables as the web process.
Require archive before deletion never prunesCleanup waits for a successful archive when deletion protection is enabled. Fix the archive failure first.
Object exists but cannot be inspectedConfirm it is a compressed JSONL file and that the operator has permission to download from the bucket.

Use jobs and verification for manual commands and the failure visibility path.