Process model
Run web traffic and background work as separate responsibilities.
The production app can be deployed on Fly, Kamal, Docker, or any platform that can run a Rails web process and a Sidekiq worker against PostgreSQL and Redis. After CI validates a version-changing main commit, the release workflow builds one canonical image, verifies the same digest through public ghcr.io/taimoorq/logister, docker.io/taimoorq/logister, and optional quay.io/taimoorq/logister references, deploys that digest to Fly, and checks the public runtime identity.
| Process | Purpose | Typical command |
|---|---|---|
| Web | Serves the UI, auth screens, project pages, public pages, health checks, and API ingestion endpoints. | ./bin/rails server behind Puma/Thruster or your platform web command |
| Worker | Runs asynchronous jobs such as optional ClickHouse writes, Action Mailer delivery, project notifications, digest scheduling, monitor sweeps, health sweeps, and retention/archive tasks. | bundle exec sidekiq -C config/sidekiq.yml |
| Release or migration step | Prepares the database before the new web process handles traffic. | ./bin/rails db:prepare |
Planning note
Low-volume installs can start with one web process and one worker. As traffic grows, scale the web and worker process counts separately so ingestion and background jobs do not compete with user-facing requests.
Required secrets
These are the minimum variables before the first administrator can use the interface.
| Variable | Purpose |
|---|---|
RAILS_MASTER_KEY | Decrypts the app credentials and the key used to protect encrypted installation settings. |
DATABASE_URL | Connects the Rails app to PostgreSQL for runtime data and durable onboarding state. |
DATABASE_MIGRATION_URL | Optional direct PostgreSQL URL used only by the release migration step when your provider separates runtime pooling from migration/admin connections. |
LOGISTER_SETUP_TOKEN | Long random one-time value used at /setup to create and confirm the first database-backed administrator. Remove it after the instance is claimed. |
Production also needs normal Rails platform settings such as RAILS_ENV=production, PORT when your host requires it, and HTTPS termination through your platform, load balancer, or reverse proxy. After the web process starts, use Admin → Installation or environment variables for Redis, the canonical URL, Sidekiq concurrency, and optional integrations. The onboarding and add-on guide shows both paths in the running interface.
Sample env reference
Use .env.sample as a map, then store real values in your provider.
Local development can copy .env.sample to .env. Production can set these values in Fly secrets, Kamal secrets, Docker secrets, or another secret manager, or save supported runtime settings under Admin → Installation. A nonblank environment variable always wins; the admin page identifies that variable while keeping an encrypted editable fallback for later. Do not commit real credentials to a fork of this public repository.
| Entry | What it controls | What self-hosters should do |
|---|---|---|
RAILS_ENV | Rails runtime mode. | Use development locally and production in deployed environments. Most platforms set this through their app config. |
RAILS_MAX_THREADS | Puma thread count and Rails database pool sizing. | Start with the sample value, then raise only when the database pool and host memory can support more concurrent requests. |
RAILS_LOG_LEVEL | Production logging verbosity. | Use info by default. Temporarily use debug only when investigating issues because debug logs can include sensitive request context. |
PORT | Port Puma listens on. | Set only when your platform requires a specific port. Fly uses 8080 through fly.toml; local Rails defaults to 3000. |
WEB_CONCURRENCY | Optional Puma worker process count. | Leave unset for small installs. Raise only when the host has enough memory and PostgreSQL can support the extra connections. |
RAILS_MASTER_KEY | Decrypts Rails credentials in production. | Generate or copy the master key for your deployment credentials, then store it in your provider secret manager. Never commit config/master.key. |
DATABASE_URL | Primary PostgreSQL connection for web and worker runtime traffic. | Provision PostgreSQL, then use the connection URL from the provider. For Fly, use Fly Postgres or Managed Postgres. For Prisma Postgres or pooled providers, use the pooled/runtime URL here. |
DATABASE_MIGRATION_URL | Optional direct PostgreSQL connection for release migrations. | Set this only when the provider gives separate pooled and direct URLs. Use the direct/admin URL so db:prepare can run migrations reliably. |
LOGISTER_SETUP_TOKEN | One-time first-administrator claim token. | Generate a long random value, enter it at /setup, then remove it from every web process after the instance is claimed. Logister compares it securely and never stores it. |
REDIS_URL | Rails cache and Sidekiq job queue backend. | Provision Redis or Valkey. Use redis:// locally and rediss:// when your managed Redis provider requires TLS. |
SIDEKIQ_CONCURRENCY | Number of jobs each Sidekiq worker process can run in parallel. | Start with 5. Raise only when Redis, PostgreSQL, SMTP, and ClickHouse can tolerate more concurrent background work. |
LOGISTER_PUBLIC_URL | Canonical app URL used in emails, generated links, sitemap, and metadata. | Set this after DNS and HTTPS are ready, for example https://logister.example.com. Avoid localhost in production or email links will be wrong. |
LOGISTER_DOCS_URL | Base URL for in-app documentation links and generated docs metadata. | Use https://logister.org/docs for the canonical public docs, or point to your own Cloudflare Pages docs fork. Set the same value as a GitHub Actions variable when building the static docs so cloudflare-docs/sitemap.xml and cloudflare-docs/robots.txt use your docs URL. |
LOGISTER_ADMIN_EMAILS | Optional break-glass allow-list for existing users. | The account created at /setup receives a durable database-backed admin role. Use this variable only when an existing confirmed user also needs environment-controlled admin access. |
LOGISTER_API_KEY_PREFIX | Prefix for newly generated project ingestion tokens. | Leave as logister unless your organization wants branded token prefixes. Changing it does not rotate existing tokens. |
LOGISTER_PUBLIC_API_RATE_LIMIT_REQUESTS | Accepted public API requests allowed per API token and endpoint in each window. | Defaults to 1200. Raise only when Redis and PostgreSQL can absorb the extra ingestion traffic. |
LOGISTER_PUBLIC_API_RATE_LIMIT_PERIOD_SECONDS | Public API rate-limit window length in seconds. | Defaults to 60. Keep this aligned with client retry behavior. |
LOGISTER_PUBLIC_API_AUTH_FAILURE_RATE_LIMIT_REQUESTS | Missing, invalid, revoked, or archived-project token attempts allowed per source IP in each window. | Defaults to 120. Lower it for exposed public instances that receive noisy unauthorized traffic. |
LOGISTER_CLI_READ_RATE_LIMIT_REQUESTS, LOGISTER_CLI_READ_RATE_LIMIT_PERIOD_SECONDS | Authenticated CLI requests allowed per user CLI token and fixed window. | Defaults to 600 requests per 60 seconds. Keep enough headroom for two-second follow polling and lower it only when expensive reads need tighter protection. |
LOGISTER_CLI_PRE_AUTH_RATE_LIMIT_REQUESTS | Invalid CLI bearer attempts allowed per source IP and CLI rate-limit window before token lookup. | Defaults to 1200. Cache keys contain a one-way source-IP hash, not the raw IP or submitted token. |
LOGISTER_CLI_POSTGRES_STATEMENT_TIMEOUT_MS | Request-wide PostgreSQL execution ceiling for authenticated CLI responses. | Defaults to 15000 milliseconds and accepts 100 through 120000. Invalid values use the default. Timed-out reads return a sanitized 503 query_timeout; clients should narrow the time range or filters. |
LOGISTER_GITHUB_APP_ID | Numeric GitHub App ID for source integration. | Set when self-hosted Logister should resolve private repository source excerpts, sync repositories, or create GitHub issues. See the GitHub App guide. |
LOGISTER_GITHUB_APP_PRIVATE_KEY | GitHub App private key PEM. | Store it as a secret. Multiline PEM values work when your provider supports them; escaped \n sequences also work. |
LOGISTER_GITHUB_WEBHOOK_SECRET | Secret used to verify GitHub webhook signatures. | Use the same random value in GitHub and Logister. Missing values make webhooks return 503; mismatches return 401. |
LOGISTER_GITHUB_APP_SLUG, LOGISTER_GITHUB_APP_INSTALL_URL | Install URL discovery for the GitHub App. | Set the slug so Logister can derive the install URL, or set the full install URL directly for GitHub Enterprise Server or unusual App paths. |
LOGISTER_GITHUB_API_URL, LOGISTER_GITHUB_WEB_URL, LOGISTER_GITHUB_API_VERSION | Optional GitHub endpoint and REST API version overrides. | Leave defaults for github.com. For GitHub Enterprise Server, point the web URL at the browser host and the API URL at /api/v3. |
LOGISTER_GITHUB_STATELESS_S2S_TOKEN | Optional GitHub App installation-token rollout override. | Leave unset for normal operation. Set enabled to proactively request GitHub's stateless S2S installation tokens, or disabled only as a temporary fallback while diagnosing a GitHub rollout issue. |
| Project-level public API overrides | Optional per-project rate-limit overrides stored in the application database. | Only database-backed app administrators or break-glass admins from LOGISTER_ADMIN_EMAILS can set these from project settings. Project owners and shared project members cannot change them. |
LOGISTER_API_KEY, LOGISTER_ENDPOINT, LOGISTER_DEPLOYMENT_ENDPOINT, LOGISTER_RELEASE, LOGISTER_REPOSITORY, LOGISTER_COMMIT_SHA, LOGISTER_BRANCH | Optional self-observability for the Logister app itself through logister-ruby. | Use when you want this Logister instance to report its own app errors, rejected client submissions, request spans, slow SQL metrics, ClickHouse ingest failures, digest scheduler check-ins, and source/deployment context to a Logister project. Generate the key from the destination project; using a separate project is usually cleaner than mixing operational telemetry with customer app events. After configuring it, run bin/rails logister:sample_telemetry to send one sample error, log, metric, transaction, span set, check-in, and deployment record. |
LOGISTER_CAPTURE_REQUEST_SPANS, LOGISTER_CAPTURE_DB_METRICS, LOGISTER_CAPTURE_WEB_REQUEST_TRANSACTIONS | Self-monitoring controls for request spans, slow SQL metrics, and slow Rails request transactions. | Leave enabled for production diagnosis unless event volume becomes noisy. Tune with LOGISTER_DB_METRIC_MIN_DURATION_MS, LOGISTER_DB_METRIC_SAMPLE_RATE, LOGISTER_WEB_REQUEST_MIN_DURATION_MS, and LOGISTER_WEB_REQUEST_LOG_MIN_DURATION_MS. |
LOGISTER_EMAIL_FROM | Sender address for auth mail, invitations, project alerts, digests, and operational notices. | Use a mailbox on a domain you control, verify that sender or domain with your SMTP provider, and publish SPF, DKIM, and DMARC records. |
SES_REGION | Amazon SES region used to derive the default SMTP hostname. | Choose the region where you verified your SES identity, such as us-east-1. |
SES_SMTP_USERNAME / SES_SMTP_PASSWORD | SMTP credentials for Amazon SES delivery. | Create SMTP credentials in the Amazon SES console, store them as secrets, and move the SES account out of sandbox mode before sending to arbitrary recipients. |
SES_SMTP_ADDRESS, SES_SMTP_PORT, SES_SMTP_DOMAIN | Optional SMTP endpoint overrides. | Leave unset for standard SES, or set them when using a different SMTP-compatible provider. |
SES_SMTP_ENABLE_STARTTLS_AUTO, SES_SMTP_OPEN_TIMEOUT, SES_SMTP_READ_TIMEOUT | SMTP TLS and timeout tuning. | Keep the defaults unless your SMTP provider requires different transport settings. |
SES_CONFIGURATION_SET | Optional SES delivery-metrics tag for Logister mail. | Create a configuration set in SES if you want bounce, complaint, open, or delivery metrics separated for Logister messages. |
LOGISTER_CLICKHOUSE_MODE | Controls the staged ClickHouse path: disabled, dual_write, or read_preferred. | Use dual_write only during schema setup, backfill, and coverage verification; dashboards still read PostgreSQL in that mode. Switch to read_preferred after coverage passes. The legacy LOGISTER_CLICKHOUSE_ENABLED=true alias selects only dual write and cannot enable ClickHouse reads. |
LOGISTER_CLICKHOUSE_URL, LOGISTER_CLICKHOUSE_DATABASE, LOGISTER_CLICKHOUSE_EVENTS_TABLE, LOGISTER_CLICKHOUSE_SPANS_TABLE | ClickHouse endpoint, database, raw event table, and raw span table targets. | Use values from your ClickHouse server or ClickHouse Cloud service. Load the schema from the ClickHouse guide before enabling writes, then verify readiness with bin/rails logister:clickhouse:schema:status or /health/clickhouse. |
LOGISTER_CLICKHOUSE_USERNAME / LOGISTER_CLICKHOUSE_PASSWORD | ClickHouse runtime credentials. | Create a least-privilege user that can insert into and query the Logister event and span tables, then store the password as a secret. |
LOGISTER_CLICKHOUSE_MIGRATION_USERNAME / LOGISTER_CLICKHOUSE_MIGRATION_PASSWORD | Optional release-only ClickHouse DDL credentials. | Use a separate operator account with CREATE, ALTER, and DROP VIEW access. bin/release uses it for idempotent schema repair when ClickHouse is enabled, without granting DDL permissions to the runtime account. |
LOGISTER_CLICKHOUSE_FAILURE_THROTTLE_SECONDS | Throttle window for Logister self-monitoring when ClickHouse writes fail. | Leave the default unless an operator wants more or fewer internal failure reports. Failures are reported through logister-ruby as logs and count metrics when self-observability is configured. |
ACTIVE_STORAGE_SERVICE | Active Storage service used by production file/archive writes. | Use local for a simple private install with mounted disk, or amazon when using S3-compatible archive storage. |
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AWS_S3_BUCKET | Amazon S3 credentials and bucket target for the built-in amazon Active Storage service. | Create a private bucket and a least-privilege IAM user or role that can write only the archive prefix Logister needs. Store the key and secret in your provider secret manager. |
AWS_S3_ENDPOINT, AWS_S3_FORCE_PATH_STYLE | Optional S3-compatible provider settings. | Set these only for non-AWS object stores such as MinIO, R2-compatible gateways, or providers that require a custom endpoint or path-style addressing. |
LOGISTER_ARCHIVE_PREFIX, LOGISTER_ARCHIVE_STORAGE_SERVICE | Telemetry archive object prefix and optional storage-service override. | Use a low-cardinality prefix such as telemetry. Leave the storage override blank to use ACTIVE_STORAGE_SERVICE, or set it when archives should use a different Active Storage service. |
LOGISTER_TURNSTILE_ENABLED | Enables Cloudflare Turnstile on sign-in, sign-up, password, and confirmation forms. | Turn on for public instances that need bot protection. Keep off for private/internal installs unless you want the extra challenge. |
LOGISTER_TURNSTILE_SITE_KEY / LOGISTER_TURNSTILE_SECRET_KEY | Cloudflare Turnstile browser key and server verification secret. | Create a Turnstile widget in Cloudflare, add your app domain, put the site key in config, and keep the secret key in your secret store. |
LOGISTER_COOKIE_CONSENT_ENABLED | Controls whether the cookie consent banner renders. | Set false for private installs without optional analytics. Keep true when you run public product pages with analytics. |
LOGISTER_ANALYTICS_ENABLED | Allows analytics tags outside production for verification. | Leave false unless you need to test analytics in development or staging. |
PROBO_COOKIE_BANNER_SCRIPT_URL, PROBO_COOKIE_BANNER_ID, PROBO_COOKIE_BANNER_BASE_URL, PROBO_COOKIE_BANNER_PROXY_ENABLED, PROBO_COOKIE_BANNER_POSITION | Probo Cookie Banner loader and banner configuration. | Create or host a Probo-compatible banner configuration, then store the banner ID and upstream base URL in deployment config. The Rails app proxies the upstream by default so browser banner requests stay same-origin; set PROBO_COOKIE_BANNER_PROXY_ENABLED=false only when the upstream supports credentialed CORS for your app domain. |
LOGISTER_ANALYTICS_COOKIE_CATEGORY | Consent category slug required before analytics scripts load. | Match this to the category slug configured in your consent banner, usually analytics. |
GOOGLE_TAG_ID | Optional Google Analytics measurement ID. | Create a GA4 web data stream and use its G-... measurement ID. Use your own property for forks and self-hosted instances. |
CLOUDFLARE_WEB_ANALYTICS_TOKEN | Optional Cloudflare Web Analytics token. | Create a Cloudflare Web Analytics site token for your app domain. Use this instead of, or alongside, Google Analytics only when consent requirements are satisfied. |
CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_PAGES_PROJECT | Cloudflare deployment settings for the static docs and docs Worker proxy. | Set the token and account ID as GitHub Actions secrets for both docs workflows. The Pages workflow also needs CLOUDFLARE_PAGES_PROJECT. Keep cloudflare-docs-proxy/wrangler.toml pointed at the Pages origin that serves the static docs. Use the optional LOGISTER_DOCS_URL and DOCS_SITEMAP_LASTMOD repository variables when you want the docs build step to generate fork-specific metadata. |
DOCS_PROBO_COOKIE_BANNER_ID, DOCS_PROBO_COOKIE_BANNER_BASE_URL, DOCS_PROBO_COOKIE_BANNER_POSITION, DOCS_ANALYTICS_COOKIE_CATEGORY | Cloudflare Pages runtime consent-banner configuration. | Set these as Cloudflare Pages secrets if you host your own docs site with consent-gated analytics. |
DOCS_GOOGLE_TAG_ID, DOCS_CLOUDFLARE_WEB_ANALYTICS_TOKEN | Cloudflare Pages docs analytics IDs. | Use your own GA4 or Cloudflare Web Analytics values for your docs fork. These become browser-visible runtime config. |
Keep secrets out of git
Values such as RAILS_MASTER_KEY, database passwords, Redis passwords, SES SMTP passwords, Turnstile secret keys, ClickHouse passwords, AWS access keys, S3-compatible storage secrets, and Cloudflare API tokens must live in a provider secret store. The sample file intentionally uses blanks or placeholders.
Scaling knobs
Use these variables when you need more throughput.
| Variable | Default | Use when |
|---|---|---|
RAILS_MAX_THREADS | 3 | You want each web process to handle more concurrent requests. Match database pool sizing to your thread count. |
WEB_CONCURRENCY | 1 | You want multiple Puma workers per web machine. Use only when the host has enough CPU and memory. |
SIDEKIQ_CONCURRENCY | 5 | You want each worker process to run more jobs in parallel. Raise carefully with Redis and database capacity in mind. |
RAILS_LOG_LEVEL | info | You need more or less production log detail during operations. |
Optional services
Enable additional integrations only when you need them.
ClickHouse
ClickHouse is optional. Keep it off until you need dedicated analytics storage or higher-volume event analysis. When you turn it on, set the variables in the ClickHouse guide, load the event/span schema, keep the worker running, and complete the coverage-gated cutover to read_preferred. A healthy /health/clickhouse response with clickhouse_mode set to dual_write confirms connectivity and schema, but dashboards still read PostgreSQL.
GitHub App source integration
The GitHub App is optional. Use it when private repository source excerpts, CODEOWNERS hints, project-scoped repository connections, deployment links, or GitHub issue creation would make error triage faster. The setup is App-based instead of personal-token-based; see the GitHub App guide for exact GitHub settings and Logister variables.
S3-compatible archive storage
Archive storage is optional. Use it when you want compressed JSONL exports of older hot telemetry before pruning high-volume non-error events from PostgreSQL. The built-in path uses Active Storage, so self-hosters can keep local disk for small private installs or swap to Amazon S3 and S3-compatible providers without changing app code. The product behavior is documented in the data retention guide.
shell
ACTIVE_STORAGE_SERVICE=amazon
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION=us-east-1
AWS_S3_BUCKET=
LOGISTER_ARCHIVE_PREFIX=telemetry
# Optional for non-AWS S3-compatible providers:
# AWS_S3_ENDPOINT=
# AWS_S3_FORCE_PATH_STYLE=true
# LOGISTER_ARCHIVE_STORAGE_SERVICE=amazonUse archive storage for object storage setup and retention jobs for dry runs, manual commands, and verification.
Email is optional for a private single-user trial, but recommended for real users because account confirmation, password reset, immediate project alerts, digest summaries, monitor alerts, and operational notices depend on SMTP delivery.
LOGISTER_EMAIL_FROMsets the branded sender for auth, system mail, and project notifications.SES_REGIONselects the Amazon SES region, defaulting tous-east-1.SES_SMTP_USERNAMEandSES_SMTP_PASSWORDenable Amazon SES SMTP delivery in production.SES_CONFIGURATION_SEToptionally tags Logister notification mail for SES delivery metrics.SES_SMTP_ADDRESS,SES_SMTP_PORT,SES_SMTP_DOMAIN,SES_SMTP_ENABLE_STARTTLS_AUTO,SES_SMTP_OPEN_TIMEOUT, andSES_SMTP_READ_TIMEOUTare available when your SMTP provider needs custom settings.- Verify the sender domain in SES and publish SPF, DKIM, and DMARC records so notification mail is less likely to be filtered as spam.
For the product settings users see after SMTP works, read email notifications.
Cloudflare Turnstile
shell
LOGISTER_TURNSTILE_ENABLED=true
LOGISTER_TURNSTILE_SITE_KEY=
LOGISTER_TURNSTILE_SECRET_KEY=The sign-in, sign-up, password, and confirmation flows validate Turnstile when enabled.
Consent-gated analytics
shell
LOGISTER_COOKIE_CONSENT_ENABLED=true
LOGISTER_ANALYTICS_ENABLED=true
PROBO_COOKIE_BANNER_SCRIPT_URL=https://cdn.jsdelivr.net/npm/@probo/cookie-banner/dist/cookie-banner.iife.js
PROBO_COOKIE_BANNER_ID=
PROBO_COOKIE_BANNER_BASE_URL=https://your-probo.example/api/cookie-banner/v1
PROBO_COOKIE_BANNER_PROXY_ENABLED=true
PROBO_COOKIE_BANNER_POSITION=bottom-left
LOGISTER_ANALYTICS_COOKIE_CATEGORY=analytics
GOOGLE_TAG_ID=G-XXXXXXXXXX
CLOUDFLARE_WEB_ANALYTICS_TOKEN=Provider files
Keep deploy config in version control, but keep secrets out of the repo.
Dockerfilefor production app images. The release workflow publishesghcr.io/taimoorq/logister:<version>anddocker.io/taimoorq/logister:<version>, pluslatestand short-SHA tags in both registries. It also publishesquay.io/taimoorq/logister:<version>,latest, and short-SHA tags whenQUAY_USERNAMEandQUAY_TOKENare configured. Run separate web and worker containers from the same image; see the Docker self-hosting option for the full shape.fly.toml.exampleas the Fly.io template self-hosters can copy before setting their own app name, region, and scale.fly.tomlas the tracked production Fly.io config for this public instance, with separate app and worker processes.config/deploy.ymlas a Kamal-oriented starting point if you prefer a server-based Docker deployment.docker-compose.ymlfor local PostgreSQL, Redis, and ClickHouse services. Treat its defaults as local/demo values, not production secrets.
Important
Keep secrets in your deploy provider or CI secret store, not in tracked config files.
Production checklist
Run this checklist before calling the rollout complete.
- Set the Rails master key, PostgreSQL URL, and one-time setup token.
- Run migrations during deploy and start the web process.
- Claim
/setup, remove the setup token, and follow Admin → Installation. - Save Redis settings, restart web and worker processes, and require the Redis & jobs check to see at least one Sidekiq process.
- Configure outbound email before relying on signup confirmations, password resets, invitations, or project notifications.
- Configure S3-compatible archive storage if you plan to export and prune older hot telemetry.
- Keep ClickHouse in dual-write mode only until schema repair, backfill, and coverage verification pass; then switch to
read_preferredwhen supported dashboards should use it. - Remove the legacy
LOGISTER_CLICKHOUSE_ENABLEDoverride when managing the mode withLOGISTER_CLICKHOUSE_MODEor the admin UI. - Keep provider config tracked separately from secrets, and review the redacted installation history after operational changes.
Deploy verification
Check the operator path, not just the web process.
Post-deploy verification
One-time setup token has been removed after claim
First administrator can sign in and open Admin -> Installation
Required General check passes
Required Redis & jobs check sees Redis and at least one Sidekiq process
Project creation succeeds
API key generation works and shows the token once
Background job processes can connect to Redis
One test event lands in the project inbox
First-occurrence alerts enqueue when a new error group is created
ClickHouse health reports ready if optional ClickHouse is enabled
ClickHouse mode reports read_preferred when supported dashboards should use it
Archive export task can write to object storage if optional archives are enabled