When to choose Log

We build OpenData and this is our best good-faith comparison. Corrections are welcome. Log is built to deliver ordered, replayable events for high cardinality entities: a stream per user, session, agent, or device. With Log, every key is its own ordered stream and there are no topics or partitions to manage.

​

Choose Log if you care about…

…a stream per user, session, agent, or device, created by just writing to it. There are no topics to declare and no partition counts to size, whether you have a thousand streams or a million. …reading specific streams. Per-key scans are range reads on an LSM tree, and they stay fast as key cardinality grows. In our benchmarks, going from 100K to 1M keys left single-key poll latency flat. …serving tens of thousands of concurrent followers. One node sustained 50,000 concurrent followers across 1M keys at 20 MB/s ingest with sub-50ms p50, about $224/month all-in. Each added reader node serves another ~50K followers for ~$174/month. …replay from any point in history with retention at S3 prices. Any offset in any stream is a range read; retention is dropping entire time-based segments. …never operating a broker cluster. Log runs as one binary with an object store bucket. A single node is durable, read replicas pull straight from S3 (and can be scoped to key ranges), and splitting a node is a metadata operation: consumers resume at the same sequence numbers.

​

How the alternatives stack up

✓ = yes · ~ = partially · ✗ = no ¹ Kafka-protocol systems route by topic-partition: one key’s history means scanning its partition, and stream-per-entity means an unworkable partition count, on disks or on S3.
² NATS handles millions of subjects at low latency, but replayable history is bounded by the size of your replicated disks.
³ s2-lite is the nearest open-source neighbor to Log: MIT, stream-per-key, also built on SlateDB. It’s a young single-node companion to S2’s cloud, without published high-cardinality serving numbers.
⁴ Each follower tailing its own “stream” multiplies the problem in ¹.
⁵ Kafka tiered storage parks old segments on S3, but retention and serving remain broker concerns.
⁶ JetStream persists to RAFT-replicated local disks you manage.
⁷ WarpStream’s agents are stateless but require its proprietary hosted control plane; AutoMQ’s core is BSL. The systems that can do per-entity streams are managed services or very young. The mature open-source systems are funnels with the wrong data model for routing. Log is the self-hosted system built for routing.

​

The tradeoffs

• Latency: durable end-to-end latency is ~2s by default, though tuning can bring it lower. Polls run p50 ~30ms / p99 ~300ms at 25 MiB/s, and degrade if ingestion rates thrash the cache.
• You track your own offsets: consumer groups and efficient tail-following are active roadmap items; today, store per-key sequence numbers yourself.
• One writer node: write scaling is vertical for now; readers scale horizontally.

---

Last reviewed July 2026. Please tell us if a claim has gone stale.