> ## Documentation Index
> Fetch the complete documentation index at: https://opendata.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenData: Writing Data

The OpenData family of databases share a common foundation to provide a unified
experience and similar operational characteristics. Nearly all the databases
share a similar read/write path.

The write path accepts data and writes it both to a durable WAL as well as into
in-memory data structures that are eventually flushed to object storage as
SSTs in an LSM tree:

<pre className="ascii-art">
  {`
    ┌─────────────────────────────────────────────────────────────────────┐
    │▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ opendata write path ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
    ├─────────────────────────────────────────────────────────────────────┤
    │                                                                     │
    │                                                                     │
    │                            ╔═════════╗       ╔SlateDB══════════╗    │
    │                            ║         ║▒      ║                 ║▒   │
    │                       ┌───▶║   WAL   ║▒      ║                 ║▒   │
    │                       │    ║         ║▒      ║                 ║▒   │
    │                       │    ╚═════════╝▒      ║  ┌───────────┐  ║▒   │
    │  ╔═══════════════╗    │     ▒▒▒▒▒▒▒▒▒▒▒      ║  │ LSM Tree  │  ║▒   │
    │  ║  write(data)  ║────┤                      ║  │ on Object │  ║▒   │
    │  ╚═══════════════╝    │                      ║  │  Storage  │  ║▒   │
    │                       │    ╔═════════╗       ║  └───────────┘  ║▒   │
    │                       │    ║         ║▒      ║                 ║▒   │
    │                       └───▶║  Delta  ║──────▶║                 ║▒   │
    │                            ║         ║▒      ║                 ║▒   │
    │                            ╚═════════╝▒      ╚═════════════════╝▒   │
    │                             ▒▒▒▒▒▒▒▒▒▒▒       ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒   │
    │                                                                     │
    └─────────────────────────────────────────────────────────────────────┘
    `}
</pre>

Writes are assigned a canonical ordering before being acknowledged to ensure
that data is written consistently. From there, small WAL (write-ahead log)
files are created and flushed frequently to object storage to ensure durability
within a configurable time delay. These files are used to recover the database
in the event of a system failure.

Simultaneously, data is written into an in-memory data structure we call a
`Delta`. This is used to make new data immediately available for reads. Once
the `Delta` is full, it is flushed to object storage as a SlateDB SST. While
the `Delta` implementation is domain-specific (unique to each database), the
management mechanism is common across all databases, ensuring that the ingest
semantics are correct and well tested.

<Note>
  LSM Tree maintence tasks such as comapction and garbage collection are handled
  asynchronously and can run on a separate runtime (or even on a different
  machine). While each database can implement its own comapction strategy, we
  delegate the compaction execution to SlateDB.

  See [Compaction in SlateDB](https://slatedb.io/rfcs/0002-compaction/).
</Note>

### Flexible Durability Guarantees

The common write coordination mechanism allows users to control the required
durability before acknowledging the write. This allows users to trade off
durability for write latency. This tradeoff is fundamental to the design of
object-store native systems.

At one end of the spectrum, writes can be acknowledged as soon as the data
is buffered in the in-memory Delta. This offers the lowest latency but means
that data written since the last WAL flush is at risk if the process crashes.
At the other end, writes can wait until the WAL is flushed to object storage
before being acknowledged, providing full durability at the cost of higher
write latency.

| Durability Level | Ack Condition                   | Latency | Data at Risk                  |
| ---------------- | ------------------------------- | ------- | ----------------------------- |
| **Buffered**     | Data written to in-memory Delta | Lowest  | Writes since last WAL flush   |
| **WAL-durable**  | WAL flushed to object storage   | Higher  | None (survives process crash) |

Since WAL files are flushed frequently on a configurable interval, the
buffered mode still provides durability within a bounded time window. Choosing
the right level depends on the use case.

A third option decouples durability from the writer entirely: clients can
write into an object-storage queue, and the main writer drains that queue
asynchronously. The write returns as soon as the batch is durable in object
storage, so ingest keeps working across writer restarts or failovers. See
[Stateless Ingest](/timeseries/ingest) for the Timeseries implementation.

### Backpressure & Flow Control

When writes arrive faster than Deltas can be flushed to object storage, the
system applies backpressure to prevent unbounded memory growth. Once the
number of unflushed Deltas exceeds a configurable threshold, new writes are
stalled until a flush completes and frees memory.

This flow control mechanism ensures that the system degrades gracefully under
load rather than running out of memory. The backpressure threshold is tuned
alongside the Delta size and flush interval to balance write throughput
against memory usage.