Skip to main content
Reading data from OpenData databases heavily leverages data caches to avoid unnecessary GET API calls to object storage. Reading data relies on a combination of domain-specific logic as well as shared storage access primitives. Each database implements its own indexes and data layouts for optimal read performance, but all use SlateDB queries to access the data through multiple levels of caches.

Read Freshness

Read freshness can be measured in terms of object storage round trips between a write being accepted and it being visible to a query:
Write PathRead TargetS3 Round TripsDescription
Direct write to writerWriter0Data is written to the in-memory Delta and is immediately readable on the writer with no object storage interaction.
Zonal ingestWriter1Data is written to object storage by a stateless ingestor and must be picked up by the writer before it is readable.
AnyRead replica+1Read replicas discover new data by watching object storage for manifest updates, adding one additional round trip on top of whatever the write path requires.
Each round trip typically adds around 100ms of latency plus the polling interval, which is configurable. In practice, the polling interval dominates the freshness delay.