Skip to main content
Buffer is a stateless write-path library that uses object storage as a durable queue between producers and a database. Producers accept opaque byte entries, batch them, and flush them to object storage. A single binary manifest in object storage coordinates producers and consumers using queue semantics. Because the data and the manifest live in object storage, Buffer is stateless and horizontally scalable. Object storage takes care of durability and availability.

Architecture

How Buffer works by leveraging object storage

Configuration

How to configure producers and consumers

GitHub

View the source code, open issues, and contribute