This document covers the Prometheus storage system, including the local Time Series Database (TSDB) implementation, storage abstractions, and remote storage integration. Prometheus storage is optimized for time series workloads with high write throughput and efficient query performance.
Related pages: Data Collection covers how metrics are ingested, Query Engine explains how data is queried, and Rules and Alerting describes how derived metrics are computed.
Prometheus uses a custom time series database (TSDB) that combines in-memory storage for recent data with immutable on-disk blocks for historical data. The architecture is designed to handle millions of active time series with high ingestion rates while maintaining low query latency.
Prometheus Storage System Architecture
Sources: tsdb/db.go236-293 tsdb/head.go68-146 tsdb/block.go85-150 storage/remote/queue_manager.go420-463
The storage system consists of several key components:
tsdb.DB: Main storage coordinator managing Head and Blocks. tsdb/db.go236-293tsdb.Head: In-memory storage for recent data (typically 2 hours). tsdb/head.go68-146Block: Immutable on-disk storage for historical data. tsdb/block.go85-150storage.Querier: Interface for reading data across Head and Blocks. storage/interface.go123-132remote.QueueManager: Manages remote write to external systems. storage/remote/queue_manager.go420-463Data Flow: Write and Read Paths
Sources: tsdb/head_append.go163-180 tsdb/db.go1113-1195 tsdb/querier.go108-119
Prometheus defines core storage abstractions in the storage package that enable different implementations, testing, and composition patterns like fanout storage.
For details, see Storage Interfaces.
| Interface | Purpose | Primary Methods | Implementation |
|---|---|---|---|
storage.Storage | Complete storage system | Appender(), Querier(), ChunkQuerier(), StartTime(), Close() | tsdb.DB |
storage.Appendable | Write interface | Appender(ctx) | tsdb.DB, tsdb.Head |
storage.Queryable | Read interface | Querier(mint, maxt), ChunkQuerier(mint, maxt) | tsdb.DB, Block |
storage.Appender | Transaction writes | Append(), AppendHistogram(), AppendExemplar(), Commit(), Rollback() | headAppender |
storage.Querier | Sample queries | Select(), LabelValues(), LabelNames() | blockQuerier, headQuerier |
Sources: storage/interface.go61-111 storage/interface.go123-170
The Prometheus TSDB uses a two-tier architecture: an in-memory Head for recent data and immutable Block files for historical data. This design balances immediate write performance with efficient long-term storage and querying.
For details, see TSDB Architecture.
The tsdb.DB struct coordinates all storage operations, including Head management, block loading, and compaction. It acts as the central entry point for all storage interactions. tsdb/db.go236-293
Sources: tsdb/db.go236-293 tsdb/db.go99-226
The Head manages the most recent time window of data in memory. It uses a stripeSeries structure to provide concurrent access to series through hash-based striping. The Head is responsible for ingesting new samples, managing their lifecycle in memory, and handling out-of-order data.
For details, see Head and In-Memory Storage.
Sources: tsdb/head.go68-146 tsdb/head.go1820-1880
The Write-Ahead Log (WAL) provides durability by persisting all writes before they're committed to in-memory structures. This ensures that no data is lost in case of a crash. Prometheus also maintains a Write-Behind Log (WBL) specifically for out-of-order samples, allowing them to be written to disk and replayed later.
For details, see WAL and Durability.
Sources: tsdb/head_wal.go79-250 tsdb/wlog/wlog.go
The initial two-hour blocks created from the Head are eventually compacted into longer blocks in the background to improve query efficiency and storage utilization. The LeveledCompactor orchestrates this process, merging smaller blocks into larger ones according to a predefined retention policy.
For details, see Compaction and Block Management.
Sources: tsdb/compact.go195-370 tsdb/db.go1113-1195
Prometheus optionally integrates with remote storage systems. It can write samples to remote endpoints via a QueueManager and read from them using the fanout querier pattern, allowing queries to span both local and remote data.
For details, see Remote Storage Integration.
Sources: storage/remote/queue_manager.go420-556 storage/fanout.go50-106
Prometheus supports various data types including float samples, native histograms, and exemplars. These are encoded into chunks using specialized formats like XOR for floats, enabling efficient storage and retrieval.
For details, see Data Types and Formats.
Sources: storage/interface.go21-54 tsdb/head_append.go330-342
Prometheus is highly efficient, storing an average of only 1-2 bytes per sample. docs/storage.md108-113
| Operation | Complexity | Typical Performance |
|---|---|---|
| Sample append | O(1) amortized | ~500k samples/sec per core |
| Series creation | O(1) hash lookup | ~100k series/sec |
| Query (recent data) | O(matching series) | Sub-millisecond for 1k series |
Refresh this wiki