InfluxDB Internals
5 likes2,281 views
The document discusses the internals of InfluxDB, focusing on its design as a time series database. It highlights the unique requirements of time series data, including high throughput ingestion, fast queries, and special storage needs. Key advancements in InfluxDB, such as TSI for cardinality management and IFQL for improved query execution, are also outlined.
![InfluxDB: on-disk (filesystem)
CREATE RETENTION POLICY <retention_policy_name> ON
<database_name> DURATION <duration> REPLICATION <n> [SHARD
DURATION <duration>] [DEFAULT]
Database directory /db
Retention Policy directory /db/rp
Shard Group (time bounded) (Logical)
Shard directory (db/rp/Id#)
TSM0001.tsm (data file)
TSM0002.tsm (data file)](https://image.slidesharecdn.com/ryanbetts-180215155933/85/InfluxDB-Internals-13-320.jpg)
InfluxDB Internals
- 1. InfluxDB Internals Platform Engineering Team @ryanbetts / ryan@influxdata.com
- 2. How great are databases? • I like making things with smart, clever, kind people. • I’ve been working on high-throughput, realtime data for the last 10 years.
- 3. • What’s so special about time series • Time series database designs • InfluxDB internals
- 5. RDBMS NoSQL TSDB Correctness ACID BASE BASE Schema DDL DDL / documents on-write Writing data DML POST/PUT line protocol Reading data SQL GET + filter filter, window, group, join
- 6. TSDB unique combination • Ingest: thousands to millions of points per second • Store: fast accumulating, append-mostly data, lots of repetition, often with time-to-live • Query: analytic queries with fast filtering, windowing • Scale: availability, storage, query
- 7. Facebook Gorilla • TTL eviction • Columnar compression • Write availability > query correctness • Metric-based schema • Separate query processing from access-path
- 8. Druid • Roll-up at ingest • Columnar storage & time-based segments • Indexes on dimension for fast filtering • Separation of real time and historical data nodes
- 9. Bullet Journals • Fast event recording • Ordered by time • Indexed by dimensions • Weekly / Monthly roll-up
- 11. InfluxDB: Adding data (1) POST ’http://localhost:8086/write?db=mydb' --data- binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000’
- 12. InfluxDB: Adding data (2) fsync( ) batch to WAL Add to in- memory cache Snapshot cache to TSM Add to index
- 13. InfluxDB: on-disk (filesystem) CREATE RETENTION POLICY <retention_policy_name> ON <database_name> DURATION <duration> REPLICATION <n> [SHARD DURATION <duration>] [DEFAULT] Database directory /db Retention Policy directory /db/rp Shard Group (time bounded) (Logical) Shard directory (db/rp/Id#) TSM0001.tsm (data file) TSM0002.tsm (data file)
- 15. InfluxDB: Adding data (DB) fsync( ) batch to WAL Add to in- memory cache Snapshot to TSM Add to index
- 16. InfluxDB: Adding data (index) • Measurement name -> field keys • Measurement name -> series • Measurement name -> tag keys -> tag value -> series • Series -> shards • (Also sketches of series and measurements for fast cardinality estimation)
- 17. InfluxDB: TSI • Roaring-bitmaps to short- cut series creation on insert • Iterators for index mappings • Index is per-shard; series id file is per-database • Partitioned for lock-splitting
- 18. TSI TSI
- 19. InfluxDB: InfluxQL Queries 1. Parses time range and expressions for filtering data 2. Look-up shards to access using the list of measurements and the time frame 3. Create the iterators for each shard 4. Merge the shard iterator outputs select user, system from cpu where time > now() - 1h and host = 'serverA
- 20. InfluxQL: Query with IFQL 1. Stand-alone `ifqld` coordinator nodes 2. Streaming storage iterators that support rate-limits 3. Separation of query planning and query distribution 4. Extensible, functional language 5. Unification of InfluxQL and TICKScript
- 21. A brief sidebar on append-mostly databases No one tells you about: * Wrong data * Old (back-filled) data
- 22. InfluxDB Clustering • Strongly consistent meta-cluster (based on RAFT) • User configured replication factor • Replication and shard aware query planner • Hinted-Handoff queues on each data node • (WIP) Anti-entropy consistency repair
- 23. Conclusions • Time series data has unique storage and query requirements that impact database design. • Evolution of InfluxDB: 1. TSI: remove the in-memory size limit on cardinality 2. IFQL: faster feature velocity; safer execution. 3. Anti-entropy repair: easier, more robust scale-out.