Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
0 likes418 views
Katy Farmer discusses using SQL with InfluxDB 2.0 along with Flux for managing time series data and other data types, targeting industrial IoT applications. The document highlights examples of data import, querying, joining streams, and specific use cases involving robots and their performance metrics. The presentation concludes with a reminder to follow data schema and rules during storage operations.
![© InfluxData. All rights reserved.
Step : Joining Data Streams
import "sql"
info = sql.from(
driverName: "postgres",
dataSourceName: "postgresql://localhost/robots?sslmode=disable",
query:"SELECT * FROM robots;")
cpu = from(bucket: "robots")
|> range(start: -10d)
|> filter(fn: (r) => r._measurement == "cpu")
|> filter(fn: (r) => r._field == "system_usage")
|> mean()
join(tables: {cpu:cpu, info:info}, on: ["id"])](https://image.slidesharecdn.com/day2-b-400pm-usingthesqldatasourcein2-191008213642/85/Creating-and-Using-the-Flux-SQL-Datasource-Katy-Farmer-InfluxData-13-320.jpg)
![© InfluxData. All rights reserved.
import "sql"
info = sql.from(
driverName: "postgres",
dataSourceName: "postgresql://localhost/robots?sslmode=disable",
query:"SELECT * FROM robots;")
cpu = from(bucket: "robots")
|> range(start: -10d)
|> filter(fn: (r) => r._measurement == "cpu")
|> filter(fn: (r) => r._field == "system_usage")
|> mean()
join(tables: {cpu:cpu, info:info}, on: ["id"])
|> keep(columns: ["id", "_value", "created_at"])
|> sql.to(
driverName: "postgres",
dataSourceName: "postgresql://localhost/robots?sslmode=disable",
table: "averages")](https://image.slidesharecdn.com/day2-b-400pm-usingthesqldatasourcein2-191008213642/85/Creating-and-Using-the-Flux-SQL-Datasource-Katy-Farmer-InfluxData-16-320.jpg)
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
- 1. Katy Farmer Developer Advocate @thekatertot Using the SQL Datasource in 2.0
- 2. InfluxDB . + Flux + SQL (+ you!) Multiple Datasources InfluxDB . + Flux + SQL (+ you!) Multiple Datasources
- 3. Why? • Connect your time series data with other types of data for enrichment (and for fun) • Single responsibility databases Why? • Connect your time series data with other types of data for enrichment (and for fun) • Single responsibility databases
- 4. © InfluxData. All rights reserved. Industrial IoT Relational Data All sensor metrics (temperature, pressure, water level, Time Series Data Hardware information (model, year, etc.), date deployed, region, id © InfluxData. All rights reserved. Industrial IoT Relational Data All sensor metrics (temperature, pressure, water level, Time Series Data Hardware information (model, year, etc.), date deployed, region, id © InfluxData. All rights reserved. Industrial IoT Relational Data All sensor metrics (temperature, pressure, water level, Time Series Data Hardware information (model, year, etc.), date deployed, region, id
- 5. © InfluxData. All rights reserved. User Tracking Relational Data Logins, page visits, clicks, duration on page Time Series Data Name, email, location, history (browsing, purchases, etc.) © InfluxData. All rights reserved. User Tracking Relational Data Logins, page visits, clicks, duration on page Time Series Data Name, email, location, history (browsing, purchases, etc.) © InfluxData. All rights reserved. User Tracking Relational Data Logins, page visits, clicks, duration on page Time Series Data Name, email, location, history (browsing, purchases, etc.)
- 6. Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp) Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp) Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp) Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp)
- 7. Example: Robots (Industrial IoT) Robot jobs vary. Some measure parts, some place parts on belts, and others rotate the parts for inspection. But not all robots perform the same. Some of them crash. What is the average CPU usage of each robot? Example: Robots (Industrial IoT) Robot jobs vary. Some measure parts, some place parts on belts, and others rotate the parts for inspection. But not all robots perform the same. Some of them crash. What is the average CPU usage of each robot? Example: Robots (Industrial IoT) Robot jobs vary. Some measure parts, some place parts on belts, and others rotate the parts for inspection. But not all robots perform the same. Some of them crash. What is the average CPU usage of each robot?
- 8. © InfluxData. All rights reserved. Step : Importing SQL Data with Flux import "sql" sql.from( driverName: "postgres", dataSourceName: "postgresql://user:password@localhost", query:"SELECT * FROM mydb;")
- 9. © InfluxData. All rights reserved. Step : Importing SQL Data with Flux import "sql" sql.from( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", query:"SELECT * FROM robots;")
- 10. © InfluxData. All rights reserved.© InfluxData. All rights reserved.
- 11. © InfluxData. All rights reserved. Step : Write your first Flux query from(bucket: "robots") |> range(start: -10d) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "system_usage") |> mean()
- 12. © InfluxData. All rights reserved.© InfluxData. All rights reserved.
- 13. © InfluxData. All rights reserved. Step : Joining Data Streams import "sql" info = sql.from( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", query:"SELECT * FROM robots;") cpu = from(bucket: "robots") |> range(start: -10d) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "system_usage") |> mean() join(tables: {cpu:cpu, info:info}, on: ["id"])
- 14. © InfluxData. All rights reserved.© InfluxData. All rights reserved.
- 15. © InfluxData. All rights reserved. Step : Storing Results sql.to() requirements: ❏ Data in the stream must have same column names as the SQL DB ❏ Drop any columns that won’t be stored ❏ Remember your schema! Any rules (not null, unique, etc.) must be followed or the write will not succeed.
- 16. © InfluxData. All rights reserved. import "sql" info = sql.from( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", query:"SELECT * FROM robots;") cpu = from(bucket: "robots") |> range(start: -10d) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "system_usage") |> mean() join(tables: {cpu:cpu, info:info}, on: ["id"]) |> keep(columns: ["id", "_value", "created_at"]) |> sql.to( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", table: "averages")
- 17. © InfluxData. All rights reserved. Things I did wrong © InfluxData. All rights reserved. Things I did wrong
- 18. © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot