Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, Reddit
0 likes104 views
Reddit used kSQLdb to build a live event debugger that allowed QA teams to test event data with only a few seconds delay, removing the barriers of having to wait hours to query a data warehouse. By ingesting event data into kSQLdb's real-time streams, developers could use SQL queries to quickly search for specific event types or testing sessions and see the results in a web UI, improving their ability to test and debug events during development without any prior database knowledge.
Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, Reddit
- 1. Live Event Debugging With kSQLdb at Reddit Hannah Hagen, Paul Kiernan Replace with circle-cropped image (LINK) or Snoovatarof presenter
- 2. 2023
- 3. Release new feature The Old Testing Process QA (fires events) WAIT 2 HOURS Query the data warehouse
- 4. Drawback of 2 hour delay ➔ Testing was not coupled with development
- 5. What we really wanted (and didn’t have) ➔ Quick and efficient testing procedure (no delays!) ➔ No prior knowledge required (e.g. how to access our data warehouse, etc) ➔ In summary, we wanted to remove the barriers to test event data.
- 6. Why kSQLdb? ➔ Real-time event stream ➔ We already had Kafka-based infrastructure. ➔ Easy set up. Just need to run a few “create stream” and “create table” statements and deploy a kSQLdb cluster. ➔ SQL-like filters meant developers can search for the event type or the testing session they care about.
- 15. End Result ➔ Couple seconds delay between firing event and seeing it in Web UI ➔ UI-driven QA lows barriers for testing event data