![[Meetup ms] Kafka Streams](https://image.slidesharecdn.com/meetup-mskafkastreams-180704165301/85/Meetup-ms-Kafka-Streams-30-320.jpg)
![[Meetup ms] Kafka Streams](https://image.slidesharecdn.com/meetup-mskafkastreams-180704165301/85/Meetup-ms-Kafka-Streams-33-320.jpg)
![[Meetup ms] Kafka Streams](https://image.slidesharecdn.com/meetup-mskafkastreams-180704165301/85/Meetup-ms-Kafka-Streams-37-320.jpg)
![[Meetup ms] Kafka Streams](https://image.slidesharecdn.com/meetup-mskafkastreams-180704165301/85/Meetup-ms-Kafka-Streams-48-320.jpg)
[Meetup ms] Kafka Streams
- 1. Kafka Streams Quando streams encontram tabelas
- 3. Matheus Alagia Eng. de Computação https://www.linkedin.com/in/matheusalagia/ https://ubots.com.br
- 4. Microservices Independent deploy Low coupling Horizontal Scalability Technological choices Business time Independent teams Technological evolution Resilient
- 5. Microservices is really just distributed systems!
- 6. Increasingly we build ecosystem
- 7. The Hardest Part About Microservices: Your Data
- 8. Pessoa Conta Corrente CPF, Nome, Endereço, Renda, etc. Número conta, Tipo, Lançamentos, etc Services with different database
- 9. Pessoa Conta Corrente CPF, Nome, Endereço, Renda, etc. Número conta, Tipo, Lançamentos, etc Relatório CPF, Nome, Número Conta, Lançamentos Services with different database
- 10. Pessoa Conta Corrente CPF, Nome, Endereço, Renda, etc. Número conta, Tipo, Lançamentos, etc Relatório CPF, Nome, Número Conta, Lançamentos REST
- 11. Pessoa Conta Corrente CPF, Nome, Endereço, Renda, etc. Número conta, Tipo, Lançamentos, etc Relatório CPF, Nome, Número Conta, Lançamentos Database integration
- 12. Pessoa Relatório Services with different database Conta Cartão Previdência Título Serasa Crédito
- 13. Streaming processing can be the answer...
- 14. What is Streaming Processing? “Is some kind of computation over a Data Stream. First and foremost, a data stream is an abstraction representing an unbounded dataset. Unbounded means infinite and ever growing.” Kafka: The Definitive Guide
- 16. Stream processing is a programming paradigm... Request-Response Batch ProcessingStreaming Processing Throughput Latency
- 17. The world always changes, and sometimes we are interested in the events that caused those changes, whereas other times we are interested in the current state of the world…. Stream-Table Duality
- 18. Stream-Table Duality Example LANÇAMENTOS CONTA CORRENTE Conta: 1 Valor: 100 Conta: 1 Valor: 200 Conta: 5 Valor: 200 Conta: 1 Valor: -50
- 19. Stream-Table Duality Example LANÇAMENTOS CONTA CORRENTE Conta: 1 Valor: 100 Conta: 1 Valor: 200 Conta: 5 Valor: 200 Conta: 1 Valor: -50 1: 250 5: 200
- 20. Systems that allow you to transition back and forth between the two ways of looking at data are more powerful than systems that support just one. - Neha Narkhede (Kafka: The definitive guide)
- 21. Kafka vs Kafka Stream ● Distributed log ● High available ● ⅓ Fortune 500 ● APIS: ○ Producer ○ Consumer ○ Connect ○ Streams ● Part of Kafka ecosystem ● Just a lib ● Simple API ● DSL
- 24. Code
- 25. External Lookup: Stream-Table Join
- 26. Processing with Local State
- 28. Stateless is good!! ● Services start instantly ● Thread safe ● Scaled out linearly ● Not shared state
- 29. But sometimes we need state…. Moving the state to a database just push the problem to another layer.
- 31. Statefull can be good too... ● Services start rapidly ● Thread safe ● Scaled out linearly ● Not shared state
- 32. KAFKA STREAMS ● Services start rapidly V ● Thread safe V ● Scaled out linearly V ● Not shared state V ● HIGH THROUGHPUT
- 34. Stream
- 35. Table
- 36. Stream
- 38. Stream-Processing Concepts Time Event time Local state Log append time State Processing time External state Time Windows Slide window Tumbling window Hopping Window
- 39. Stream-Processing Concepts Time Event time Local state Log append time State Processing time External state Time Windows Slide window Tumbling window Hopping Window
- 40. Stream-Processing Concepts Time Event time Local state Log append time State Processing time External state Time Windows Slide window Tumbling window Hopping Window
- 41. Join ● Stream - Stream ● Stream - Table ● Table - Table
- 42. Stream - Stream (Windowed-join)
- 43. Stream - Table
- 44. Under the wood
- 45. Under the wood
- 46. Under the wood
- 49. Conclusion ● Stream processing is a powerful tool ● Kafka Streams is simple ● Designed for microservices
- 50. Obrigado!
- 51. Perguntas?