Game of Streams: How to Tame and Get the Most from Your Messaging Platforms
1 like•337 views
The document discusses using messaging platforms in distributed architectures and provides an overview of Spring Cloud Stream. It describes how Spring Cloud Stream can be used with different messaging platforms like Kafka and RabbitMQ. The document also provides code samples for a source service that generates aircraft position data and sends it to a processor service, which transforms the data and filters out records before sending it to a sink service.
![@mkheck www.thehecklers.com
Source service
code
(imperative)
@SpringBootApplication
public class PsourceApplication {
public static void main(String[] args) {
SpringApplication.run(PsourceApplication.class, args);
}
}
@Configuration
class PositionFeed {
private final WebClient client = WebClient.create("http://localhost:8080");
@Bean
Supplier<List<Aircraft>> sendAircraftPositions() {
return () -> {
List<Aircraft> aircraftList = client.get()
.retrieve()
.bodyToFlux(Aircraft.class)
.filter(ac -> !ac.getReg().isEmpty())
.toStream()
.collect(Collectors.toList());
aircraftList.forEach(System.out::println);
return aircraftList;
};
}
}
@Data
@JsonIgnoreProperties(ignoreUnknown = true)
class Aircraft {
private Long id;
private String callsign, reg, flightno, type;
private int altitude, heading, speed;
private double lat, lon;
}](https://image.slidesharecdn.com/gameofstreams20202-200910223532/85/Game-of-Streams-How-to-Tame-and-Get-the-Most-from-Your-Messaging-Platforms-24-320.jpg)
![@mkheck www.thehecklers.com
Source service
code
(reactive)
@SpringBootApplication
public class PsourceApplication {
public static void main(String[] args) {
SpringApplication.run(PsourceApplication.class, args);
}
}
@Configuration
class PositionFeed {
private final WebClient client = WebClient.create("http://localhost:8080");
@PollableBean
Supplier<Flux<Aircraft>> sendAircraftPositions() {
return () -> client.get()
.retrieve()
.bodyToFlux(Aircraft.class)
.filter(ac -> !ac.getReg().isEmpty())
.log();
}
}
@Data
@JsonIgnoreProperties(ignoreUnknown = true)
class Aircraft {
private Long id;
private String callsign, reg, flightno, type;
private int altitude, heading, speed;
private double lat, lon;
}](https://image.slidesharecdn.com/gameofstreams20202-200910223532/85/Game-of-Streams-How-to-Tame-and-Get-the-Most-from-Your-Messaging-Platforms-25-320.jpg)
![@mkheck www.thehecklers.com
Sink service
code
(imperative)
@SpringBootApplication
public class PsinkApplication {
public static void main(String[] args) {
SpringApplication.run(PsinkApplication.class, args);
}
}
@Configuration
class AircraftLogger {
@Bean
Consumer<List<EssentialAircraft>> logIt() {
return listEssentialAC -> listEssentialAC.forEach(System.out::println);
}
}
@Data
@AllArgsConstructor
class EssentialAircraft {
private String callsign, reg, type;
private String mfr;
}](https://image.slidesharecdn.com/gameofstreams20202-200910223532/85/Game-of-Streams-How-to-Tame-and-Get-the-Most-from-Your-Messaging-Platforms-32-320.jpg)
![@mkheck www.thehecklers.com
Sink service
code
(reactive)
@SpringBootApplication
public class PsinkApplication {
public static void main(String[] args) {
SpringApplication.run(PsinkApplication.class, args);
}
}
@Configuration
class AircraftLogger {
@Bean
Consumer<Flux<EssentialAircraft>> logIt() {
return eACFlux -> eACFlux.subscribe(System.out::println);
}
}
@Data
@AllArgsConstructor
class EssentialAircraft {
private String callsign, reg, type;
private String mfr;
}](https://image.slidesharecdn.com/gameofstreams20202-200910223532/85/Game-of-Streams-How-to-Tame-and-Get-the-Most-from-Your-Messaging-Platforms-33-320.jpg)
Game of Streams: How to Tame and Get the Most from Your Messaging Platforms
- 1. Game of Streams 🐉 How to tame & get the most from your messaging platforms Mark Heckler Professional Problem Solver, Spring Developer & Advocate www.thehecklers.com [email protected] [email protected] @mkheck
- 2. @mkheck www.thehecklers.com “Please do LESS with MORE!” 💰💰💰
- 3. @mkheck www.thehecklers.com Who am I? • Author • Architect & Developer • Java Champion, Rockstar • Professional Problem Solver • Spring Developer & Advocate • Creador y curador de • Pilot
- 4. @mkheck www.thehecklers.com New book! Now available for early access https://bit.ly/springbootbook
- 5. @mkheck www.thehecklers.com Takeaways Why use messaging platforms/where do they fit in a distributed architecture? Examples of leading messaging platforms What is Spring Cloud Stream? Why use it?
- 6. @mkheck www.thehecklers.com Takeaways Why use messaging platforms/where do they fit in a distributed architecture? Examples of leading messaging platforms What is Spring Cloud Stream? Why use it?
- 7. @mkheck www.thehecklers.com Takeaways Why use messaging platforms/where do they fit in a distributed architecture? Examples of leading messaging platforms What is Spring Cloud Stream? Why use it?
- 8. @mkheck www.thehecklers.com Takeaways Why use messaging platforms/where do they fit in a distributed architecture? Examples of leading messaging platforms What is Spring Cloud Stream? Why use it?
- 10. @mkheck www.thehecklers.com SCSt + ${messaging.platform} =
- 15. @mkheck www.thehecklers.com SinkProcessor Stream Revisited: Legacy Source
- 16. @mkheck www.thehecklers.com ConsumerFunction Stream Revisited: Evolution Supplier
- 17. @mkheck www.thehecklers.com And a few really useful bits & bobs (if time permits, if not, please star the repo!)
- 20. @mkheck www.thehecklers.com Resources https://github.com/mkheck/game-of-streams-aircraft-edition https://spring.io/projects/spring-cloud-stream [email protected], [email protected] @mkheck on Twitter Kotlin fan? https://github.com/mkheck/game-of-streams-aircraft-edition-kotlin
- 21. @mkheck www.thehecklers.com In case of emergency, break glass
- 23. @mkheck www.thehecklers.com Source service properties server.port=0 spring.cloud.stream.default-binder=rabbit #spring.cloud.stream.poller.fixed-delay=5000 spring.cloud.stream.bindings.sendAircraftPositions-out-0.destination=processor #spring.cloud.stream.bindings.sendAircraftPositions-out-0.binder=kafka spring.cloud.stream.kafka.binder.min-partition-count=4 spring.cloud.stream.kafka.binder.auto-add-partitions=true # DO NOT DO THIS IN PRODUCTION, FOR DEMO PURPOSES ONLY!!! management.endpoints.web.exposure.include=*
- 24. @mkheck www.thehecklers.com Source service code (imperative) @SpringBootApplication public class PsourceApplication { public static void main(String[] args) { SpringApplication.run(PsourceApplication.class, args); } } @Configuration class PositionFeed { private final WebClient client = WebClient.create("http://localhost:8080"); @Bean Supplier<List<Aircraft>> sendAircraftPositions() { return () -> { List<Aircraft> aircraftList = client.get() .retrieve() .bodyToFlux(Aircraft.class) .filter(ac -> !ac.getReg().isEmpty()) .toStream() .collect(Collectors.toList()); aircraftList.forEach(System.out::println); return aircraftList; }; } } @Data @JsonIgnoreProperties(ignoreUnknown = true) class Aircraft { private Long id; private String callsign, reg, flightno, type; private int altitude, heading, speed; private double lat, lon; }
- 25. @mkheck www.thehecklers.com Source service code (reactive) @SpringBootApplication public class PsourceApplication { public static void main(String[] args) { SpringApplication.run(PsourceApplication.class, args); } } @Configuration class PositionFeed { private final WebClient client = WebClient.create("http://localhost:8080"); @PollableBean Supplier<Flux<Aircraft>> sendAircraftPositions() { return () -> client.get() .retrieve() .bodyToFlux(Aircraft.class) .filter(ac -> !ac.getReg().isEmpty()) .log(); } } @Data @JsonIgnoreProperties(ignoreUnknown = true) class Aircraft { private Long id; private String callsign, reg, flightno, type; private int altitude, heading, speed; private double lat, lon; }
- 28. @mkheck www.thehecklers.com Processor service code (imperative) @Configuration class AircraftTransformer { @Bean Function<List<Aircraft>, List<EssentialAircraft>> transformAC() { return listAC -> { List<EssentialAircraft> listEssential = new ArrayList<>(); listAC.forEach(ac -> listEssential.add(new EssentialAircraft(ac.getCallsign(), ac.getReg(), ac.getType()))); listEssential.forEach(System.out::println); return listEssential; }; } } @Data @AllArgsConstructor class Aircraft { private Long id; private String callsign, reg, flightno, type; private int altitude, heading, speed; private double lat, lon; } @Data class EssentialAircraft { private String callsign, reg, type, mfr; public EssentialAircraft(String callsign, String reg, String type) { this.callsign = callsign; this.reg = reg; this.type = type; setMfr(); } private void setMfr() { mfr = switch (type.substring(0, 2)) { case "A2", "A3" -> "Airbus"; case "BE", "B3" -> "Beechcraft"; case "B7" -> "Boeing"; case "C1", "C2", "C4", "C6", "C7", "T2" -> "Cessna"; case "CR", "E7" -> "Embraer"; case "LJ" -> "Learjet"; case "MD" -> "McDonnell Douglas"; case "PA" -> "Piper"; default -> "Other"; }; } }
- 29. @mkheck www.thehecklers.com Processor service code (reactive) @Configuration class AircraftTransformer { @Bean Function<Flux<Aircraft>, Flux<EssentialAircraft>> transformAC() { return aircraftFlux -> aircraftFlux.map(ac -> new EssentialAircraft(ac.getCallsign(), ac.getReg(), ac.getType())); //.log(); } @Bean Function<Flux<EssentialAircraft>, Flux<EssentialAircraft>> filterNullCallsigns() { return flux -> flux.filter(ac -> null != ac.getCallsign()) .log(); } } @Data @AllArgsConstructor class Aircraft { private Long id; private String callsign, reg, flightno, type; private int altitude, heading, speed; private double lat, lon; } @Data class EssentialAircraft { private String callsign, reg, type, mfr; public EssentialAircraft(String callsign, String reg, String type) { this.callsign = callsign; this.reg = reg; this.type = type; setMfr(); } private void setMfr() { mfr = switch (type.substring(0, 2)) { case "A2", "A3" -> "Airbus"; case "BE", "B3" -> "Beechcraft"; case "B7" -> "Boeing"; case "C1", "C2", "C4", "C6", "C7", "T2" -> "Cessna"; case "CR", "E7" -> "Embraer"; case "LJ" -> "Learjet"; case "MD" -> "McDonnell Douglas"; case "PA" -> "Piper"; default -> "Other"; }; } }
- 31. @mkheck www.thehecklers.com Sink service properties server.port=0 spring.cloud.stream.bindings.logIt-in-0.destination=sink spring.cloud.stream.bindings.logIt-in-0.group=sink spring.cloud.stream.bindings.logIt-in-0.binder=rabbit spring.cloud.stream.kafka.binder.min-partition-count=4 spring.cloud.stream.kafka.binder.auto-add-partitions=true # Once more (I promise): DO NOT DO THIS IN PRODUCTION!!! management.endpoints.web.exposure.include=*
- 32. @mkheck www.thehecklers.com Sink service code (imperative) @SpringBootApplication public class PsinkApplication { public static void main(String[] args) { SpringApplication.run(PsinkApplication.class, args); } } @Configuration class AircraftLogger { @Bean Consumer<List<EssentialAircraft>> logIt() { return listEssentialAC -> listEssentialAC.forEach(System.out::println); } } @Data @AllArgsConstructor class EssentialAircraft { private String callsign, reg, type; private String mfr; }
- 33. @mkheck www.thehecklers.com Sink service code (reactive) @SpringBootApplication public class PsinkApplication { public static void main(String[] args) { SpringApplication.run(PsinkApplication.class, args); } } @Configuration class AircraftLogger { @Bean Consumer<Flux<EssentialAircraft>> logIt() { return eACFlux -> eACFlux.subscribe(System.out::println); } } @Data @AllArgsConstructor class EssentialAircraft { private String callsign, reg, type; private String mfr; }
- 34. @mkheck www.thehecklers.com Sample data PlaneFinder API Gateway