Spring Cloud Stream Consumer 是 Spring Cloud Stream 框架中用于消费消息的核心组件。它允许开发者以声明式的方式处理来自消息中间件(如 Kafka、RabbitMQ)的消息,同时提供了丰富的配置选项和功能支持。以下是关于 Spring Cloud Stream Consumer 的详细介绍:
1. 核心功能
Spring Cloud Stream Consumer 提供了以下主要功能:
- 消息消费:通过定义消费者函数或方法,处理来自消息中间件的消息。
- 函数式编程支持:支持使用函数式编程模型(如
Consumer<>
)处理消息,简化代码并减少样板代码。 - 分区消费:支持分区消费机制,通过配置
spring.cloud.stream.bindings.<channelName>.consumer.partitioned
属性,实现消息的负载均衡和并发处理。 - 消息分组:通过配置
spring.cloud.stream.bindings.<channelName>.group
属性,确保同一组内的消费者竞争消费消息,避免重复消费。 - 并发控制:通过
spring.cloud.stream.default.consumer.concurrency
属性,调整消费者的并发度,优化消息处理性能。
2. 使用场景
Spring Cloud Stream Consumer 适用于以下场景:
- 事件驱动架构:处理来自消息队列的事件,如订单创建、支付成功等。
- 实时数据处理:消费实时数据流,如日志处理、监控数据等。
- 分布式系统:在微服务架构中,作为消息的消费者,实现服务间的解耦和异步通信。
3. 配置与使用
3.1 基本配置
在 application.yml
中配置消费者绑定的消息通道:
spring:
cloud:
stream:
bindings:
input: # 消费者通道名称
destination: my-topic # 消息主题
group: my-group # 消费者组
3.2 定义消费者
使用函数式编程模型定义消费者:
@Bean
public Consumer<String> myConsumer() {
return message -> {
System.out.println("Received message: " + message);
};
}
3.3 分区消费
启用分区消费:
spring:
cloud:
stream:
bindings:
input:
consumer:
partitioned: true
3.4 并发控制
调整消费者的并发度:
spring:
cloud:
stream:
default:
consumer:
concurrency: 3
4. 高级特性
4.1 事务管理
在函数式编程模型中,@Transactional
注解不被直接支持。可以通过以下方式实现事务管理:
- 使用支持事务的 Binder(如 Kafka、RabbitMQ)。
- 手动实现事务管理,例如使用
TransactionTemplate
。
4.2 消息分组
通过配置消费者组,确保消息只被一个消费者实例处理:
spring:
cloud:
stream:
bindings:
input:
group: my-group
4.3 错误处理
通过自定义错误处理器,处理消费过程中出现的异常:
@Bean
public Consumer<Message<String>> myConsumer() {
return message -> {
try {
// 处理消息
} catch (Exception e) {
// 处理异常
}
};
}
5. 总结
Spring Cloud Stream Consumer 是构建消息驱动微服务的重要组件,提供了丰富的功能和灵活的配置选项。通过函数式编程模型、分区消费、消息分组和并发控制等特性,开发者可以轻松实现高效、可靠的消息处理逻辑。更多详细信息,可以参考 Spring Cloud Stream 官方文档 和相关示例代码。
如果需要更详细的代码示例或配置说明,可以参考以下资源:
如果有多个消息消费者,那么消息生产者发送的消息会被多个消费者都接收到,这种情况在某些实际场景下是有很大问题的,比如在如下场景中,订单系统做集群部署,都会从 RabbitMQ 中获取订单信息,如果一个订单消息同时被两个服务消费,系统肯定会出现问题。为了避免这种情况,Stream 提供了消息分组来解决该问题。
package com.didispace;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.stream.annotation.rxjava.EnableRxJavaProcessor;
import org.springframework.cloud.stream.annotation.rxjava.RxJavaProcessor;
import org.springframework.context.annotation.Bean;
//@EnableRxJavaProcessor
//@EnableBinding(value = {Processor.class})
public class App1 {
private static Logger logger = LoggerFactory.getLogger(HelloApplication.class);
// @StreamListener(Processor.INPUT)
// @SendTo(Processor.OUTPUT)
// public Object receiveFromInput(Object payload) {
// logger.info("Received: " + payload);
// return "From Input Channel Return - " + payload;
// }
/**原生实现**/
// @ServiceActivator(inputChannel= Processor.INPUT, outputChannel = Processor.OUTPUT)
// public Object receiveFromInput(Object payload) {
// logger.info("Received: " + payload);
// return "From Input Channel Return - " + payload;
// }
/**rxjava实现**/
@Bean
public RxJavaProcessor<String,String> processor() {
return inputStream -> inputStream.map(data -> {
logger.info("Received: " + data);
return data;
}).buffer(5).map(data -> String.valueOf("From Input Channel Return - " + data));
}
}
package com.didispace;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.converter.MessageConverter;
@SpringBootApplication
public class HelloApplication {
public static void main(String[] args) {
SpringApplication.run(HelloApplication.class, args);
}
}
package com.didispace;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;
//@EnableBinding(value = {Sink.class})
public class SinkReceiver1 {
private static Logger logger = LoggerFactory.getLogger(HelloApplication.class);
@StreamListener(Sink.INPUT)
public void receive(Object payload) {
logger.info("Received: " + payload);
}
}
package com.didispace;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;
//@EnableBinding(value = {Sink.class})
public class SinkReceiver2 {
private static Logger logger = LoggerFactory.getLogger(HelloApplication.class);
@StreamListener(Sink.INPUT)
public void receive(User user) {
logger.info("Received: " + user);
}
/**原生实现需要实现转换**/
// @ServiceActivator(inputChannel=Sink.INPUT)
// public void receive(User user) {
// logger.info("Received: " + user);
// }
//
// @Transformer(inputChannel = Sink.INPUT, outputChannel = Sink.INPUT)
// public User transform(String message) throws Exception {
// ObjectMapper objectMapper = new ObjectMapper();
// User user = objectMapper.readValue(message, User.class);
// return user;
// }
}
package com.didispace;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;
import org.springframework.integration.annotation.ServiceActivator;
//@EnableBinding(value = {Sink.class})
public class SinkReceiver3 {
private static Logger logger = LoggerFactory.getLogger(HelloApplication.class);
@StreamListener(Sink.INPUT)
public void receive(User user) {
logger.info("Received: " + user);
}
// @Bean
// @InboundChannelAdapter(value = Sink.INPUT, poller = @Poller(fixedDelay = "2000"))
// public MessageSource<String> timerMessageSource() {
// Map<String, Object> headers = new HashMap<>();
// headers.put("content-type", "application/user");
// return () -> new GenericMessage<>("{\"name\":\"didi\", \"age\":30}", headers);
// }
// Need @Transformer
// @ServiceActivator(inputChannel=Sink.INPUT)
// public void receive(User user) {
// logger.info("Received: " + user);
// }
// @Transformer(inputChannel = Sink.INPUT, outputChannel = Sink.INPUT)
// public User transform(String message) throws Exception {
// ObjectMapper objectMapper = new ObjectMapper();
// User user = objectMapper.readValue(message, User.class);
// return user;
// }
}
package com.didispace;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;
@EnableBinding(value = {Sink.class})
public class SinkReceiver4 {
private static Logger logger = LoggerFactory.getLogger(HelloApplication.class);
@StreamListener(Sink.INPUT)
public void receive(User user) {
logger.info("Received: " + user);
}
// 配置属性如下:
// # Comsumer Group:input
// spring.cloud.stream.bindings.input.group=Service-A
//
// # Partition
// spring.cloud.stream.bindings.input.destination=greetings
// spring.cloud.stream.bindings.input.consumer.partitioned=true
// spring.cloud.stream.instanceCount=2
// spring.cloud.stream.instanceIndex=0
}
package com.didispace;
import java.io.Serializable;
public class User implements Serializable {
private String name;
private Integer age;
public User() {
}
public User(String name, Integer age) {
this.name = name;
this.age = age;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
@Override
public String toString() {
return "name=" + name + ", age=" + age;
}
}
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.didispace</groupId>
<artifactId>stream-consumer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>stream-consumer</name>
<description>Demo project for Spring Boot</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.7.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-stream-rabbit</artifactId>
</dependency>
<!--<dependency>-->
<!--<groupId>org.springframework.cloud</groupId>-->
<!--<artifactId>spring-cloud-stream-binder-kafka</artifactId>-->
<!--</dependency>-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-rxjava</artifactId>
<version>1.0.2.RELEASE</version>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Brixton.SR6</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>