OpenTelemetry学习笔记(八)：模拟从 Kafka 读取日志、解析日志、转换为 OpenTelemetry Trace 数据，并导出到 Elastic APM展示

以下是一个完整的 生产级 Java 代码示例，模拟从 Kafka 读取日志、解析日志、转换为 OpenTelemetry Trace 数据，并导出到 Elastic APM（Elastic Observability）的实现。

1. 项目结构

src/
├── main/
│   ├── java/
│   │   ├── KafkaLogConsumer.java       # Kafka 消费者
│   │   ├── LogParser.java             # 日志解析工具
│   │   ├── OpenTelemetryExporter.java # OpenTelemetry 初始化与导出
│   │   └── Main.java                  # 启动类
│   └── resources/
│       └── log4j2.xml                 # 日志配置（可选）
└── pom.xml                           # Maven 依赖

2. Maven 依赖 (pom.xml)

<dependencies>
    <!-- Kafka Client -->
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>3.6.1</version>
    </dependency>

    <!-- OpenTelemetry SDK & OTLP Exporter -->
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-sdk</artifactId>
        <version>1.34.0</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-exporter-otlp</artifactId>
        <version>1.34.0</version>
    </dependency>

    <!-- Logging (可选) -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.23.1</version>
    </dependency>
</dependencies>

3. Kafka 消费者 (KafkaLogConsumer.java)

import org.apache.kafka.clients.consumer.*;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.time.Duration;
import java.util.Collections;
import java.util.Properties;

public class KafkaLogConsumer {
    private static final String KAFKA_BOOTSTRAP_SERVERS = "localhost:9092";
    private static final String KAFKA_TOPIC = "bank-transaction-logs";
    private static final String KAFKA_GROUP_ID = "log-processor-group";

    private final KafkaConsumer<String, String> consumer;
    private final LogParser logParser;
    private final OpenTelemetryExporter exporter;

    public KafkaLogConsumer() {
        // 1. 初始化 Kafka 消费者
        Properties props = new Properties();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_BOOTSTRAP_SERVERS);
        props.put(ConsumerConfig.GROUP_ID_CONFIG, KAFKA_GROUP_ID);
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");
        props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false"); // 手动提交偏移量

        this.consumer = new KafkaConsumer<>(props);
        this.logParser = new LogParser();
        this.exporter = new OpenTelemetryExporter();
    }

    /**
     * 启动 Kafka 消费者并处理日志
     */
    public void startConsuming() {
        consumer.subscribe(Collections.singletonList(KAFKA_TOPIC));
        System.out.println("Started consuming logs from Kafka topic: " + KAFKA_TOPIC);

        try {
            while (true) {
                ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
                for (ConsumerRecord<String, String> record : records) {
                    String logLine = record.value();
                    System.out.println("Processing log: " + logLine);

                    // 2. 解析日志并生成 Trace 数据
                    exporter.exportLogAsTrace(logLine);

                    // 3. 手动提交偏移量（确保日志处理成功）
                    consumer.commitSync();
                }
            }
        } finally {
            consumer.close();
            exporter.shutdown();
        }
    }
}

4. 日志解析工具 (LogParser.java)

import java.util.HashMap;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class LogParser {
    private static final Pattern LOG_PATTERN = Pattern.compile("([\\w]+)[:=]([^|]+)");

    /**
     * 解析日志行到 Map
     */
    public Map<String, String> parseLog(String logLine) {
        Map<String, String> logData = new HashMap<>();
        Matcher matcher = LOG_PATTERN.matcher(logLine);

        while (matcher.find()) {
            String key = matcher.group(1).trim();
            String value = matcher.group(2).trim();
            logData.put(key, value);
        }

        return logData;
    }

    /**
     * 解析嵌套字段（如 SUBED:StartTime=...）
     */
    public Map<String, String> parseNestedFields(Map<String, String> logData, String prefix) {
        Map<String, String> nestedFields = new HashMap<>();
        for (String key : logData.keySet()) {
            if (key.startsWith(prefix + ":")) {
                String nestedKey = key.substring(prefix.length() + 1);
                nestedFields.put(nestedKey, logData.get(key));
            }
        }
        return nestedFields;
    }
}

5. OpenTelemetry 导出器 (OpenTelemetryExporter.java)

import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.context.Scope;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.BatchSpanProcessor;
import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter;
import io.opentelemetry.semconv.resource.attributes.ResourceAttributes;

import java.time.Instant;
import java.util.Map;

public class OpenTelemetryExporter {
    private final OpenTelemetry openTelemetry;
    private final Tracer tracer;

    public OpenTelemetryExporter() {
        // 1. 初始化 OpenTelemetry SDK
        Resource resource = Resource.getDefault()
            .merge(Resource.create(Attributes.of(
                ResourceAttributes.SERVICE_NAME, "bank-transaction-service",
                ResourceAttributes.DEPLOYMENT_ENVIRONMENT, "production"
            )));

        // 2. 配置 OTLP Exporter（导出到 Elastic APM）
        OtlpGrpcSpanExporter exporter = OtlpGrpcSpanExporter.builder()
            .setEndpoint("http://localhost:8200") // Elastic APM Server
            .build();

        // 3. 设置 BatchSpanProcessor（批量处理 Span）
        SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
            .addSpanProcessor(BatchSpanProcessor.builder(exporter).build())
            .setResource(resource)
            .build();

        this.openTelemetry = OpenTelemetrySdk.builder()
            .setTracerProvider(tracerProvider)
            .build();

        this.tracer = openTelemetry.getTracer("bank-log-processor");
    }

    /**
     * 将日志转换为 Trace 并导出
     */
    public void exportLogAsTrace(String logLine) {
        LogParser parser = new LogParser();
        Map<String, String> logData = parser.parseLog(logLine);
        String label = logData.get("Lable"); // START/END/SUBED

        if ("START".equals(label)) {
            // 4. 创建主 Span
            String transName = logData.get("TransName");
            String transId = logData.get("TransID");
            long startTime = parseTimestamp(logData.get("TimeStamp"));

            Span mainSpan = tracer.spanBuilder(transName != null ? transName : "unknown_transaction")
                .setStartTimestamp(startTime)
                .setAttribute("transaction.id", transId)
                .setAttribute("transaction.code", logData.get("TransCode"))
                .setAttribute("business.date", logData.get("Date"))
                .setAttribute("business.branch", logData.get("JiGou"))
                .setAttribute("business.teller", logData.get("Teller"))
                .setAttribute("business.customer", logData.get("FaRen"))
                .setAttribute("form.id", logData.get("FormId"))
                .startSpan();

            try (Scope scope = mainSpan.makeCurrent()) {
                // 5. 处理子事务（如 SUBED 日志）
                if (logData.containsKey("SUBED:StartTime")) {
                    processSubTransaction(logData);
                }
            } finally {
                mainSpan.end();
            }
        }
    }

    /**
     * 处理子事务（SUBED 日志）
     */
    private void processSubTransaction(Map<String, String> logData) {
        LogParser parser = new LogParser();
        Map<String, String> subFields = parser.parseNestedFields(logData, "SUBED");
        long startTime = parseTimestamp(subFields.get("StartTime"));
        long endTime = parseTimestamp(subFields.get("EndTime"));

        Span subSpan = tracer.spanBuilder("sub_transaction")
            .setStartTimestamp(startTime)
            .setAttribute("transaction.code", subFields.get("TransCode"))
            .setAttribute("duration_ms", endTime - startTime)
            .setAttribute("status.message", subFields.get("ResDes"))
            .startSpan();

        try (Scope scope = subSpan.makeCurrent()) {
            // 模拟子事务逻辑
        } finally {
            subSpan.end(endTime);
        }
    }

    /**
     * 解析时间戳（支持多种格式）
     */
    private long parseTimestamp(String timestamp) {
        try {
            if (timestamp == null) {
                return System.currentTimeMillis();
            }
            if (timestamp.contains("-")) { // ISO8601 格式
                return Instant.parse(timestamp).toEpochMilli();
            } else { // 原始长整型时间戳（如 20250710092656478）
                return Instant.parse(
                    timestamp.substring(0, 4) + "-" + timestamp.substring(4, 6) + "-" + timestamp.substring(6, 8) + "T" +
                    timestamp.substring(8, 10) + ":" + timestamp.substring(10, 12) + ":" + timestamp.substring(12, 14) + "." + timestamp.substring(14) + "Z"
                ).toEpochMilli();
            }
        } catch (Exception e) {
            return System.currentTimeMillis();
        }
    }

    /**
     * 关闭 OpenTelemetry SDK
     */
    public void shutdown() {
        if (openTelemetry instanceof OpenTelemetrySdk) {
            ((OpenTelemetrySdk) openTelemetry).getTracerProvider().close();
        }
    }
}

6. 启动类 (Main.java)

public class Main {
    public static void main(String[] args) {
        // 1. 启动 Kafka 消费者
        KafkaLogConsumer consumer = new KafkaLogConsumer();
        consumer.startConsuming();
    }
}

7. 运行与验证

1. 启动 Kafka

docker-compose up -d kafka

（或本地安装 Kafka）

2. 启动 Elastic APM Server

docker run -d --name elastic-apm -p 8200:8200 docker.elastic.co/apm/apm-server:8.12.0

3. 发送测试日志到 Kafka

kafka-console-producer --topic bank-transaction-logs --bootstrap-server localhost:9092

输入测试日志：

Lable:START|TimeStamp:20250710092656478|TransID:123|TransCode:ceipt|TransName:个人定期一本通支取|Date:20250710|JiGou:321027026|Teller:321027026210|FaRen:101|FormId:STD16770
Lable:SUBED|SUBED:StartTime=2025-07-10 09:25:49.579|SUBED:EndTime=2025-07-10 09:25:49.602|SUBED:TransCode=21000000000019|SUBED:ResDes=业务脚本处理成功
Lable:END|TimeStamp:20250710092657910|TransID:123|TransCode:ceipt|ResCode:000000|ResDes:成功|UseTime:160309ms

4. 在 Kibana 中查看 Trace

1. 访问 http://localhost:5601/app/apm/services。
2. 选择 bank-transaction-service。
3. 查看 Trace 详情，确认主子事务关联正确。

8. 关键优化点

1. Kafka 消费者优化：

   • 使用 enable.auto.commit=false + commitSync() 确保日志处理成功后再提交偏移量。
   • 增加 max.poll.records 和 max.poll.interval.ms 配置。

2. OpenTelemetry 优化：

   • 使用 BatchSpanProcessor 减少网络开销。
   • 配置 Resource 属性（如 service.name、deployment.environment）。

3. 错误处理：

   • 捕获 Kafka 消费异常（ConsumerRebalanceListener）。
   • 捕获 OpenTelemetry 导出异常（如网络问题）。

4. 日志监控：

   • 使用 Log4j2 记录处理日志（如 log.info("Processed log: {}", logLine)）。

9. 总结

• Kafka 消费 → 日志解析 → OpenTelemetry Trace 生成 → Elastic APM 可视化。
• 适用于生产环境，支持高吞吐、错误处理和可观测性。
• 可扩展为 Flink/Spark Streaming 处理大规模日志。

如果需要进一步优化（如 Kafka 消费者并行处理、OpenTelemetry 采样策略），可以在此基础上扩展。