FLUME启动报错ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirect

最新推荐文章于 2024-11-16 12:15:46 发布

呼和浩特-大数据-白胖子

最新推荐文章于 2024-11-16 12:15:46 发布

阅读量3.8k

点赞数

CC 4.0 BY-SA版权

分类专栏： hadoop 文章标签： FLUME

本文链接：https://blog.csdn.net/ddbbff2005/article/details/88428563

hadoop 专栏收录该内容

33 篇文章

订阅专栏

本文详细记录了在使用Apache Flume过程中遇到的字符集问题，解析了错误信息并提供了具体的解决方案，即通过配置文件统一字符集为GBK，避免因默认UTF-8引起的MalformedInputException异常。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

昨晚做项目，一切正常，今天重新执行，报错如下

[ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source source1: { spoolDir: /home/hdp01/apps-log }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
        at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)
        at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)
        at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:90)
        at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:252)
        at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

百度后发现是flume上次文件字符集问题，FLUME默认UTF-8。
解决方法，统一字符集。
下面是配置字符集方法
agent1.sources.source1.inputCharset = GBK