flink--自定义sink

最新推荐文章于 2024-11-07 11:10:09 发布

原创最新推荐文章于 2024-11-07 11:10:09 发布 · 482 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#flink #java

flink 专栏收录该内容

10 篇文章

订阅专栏

本文展示了如何在 Flink 中自定义 JDBC Sink，从文本文件读取数据，转换为 SensorReading 类型，然后将数据写入 MySQL 数据库，实现数据更新与插入操作。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

自定义sink

public class SinkToJDBC {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env =StreamExecutionEnvironment.getExecutionEnvironment();
        DataStreamSource<String> inputStream =
                env.readTextFile("D:\\Tool\\Idea2020\\FlinkTest\\src\\main\\resources\\file.txt");
        //转换类型
        SingleOutputStreamOperator<SensorReading> dataStream = inputStream.map(new MapFunction<String, SensorReading>() {
            @Override
            public SensorReading map(String s) throws Exception {
                String[] splits = s.split(",");
                return new SensorReading(splits[0],new Long(splits[1]),new Double(splits[2]));
            }
        });

        dataStream.addSink(new MyJDBCSink());

        env.execute();

    }
    //自定义sinkfunction
    public static class MyJDBCSink extends RichSinkFunction<SensorReading> {

        Connection conn = null;
        PreparedStatement insertStmt =null;
        PreparedStatement updateStmt =null;
        @Override
        public void open(Configuration parameters) throws Exception {
            conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/***?serverTimezone=Asia/Shanghai","root","****");
            insertStmt=conn.prepareStatement("insert into sensor_temp(id,temp) values(?,?)");
            updateStmt=conn.prepareStatement("update sensor_temp set temp= ? where id = ?");
        }

        @Override
        public void invoke(SensorReading value, Context context) throws Exception {
            //直接执行更新语句
            updateStmt.setDouble(1,value.getTemperature());
            updateStmt.setString(2,value.getId());
            updateStmt.execute();
            //if failed
            if(updateStmt.getUpdateCount() == 0){
                insertStmt.setDouble(2,value.getTemperature());
                insertStmt.setString(1,value.getId());
                insertStmt.execute();
            }
        }

        @Override
        public void close() throws Exception {
            insertStmt.close();
            updateStmt.close();
            conn.close();
        }
    }

answer:
在这里插入图片描述