Paimon Flink本地读取数据报错

源码挖掘机

已于 2024-08-20 14:40:29 修改

阅读量941

点赞数 7

CC 4.0 BY-SA版权

文章标签： flink 大数据

于 2024-08-20 14:39:36 首次发布

本文链接：https://blog.csdn.net/bigdatakenan/article/details/141357706

1.idea本地读取paimon

用idea在本地读取paimon的表时需要添加的依赖除了官网提出的和hadoop相关的，paimon-flink之类相关的除外还需要其他额外依赖

import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;

public class Test {
    public static void main(String[] args) {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);

        tableEnv.executeSql(
                "CREATE CATALOG my_jdbc WITH (\n" +
                "'type' = 'paimon',\n" +
                "'metastore' = 'jdbc',\n" +
                "'uri' = 'jdbc:mysql://test3:3306/paimon',\n" +
                "'jdbc.user' = 'root',\n" +
                "'jdbc.password' = 'N^57F9m2RI#rp8',\n" +
                "'catalog-key'='jdbc',\n" +
                "'warehouse' = 'hdfs://test1:8020/user/paimon'\n" +
                "                  )");

        tableEnv.executeSql("use catalog my_jdbc");


        tableEnv.sqlQuery("select * from test").execute().print();

    }
}

以上是我的代码，用的jdbc catalog读取hadoop上的数据，下面是最开始的依赖：


        <dependency>
            <groupId>org.apache.paimon</groupId>
            <artifactId>paimon-flink-1.18</artifactId>
            <version>0.8.0</version>
        </dependency>

        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>3.3.6</version>
        </dependency>


        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.28</version>
        </dependency>


        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-core</artifactId>
            <version>1.18.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-table-planner -->
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-planner_2.12</artifactId>
            <version>1.18.0</version>
        </dependency>


        <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-table-api-java -->
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-api-java</artifactId>
            <version>1.18.0</version>
        </dependency>


        <dependency>
            <groupId>org.apache.flink</groupId&