flink streampark初步使用

原创

已于 2024-07-23 09:59:29 修改 · 518 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#flink #大数据

于 2024-07-23 09:57:33 首次发布

1、版本信息

flink：flink-1.18.1-bin-scala_2.12

streampark：apache-streampark_2.12-2.1.4

2、部署flink

解压flink-1.18.1-bin-scala_2.12.tgz

# vi flink-conf.yaml

修改rest.address=0.0.0.0

启动flink

# ./start-cluster.sh

3、部署streampark

...待增加步骤

4、进入streampark

使用Flink Sql

CREATE TABLE t_source (
  id INT,
  data STRING,
  PRIMARY KEY (id) NOT ENFORCED
) WITH (
  'connector' = 'mysql-cdc',
  'hostname' = '192.168.0.2',
  'port' = '8306',
  'table-name' = 't_source',
  'database-name' = '***',
  'username' = '***',
  'password' = '***'
);


CREATE TABLE sink_table (
  id INT,
  data STRING,
  PRIMARY KEY (id) NOT ENFORCED
) WITH (
  'connector' = 'jdbc',
  'url' = 'jdbc:mysql://***:8306/streampark?serverTimezone=UTC&characterEncoding=utf8&useUnicode=true&useSSL=false',
  'table-name' = 't_sink',
  'username' = '***',
  'password' = '***'
);



INSERT INTO sink_table
SELECT id, data
FROM t_source;

作业依赖：

<dependency>

    <groupId>com.ververica</groupId>

    <artifactId>flink-sql-connector-mysql-cdc</artifactId>

    <version>2.4.0</version>

  </dependency>

  <dependency>

    <groupId>mysql</groupId>

    <artifactId>mysql-connector-java</artifactId>

    <version>8.0.11</version>

  </dependency>