Data人生-CSDN博客

原创 Sqoop将MySQL数据增量导入Hive示例

Sqoop将MySQL数据增量导入Hive示例 #!/bin/bash export sqoop_home=/usr/bin/sqoop if [ $# -eq 1 ] then datestr=$1 else datestr=`date -d '-1 day' +%Y -%m-%d` fi start_time=`date +%Y-%m-%d` year_stat=`date +%Y` month_stat=`date +%m` jdbc_url='jdbc:mysql://ha

2022-04-16 23:06:27 2823

原创 sqoop操作示例

##1、导入mysql到HDFS sqoop import \ --connect jdbc:mysql://172.17.0.202:3306/test --username root --password 123456 \ --table emp --target-dir /sqoop/emp -delete-target-dir -m 1 --fields-terminated-by '\t' sqoop import \ --connect jdbc:mysql://172.17.0.202:3

2022-04-10 15:25:21 260

原创 Flink:Source代码中传参

import org.apache.flink.api.java.operators.DataSource; import org.apache.flink.api.java.utils.ParameterTool; import org.apache.flink.streaming.api.datastream.DataStreamSource; import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; im

2022-02-09 15:03:52 1158

原创 Flink读取hbase的数据然后对数据进行业务操作以后写入到hbase中

import cn.flink.streaming.sink.VehicleDetailSinkFunction; import cn.flink.streaming.source.VechileDetailSourceFunction; import org.apache.flink.api.common.restartstrategy.RestartStrategies; import org.apache.flink.api.java.tuple.Tuple9; import org.apache.f

2021-09-28 15:52:55 1060

原创 flink实时产生的数据流式写入到kafka中

flink实时产生的数据流式写入到kafka中 package cn.itcast.day03.sink; import cn.itcast.day03.source.custom.MyNoParallelSource; import cn.itcast.day03.source.custom.Order; import org.apache.flink.api.common.functions.MapFunction; import org.apache.flink.streaming.api.data

2021-01-11 15:35:01 1413 1

原创将实时产生的数据流式写入到hdfs中

Flink将实时产生的数据流式写入到hdfs中 import cn.itcast.day03.source.custom.MyNoParallelSource; import cn.itcast.day03.source.custom.Order; import org.apache.flink.api.common.serialization.SimpleStringEncoder; import org.apache.flink.core.fs.Path; import org.apache.flink

2021-01-11 10:45:18 1455

原创 Flink消费kafka数据

Flink消费kafka数据 import org.apache.flink.api.common.serialization.SimpleStringSchema; import org.apache.flink.streaming.api.datastream.DataStreamSource; import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; import org.apache.flink.str

2021-01-10 20:34:55 4359

不会写代码