Paimon, 大数据

paimon-02 环境准备

env.java.opts.all: -Dfile.encoding=UTF-8
classloader.check-leaked-classloader: false

taskmanager.numberOfTaskSlots: 4

execution.checkpointing.interval: 10s
state.backend.type: rocksdb
state.checkpoints.dir: hdfs://mycluster/chk
state.savepoints.dir: hdfs://mycluster/sp
state.backend.incremental: true

上传jar包

paimon-flink-1.17-0.5-20230829.001915-137.jar
paimon-flink-action-0.5-20230829.001915-123.jar
paimon-hive-connector-3.1-0.5-20230829.001915-135.jar
paimon-spark-3.1-0.5-20230829.001915-135.jar

启动flink

解决包依赖问题

cp /data/hadoop-3.1.3/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.3.jar /data/flink-1.17.1/lib

yarn-session模式启动

bin/yarn-session.sh -d

启动sql-client客户端

bin/sql-client -s yarn-session

设置显示结果模式

SET 'sql-client.execution.result-mode'='tableau';

About 蓝染君

喜爱编程开发的程序猿