paimon-03 Catalog

Posted by 蓝染君

On 2023年 9月 28日

已关闭评论

CREATE CATALOG fs_catalog WITH (
    'type' = 'paimon',
    'warehouse' = 'hdfs://mycluster/paimon/fs'
);

Hive Catalog

启动hive metastore

nohup hive --service metastore &

创建hive catalog

CREATE CATALOG hive_catalog WITH (
    'type' = 'paimon',
    'metastore' = 'hive',
    'uri' = 'thrift://hadoop001:9083',
    'hive-conf-dir' = '/data/hive-3.1.2/conf',
    'warehouse' = 'hdfs://mycluster/paimon/hive'
);

注意事项

使用hive catalog通过alter table更改不兼容的列类型时需要配置

hive.metastore.disallow.inknown.col.type.changes=false;

如果使用hive3 请禁用hive acid

hive.strict.managed.tables=false;
hive.create.as.insert.only=false;
metastore.create.as.acid=false;

paimon中可以看到hive的库名但是看不到hive表

sql-client初始化脚本

CREATE CATALOG fs_catalog WITH (
    'type' = 'paimon',
    'warehouse' = 'hdfs://mycluster/paimon/fs'
);

CREATE CATALOG hive_catalog WITH (
    'type' = 'paimon',
    'metastore' = 'hive',
    'uri' = 'thrift://hadoop001:9083',
    'hive-conf-dir' = '/data/hive-3.1.2/conf',
    'warehouse' = 'hdfs://mycluster/paimon/hive'
);

USE CATALOG hive_catalog;

SET 'sql-client.execution.result-mode'='tableau';

执行sql-client

bin/sql-client.sh -s yarn-session -i conf/sql-client-init.sql

Blog

About 蓝染君