kafka-03 Kafka快速入门

集群规划

hadoop001	hadoop002	hadoop003
zk	zk	zk
kafka	kafka	kafka

安装部署

下载jar包，解压到/opt/module并重命名为kafka

编辑配置文件 conf/server.properties

broker.id=0  # 设置每台broker服务器的id 唯一值
delete.topic.enable=true
log.dirs=/opt/module/kafka-2.4.1/data  # 设置kafka数据存储路径
log.retention.hours=168  # 数据保留时长
log.segment.bytes=1073741824  # 每个文件的大小
# zookeeper集群
zookeeper.connect=hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka

配置环境变量

export KAFKA_HOME=/opt/module/kafka-2.4.1
export PATH=$PATH:$KAFKA_HOME/bin
# 环境生效
source /etc/profile
# 分发配置
xsync kafka-2.4.1/

启动zookeeper

#!/bin/bash

case $1 in
"start"){
        for i in hadoop001 hadoop002 hadoop003
        do
        echo ---------- zookeeper $i 启动 ------------
                ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh start"
        done
};;
"stop"){
        for i in hadoop001 hadoop002 hadoop003
        do
        echo ---------- zookeeper $i 停止 ------------
                ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh stop"
        done
};;
"status"){
        for i in hadoop001 hadoop002 hadoop003
        do
        echo ---------- zookeeper $i 状态 ------------
                ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh status"
        done
};;
esac

启动kafka

#!/bin/bash

case $1 in
"start"){
    for i in hadoop001 hadoop002 hadoop003
    do
        echo " --------启动 $i Kafka-------"
        ssh $i "/opt/module/kafka-2.4.1/bin/kafka-server-start.sh -daemon /opt/module/kafka-2.4.1/config/server.properties"
    done
};;
"stop"){
    for i in hadoop001 hadoop002 hadoop003
    do
        echo " --------停止 $i Kafka-------"
        ssh $i "/opt/module/kafka-2.4.1/bin/kafka-server-stop.sh stop"
    done
};;
esac

启动日志文件logs/server.log

Blog

kafka-03 Kafka快速入门