1、解压复制文件
[centos@hadoop10 module]$ mv kafka_2.12-3.0.0/ /data/module/kafka
[centos@hadoop10 kafka]$ vim ./config/server.properties
设置文件目录
[centos@hadoop10 kafka]$ mkdir datas
2、更改配置目录
路径 :data/module/kafka/config/server.properties
log.dirs=/data/module/kafka/datasck
zookeeper.connect=hadoop10:2181,hadoop11:2181,hadoop12:2181/kafka
num.partitions=1 //区分机器
3、设置环境变量
[centos@hadoop10 bin]$ vim /etc/profile.d/my_env.sh
KAFKA_HOME
export KAFKA_HOME=/data/module/kafka
export PATH=$PATH:$KAFKA_HOME/bin
4、分发配置
xsync kafka
5、批量启停文件
[atguigu@hadoop102 bin]$ vim kf.sh
脚本如下:
#! /bin/bash
case $1 in
"start"){
for i in hadoop10 hadoop11 hadoop12
do
echo " --------启动 $i Kafka-------"
ssh $i "/data/module/kafka/bin/kafka-server-start.sh -daemon /data/module/kafka/config/server.properties"
done
};;
"stop"){
for i in hadoop10 hadoop11 hadoop12
do
echo " --------停止 $i Kafka-------"
ssh $i "/data/module/kafka/bin/kafka-server-stop.sh "
done
};;
esac
6、问题
可能出现的第一种错误发生以下错误:
[2023-08-19 08:21:00,955] INFO [ReplicaManager broker=0] Shut down completely (kafka.server.ReplicaManager)
[2023-08-19 08:21:00,955] INFO [BrokerToControllerChannelManager broker=0 name=alterIsr]: Shutting down (kafka.server.BrokerToControllerRequestThread)
[2023-08-19 08:21:00,955] INFO [BrokerToControllerChannelManager broker=0 name=alterIsr]: Stopped (kafka.server.BrokerToControllerRequestThread)
[2023-08-19 08:21:00,956] INFO [BrokerToControllerChannelManager broker=0 name=alterIsr]: Shutdown completed (kafka.server.BrokerToControllerRequestThread)
[2023-08-19 08:21:00,963] INFO Broker to controller channel manager for alterIsr shutdown (kafka.server.BrokerToControllerChannelManagerImpl)
[2023-08-19 08:21:00,964] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Shutting down (kafka.server.BrokerToControllerRequestThread)
[2023-08-19 08:21:00,965] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Stopped (kafka.server.BrokerToControllerRequestThread)
[2023-08-19 08:21:00,965] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Shutdown completed (kafka.server.BrokerToControllerRequestThread)
[2023-08-19 08:21:00,966] INFO Broker to controller channel manager for forwarding shutdown (kafka.server.BrokerToControllerChannelManagerImpl)
[2023-08-19 08:21:00,966] INFO Shutting down. (kafka.log.LogManager)
[2023-08-19 08:21:00,993] INFO Shutdown complete. (kafka.log.LogManager)
[2023-08-19 08:21:00,995] INFO [feature-zk-node-event-process-thread]: Shutting down (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2023-08-19 08:21:01,008] INFO [feature-zk-node-event-process-thread]: Stopped (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2023-08-19 08:21:01,009] INFO [feature-zk-node-event-process-thread]: Shutdown completed (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2023-08-19 08:21:01,010] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2023-08-19 08:21:01,013] WARN An exception was thrown while closing send thread for session 0x1000069232f0004. (org.apache.zookeeper.ClientCnxn)
EndOfStreamException: Unable to read additional data from server sessionid 0x1000069232f0004, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2023-08-19 08:21:01,115] INFO Session: 0x1000069232f0004 closed (org.apache.zookeeper.ZooKeeper)
[2023-08-19 08:21:01,115] INFO EventThread shut down for session: 0x1000069232f0004 (org.apache.zookeeper.ClientCnxn)
[2023-08-19 08:21:01,115] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
原因 kafka server.properties配置配做 broker.id=1 broker.id=2 清空/datas
第二种错误:
could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient
关闭防火墙 systemctl stop firewalld