2024-06-05 13:56:00,895 - kafka.conn - INFO - Probing node bootstrap-0 broker version 2024-06-05 13:56:00,895 - kafka.conn - INFO - <BrokerConnection node_id=bootstrap-0 host=11.11.22.158:9092 <connecting> [IPv4 ('11.11.22.158', 9092)]>: Connection complete. 2024-06-05 13:56:00,996 - kafka.conn - ERROR - <BrokerConnection node_id=bootstrap-0 host=11.11.22.158:9092 <connected> [IPv4 ('11.11.22.158', 9092)]>: socket disconnected 2024-06-05 13:56:00,997 - kafka.conn - INFO - <BrokerConnection node_id=bootstrap-0 host=11.11.22.158:9092 <connected> [IPv4 ('11.11.22.158', 9092)]>: Closing connection. KafkaConnectionError: socket disconnected 2024-06-05 13:56:00,997 - kafka.conn - INFO - Broker is not v(0, 10) -- it did not recognize ApiVersionRequest_v0 2024-06-05 13:56:00,998 - kafka.conn - INFO - <BrokerConnection node_id=bootstrap-0 host=11.11.22.158:9092 <connecting> [IPv4 ('11.11.22.158', 9092)]>: connecting to 11.11.22.158:9092 [('11.11.22.158', 9092) IPv4] 2024-06-05 13:56:00,999 - kafka.conn - INFO - <BrokerConnection node_id=bootstrap-0 host=11.11.22.158:9092 <connecting> [IPv4 ('11.11.22.158', 9092)]>: Connection complete. 2024-06-05 13:56:01,210 - kafka.conn - INFO - Broker version identified as 0.9 2024-06-05 13:56:01,210 - kafka.conn - INFO - Set configuration api_version=(0, 9) to skip auto check_version requests on startup 2024-06-05 13:56:01,212 - kafka.coordinator.consumer - WARNING - group_id is None: disabling auto-commit. 2024-06-05 13:56:01,212 - kafka.consumer.subscription_state - INFO - Updating subscribed topics to: ('N3S_EHDB_insure_KFK',) 2024-06-05 13:56:01,212 - root - INFO - start ... 2024-06-05 13:56:01,214 - kafka.consumer.subscription_state - INFO - Updated partition assignment: [TopicPartition(topic='N3S_EHDB_insure_KFK', partition=0)] 2024-06-05 13:56:01,216 - kafka.conn - INFO - <BrokerConnection node_id=136 host=11.11.22.143:9092 <connecting> [IPv4 ('11.11.22.143', 9092)]>: connecting to 11.11.22.143:9092 [('11.11.22.143', 9092) IPv4] 2
从您提供的日志信息来看,这是一个与Kafka客户端和Kafka集群交互的日志。下面是对日志中关键部分的解释:
探测Kafka节点版本
2024-06-05 13:56:00,895 - kafka.conn - INFO - Probing node bootstrap-0 broker version
客户端正在尝试探测bootstrap-0这个Kafka broker的版本。
连接建立
2024-06-05 13:56:00,895 - kafka.conn - INFO - <BrokerConnection ...>: Connection complete.
客户端成功连接到了Kafka broker。
连接断开
2024-06-05 13:56:00,996 - kafka.conn - ERROR - <BrokerConnection ...>: socket disconnected
...
2024-06-05 13:56:00,997 - kafka.conn - INFO - <BrokerConnection ...>: Closing connection. KafkaConnectionError: socket disconnected
客户端在尝试与broker交互时,socket连接断开了。这可能是由于多种原因造成的,比如网络问题、broker配置问题或客户端配置问题等。
版本识别问题
2024-06-05 13:56:00,997 - kafka.conn - INFO - Broker is not v(0, 10) -- it did not recognize ApiVersionRequest_v0
客户端试图发送一个ApiVersionRequest_v0请求来确定broker支持的API版本,但broker没有识别这个请求。这通常意味着客户端和broker之间的版本不兼容。
重新连接与版本识别
...
2024-06-05 13:56:01,210 - kafka.conn - INFO - Broker version identified as 0.9
...
客户端重新连接后,成功识别出broker的版本为0.9。然后,客户端设置了相应的配置来跳过在启动时自动检查版本。
消费者组ID警告
2024-06-05 13:56:01,212 - kafka.coordinator.consumer - WARNING - group_id is None: disabling auto-commit.
这里警告说,消费者组ID (group_id) 是None,因此禁用了自动提交。在Kafka中,消费者组是多个消费者实例的集合,它们共同消费一个或多个主题的数据。如果没有指定group_id,消费者实例将不会属于任何消费者组,并且不会自动提交其消费的偏移量。
更新订阅的主题
2024-06-05 13:56:01,212 - kafka.consumer.subscription_state - INFO - Updating subscribed topics to: ('N',)
客户端更新了其订阅的主题列表,只包含一个名为N3S的主题。
分区分配
2024-06-05 13:56:01,214 - kafka.consumer.subscription_state - INFO - Updated partition assignment: [TopicPartition(topic='N3S_EHDB_insure_KFK', partition=0)]
消费者客户端被分配了主题N3S_EHDB_insure_KFK的分区0进行消费。
连接到另一个broker
2024-06-05 13:56:01,216 - kafka.conn - INFO - <BrokerConnection node_id=136 host=11.11.22.143:9092 <connecting> [IPv4 ('11.11.22.143', 9092)]>: connecting to 11.11.22.143:9092 [('11.11.22.143', 9092) IPv4]
客户端正在尝试连接到ID为136的另一个Kafka broker(位于11.11.22.143:9092)。