一、问题背景:
发现hadoop集群里4个节点的数据不均衡,通过查询datanode日志发现具体报错如下:
ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-64296 6697-10.180.200.21-1692629878350 (Datanode Uuid 6dc7aae0-583f-40b2-a1db-a8c248e9d34f) service to 10.180.200.21/10.180.200.21 :9871 Datanode denied communication with namenode because hostname cannot be resolved (ip=10.180.200.24, hostname=10.180.200 .24): DatanodeRegistration(0.0.0.0:9866, datanodeUuid=6dc7aae0-583f-40b2-a1db-a8c248e9d34f, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-056a7166-f12d-4a61-b1df-c6a010cfd618;nsid=776942410;c=1692629878350)
二、解决步骤
- 编辑/etc/hosts文件,将各个节点的hostname 和 ip写进去,比如:
127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 10.180.200.21 yourhostname0 10.180.200.22 yourhostname1 10.180.200.23 yourhostname2 10.180.200.24 yourhostname3
2.重新布置hadoop
stop-dfs.sh rm -rf /**/your_hadoop_home_data/* hdfs namenode -format start-dfs.sh
3.再次查看datanode节点已经不报错了
4.再次查看各个节点的数据分布情况,可以看到数据已经均衡了
5.完结。