AI 助理
备案控制台
开发者社区 大数据 文章 正文

Hadoop集群配置https实战案例

2024-09-04 73
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
简介： 本文提供了一个实战案例，详细介绍了如何在Hadoop集群中配置HTTPS，包括生成私钥和证书文件、配置keystore和truststore、修改hdfs-site.xml和ssl-client.xml文件，以及重启Hadoop集群的步骤，并提供了一些常见问题的故障排除方法。

作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。

一.生成私钥及证书文件并拷贝到Hadoop节点

1>.生成私钥和证书文件

[root@hadoop101.yinzhengjie.com ~]# ll
total 4
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# openssl req -new -x509 -keyout hdfs_ca_key -out hdfs_ca_cert -days 36500 -subj '/C=CN/ST=beijing/L=changping/O=bigdata/OU=devops/CN=yinzhengjie.com'
Generating a 2048 bit RSA private key
..+++
..............................................................................................+++
writing new private key to 'hdfs_ca_key'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# openssl req -new -x509 -keyout hdfs_ca_key -out hdfs_ca_cert -days 36500 -subj '/C=CN/ST=beijing/L=changping/O=bigdata/OU=devops/CN=yinzhengjie.com'

2>.将私钥和自签证书拷贝到Hadoop集群节点

[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_cert dest=${HADOOP_HOME}/etc/hadoop/conf"
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.21-12128-10738301489038/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.25-12130-180011782497170/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.22-12125-24658832547026/source", 
    "state": "file", 
    "uid": 0
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.18-12127-102283570422989/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.25-12129-1128583651866/source", 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_cert dest=${HADOOP_HOME}/etc/hadoop/conf"

[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_key dest=${HADOOP_HOME}/etc/hadoop/conf"
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.65-12397-87419418248032/source", 
    "state": "file", 
    "uid": 0
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.64-12396-52377313403452/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.63-12394-76433468569818/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.66-12400-246529012746559/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.65-12399-1022475281042/source", 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_key dest=${HADOOP_HOME}/etc/hadoop/conf"

[root@hadoop101.yinzhengjie.com ~]# ansible all -m shell -a "ls -l ${HADOOP_HOME}/etc/hadoop/conf"
hadoop104.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
hadoop102.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
hadoop101.yinzhengjie.com | CHANGED | rc=0 >>
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
-rw-r--r-- 1 root root  115 Aug 13 18:55 host-rack.txt
-rwxr-xr-x 1 root root  463 Aug 13 18:54 toplogy.py
hadoop105.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
hadoop103.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m shell -a "ls -l ${HADOOP_HOME}/etc/hadoop/conf"

二.生成"keystore"和"trustores"文件

1>.生成keystore文件

[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias localhost -validity 36500 -genkey -keyalg RSA -keysize 2048 -dname "CN=hadoop101.yinzhengjie.com, OU=devops, O=bigdata,
 L=changping, ST=beijing, C=CN"Enter keystore password:  
Re-enter new password: 
Enter key password for <localhost>
    (RETURN if same as keystore password):  
Re-enter new password: 

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore -destkeystore ke
ystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 16
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
[root@hadoop101.yinzhengjie.com ~]#

keytool -keystore keystore -alias localhost -validity 36500 -genkey -keyalg RSA -keysize 2048 -dname "CN=hadoop101.yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN"

2>.生成truststore文件同时导入CA证书 

[root@hadoop101.yinzhengjie.com ~]# ll
total 16
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# keytool -keystore truststore -alias CARoot -import -file hdfs_ca_cert
Enter keystore password:  
Re-enter new password: 
Owner: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Issuer: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Serial number: 858ca3893e608eba
Valid from: Tue Oct 06 20:22:09 CST 2020 until: Thu Sep 12 20:22:09 CST 2120
Certificate fingerprints:
     MD5:  03:6C:16:3E:83:2A:15:79:11:3C:32:3F:12:3B:28:8C
     SHA1: CE:D0:D5:0C:93:FE:06:FB:87:D4:3A:22:B0:21:51:20:27:8F:81:B7
     SHA256: 25:3B:29:F4:16:FA:98:46:02:79:38:6B:3C:DA:7B:44:2F:9A:1B:21:F5:E2:AD:A1:48:93:12:FC:AC:97:0A:8A
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore truststore -alias CARoot -import -file hdfs_ca_cert

3>.从 keystore 中导出 cert

[root@hadoop101.yinzhengjie.com ~]#  ll
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# keytool -certreq -alias localhost -keystore keystore -file cert
Enter keystore password:  

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore -destkeystore ke
ystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 24
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -certreq -alias localhost -keystore keystore -file cert

4>.用 CA 对 cert 签名，生成自签证书

[root@hadoop101.yinzhengjie.com ~]# ll
total 24
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# openssl x509 -req -CA hdfs_ca_cert -CAkey hdfs_ca_key -in cert -out cert_signed -days 36500 -CAcreateserial
Signature ok
subject=/C=CN/ST=beijing/L=changping/O=bigdata/OU=devops/CN=hadoop101.yinzhengjie.com
Getting CA Private Key
Enter pass phrase for hdfs_ca_key:
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 32
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1241 Oct  6 21:34 cert_signed
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root   17 Oct  6 21:34 hdfs_ca_cert.srl
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# openssl x509 -req -CA hdfs_ca_cert -CAkey hdfs_ca_key -in cert -out cert_signed -days 36500 -CAcreateserial

5>.将 CA 的 cert 和导入 keystore文件

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias CARoot -import -file hdfs_ca_cert
Enter keystore password:  
Owner: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Issuer: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Serial number: 858ca3893e608eba
Valid from: Tue Oct 06 20:22:09 CST 2020 until: Thu Sep 12 20:22:09 CST 2120
Certificate fingerprints:
     MD5:  03:6C:16:3E:83:2A:15:79:11:3C:32:3F:12:3B:28:8C
     SHA1: CE:D0:D5:0C:93:FE:06:FB:87:D4:3A:22:B0:21:51:20:27:8F:81:B7
     SHA256: 25:3B:29:F4:16:FA:98:46:02:79:38:6B:3C:DA:7B:44:2F:9A:1B:21:F5:E2:AD:A1:48:93:12:FC:AC:97:0A:8A
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format us
ing "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias CARoot -import -file hdfs_ca_cert

6>.用 CA 自签名之后的 cert 也导入 keystore文件

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias localhost -import -file cert_signed
Enter keystore password:  
Certificate reply was installed in keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format us
ing "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias localhost -import -file cert_signed

7>.将keystore,trustores存入到自定义目录，便于配置时方便找到它们

[root@hadoop101.yinzhengjie.com ~]# ll  ${HADOOP_HOME}/etc/hadoop/conf
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
-rw-r--r-- 1 root root  115 Aug 13 18:55 host-rack.txt
-rwxr-xr-x 1 root root  463 Aug 13 18:54 toplogy.py
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 36
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1241 Oct  6 21:34 cert_signed
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root   17 Oct  6 21:34 hdfs_ca_cert.srl
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 4181 Oct  6 21:42 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# cp keystore truststore  ${HADOOP_HOME}/etc/hadoop/conf
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll  ${HADOOP_HOME}/etc/hadoop/conf
total 32
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
-rw-r--r-- 1 root root  115 Aug 13 18:55 host-rack.txt
-rw-r--r-- 1 root root 4181 Oct  6 21:47 keystore
-rwxr-xr-x 1 root root  463 Aug 13 18:54 toplogy.py
-rw-r--r-- 1 root root 1016 Oct  6 21:47 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# cp keystore truststore ${HADOOP_HOME}/etc/hadoop/conf

8>.在集群其它节点重复[1-7]的所有步骤

　　温馨提示:  
　　　　注意观察上面的每一个步骤，比如FQDN，每个节点对应的主机名并不一致哟~

三.修改"hdfs-site.xml"配置文件

1>.修改"hdfs-site.xml"文件内容

[root@hadoop101.yinzhengjie.com ~]# vim /yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml 
<configuration>
 ...
　　<property>
　　　　<name>dfs.http.policy</name>
　　　　<value>HTTPS_ONLY</value>
　　　　<description>确定HDFS是否支持HTTPS(SSL)这将为HDFS守护程序配置HTTP端点:默认值为"HTTP_ONLY"(仅在http上提供服务),"HTTPS_ONLY"(仅在https上提供服务),"HTTP_AND_HTTPS"(同时提供服务在http和https上)</description>
　　</property>
...
</configuration>
[root@hadoop101.yinzhengjie.com ~]# 


温馨提示:
　　DataNode节点需要将"dfs.http.policy"的属性设置未"HTTPS_ONLY"，但是NameNode和Secondary NameNode需要将该值设置为"HTTP_AND_HTTPS"。
　　如果你将HDFS集群的所有节点的属性都设置成"HTTPS_ONLY",你会发现NameNode和Secondary NameNode的进程可以启动成功，但是它们的WebUI实例启动失败(也就是说你不能访问NameNode的50070端口以及Secondary NameNode的50090端口啦~)！

2>.将配置同步到集群其它datanode节点

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/hdfs-site.xml dest=${HADOOP_HOME}/etc/hadoop/"
hadoop101.yinzhengjie.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 190, 
    "group": "systemd-journal", 
    "mode": "0644", 
    "owner": "12334", 
    "path": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "size": 11470, 
    "state": "file", 
    "uid": 12334
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.64-14771-240997395125955/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.67-14775-126005848371918/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.62-14769-272249870401292/source", 
    "state": "file", 
    "uid": 0
}
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.64-14772-148620330070479/source", 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/hdfs-site.xml dest=${HADOOP_HOME}/etc/hadoop/"

四.配置ssl-client.xml文件

1>.基于模板文件创建ssl-client.xml文件

[root@hadoop101.yinzhengjie.com ~]# cp ${HADOOP_HOME}/etc/hadoop/ssl-client.xml.example ${HADOOP_HOME}/etc/hadoop/ssl-client.xml

2>.修改ssl-client.xml文件

[root@hadoop101.yinzhengjie.com ~]# vim ${HADOOP_HOME}/etc/hadoop/ssl-client.xml
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# cat ${HADOOP_HOME}/etc/hadoop/ssl-client.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<configuration>

<property>
  <name>ssl.client.truststore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/truststore</value>
  <description>Truststore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.truststore.password</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.client.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.client.keystore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/keystore</value>
  <description>Keystore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.keystore.password</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.keypassword</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

</configuration>
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

3>.将ssl-client.xml文件同步到集群其它节点

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-client.xml dest=${HADOOP_HOME}/etc/hadoop/"
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.59-13171-228481578922367/source", 
    "state": "file", 
    "uid": 0
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.59-13170-263679465385553/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.63-13174-270673831631234/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.58-13168-216772060676884/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "mode": "0644", 
    "owner": "root", 
    "path": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "size": 2459, 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-client.xml dest=${HADOOP_HOME}/etc/hadoop/"

五.配置ssl-server.xml文件

1>.基于模板文件创建ssl-server.xml文件

[root@hadoop101.yinzhengjie.com ~]# cp ${HADOOP_HOME}/etc/hadoop/ssl-server.xml.example ${HADOOP_HOME}/etc/hadoop/ssl-server.xml

2>.修改ssl-server.xml文件

[root@hadoop101.yinzhengjie.com ~]# vim ${HADOOP_HOME}/etc/hadoop/ssl-server.xml
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# cat ${HADOOP_HOME}/etc/hadoop/ssl-server.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<configuration>

<property>
  <name>ssl.server.truststore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/truststore</value>
  <description>Truststore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.truststore.password</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.server.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.server.keystore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/keystore</value>
  <description>Keystore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.password</name>
  <value>yinzhengjie</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.keypassword</name>
  <value>yinzhengjie</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.exclude.cipher.list</name>
  <value>TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_DES_CBC_SHA,SSL_DHE_RSA_WITH_DES_CBC_SHA,
  SSL_RSA_EXPORT_WITH_RC4_40_MD5,SSL_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_RC4_128_MD5</value>
  <description>Optional. The weak security cipher suites that you want excluded
  from SSL communication.</description>
</property>

</configuration>
[root@hadoop101.yinzhengjie.com ~]#

3>.将ssl-server.xml文件同步到集群其它节点

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-server.xml dest=${HADOOP_HOME}/etc/hadoop/"
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.3-13384-31361930800381/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.25-13388-191717002149571/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.19-13382-170842224507642/source", 
    "state": "file", 
    "uid": 0
}
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.25-13385-66836978382906/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "mode": "0644", 
    "owner": "root", 
    "path": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "size": 2840, 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-server.xml dest=${HADOOP_HOME}/etc/hadoop/"

六.重启启动Hadoop集群

1>.重启HDFS集群

[root@hadoop101.yinzhengjie.com ~]# manage-hdfs.sh restart
hadoop101.yinzhengjie.com | CHANGED | rc=0 >>
stopping namenode
hadoop105.yinzhengjie.com | CHANGED | rc=0 >>
stopping secondarynamenode
hadoop104.yinzhengjie.com | CHANGED | rc=0 >>
no datanode to stop
hadoop102.yinzhengjie.com | CHANGED | rc=0 >>
no datanode to stop
hadoop103.yinzhengjie.com | CHANGED | rc=0 >>
no datanode to stop
Stoping HDFS:                                              [  OK  ]
hadoop101.yinzhengjie.com | CHANGED | rc=0 >>
starting namenode, logging to /yinzhengjie/softwares/hadoop-2.10.0-fully-mode/logs/hadoop-root-namenode-hadoop101.yinzhengjie.com.out
hadoop105.yinzhengjie.com | CHANGED | rc=0 >>
starting secondarynamenode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-secondarynamenode-hadoop105.yinzhengjie.com.out
hadoop104.yinzhengjie.com | CHANGED | rc=0 >>
starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop104.yinzhengjie.com.out
hadoop102.yinzhengjie.com | CHANGED | rc=0 >>
starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop102.yinzhengjie.com.out
hadoop103.yinzhengjie.com | CHANGED | rc=0 >>
starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop103.yinzhengjie.com.out
Starting HDFS:                                             [  OK  ]
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# manage-hdfs.sh restart

2>.查看NameNode的Web UI端口

3>.查看DataNode的Web UI

七.启动集群后可能出现的错误

1>.javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Clock skew too great (37) - PROCESS_TGS)]

　　如下图所示，是由于集群时间差距过导致的，关于集群时间同步的组件比如nptd或者chrony均可以解决该问题。我推荐大家使用chrony组件来进行集群内时间同步。

　　博主推荐阅读:
　　　　https://developer.aliyun.com/article/1604349

2>.其它故障排除案例

　　博主推荐阅读:
　　　　https://developer.aliyun.com/article/1604488
　　　　https://developer.aliyun.com/article/1604481
文章标签：
分布式计算
Hadoop
Devops
Shell
关键词：
hadoop集群
hadoop案例
hadoop配置
hadoop实战
hadoop集群实战
尹正杰
目录
相关文章
程序员皮皮林
|
7天前
|
安全 应用服务中间件 网络安全
49.3k star，本地 SSL 证书生成神器，轻松解决 HTTPS 配置痛点
mkcert是一款由Filippo Valsorda开发的免费开源工具，专为生成受信任的本地SSL/TLS证书而设计。它通过简单的命令自动生成并安装本地信任的证书，使本地环境中的HTTPS配置变得轻松无比。mkcert支持多个操作系统，已获得49.2K的GitHub Star，成为开发者首选的本地SSL工具。
程序员皮皮林
40 10
武子康
|
1月前
|
分布式计算 Hadoop Shell
Hadoop-35 HBase 集群配置和启动 3节点云服务器 集群效果测试 Shell测试
Hadoop-35 HBase 集群配置和启动 3节点云服务器 集群效果测试 Shell测试
武子康
68 4
武子康
|
1月前
|
XML 分布式计算 资源调度
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece（一）
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece（一）
武子康
146 5
武子康
|
1月前
|
SQL 存储 分布式计算
Hadoop-16-Hive HiveServer2 HS2 允许客户端远程执行HiveHQL HCatalog 集群规划 实机配置运行
Hadoop-16-Hive HiveServer2 HS2 允许客户端远程执行HiveHQL HCatalog 集群规划 实机配置运行
武子康
41 3
武子康
|
1月前
Hadoop-09-HDFS集群 JavaClient 代码上手实战！详细附代码 安装依赖 上传下载文件 扫描列表 PUT GET 进度条显示（二）
Hadoop-09-HDFS集群 JavaClient 代码上手实战！详细附代码 安装依赖 上传下载文件 扫描列表 PUT GET 进度条显示（二）
武子康
39 3
武子康
|
1月前
|
XML 资源调度 网络协议
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece（二）
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece（二）
武子康
84 4
武子康
|
1月前
|
分布式计算 资源调度 Hadoop
大数据-01-基础环境搭建 超详细 Hadoop Java 环境变量 3节点云服务器 2C4G XML 集群配置 HDFS Yarn MapRedece
大数据-01-基础环境搭建 超详细 Hadoop Java 环境变量 3节点云服务器 2C4G XML 集群配置 HDFS Yarn MapRedece
武子康
72 4
武子康
|
1月前
|
SQL 分布式计算 Hadoop
Hadoop-34 HBase 安装部署 单节点配置 hbase-env hbase-site 超详细图文 附带配置文件
Hadoop-34 HBase 安装部署 单节点配置 hbase-env hbase-site 超详细图文 附带配置文件
武子康
69 2
武子康
|
1月前
|
SQL 存储 数据管理
Hadoop-15-Hive 元数据管理与存储 Metadata 内嵌模式 本地模式 远程模式 集群规划配置 启动服务 3节点云服务器实测
Hadoop-15-Hive 元数据管理与存储 Metadata 内嵌模式 本地模式 远程模式 集群规划配置 启动服务 3节点云服务器实测
武子康
54 2
武子康
|
1月前
|
分布式计算 Java Hadoop
Hadoop-09-HDFS集群 JavaClient 代码上手实战！详细附代码 安装依赖 上传下载文件 扫描列表 PUT GET 进度条显示（一）
Hadoop-09-HDFS集群 JavaClient 代码上手实战！详细附代码 安装依赖 上传下载文件 扫描列表 PUT GET 进度条显示（一）
武子康
39 2