AI 助理
备案控制台
开发者社区 大数据 文章 正文

Hadoop集群配置https实战案例

2024-09-04 20
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
简介： 本文提供了一个实战案例，详细介绍了如何在Hadoop集群中配置HTTPS，包括生成私钥和证书文件、配置keystore和truststore、修改hdfs-site.xml和ssl-client.xml文件，以及重启Hadoop集群的步骤，并提供了一些常见问题的故障排除方法。

作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。

一.生成私钥及证书文件并拷贝到Hadoop节点

1>.生成私钥和证书文件

[root@hadoop101.yinzhengjie.com ~]# ll
total 4
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# openssl req -new -x509 -keyout hdfs_ca_key -out hdfs_ca_cert -days 36500 -subj '/C=CN/ST=beijing/L=changping/O=bigdata/OU=devops/CN=yinzhengjie.com'
Generating a 2048 bit RSA private key
..+++
..............................................................................................+++
writing new private key to 'hdfs_ca_key'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# openssl req -new -x509 -keyout hdfs_ca_key -out hdfs_ca_cert -days 36500 -subj '/C=CN/ST=beijing/L=changping/O=bigdata/OU=devops/CN=yinzhengjie.com'

2>.将私钥和自签证书拷贝到Hadoop集群节点

[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_cert dest=${HADOOP_HOME}/etc/hadoop/conf"
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.21-12128-10738301489038/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.25-12130-180011782497170/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.22-12125-24658832547026/source", 
    "state": "file", 
    "uid": 0
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.18-12127-102283570422989/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a8f408afafd58c73986a8d2a305c72c524be1064", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_cert", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "1eccd159e93db3ecbd9f8a6f5da37516", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1346, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987750.25-12129-1128583651866/source", 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_cert dest=${HADOOP_HOME}/etc/hadoop/conf"

[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_key dest=${HADOOP_HOME}/etc/hadoop/conf"
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.65-12397-87419418248032/source", 
    "state": "file", 
    "uid": 0
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.64-12396-52377313403452/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.63-12394-76433468569818/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.66-12400-246529012746559/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "a4a65fd1fe2d89af140ac1acc36ec6b62f9b5806", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs_ca_key", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "6c00b94f93f9424895e5954fdf4be26e", 
    "mode": "0644", 
    "owner": "root", 
    "size": 1834, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601987788.65-12399-1022475281042/source", 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=~/hdfs_ca_key dest=${HADOOP_HOME}/etc/hadoop/conf"

[root@hadoop101.yinzhengjie.com ~]# ansible all -m shell -a "ls -l ${HADOOP_HOME}/etc/hadoop/conf"
hadoop104.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
hadoop102.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
hadoop101.yinzhengjie.com | CHANGED | rc=0 >>
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
-rw-r--r-- 1 root root  115 Aug 13 18:55 host-rack.txt
-rwxr-xr-x 1 root root  463 Aug 13 18:54 toplogy.py
hadoop105.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
hadoop103.yinzhengjie.com | CHANGED | rc=0 >>
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m shell -a "ls -l ${HADOOP_HOME}/etc/hadoop/conf"

二.生成"keystore"和"trustores"文件

1>.生成keystore文件

[root@hadoop101.yinzhengjie.com ~]# ll
total 12
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias localhost -validity 36500 -genkey -keyalg RSA -keysize 2048 -dname "CN=hadoop101.yinzhengjie.com, OU=devops, O=bigdata,
 L=changping, ST=beijing, C=CN"Enter keystore password:  
Re-enter new password: 
Enter key password for <localhost>
    (RETURN if same as keystore password):  
Re-enter new password: 

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore -destkeystore ke
ystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 16
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
[root@hadoop101.yinzhengjie.com ~]#

keytool -keystore keystore -alias localhost -validity 36500 -genkey -keyalg RSA -keysize 2048 -dname "CN=hadoop101.yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN"

2>.生成truststore文件同时导入CA证书 

[root@hadoop101.yinzhengjie.com ~]# ll
total 16
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# keytool -keystore truststore -alias CARoot -import -file hdfs_ca_cert
Enter keystore password:  
Re-enter new password: 
Owner: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Issuer: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Serial number: 858ca3893e608eba
Valid from: Tue Oct 06 20:22:09 CST 2020 until: Thu Sep 12 20:22:09 CST 2120
Certificate fingerprints:
     MD5:  03:6C:16:3E:83:2A:15:79:11:3C:32:3F:12:3B:28:8C
     SHA1: CE:D0:D5:0C:93:FE:06:FB:87:D4:3A:22:B0:21:51:20:27:8F:81:B7
     SHA256: 25:3B:29:F4:16:FA:98:46:02:79:38:6B:3C:DA:7B:44:2F:9A:1B:21:F5:E2:AD:A1:48:93:12:FC:AC:97:0A:8A
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore truststore -alias CARoot -import -file hdfs_ca_cert

3>.从 keystore 中导出 cert

[root@hadoop101.yinzhengjie.com ~]#  ll
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# keytool -certreq -alias localhost -keystore keystore -file cert
Enter keystore password:  

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore -destkeystore ke
ystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 24
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -certreq -alias localhost -keystore keystore -file cert

4>.用 CA 对 cert 签名，生成自签证书

[root@hadoop101.yinzhengjie.com ~]# ll
total 24
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# openssl x509 -req -CA hdfs_ca_cert -CAkey hdfs_ca_key -in cert -out cert_signed -days 36500 -CAcreateserial
Signature ok
subject=/C=CN/ST=beijing/L=changping/O=bigdata/OU=devops/CN=hadoop101.yinzhengjie.com
Getting CA Private Key
Enter pass phrase for hdfs_ca_key:
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 32
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1241 Oct  6 21:34 cert_signed
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root   17 Oct  6 21:34 hdfs_ca_cert.srl
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 2278 Oct  6 20:51 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# openssl x509 -req -CA hdfs_ca_cert -CAkey hdfs_ca_key -in cert -out cert_signed -days 36500 -CAcreateserial

5>.将 CA 的 cert 和导入 keystore文件

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias CARoot -import -file hdfs_ca_cert
Enter keystore password:  
Owner: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Issuer: CN=yinzhengjie.com, OU=devops, O=bigdata, L=changping, ST=beijing, C=CN
Serial number: 858ca3893e608eba
Valid from: Tue Oct 06 20:22:09 CST 2020 until: Thu Sep 12 20:22:09 CST 2120
Certificate fingerprints:
     MD5:  03:6C:16:3E:83:2A:15:79:11:3C:32:3F:12:3B:28:8C
     SHA1: CE:D0:D5:0C:93:FE:06:FB:87:D4:3A:22:B0:21:51:20:27:8F:81:B7
     SHA256: 25:3B:29:F4:16:FA:98:46:02:79:38:6B:3C:DA:7B:44:2F:9A:1B:21:F5:E2:AD:A1:48:93:12:FC:AC:97:0A:8A
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 21 60 19 83 94 27 E2 02   51 F7 EB C5 82 69 40 E1  !`...'..Q....i@.
0010: 68 97 94 FA                                        h...
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format us
ing "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias CARoot -import -file hdfs_ca_cert

6>.用 CA 自签名之后的 cert 也导入 keystore文件

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias localhost -import -file cert_signed
Enter keystore password:  
Certificate reply was installed in keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format us
ing "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12".[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# keytool -keystore keystore -alias localhost -import -file cert_signed

7>.将keystore,trustores存入到自定义目录，便于配置时方便找到它们

[root@hadoop101.yinzhengjie.com ~]# ll  ${HADOOP_HOME}/etc/hadoop/conf
total 20
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
-rw-r--r-- 1 root root  115 Aug 13 18:55 host-rack.txt
-rwxr-xr-x 1 root root  463 Aug 13 18:54 toplogy.py
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll
total 36
-rw-r--r-- 1 root root 1117 Oct  6 21:18 cert
-rw-r--r-- 1 root root 1241 Oct  6 21:34 cert_signed
-rw-r--r-- 1 root root 1346 Oct  6 20:22 hdfs_ca_cert
-rw-r--r-- 1 root root   17 Oct  6 21:34 hdfs_ca_cert.srl
-rw-r--r-- 1 root root 1834 Oct  6 20:22 hdfs_ca_key
-rw------- 1 root root 3362 Oct  6 18:33 hdfs.keytab
-rw-r--r-- 1 root root 4181 Oct  6 21:42 keystore
-rw-r--r-- 1 root root 1016 Oct  6 20:59 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# cp keystore truststore  ${HADOOP_HOME}/etc/hadoop/conf
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# ll  ${HADOOP_HOME}/etc/hadoop/conf
total 32
-rw-r--r-- 1 root root 1346 Oct  6 20:35 hdfs_ca_cert
-rw-r--r-- 1 root root 1834 Oct  6 20:36 hdfs_ca_key
-rw-r--r-- 1 root root 3362 Oct  6 18:39 hdfs.keytab
-rw-r--r-- 1 root root  115 Aug 13 18:55 host-rack.txt
-rw-r--r-- 1 root root 4181 Oct  6 21:47 keystore
-rwxr-xr-x 1 root root  463 Aug 13 18:54 toplogy.py
-rw-r--r-- 1 root root 1016 Oct  6 21:47 truststore
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# cp keystore truststore ${HADOOP_HOME}/etc/hadoop/conf

8>.在集群其它节点重复[1-7]的所有步骤

　　温馨提示:  
　　　　注意观察上面的每一个步骤，比如FQDN，每个节点对应的主机名并不一致哟~

三.修改"hdfs-site.xml"配置文件

1>.修改"hdfs-site.xml"文件内容

[root@hadoop101.yinzhengjie.com ~]# vim /yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml 
<configuration>
 ...
　　<property>
　　　　<name>dfs.http.policy</name>
　　　　<value>HTTPS_ONLY</value>
　　　　<description>确定HDFS是否支持HTTPS(SSL)这将为HDFS守护程序配置HTTP端点:默认值为"HTTP_ONLY"(仅在http上提供服务),"HTTPS_ONLY"(仅在https上提供服务),"HTTP_AND_HTTPS"(同时提供服务在http和https上)</description>
　　</property>
...
</configuration>
[root@hadoop101.yinzhengjie.com ~]# 


温馨提示:
　　DataNode节点需要将"dfs.http.policy"的属性设置未"HTTPS_ONLY"，但是NameNode和Secondary NameNode需要将该值设置为"HTTP_AND_HTTPS"。
　　如果你将HDFS集群的所有节点的属性都设置成"HTTPS_ONLY",你会发现NameNode和Secondary NameNode的进程可以启动成功，但是它们的WebUI实例启动失败(也就是说你不能访问NameNode的50070端口以及Secondary NameNode的50090端口啦~)！

2>.将配置同步到集群其它datanode节点

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/hdfs-site.xml dest=${HADOOP_HOME}/etc/hadoop/"
hadoop101.yinzhengjie.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 190, 
    "group": "systemd-journal", 
    "mode": "0644", 
    "owner": "12334", 
    "path": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "size": 11470, 
    "state": "file", 
    "uid": 12334
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.64-14771-240997395125955/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.67-14775-126005848371918/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.62-14769-272249870401292/source", 
    "state": "file", 
    "uid": 0
}
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "0adc0d87b88313df17ba1deb2d68a359c8dc9be4", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/hdfs-site.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "87d3fa9d1b93d03fd2b509508086def4", 
    "mode": "0644", 
    "owner": "root", 
    "size": 11470, 
    "src": "/root/.ansible/tmp/ansible-tmp-1602000461.64-14772-148620330070479/source", 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/hdfs-site.xml dest=${HADOOP_HOME}/etc/hadoop/"

四.配置ssl-client.xml文件

1>.基于模板文件创建ssl-client.xml文件

[root@hadoop101.yinzhengjie.com ~]# cp ${HADOOP_HOME}/etc/hadoop/ssl-client.xml.example ${HADOOP_HOME}/etc/hadoop/ssl-client.xml

2>.修改ssl-client.xml文件

[root@hadoop101.yinzhengjie.com ~]# vim ${HADOOP_HOME}/etc/hadoop/ssl-client.xml
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# cat ${HADOOP_HOME}/etc/hadoop/ssl-client.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<configuration>

<property>
  <name>ssl.client.truststore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/truststore</value>
  <description>Truststore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.truststore.password</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.client.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.client.keystore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/keystore</value>
  <description>Keystore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.keystore.password</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.keypassword</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

</configuration>
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]#

3>.将ssl-client.xml文件同步到集群其它节点

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-client.xml dest=${HADOOP_HOME}/etc/hadoop/"
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.59-13171-228481578922367/source", 
    "state": "file", 
    "uid": 0
}
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.59-13170-263679465385553/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.63-13174-270673831631234/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "f3ab7ce75cb96fec12999a507d1f1032", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2459, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601998791.58-13168-216772060676884/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "checksum": "335bcafcd6baf119d58fdd5f6510bed58b9cf31c", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "gid": 0, 
    "group": "root", 
    "mode": "0644", 
    "owner": "root", 
    "path": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-client.xml", 
    "size": 2459, 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-client.xml dest=${HADOOP_HOME}/etc/hadoop/"

五.配置ssl-server.xml文件

1>.基于模板文件创建ssl-server.xml文件

[root@hadoop101.yinzhengjie.com ~]# cp ${HADOOP_HOME}/etc/hadoop/ssl-server.xml.example ${HADOOP_HOME}/etc/hadoop/ssl-server.xml

2>.修改ssl-server.xml文件

[root@hadoop101.yinzhengjie.com ~]# vim ${HADOOP_HOME}/etc/hadoop/ssl-server.xml
[root@hadoop101.yinzhengjie.com ~]# 
[root@hadoop101.yinzhengjie.com ~]# cat ${HADOOP_HOME}/etc/hadoop/ssl-server.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<configuration>

<property>
  <name>ssl.server.truststore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/truststore</value>
  <description>Truststore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.truststore.password</name>
  <value>yinzhengjie</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.server.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.server.keystore.location</name>
  <value>/yinzhengjie/softwares/hadoop/etc/hadoop/conf/keystore</value>
  <description>Keystore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.password</name>
  <value>yinzhengjie</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.keypassword</name>
  <value>yinzhengjie</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.exclude.cipher.list</name>
  <value>TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_DES_CBC_SHA,SSL_DHE_RSA_WITH_DES_CBC_SHA,
  SSL_RSA_EXPORT_WITH_RC4_40_MD5,SSL_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_RC4_128_MD5</value>
  <description>Optional. The weak security cipher suites that you want excluded
  from SSL communication.</description>
</property>

</configuration>
[root@hadoop101.yinzhengjie.com ~]#

3>.将ssl-server.xml文件同步到集群其它节点

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-server.xml dest=${HADOOP_HOME}/etc/hadoop/"
hadoop103.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.3-13384-31361930800381/source", 
    "state": "file", 
    "uid": 0
}
hadoop105.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.25-13388-191717002149571/source", 
    "state": "file", 
    "uid": 0
}
hadoop102.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.19-13382-170842224507642/source", 
    "state": "file", 
    "uid": 0
}
hadoop104.yinzhengjie.com | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": true, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "68d0c7a7b6da73a7ceab1172404ff380", 
    "mode": "0644", 
    "owner": "root", 
    "size": 2840, 
    "src": "/root/.ansible/tmp/ansible-tmp-1601999063.25-13385-66836978382906/source", 
    "state": "file", 
    "uid": 0
}
hadoop101.yinzhengjie.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "checksum": "ca744794ae2944ad05b0933750193b1e860ee8cd", 
    "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "gid": 0, 
    "group": "root", 
    "mode": "0644", 
    "owner": "root", 
    "path": "/yinzhengjie/softwares/hadoop/etc/hadoop/ssl-server.xml", 
    "size": 2840, 
    "state": "file", 
    "uid": 0
}
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a "src=${HADOOP_HOME}/etc/hadoop/ssl-server.xml dest=${HADOOP_HOME}/etc/hadoop/"

六.重启启动Hadoop集群

1>.重启HDFS集群

[root@hadoop101.yinzhengjie.com ~]# manage-hdfs.sh restart
hadoop101.yinzhengjie.com | CHANGED | rc=0 >>
stopping namenode
hadoop105.yinzhengjie.com | CHANGED | rc=0 >>
stopping secondarynamenode
hadoop104.yinzhengjie.com | CHANGED | rc=0 >>
no datanode to stop
hadoop102.yinzhengjie.com | CHANGED | rc=0 >>
no datanode to stop
hadoop103.yinzhengjie.com | CHANGED | rc=0 >>
no datanode to stop
Stoping HDFS:                                              [  OK  ]
hadoop101.yinzhengjie.com | CHANGED | rc=0 >>
starting namenode, logging to /yinzhengjie/softwares/hadoop-2.10.0-fully-mode/logs/hadoop-root-namenode-hadoop101.yinzhengjie.com.out
hadoop105.yinzhengjie.com | CHANGED | rc=0 >>
starting secondarynamenode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-secondarynamenode-hadoop105.yinzhengjie.com.out
hadoop104.yinzhengjie.com | CHANGED | rc=0 >>
starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop104.yinzhengjie.com.out
hadoop102.yinzhengjie.com | CHANGED | rc=0 >>
starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop102.yinzhengjie.com.out
hadoop103.yinzhengjie.com | CHANGED | rc=0 >>
starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop103.yinzhengjie.com.out
Starting HDFS:                                             [  OK  ]
[root@hadoop101.yinzhengjie.com ~]#

[root@hadoop101.yinzhengjie.com ~]# manage-hdfs.sh restart

2>.查看NameNode的Web UI端口

3>.查看DataNode的Web UI

七.启动集群后可能出现的错误

1>.javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Clock skew too great (37) - PROCESS_TGS)]

　　如下图所示，是由于集群时间差距过导致的，关于集群时间同步的组件比如nptd或者chrony均可以解决该问题。我推荐大家使用chrony组件来进行集群内时间同步。

　　博主推荐阅读:
　　　　https://developer.aliyun.com/article/1604349

2>.其它故障排除案例

　　博主推荐阅读:
　　　　https://developer.aliyun.com/article/1604488
　　　　https://developer.aliyun.com/article/1604481
文章标签：
分布式计算
Hadoop
Devops
Shell
关键词：
hadoop集群
hadoop集群案例
配置HTTPS案例
hadoop实战
配置HTTPS实战
尹正杰
目录
相关文章
尹正杰
|
12天前
|
Linux Docker Windows
Docker配置https证书案例
本文介绍了如何为Docker的Harbor服务配置HTTPS证书，包括安装Docker和Harbor、修改配置文件以使用证书、生成自签名证书、配置证书以及验证配置的步骤。
尹正杰
19 2
Docker配置https证书案例
尹正杰
|
12天前
|
机器学习/深度学习 分布式计算 安全
Hadoop集群常见报错汇总
这篇博客总结了Hadoop集群中可能遇到的各种常见错误，包括Kerberos认证问题、配置错误、权限问题等，并为每个问题提供了详细的错误复现、原因分析以及相应的解决方案。
尹正杰
26 1
Hadoop集群常见报错汇总
vohelon
|
1天前
|
监控 安全 应用服务中间件
如何配置HTTPS协议？
如何配置HTTPS协议？
vohelon
7 4
尹正杰
|
12天前
|
分布式计算 监控 Hadoop
监控Hadoop集群实战篇
介绍了监控Hadoop集群的方法，包括监控Linux服务器、Hadoop指标、使用Ganglia监控Hadoop集群、Hadoop日志记录、通过Hadoop的Web UI进行监控以及其他Hadoop组件的监控，并提供了相关监控工具和资源的推荐阅读链接。
尹正杰
28 2
wljslmz
|
1月前
|
存储 分布式计算 Hadoop
什么是 Hadoop？它如何解决大数据问题？
【8月更文挑战第12天】
wljslmz
66 4
什么是 Hadoop？它如何解决大数据问题？
土木林森
|
15天前
|
图形学 数据可视化 开发者
超实用Unity Shader Graph教程：从零开始打造令人惊叹的游戏视觉特效，让你的作品瞬间高大上，附带示例代码与详细步骤解析！
【8月更文挑战第31天】Unity Shader Graph 是 Unity 引擎中的强大工具，通过可视化编程帮助开发者轻松创建复杂且炫酷的视觉效果。本文将指导你使用 Shader Graph 实现三种效果：彩虹色渐变着色器、动态光效和水波纹效果。首先确保安装最新版 Unity 并启用 Shader Graph。创建新材质和着色器图谱后，利用节点库中的预定义节点，在编辑区连接节点定义着色器行为。
土木林森
56 0
william.zhang(张)
|
20天前
|
分布式计算 资源调度 Hadoop
Hadoop入门基础（五）：Hadoop 常用 Shell 命令一网打尽，提升你的大数据技能！
Hadoop入门基础（五）：Hadoop 常用 Shell 命令一网打尽，提升你的大数据技能！
william.zhang(张)
33 2
1941623231718325
|
21天前
|
存储 SQL 分布式计算
Hadoop生态系统概述：构建大数据处理与分析的基石
【8月更文挑战第25天】Hadoop生态系统为大数据处理和分析提供了强大的基础设施和工具集。通过不断扩展和优化其组件和功能，Hadoop将继续在大数据时代发挥重要作用。
1941623231718325
43 3
土木林森
|
22天前
|
资源调度 分布式计算 Hadoop
揭秘Hadoop Yarn背后的秘密！它是如何化身‘资源大师’，让大数据处理秒变高效大戏的？
【8月更文挑战第24天】在大数据领域，Hadoop Yarn（另一种资源协调者）作为Hadoop生态的核心组件，扮演着关键角色。Yarn通过其ResourceManager、NodeManager、ApplicationMaster及Container等组件，实现了集群资源的有效管理和作业调度。当MapReduce任务提交时，Yarn不仅高效分配所需资源，还能确保任务按序执行。无论是处理Map阶段还是Reduce阶段的数据，Yarn都能优化资源配置，保障任务流畅运行。此外，Yarn还在Spark等框架中展现出灵活性，支持不同模式下的作业执行。未来，Yarn将持续助力大数据技术的发展与创新。
土木林森
27 2
土木林森
|
1月前
|
分布式计算 Hadoop 大数据
Spark 与 Hadoop 的大数据之战：一场惊心动魄的技术较量，决定数据处理的霸权归属！
【8月更文挑战第7天】无论是 Spark 的高效内存计算，还是 Hadoop 的大规模数据存储和处理能力，它们都为大数据的发展做出了重要贡献。
土木林森
61 2

热门文章

最新文章

  • 1
    如何使用被信任的证书来配置SLB HTTPS协议，使slb站点支持ssl
  • 2
    HTTP协议状态码详解（HTTP Status Code）
  • 3
    【Java Servlet 开发系列之二】创建WebApp详细步骤，通过Servlet实现http简单交互
  • 4
    艾伟：WCF 3.5对HTTP编程的增强
  • 5
    智享系列之如何让你的网站免费使用阿里云的Https服务
  • 6
    Android http 的使用
  • 7
    WebSocket的原理，以及和Http的关系
  • 8
    详细讲解https证书openssl公钥私钥以及ssh公钥私钥的生成以及使用
  • 9
    部分HTTPS网站受影响：Firefox暂停“弃用SHA-1加密”计划
  • 10
    Spring Boot 的 HTTP 客户端框架
  • 1
    Hadoop节点网络性能的带宽测试选择测试工具
    53
  • 2
    Hadoop节点网络性能的带宽测试设置测试环境
    51
  • 3
    Hadoop节点网络性能的带宽测试
    71
  • 4
    Hadoop节点网络性能测试时延测试
    91
  • 5
    Hadoop节点网络性能的带宽测试
    74
  • 6
    Hadoop节点网络性能测试准备测试工具
    61
  • 7
    【专栏】Hadoop，开源大数据处理框架：驭服数据洪流的利器
    131
  • 8
    Hadoop节点网络性能测试
    73
  • 9
    Hadoop节点网络设备与交换机检查
    63
  • 10
    Hadoop节点检查物理连接
    49

    • 相关课程

    更多
  • 大数据实战项目：反爬虫系统（Lua+Spark+Redis+Hadoop框架搭建）第一阶段
  • 大数据实战项目：反爬虫系统（Lua+Spark+Redis+Hadoop框架搭建）第二阶段
  • 大数据实战项目：反爬虫系统（Lua+Spark+Redis+Hadoop框架搭建）第四阶段
  • 大数据Hadoop快速入门
  • Hadoop快速入门
  • Hadoop企业优化及扩展案例

    • 相关电子书

    更多
  • 《构建Hadoop生态批流一体的实时数仓》
  • 零基础实现hadoop 迁移 MaxCompute 之 数据
  • CIO 指南:如何在SAP软件架构中使用Hadoop

    • 相关实验场景

    更多
  • 通过HTTPS加速网关快速部署网站加密
  • 云上Web及FTP
  • 搭建Hadoop环境
    • 下一篇
    基于LNMP搭建WordPress