HBase完全分布式安装
下载HBase安装包
Apache HBase – Apache HBase Downloads
https://hbase.apache.org/downloads.html
上传至master虚拟机并解压HBase
cd /opt/ tar -zxvf hbase-2.3.3.tar.gz #修改权限 sudo chmod -R 777 /opt/hbase-2.3.3
配置环境变量
[root@master ~]# vim /etc/bashrc #HBase export HBASE_HOME=/opt/hbase-2.3.3 export PATH=$PATH:$HBASE_HOME/bin #配置生效 [root@master ~]# source /etc/bashrc
三台虚拟机均进行此操作
在这里我们可以使用 hbase -version 查看环境变量是否正确
[root@master ~]# hbase -version java version "1.8.0_261" Java(TM) SE Runtime Environment (build 1.8.0_261-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode) [root@master ~]#
配置hbase相关文件
修改及配置 hbase-2.3.3/conf 目录下的 hbase-env.sh 文件
[root@master ~]# cd /opt/hbase-3.6.2/conf [root@master ~]# vim hbase-env.sh #!/usr/bin/env bash # Where log files are stored. $HBASE_HOME/logs by default. # export HBASE_LOG_DIR=${HBASE_HOME}/logs # Enable remote JDWP debugging of major HBase processes. Meant for Core Developers # export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8070" # export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071" # export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072" # export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073" # export HBASE_REST_OPTS="$HBASE_REST_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8074" # A string representing this instance of hbase. $USER by default. # export HBASE_IDENT_STRING=$USER # The scheduling priority for daemon processes. See 'man nice'. # export HBASE_NICENESS=10 # The directory where pid files are stored. /tmp by default. export HBASE_PID_DIR=/opt/hadoop/hadoop/pids # Seconds to sleep between slave commands. Unset by default. This # can be useful in large clusters, where, e.g., slave rsyncs can # otherwise arrive faster than the master can service them. # export HBASE_SLAVE_SLEEP=0.1 # Tell HBase whether it should manage it's own instance of ZooKeeper or not. # export HBASE_MANAGES_ZK=true # The default log rolling policy is RFA, where the log file is rolled as per the size defined for the # RFA appender. Please refer to the log4j.properties file to see more details on this appender. # In case one needs to do log rolling on a date change, one should set the environment property # HBASE_ROOT_LOGGER to "<DESIRED_LOG LEVEL>,DRFA". # For example: # HBASE_ROOT_LOGGER=INFO,DRFA # The reason for changing default to RFA is to avoid the boundary case of filling out disk space as # DRFA doesn't put any cap on the log size. Please refer to HBase-5655 for more context. # Tell HBase whether it should include Hadoop's lib when start up, # the default value is false,means that includes Hadoop's lib. export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true" # Override text processing tools for use by these launch scripts. # export GREP="${GREP-grep}" # export SED="${SED-sed}" export JAVA_HOME=/opt/jdk1.8.0_261 export HBASE_HOME=/opt/hbase-2.3.3 export HBASE_MANAGES_ZK=false
其中 HBASE_MANAGES_ZK=false 表示我们使用自己安装 zookeeper 集群而不是 hbase 自带的 zookeeper 集群
修改及配置 hbase-2.3.3/conf 目录下的 hbase-site.xml 文件
[root@master ~]# vim hbase-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- /* * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding copyright ownership. The ASF licenses this file * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ --> <configuration> <property> <name>hbase.root.dir</name> <value>hdfs://master:9000/hbase</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.tmp.dir</name> <value>./tmp</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>master,slave1,slave2</value> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property> <property> <name>hbase.unsafe.stream.capability.enforce</name> <value>false</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/opt/zookeeper-3.6.2</value> </property> <property> <name>zookeeper.session.timeout</name> <value>300000</value> <!--默认: 180000 ：zookeeper 会话超时时间，单位是毫秒 --> </property> <property> <name>hbase.master.maxclockskew</name> <value>30000</value> </property> </configuration>
添加 hbase 集群服务器的 ip 或者 hostname ，vi regionservers
1. [root@master ~]# vim regionservers 2. 3. master 4. slave1 5. slave2
将以上 Hbase 文件复制到三台服务器上
scp -r /opt/hbase-2.3.3 root@slave1:/opt scp -r /opt/hbase-2.3.3 root@slave2:/opt
完成即可
启动hbase
配置完成hbase后将上面的所有文件复制到其他两台服务器上，然后使用 start-hbase.sh 命令启动hbase集群
[root@master ~]# cd /opt/hbase-2.3.3/bin/ [root@master bin]# ls considerAsDead.sh hbase-config.cmd master-backup.sh start-hbase.sh draining_servers.rb hbase-config.sh region_mover.rb stop-hbase.cmd get-active-master.rb hbase-daemon.sh regionservers.sh stop-hbase.sh graceful_stop.sh hbase-daemons.sh region_status.rb test hbase hbase-jruby replication tmp hbase-cleanup.sh hirb.rb rolling-restart.sh zookeepers.sh hbase.cmd local-master-backup.sh shutdown_regionserver.rb hbase-common.sh local-regionservers.sh start-hbase.cmd [root@master bin]# start-hbase.sh running master, logging to /opt/hbase-2.3.3/logs/hbase-root-master-master.out slave1: running regionserver, logging to /opt/hbase-2.3.3/logs/hbase-root-regionserver-slave1.out slave2: running regionserver, logging to /opt/hbase-2.3.3/logs/hbase-root-regionserver-slave2.out master: running regionserver, logging to /opt/hbase-2.3.3/logs/hbase-root-regionserver-master.out [root@master bin]#
在哪台服务器使用上述命令启动则那台服务器即为 master 节点，使用 jps命令查看启动情况
[root@master bin]# jps 7281 ResourceManager 8450 HMaster 6965 SecondaryNameNode 6535 NameNode 8619 HRegionServer 7693 QuorumPeerMain 8943 Jps
slave1
[root@slave1 bin]# jps 2180 QuorumPeerMain 2509 Jps 1919 DataNode 2351 HRegionServer [root@slave1 bin]#
slave2
[root@slave2 ~]# jps 3441 QuorumPeerMain 3875 HRegionServer 4040 Jps 3165 DataNode [root@slave2 ~]#
可以看到服务器1启动和 HMaster 和 HRegionServer 进程，服务器2和服务器3启动和HRegionServer 进程。
至此大功告成！！！困扰了我一个月的难题，终于解决了！！！
🙇
当然，我们也可以通过Web页面查看 Hbase 集群情况 : http://IP:16010
加油(ง •_•)ง
