单机手动部署OceanBase集群

​

1 部署环境
VirBox下centos虚拟机，虚拟机内存为12G， 硬盘40GB。

2 初始化服务器部署环境（这里的步骤同

修改内核参数，生产环境需要按这个修改，模拟环境下用docker试过，不改内核参数也能部署。

[root]#vim /etc/sysctl.conf
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.ip_local_port_range = 3500 65535
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_tw_reuse = 129
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_slow_start_after_idle=0
vm.swappiness = 0
vm.min_free_kbytes = 2097152
vm.max_map_count=655360
fs.aio-max-nr=1048576
[root@38c7642a747b /]# sysctl -p ##使更改的配置生效

修改会话变量设置，更改完后需要退出后重新登陆会话，检查是否生效

[root@38c7642a747b /]#vim /etc/security/limits.conf

• soft nofile 655360
• hard nofile 655360
• soft nproc 655360
• hard nproc 655360
• soft core unlimited
• hard core unlimited
• soft stack unlimited
• hard stack unlimited

[root@38c7642a747b /]# ulimit -a

检查并关闭防火墙

[root]# systemctl status firewalld

● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2022-01-05 01:30:20 EST; 4min 47s ago

[root]# systemctl disable firewalld

Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

[root]# systemctl stop firewalld
[root]# systemctl status firewalld

● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)

关闭SELinux，更改配置文件后重启生效，或者用setenforce 0命令可立即生效。

[root@localhost ~]# vi /etc/selinux/config
[root@localhost ~]# cat /etc/selinux/config

    SELINUX=disabled
    SELINUXTYPE=targeted

创建用户，由于OceanBase rpm安装包默认将软件安装在/home/admin目录下，创建名为admin的用户可以简化部署过程，这里顺便给用户增加了sudo权限。

[root@38c7642a747b /]# useradd admin
[root@38c7642a747b /]# passwd admin

[root@38c7642a747b /]# usermod admin -G wheel
[root@38c7642a747b /]# id admin

  uid=1000(admin) gid=1000(admin) groups=1000(admin),10(wheel)

3 下载并安装软件包

手动部署需要下载observer、obproxy软件包，另外也下载了obclient用户连接集群，还需要下载libobclient（obclient库），oceanbase-ce-libs库两个包，我这里下载在root用户的home目录下

[root]# wget https://mirrors.aliyun.com/oceanbase/community/stable/el/8/x86_64/oceanbase-ce-libs-3.1.0-3.el8.x86_64.rpm

  --2022-01-05 01:30:43--  

oceanbase-ce-libs-3.1.0-3 100%[===================================>] 486.11K 1.28MB/s in 0.4s

[root]# wget https://mirrors.aliyun.com/oceanbase/community/stable/el/8/x86_64/obclient-2.0.0-2.el8.x86_64.rpm

  --2022-01-05 01:33:01--  
  obclient-2.0.0-2.el8.x86_ 100%[===================================>] 190.18M  1.42MB/s    in 2m 3s

[root]# wget https://mirrors.aliyun.com/oceanbase/community/stable/el/8/x86_64/obproxy-3.1.0-1.el8.x86_64.rpm

  --2022-01-05 01:36:05--  
  obproxy-3.1.0-1.el8.x86_6 100%[===================================>]   7.81M  1.90MB/s    in 4.2s

  2022-01-05 01:36:09 (1.86 MB/s) - 'obproxy-3.1.0-1.el8.x86_64.rpm' saved [8190556/8190556]

[root]# wget https://mirrors.aliyun.com/oceanbase/community/stable/el/8/x86_64/libobclient-2.0.0-2.el8.x86_64.rpm

  --2022-01-05 01:44:42--  
  libobclient-2.0.0-2.el8.x 100%[===================================>]   1007K  1.34MB/s    in 0.7s

[root]# wget https://mirrors.aliyun.com/oceanbase/community/stable/el/8/x86_64/libobclient-2.0.0-2.el8.x86_64.rpm

  libobclient-2.0.0-2.el8.x 100%[===================================>]   1007K  1.34MB/s    in 0.7s

安装就比较简单了

[root]# rpm -ivh oc*.rpm
  warning: oceanbase-ce-3.1.0-3.el8.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID e9b4a7aa: NOKEY
  Verifying...                          ################################# [100%]
  Preparing...                          ################################# [100%]
  Updating / installing...
     1:oceanbase-ce-libs-3.1.0-3.el8    ################################# [ 50%]
     2:oceanbase-ce-3.1.0-3.el8         ################################# [100%]
[root]# rpm -ivh libobclient-2.0.0-2.el8.x86_64.rpm
  warning: libobclient-2.0.0-2.el8.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID e9b4a7aa: NOKEY
  Verifying...                          ################################# [100%]
  Preparing...                          ################################# [100%]
  Updating / installing...
     1:libobclient-2.0.0-2.el8          ################################# [100%]
[root]# rpm -ivh ob*
  warning: obclient-2.0.0-2.el8.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID e9b4a7aa: NOKEY
  Verifying...                          ################################# [100%]
  Preparing...                          ################################# [100%]
  Updating / installing...
     1:obproxy-3.1.0-1.el8              ################################# [ 50%]
     2:obclient-2.0.0-2.el8             ################################# [100%]

安装目录是下面这个样子，oceanbase和obproxy各有自己的安装目录，都处于/home/admin目录下

[root]# tree oceanbase

   oceanbase
   |-- bin
   |   |-- import_time_zone_info.py
   |   `-- observer
   |-- etc
   |   `-- timezone_V1.log
   `-- lib
       |-- libaio.so -> libaio.so.1.0.1
       |-- libaio.so.1 -> libaio.so.1.0.1
       |-- libaio.so.1.0.1
       |-- libmariadb.so -> libmariadb.so.3
       `-- libmariadb.so.3

   3 directories, 8 files

[root@38c7642a747b admin]# tree obproxy-3.1.0
   obproxy-3.1.0
   `-- bin
       |-- obproxy
       `-- obproxyd.sh

   1 directory, 2 files

4 初始化数据目录

       建虚拟机时没有添加第二个磁盘，这里就把数据目录直接放到了/home/admin下，创建目录的命令做了一些调整

[root] mkdir -p ~/oceanbase/store/obdemo/{clog,etc2,etc3,ilog,slog,sstable}
建成后的目录结构如下图

oceanbase

    └── store
      └── obdemo
        ├── clog
        ├── etc2
        ├── etc3
        ├── ilog
        ├── slog
        └── sstable
           

5 启动observer进程

    由于上面更改了目录结构，相应的启动命令也需要调整一下，去掉关于/data， /redo路径的设置

[admin]$ echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/oceanbase/lib' >> ~/.bash_profile
[admin]$ . ~/.bash_profile
[admin]$ cd ~/oceanbase && bin/observer -i lo -p 2881 -P 2882 -z zone1 -d ~/oceanbase/store/obdemo -r '127.0.0.1:2882:2881' -c 20210912 -n obdemo -o "memory_limit=8G,cache_wash_threshold=1G,__min_full_resource_pool_memory=268435456,system_memory=3G,memory_chunk_cache_size=128M,cpu_count=4,net_thread_count=4,datafile_size=10G,stack_size=1536K" -d ~/oceanbase/store/obdemo

  bin/observer -i lo -p 2881 -P 2882 -z zone1 -d /home/admin/oceanbase/store/obdemo -r 127.0.0.1:2882:2881 -c 20210912 -n obdemo -o memory_limit=8G,cache_wash_threshold=1G,__min_full_resource_pool_memory=268435456,system_memory=3G,memory_chunk_cache_size=128M,cpu_count=4,net_thread_count=4,datafile_size=10G,stack_size=1536K -d /home/admin/oceanbase/store/obdemo
  devname: enp0s3
  mysql port: 2881
  rpc port: 2882
  zone: zone1
  data_dir: /home/admin/oceanbase/store/obdemo
  rs list: 127.0.0.1:2882:2881
  cluster id: 20210912
  appname: obdemo
  optstr: memory_limit=8G,cache_wash_threshold=1G,__min_full_resource_pool_memory=268435456,system_memory=3G,memory_chunk_cache_size=128M,cpu_count=4,net_thread_count=4,datafile_size=10G,stack_size=1536K
  data_dir: /home/admin/oceanbase/store/obdemo

[admin]$ ps -ef|grep obse

  admin     1533     1 99 03:03 ?        00:00:13 bin/observer -i enp0s3 -p 2881 -P 2882 -z zone1 -d /home/admin/oceanbase/store/obdemo -r 127.0.0.1:2882:2881 -c 20210912 -n obdemo -o memory_limit=8G,cache_wash_threshold=1G,__min_full_resource_pool_memory=268435456,system_memory=3G,memory_chunk_cache_size=128M,cpu_count=4,net_thread_count=4,datafile_size=10G,stack_size=1536K -d /home/admin/oceanbase/store/obdemo
  admin     1706  1452  0 03:03 pts/0    00:00:00 grep --color=auto obse

observer进程已经启动了，开始集群自举，先连接至集群（初始密码为空）

[admin]$ mysql -h127.0.0.1 -u root -P2881 -p -c -A
启动集群自举，自举花了接近两分钟时间。

mysql> set session ob_query_timeout=1000000000;
mysql> alter system bootstrap ZONE 'zone1' SERVER '127.0.0.1:2882';

    Query OK, 0 rows affected (1 min 57.52 sec)

登陆集群系统租户，设置root密码，创建obproxy连接用户，授予必要权限

[admin]$ mysql -h127.0.0.1 -u root@sys -P2881 -p -c -A

      Enter password:
      Welcome to the MySQL monitor.  Commands end with ; or \g.
mysql> grant select on oceanbase.* to proxyro identified by 'root123'; 创建代理用户
      Query OK, 0 rows affected (0.20 sec)

mysql> alter user root identified by 'root123'; 更改root密码

      Query OK, 0 rows affected (0.07 sec)

6 启动obproxy进程

[admin] $cd ~/obproxy-3.1.0/ && bin/obproxy -r "127.0.0.1:2881" -p 2883 -o "enable_strict_kernel_release=false, enable_cluster_checkout=false,enable_metadb_used=false" -c obdemo

  bin/obproxy -r 127.0.0.1 -p 2883 -o enable_strict_kernel_release=false,enable_cluster_checkout=false,enable_metadb_used=false -c obdemo
  rs list: 127.0.0.1:2881
  listen port: 2883
  optstr: enable_strict_kernel_release=false,enable_cluster_checkout=false,enable_metadb_used=false
  cluster_name: obdemo

       observer启动正常，监听2883端口，集群内有一台服务器。下面登陆obproxy，修改密码，登陆初始密码为空

[admin]$ obclient -h127.0.0.1 -u root@proxysys -P 2883 -p ##登陆代理，初始密码为空

  Enter password:
  Welcome to the OceanBase.  Commands end with ; or \g.
  
  Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

修改obproxy sys用户密码

  MySQL [(none)]> alter proxyconfig set obproxy_sys_password='root123';
      Query OK, 0 rows affected (0.003 sec)

修改OBPROXY连接OceanBase集群用户proxyro密码，密码和前面创建用户proxyro的密码相同。

MySQL [(none)]> alter proxyconfig set observer_sys_password='root123';

  Query OK, 0 rows affected (0.002 sec)

退出后，通过obproxy登陆oceanbase系统库，检验登陆是否成功

[admin]$ obclient -h127.0.0.1 -uroot@sys -P2881 -p -c -A

  Enter password:
  Welcome to the OceanBase.  Commands end with ; or \g.
  Your MySQL connection id is 3221487675
  Server version: 5.7.25 OceanBase 3.1.0 (r3-b20901e8c84d3ea774beeaca963c67d7802e4b4e) (Built Aug 10 2021 08:10:38)
  
  Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
  
  Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  
  MySQL [(none)]> show databases;
  +--------------------+
  | Database           |
  +--------------------+
  | oceanbase          |
  | information_schema |
  | mysql              |
  | SYS                |
  | LBACSYS            |
  | ORAAUDITOR         |
  | test               |
  +--------------------+
  7 rows in set (0.003 sec)
  
  MySQL [(none)]> use oceanbase;
  Database changed
  MySQL [oceanbase]> show full processlist;
  +------------+------+--------+-----------------+-----------+---------+------+--------+-----------------------+-----------+------+--------------+
  | Id         | User | Tenant | Host            | db        | Command | Time | State  | Info                  | Ip        | Port | Proxy_sessid |
  +------------+------+--------+-----------------+-----------+---------+------+--------+-----------------------+-----------+------+--------------+
  | 3221487675 | root | sys    | 127.0.0.1:42144 | oceanbase | Query   |    0 | ACTIVE | show full processlist | 127.0.0.1 | 2881 |         NULL |
  +------------+------+--------+-----------------+-----------+---------+------+--------+-----------------------+-----------+------+--------------+
  1 row in set (0.005 sec)
  

成功登陆系统租户，show full processlist命令显示正常。

7 创建租户、数据库、表

连接系统租户

[admin@38c7642a747b ~]$ obclient -h127.0.0.1 -u root@proxysys -P 2883 -p ##登陆代理，初始密码为空

  Enter password:
  Welcome to the OceanBase.  Commands end with ; or \g.
  Your MySQL connection id is 11
  Server version: 5.6.25

  Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

  Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

查看已经配置的资源单元规格，和集群内资源使用情况

use oceanbase;

Database changed

select name,

   min_cpu,
   min_memory/1024/1024/1024 min_mem_GB,
   max_cpu,
   max_memory/1024/1024/1024 max_mem_GB,
   max_disk_size/1024/1024/1024 max_disk_size_gb
from __all_unit_config;
+-----------------+---------+----------------+---------+----------------+------------------+
| name            | min_cpu | min_mem_GB     | max_cpu | max_mem_GB     | max_disk_size_gb |
+-----------------+---------+----------------+---------+----------------+------------------+
| sys_unit_config |     2.5 | 1.250000000000 |       5 | 1.500000000000 |  20.000000000000 |
+-----------------+---------+----------------+---------+----------------+------------------

alter resource unit sys_unit_config min_cpu 1,max_cpu 2,max_disk_size '10G';
select a.zone,concat(a.svr_ip,':',a.svr_port) observer,

   cpu_total,
  cpu_assigned,
  (cpu_total-cpu_assigned) cpu_free,
   mem_total/1024/1024/1024 mem_total_gb,
   mem_assigned/1024/1024/1024 mem_assign_gb,
  (mem_total-mem_assigned)/1024/1024/1024 mem_free_gb,
  disk_total/1024/1024/1024 total_disk_gb,
  (disk_total-disk_assigned)/1024/1024/1024 disk_free_gb

from __all_virtual_server_stat a join __all_server b on (a.svr_ip=b.svr_ip and a.svr_port=b.svr_port)

 order by a.zone, a.svr_ip;

+-------+----------------+-----------+--------------+----------+----------------+----------------+----------------+-----------------+-----------------+
| zone | observer | cpu_total | cpu_assigned | cpu_free | mem_total_gb | mem_assign_gb | mem_free_gb | total_disk_gb | disk_free_gb |
+-------+----------------+-----------+--------------+----------+----------------+----------------+----------------+-----------------+-----------------+
| zone1 | 127.0.0.1:2882 | 4 | 1 | 3 | 5.000000000000 | 1.250000000000 | 3.750000000000 | 20.000000000000 | 10.000000000000 |
+-------+----------------+-----------+--------------+----------+----------------+----------------+----------------+-----------------+-----------------+
1 row in set (0.00 sec)

系统资源单元规格占用硬盘较大，占用了整个虚拟机的硬盘，需要调小一点

创建资源单元规格，资源池和租户

create resource unit myunit1 max_cpu=1,max_memory='2G',max_iops=2000,max_disk_size='6G', max_session_num=1000;

Query OK, 0 rows affected (0.03 sec)

create resource pool my_pool unit='myunit1', unit_num=1;

Query OK, 0 rows affected (0.13 sec)
select unit_count,unit_config_id,zone_list,name from __all_resource_pool;
+------------+----------------+-----------+----------+
| unit_count | unit_config_id | zone_list | name     |
+------------+----------------+-----------+----------+
|          1 |              1 | zone1     | sys_pool |
|          1 |           1001 | zone1     | my_pool  |
+------------+----------------+-----------+----------+

create tenant my_tenant resource_pool_list('my_pool'),primary_zone="RANDOM",charset='utf8' set ob_tcp_invited_nodes='%';

连接租户，创建数据库及表

[admin$ obclient -h127.0.0.1 -uroot@my_tenant -P2881 -p -c -A

 Enter password:
 Welcome to the OceanBase.  Commands end with ; or \g.
 Your MySQL connection id is 3221487944
 Server version: 5.7.25 OceanBase 3.1.0 (r3-b20901e8c84d3ea774beeaca963c67d7802e4b4e) (Built Aug 10 2021 08:10:38)
 
 Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
 MySQL [(none)]> create database mydatabase;
 Query OK, 1 row affected (0.073 sec)
 
 MySQL [(none)]> use mydatabase;
 Database changed

 MySQL [mydatabase]> create table student(
     ->     sno int not null auto_increment primary key,
     ->     sname varchar(10) not null,
     ->     sex char(1) not null,
     ->     sage tinyint(100) not null,
     ->     sdept char(4) not null)comment = '学生表';
 Query OK, 0 rows affected (0.150 sec)
 
 MySQL [mydatabase]> insert into student (sname, sex, sage, sdept) values ('liyong', 'm', 20, 'CS');
 Query OK, 1 row affected (0.047 sec)
 
 MySQL [mydatabase]> insert into student (sname, sex, sage, sdept) values ('liuchen', 'f', 19, 'CS');
 Query OK, 1 row affected (0.012 sec)
 
 MySQL [mydatabase]> insert into student (sname, sex, sage, sdept) values ('wangmin', 'f', 18, 'MA');
 Query OK, 1 row affected (0.012 sec)
 
 MySQL [mydatabase]> insert into student (sname, sex, sage, sdept) values ('zhangli', 'f', 19, 'IS');
 Query OK, 1 row affected (0.012 sec)
 
 MySQL [mydatabase]>
 MySQL [mydatabase]> select * from student;
 +-----+---------+-----+------+-------+
 | sno | sname   | sex | sage | sdept |
 +-----+---------+-----+------+-------+
 |   1 | liyong  | m   |   20 | CS    |
 |   2 | liuchen | f   |   19 | CS    |
 |   3 | wangmin | f   |   18 | MA    |
 |   4 | zhangli | f   |   19 | IS    |
 +-----+---------+-----+------+-------+
 4 rows in set (0.005 sec)

    连接自定义租户时遇到了问题，总是卡在登陆界面，关闭了容器，重启了一下就ok了。

​