MySQL数据库读写分离

本文涉及的产品
RDS MySQL Serverless 基础系列,0.5-2RCU 50GB
云数据库 RDS MySQL,集群系列 2核4GB
推荐场景:
搭建个人博客
云数据库 RDS MySQL,高可用系列 2核4GB
简介: 使用Amoeba实现MySQL数据库读写分离

1、安装jdk
root@proxy:~# java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

2、配置dbServer.xml
root@proxy:/usr/local/amoeba/conf# cat dbServers.xml
<?xml version="1.0" encoding="gbk"?>


            <!-- 
                    Each dbServer needs to be configured into a Pool,
                    If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
                     add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
                     such as 'multiPool' dbServer 
            -->

    <dbServer name="abstractServer" abstractive="true">
            <factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
                    <property name="manager">${defaultManager}</property>
                    <property name="sendBufferSize">64</property>
                    <property name="receiveBufferSize">128</property>

                    <!-- mysql port -->
                    <property name="port">3306</property>

                    <!-- mysql schema -->
                    <property name="schema">test</property>

                    <!-- mysql user -->
                    <property name="user">root</property>

                    <!-- mysql password -->
                    <property name="password">111</property>

            </factoryConfig>

            <poolConfig class="com.meidusa.amoeba.net.poolable.PoolableObjectPool">
                    <property name="maxActive">500</property>
                    <property name="maxIdle">500</property>
                    <property name="minIdle">10</property>
                    <property name="minEvictableIdleTimeMillis">600000</property>
                    <property name="timeBetweenEvictionRunsMillis">600000</property>
                    <property name="testOnBorrow">true</property>
                    <property name="testOnReturn">true</property>
                    <property name="testWhileIdle">true</property>
            </poolConfig>
    </dbServer>

    <dbServer name="master1" parent="abstractServer">
            <factoryConfig>
                    <!-- mysql ip -->
                    <property name="ipAddress">10.10.10.40</property>
            </factoryConfig>
    </dbServer>

    <dbServer name="master2" parent="abstractServer">
            <factoryConfig>
                    <!-- mysql ip -->
                    <property name="ipAddress">10.10.10.50</property>
            </factoryConfig>
    </dbServer>

    <dbServer name="slave1" parent="abstractServer">
            <factoryConfig>
                    <!-- mysql ip -->
                    <property name="ipAddress">10.10.10.41</property>
            </factoryConfig>
    </dbServer>

    <dbServer name="slave2" parent="abstractServer">
            <factoryConfig>
                    <!-- mysql ip -->
                    <property name="ipAddress">10.10.10.42</property>
            </factoryConfig>
    </dbServer>

rp池配置的是salve1和slave2节点

    <dbServer name="rp" virtual="true">
            <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
                    <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
                    <property name="loadbalance">1</property>

                    <!-- Separated by commas,such as: server1,server2,server1 -->
                    <property name="poolNames">slave1,slave2</property>
            </poolConfig>
    </dbServer>

wp池配置的是master1和master2节点

    <dbServer name="wp" virtual="true">
            <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
                    <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
                    <property name="loadbalance">1</property>

                    <!-- Separated by commas,such as: server1,server2,server1 -->
                    <property name="poolNames">master1,master2</property>
            </poolConfig>
    </dbServer>

3、配置amoeba.xml
root@proxy:/usr/local/amoeba/conf# cat amoeba.xml
<?xml version="1.0" encoding="gbk"?>


    <proxy>

            <!-- service class must implements com.meidusa.amoeba.service.Service -->
            <service name="Amoeba for Mysql" class="com.meidusa.amoeba.net.ServerableConnectionManager">
                    <!-- port -->
                    <property name="port">8066</property>

                    <!-- bind ipAddress -->
                    <!-- 
                    <property name="ipAddress">127.0.0.1</property>
                     -->

                    <property name="manager">${clientConnectioneManager}</property>

                    <property name="connectionFactory">
                            <bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
                                    <property name="sendBufferSize">128</property>
                                    <property name="receiveBufferSize">64</property>
                            </bean>
                    </property>

                    <property name="authenticator">
                            <bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">

                                    <property name="user">root</property>

                                    <property name="password">root</property>

                                    <property name="filter">
                                            <bean class="com.meidusa.amoeba.server.IPAccessController">
                                                    <property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
                                            </bean>
                                    </property>
                            </bean>
                    </property>

            </service>

            <!-- server class must implements com.meidusa.amoeba.service.Service -->
            <service name="Amoeba Monitor Server" class="com.meidusa.amoeba.monitor.MonitorServer">
                    <!-- port -->
                    <!-- default value: random number
                    <property name="port">9066</property>
                    -->
                    <!-- bind ipAddress -->
                    <property name="ipAddress">127.0.0.1</property>
                    <property name="daemon">true</property>
                    <property name="manager">${clientConnectioneManager}</property>
                    <property name="connectionFactory">
                            <bean class="com.meidusa.amoeba.monitor.net.MonitorClientConnectionFactory"></bean>
                    </property>

            </service>

            <runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
                    <!-- proxy server net IO Read thread size -->
                    <property name="readThreadPoolSize">20</property>

                    <!-- proxy server client process thread size -->
                    <property name="clientSideThreadPoolSize">30</property>

                    <!-- mysql server data packet process thread size -->
                    <property name="serverSideThreadPoolSize">30</property>

                    <!-- per connection cache prepared statement size -->
                    <property name="statementCacheSize">500</property>

                    <!-- query timeout( default: 60 second , TimeUnit:second) -->
                    <property name="queryTimeout">60</property>
            </runtime>

    </proxy>

    <!-- 
            Each ConnectionManager will start as thread
            manager responsible for the Connection IO read , Death Detection
    -->
    <connectionManagerList>
            <connectionManager name="clientConnectioneManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
                    <property name="subManagerClassName">com.meidusa.amoeba.net.ConnectionManager</property>
                    <!-- 
                      default value is avaliable Processors 
                    <property name="processors">5</property>
                     -->
            </connectionManager>
            <connectionManager name="defaultManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
                    <property name="subManagerClassName">com.meidusa.amoeba.net.AuthingableConnectionManager</property>

                    <!-- 
                      default value is avaliable Processors 
                    <property name="processors">5</property>
                     -->
            </connectionManager>
    </connectionManagerList>

            <!-- default using file loader -->
    <dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
            <property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
    </dbServerLoader>

    <queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
            <property name="ruleLoader">
                    <bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
                            <property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
                            <property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
                    </bean>
            </property>

default池只配置了master1节点

            <property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
            <property name="LRUMapSize">1500</property>
            <property name="defaultPool">master1</property>

            <property name="writePool">wp</property>
            <property name="readPool">rp</property>

            <property name="needParse">true</property>
    </queryRouter>

4、启动amoeba
root@proxy:/usr/local/amoeba# bin/amoeba start &
[1] 2511
root@proxy:/usr/local/amoeba#
The stack size specified is too small, Specify at least 160k
Could not create the Java virtual machine.

[1]+ Exit 1 bin/amoeba start
更改bin/amoeba文件
DEFAULT_OPTS="-server -Xms256m -Xmx256m -Xss128k"
更改为
DEFAULT_OPTS="-server -Xms256m -Xmx256m -Xss256k"

启动正常之后,连接amoeba

root@proxy:~# mysql -uroot -p -h10.10.10.39 -P8066
Enter password:
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 307988519
Server version: 5.1.45-mysql-amoeba-proxy-2.2.0

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

  1. Other names may be trademarks of their respective
    owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

java.util.NoSuchElementException: Could not create a validated object, cause: ValidateObject failed
root@10.10.10.39:(none) MySQL-> at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1191)

    at com.meidusa.amoeba.net.poolable.GenericObjectPool.borrowObject(GenericObjectPool.java:381)
    at com.meidusa.amoeba.mysql.handler.CommandMessageHandler.startSession(CommandMessageHandler.java:633)
    at com.meidusa.amoeba.mysql.handler.MySqlCommandDispatcher.handleMessage(MySqlCommandDispatcher.java:123)
    at com.meidusa.amoeba.mysql.net.MysqlClientConnection$2.run(MysqlClientConnection.java:291)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
    

连接报错,查看root.log的输出
2015-08-29 16:01:03,299 INFO net.BackendConnectionFactory - open socket channel to server[10.10.10.40:3306]
2015-08-29 16:01:03,303 ERROR net.MysqlServerConnection - handShake with /10.10.10.40:3306 error:Access denied for user 'root'@'10.10.10.39' (using password: NO),hashCode=661272757
2015-08-29 16:01:03,324 INFO net.BackendConnectionFactory - open socket channel to server[10.10.10.50:3306]
2015-08-29 16:01:03,327 ERROR net.MysqlServerConnection - handShake with /10.10.10.50:3306 error:Access denied for user 'root'@'10.10.10.39' (using password: NO),hashCode=505588567
2015-08-29 16:01:03,331 INFO net.BackendConnectionFactory - open socket channel to server[10.10.10.41:3306]
2015-08-29 16:01:03,334 ERROR net.MysqlServerConnection - handShake with /10.10.10.41:3306 error:Access denied for user 'root'@'10.10.10.39' (using password: NO),hashCode=530654357
2015-08-29 16:01:03,335 INFO net.BackendConnectionFactory - open socket channel to server[10.10.10.42:3306]
2015-08-29 16:01:03,338 ERROR net.MysqlServerConnection - handShake with /10.10.10.42:3306 error:Access denied for user 'root'@'10.10.10.39' (using password: NO),hashCode=848649429

查看数据库连接权限

root@localhost:mysql MySQL-> select host,user from user;
host user
% rep_user
% root
127.0.0.1 root
::1 root
localhost
localhost root
ubuntu
ubuntu root

8 rows in set (0.00 sec)

数据库权限正常,使用客户端连接测试
root@proxy:~# mysql -uroot -p -h10.10.10.40
Enter password:
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 417
Server version: 5.6.19-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial)

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

  1. Other names may be trademarks of their respective
    owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

root@10.10.10.40:(none) MySQL-> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

root@10.10.10.40:test MySQL-> show tables;
Tables_in_test
t1

1 row in set (0.00 sec)

客户端连接也正常,猜想应该是amoeba配置有问题

                    <!-- mysql password
                    <property name="password">111</property>
                    -->

发现密码被注释了,更改

                    <!-- mysql password -->
                    <property name="password">111</property>

启动一切正常

root@proxy:~# mysql -uroot -p -h10.10.10.39 -P8066

Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1113757163
Server version: 5.1.45-mysql-amoeba-proxy-2.2.0 MySQL Enterprise Server - Advanced Edition (Commercial)

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

[root@10.10.10.39:(none)][12:06:15am] MySQL-> use test;
Database changed
[root@10.10.10.39:test][12:06:18am] MySQL->
[root@10.10.10.39:test][12:06:18am] MySQL-> show tables;
+----------------+
| Tables_in_test |
+----------------+
| t1 |
+----------------+
1 row in set (0.00 sec)

5、测试读写分离
停止master2,slave1,slave2节点的复制
root@localhost:(none) MySQL-> stop slave;
Query OK, 0 rows affected (0.04 sec)

root@localhost:(none) MySQL-> show slave status G
1. row **

           Slave_IO_State: 
              Master_Host: 10.10.10.40
              Master_User: rep_user
              Master_Port: 3306
            Connect_Retry: 60
          Master_Log_File: master1-bin.000007
      Read_Master_Log_Pos: 442
           Relay_Log_File: slave-relay-bin.000008
            Relay_Log_Pos: 607
    Relay_Master_Log_File: master1-bin.000007
         Slave_IO_Running: No
        Slave_SQL_Running: No
        

使用amoeba连接,往测试表t1中插入测试数据
root@proxy:~# mysql -uroot -p -h10.10.10.39 -P8066
Enter password:
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 750410715
Server version: 5.1.45-mysql-amoeba-proxy-2.2.0 MySQL Enterprise Server - Advanced Edition (Commercial)

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

  1. Other names may be trademarks of their respective
    owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

root@10.10.10.39:(none) MySQL-> use test;
Database changed
root@10.10.10.39:test MySQL-> insert into t1 values (null,'aaaaa');
Query OK, 1 row affected (0.05 sec)

master1查看表t1

root@localhost:mysql MySQL-> select * from test.t1;
id name
1 aaaaa

1 row in set (0.00 sec)

在master2,slave1,slave2上查看表t1
root@localhost:(none) MySQL-> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
root@localhost:test MySQL-> select * from t1;
Empty set (0.00 sec)

在proxy上查看表t1表的数据
root@10.10.10.39:test MySQL-> select * from t1;
Empty set (0.05 sec)

在proxy上无法查看插入的数据,说明读是在rp池里,而rp池只有slave1和slave2,数据没有同步到从节点。

相关实践学习
如何在云端创建MySQL数据库
开始实验后,系统会自动创建一台自建MySQL的 源数据库 ECS 实例和一台 目标数据库 RDS。
全面了解阿里云能为你做什么
阿里云在全球各地部署高效节能的绿色数据中心,利用清洁计算为万物互联的新世界提供源源不断的能源动力,目前开服的区域包括中国(华北、华东、华南、香港)、新加坡、美国(美东、美西)、欧洲、中东、澳大利亚、日本。目前阿里云的产品涵盖弹性计算、数据库、存储与CDN、分析与搜索、云通信、网络、管理与监控、应用服务、互联网中间件、移动服务、视频服务等。通过本课程,来了解阿里云能够为你的业务带来哪些帮助 &nbsp; &nbsp; 相关的阿里云产品:云服务器ECS 云服务器 ECS(Elastic Compute Service)是一种弹性可伸缩的计算服务,助您降低 IT 成本,提升运维效率,使您更专注于核心业务创新。产品详情: https://www.aliyun.com/product/ecs
目录
相关文章
|
6月前
|
关系型数据库 MySQL Java
MySQL的主从复制 && SpringBoot整合Sharding-JDBC解决读写分离
MySQL的主从复制 && SpringBoot整合Sharding-JDBC解决读写分离
140 0
|
6月前
|
SQL 关系型数据库 MySQL
Mycat【Mycat部署安装(核心配置及目录结构、安装以及管理命令详解)Mycat高级特性(读写分离概述、搭建读写分离、MySQL双主双从原理)】(三)-全面详解(学习总结---从入门到深化)
Mycat【Mycat部署安装(核心配置及目录结构、安装以及管理命令详解)Mycat高级特性(读写分离概述、搭建读写分离、MySQL双主双从原理)】(三)-全面详解(学习总结---从入门到深化)
587 0
|
1月前
|
负载均衡 监控 关系型数据库
MySQL 官宣:支持读写分离了!!
【10月更文挑战第8天】MySQL的读写分离功能显著提升了数据库性能、可用性和可靠性。通过将读写操作分配至不同服务器,有效减轻单个服务器负载,提高响应速度与吞吐量,并增强系统稳定性。此外,它还支持便捷的扩展方式,可通过增加只读服务器提升读操作性能。实现读写分离的方法包括软件层面(如使用数据库中间件)和硬件层面(使用独立服务器)。使用时需注意数据一致性、负载均衡及监控管理等问题。
106 0
|
6月前
|
SQL 关系型数据库 MySQL
mysql 主从复制与读写分离
mysql 主从复制与读写分离
|
2月前
|
关系型数据库 MySQL Java
MySQL主从复制实现读写分离
MySQL主从复制(二进制日志)、 Sharding-JDBC实现读写分离
MySQL主从复制实现读写分离
|
3月前
|
SQL 关系型数据库 MySQL
(二十五)MySQL主从实践篇:超详细版读写分离、双主热备架构搭建教学
在上篇《主从原理篇》中,基本上把主从复制原理、主从架构模式、数据同步方式、复制技术优化.....等各类细枝末节讲清楚了,本章则准备真正对聊到的几种主从模式落地实践,但实践的内容通常比较枯燥乏味,因为就是调整各种配置、设置各种参数等步骤。
546 2
|
3月前
|
关系型数据库 MySQL PHP
开发者必看:MySQL主从复制与Laravel读写分离的完美搭配
开发者必看:MySQL主从复制与Laravel读写分离的完美搭配
72 2
|
3月前
|
SQL 关系型数据库 MySQL
mysql读写分离,主从同步
本文介绍了如何在Laravel项目中配置数据库读写分离,并实现MySQL主从同步。主要步骤包括:在`config/database.php`中设置读写分离配置;为主机授予从机访问权限;配置各MySQL服务器的`/etc/my.cnf`文件以确保唯一的`server-id`;以及通过SQL命令设置主从关系并启动从服务。文章还针对一些常见错误提供了排查方法。最后通过验证确认主从同步是否成功。[原文链接](https://juejin.cn/post/6901581801458958344)。版权所有者为作者佤邦帮主,转载请遵循相关规定。
|
3月前
|
cobar 关系型数据库 MySQL
使用MyCat实现MySQL主从读写分离(一)概述
【8月更文挑战第11天】MySQL读写分离通过主从复制分散负载,主库负责写操作,从库承担读查询,以复制技术确保数据一致性。此策略有效缓解锁竞争,提升查询效能并增强系统可用性。实现方式包括应用层处理,简便快捷但灵活性受限;或采用中间件如MyCAT、Vitess等,支持复杂场景但需专业团队维护。
112 0
|
3月前
|
SQL 关系型数据库 MySQL
基于proxysql实现MySQL读写分离
基于proxysql实现MySQL读写分离