Hbase 客户端抛出 RegionTooBusyException: Above memstore limit 异常,请问本次访问请求就会失败吗?如果是Kafka写数据到Hbase ,数据就会丢失吗?如何避免客户端报此异常?
Kafka写数据入Hbase报错,挺频繁的,隔断时间就会报。错误如下:
18/04/13 09:04:18 INFO client.AsyncProcess: #2758513, waiting for some tasks to finish. Expected max=0, tasksInProgress=11
18/04/13 09:04:28 INFO client.AsyncProcess: #2758513, waiting for some tasks to finish. Expected max=0, tasksInProgress=12
18/04/13 09:04:28 INFO client.AsyncProcess: #2758513, table=monitor_sql, attempt=12/35 failed=1ops, last exception: org.apache.hadoop.hbase.RegionTooBusyException: org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit, regionName=monitor_sql,000152357813618538,1523581185750.16d97b72e0329effd15a7c7f170213e4., server=yq-hadoop184136,60020,1519785020264, memstoreSize=268483248, blockingMemStoreSize=268435456
at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:3649)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2859)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2810)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:751)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:713)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
on yq-hadoop184136,60020,1519785020264, tracking started null, retrying after=20046ms, replay=1ops
18/04/13 09:04:48 INFO client.AsyncProcess: #2758513, table=monitor_sql, attempt=13/35 failed=1ops, last exception: org.apache.hadoop.hbase.RegionTooBusyException: org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit, regionName=monitor_sql,000152357813618538,1523581185750.16d97b72e0329effd15a7c7f170213e4., server=yq-hadoop184136,60020,1519785020264, memstoreSize=268483248, blockingMemStoreSize=268435456
at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:3649)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2859)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2810)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:751)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:713)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
on yq-hadoop184136,60020,1519785020264, tracking started null, retrying after=20013ms, replay=1ops
18/04/13 09:04:48 INFO client.AsyncProcess: #2758513, waiting for some tasks to finish. Expected max=0, tasksInProgress=13
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。
都已经异常了。应该会失败。
本次请求会失败,客户端会重试。客户端有一定重试次数,重试次数全部耗尽就会向上层抛异常,表示写入失败。这时候用户需要再次调用接口才行。你看到的日志是
你日志中的 attempt=12/35 就是你重试的次数,当重试达到一定次数时候,客户端会开始记录错误日志,所以你看到的这个日志只是说这次请求失败,还在继续重试。
可以参考这篇文章,了解下原因:https://yq.aliyun.com/articles ... YlRjS