园区间业务系统访问故障处理

简介:

故障现象:

客户反映园区仓库访问NC系统速度很慢,从园区ping NC服务器地址时有严重的丢包现象。如下图所示:

wKiom1gz1DKi117aAAKQFNfloAw338.png-wh_50

wKiom1gz1DOx2rwHAAFuqD6gfac287.png-wh_50


处理过程:

登录用户两个园区的核心发现,两个园区的portchannel连接正常,但是其中一个园区portchannel成员端口下存在严重的CRC校验错误,如下图所示:

wKioL1gz08HDzqxmAALCiT0OPh0626.png-wh_50

此时关闭端口ten1/5/1,数据转发从另一个端口走,关闭后再次PING NC服务器,数据包正常

wKioL1gz1KDgfrB5AAFB4YDNDbs447.png-wh_50

wKiom1gz1KCTa0kNAAFlB_3Ct-w146.png-wh_50

为进一步确认该故障,再次对ten1/5/1进行开启操作,但打开后发现,问题立即出现

SW#ping 192.168.x.x repeat 1000 source 192.168.x.x

Type escape sequence to abort.

Sending 1000, 100-byte ICMP Echos to 192.168.x.x, timeout is 2 seconds:

Packet sent with a source address of 192.168.x.x

!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!.!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!.!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!.!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!

!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!.!!!!!!!!!!!!!!!!!..!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!

!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!

Success rate is 97 percent (978/1000), round-trip min/avg/max = 1/2/20 ms

SW#sh int ten1/5/1

TenGigabitEthernet1/5/1 is up, line protocol is up (connected)

  Hardware is Ten Gigabit Ethernet Port, address is 78da.6e07.7b7a (bia 78da.6e07.7b7a)

  Description: connection to laishanchangqu

  MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,

     reliability 245/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 10Gb/s, link type is auto, media type is 10GBase-ER

  input flow-control is on, output flow-control is on

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:03, output never, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 3209000 bits/sec, 358 packets/sec

  5 minute output rate 926000 bits/sec, 685 packets/sec

     4935914010 packets input, 5278619447870 bytes, 0 no buffer

     Received 87068 broadcasts (87050 multicasts)

     0 runts, 0 giants, 0 throttles

     277020288 input errors, 277020288 CRC, 0 frame, 0 overrun, 0 ignored

     0 input packets with dribble condition detected

     9064745454 packets output, 3287137727436 bytes, 0 underruns

     0 output errors, 0 collisions, 4 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier

     0 output buffer failures, 0 output buffers swapped out

SW#conf t

Enter configuration commands, one per line.  End with CNTL/Z.

SW(config)#int ten1/5/1

SW(config-if)#shut

SW(config-if)#do sh run int ten1/5/1

Building configuration...

 

Current configuration : 171 bytes

!

interface TenGigabitEthernet1/5/1

 switchport access vlan 133

 switchport mode access

 shutdown

 channel-group 10 mode on

end

 

SW(config-if)#exit

SW(config)#exit

SW#wr

Building configuration...

Compressed configuration from 21267 bytes to 7568 bytes[OK]

SW#ping 192.168.x.x repeat 1000 source 192.168.x.x

Type escape sequence to abort.

Sending 1000, 100-byte ICMP Echos to 192.168.x.x timeout is 2 seconds:

Packet sent with a source address of 192.168.x.x

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!

Success rate is 100 percent (1000/1000), round-trip min/avg/max = 1/2/20 ms


经最终确认,NC访问问题主要是由于该端口的CRC校验错误引起。


解决建议:

1、检查两厂区之间的链路

2、检查模块

3、检查尾纤


                                                               *********************

终:

       第二天测试两个园区之间的链路,发现一根光纤的衰减在-21左右,而该接口上的万兆模块可接受的衰减最小值是-15.8,后来运营商对线路进行调优检测,最终恢复正常,测试,网络访问正常

                                                               *********************









本文转自 弯月楼主 51CTO博客,原文链接:http://blog.51cto.com/05wylz/1875378,如需转载请自行联系原作者
目录
相关文章
|
3月前
|
监控 安全 Cloud Native
公司监控电脑效率升级:精细化选择与应用
在竞争激烈的商业环境中,高效的员工表现对企业成功至关重要。为此,选用合适的电脑监控方案变得尤为重要。Libera Networks LAN/WAN Manager 提供云原生监控服务,能快速适应企业变化并集成最新威胁情报,但成本较高。WorkWin 是一款优秀的国产监控软件,提供全方位监控功能与个性化策略配置,加之深入的数据分析及严格的数据安全保障。SentinelOne 则专注于端点安全,能有效防御各种威胁,但配置较复杂。OsMonitor 能准确监测员工活动,提供灵活的监控策略及直观的报表功能。这些工具帮助企业提升工作效率的同时,兼顾了安全与隐私。
72 0
|
算法 BI
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.2故障分体系
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.2故障分体系
355 0
|
UED
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.1 故障等级定义
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.1 故障等级定义
1357 0
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3故障管理全流程
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3故障管理全流程
136 0
|
运维 NoSQL 容器
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.3 故障快恢
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.3 故障快恢
235 0
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.5 改进追踪
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.5 改进追踪
148 0
|
运维 监控 中间件
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.1故障发现
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.1故障发现
203 0
|
运维 监控
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.2故障应急
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.2故障应急
350 0
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.4 故障复盘
《云上业务稳定性保障实践白皮书》——三.故障管理体系——3.故障管理全流程——3.3.4 故障复盘
296 0
|
容灾
《医保行业容灾演练云上技术白皮书》——第四章 医保云容灾演练方案——4.4 容灾演练方案——4.4.6 A机房公共区云平台故障演练(入口断网)
《医保行业容灾演练云上技术白皮书》——第四章 医保云容灾演练方案——4.4 容灾演练方案——4.4.6 A机房公共区云平台故障演练(入口断网)