URL: https://www.opennet.me/cgi-bin/openforum/vsluhboard.cgi
Форум: vsluhforumID6
Нить номер: 22043
[ Назад ]
Исходное сообщение
"На cisco 2811 падают оба провайдера сразу!!!"
Отправлено Gordje , 15-Дек-10 06:18 
Предыстория:
Есть территориально распределённая сеть по дальнему востоку 7 (городов). Как следствие 7 цисок 2811! Все как сёстры близнецы по конфигу - задачи простые:
1. выход в инет через двух провайдеров (основной/резервный), реализовано на ip sla.
2. поддержание туннелей ipip между одним городом и всеми остальными.

ПРОБЛЕМА !!!
ТОЛЬКО на одном филиале с периодичностью в 1,5 суток падают оба провайдера, как следствие падают туннели - лечиться перезагрузкой!!! В логах много записей типа:

Dec 15 07:05:45 172.16.1.1 956: %TRACKING-5-STATE: 101 ip sla 101 reachability Up->Down
Dec 15 07:05:59 172.16.1.1 957: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel500203, changed state to up
Dec 15 07:06:07 172.16.1.1 958: %OSPF-5-ADJCHG: Process 51, Nbr 255.255.255.255 on Tunnel500101 from EXSTART to DOWN, Neighbor Down: Interface down or detached
Dec 15 07:06:15 172.16.1.1 961: %TRACKING-5-STATE: 101 ip sla 101 reachability Down->Up

очевидно что одно следствие другого! ни каких др. записей просто нет! с какого-то момента начинают сыпать такие мэсэджи, связи нет и до самой перезагрузки забивается лог файл!

ДАНО:
cisco 2811 + HWIC-4ESW
IOS c2800nm-advsecurityk9-mz.124-24.T2.bin
провайдеры подключены к HWIC-4ESW дальше VLAN'ами

СДЕЛАНО:
1. изменены тайминги ip sla с дефолтовых на побольше ~ sla принемает решение около минуты
2. полностью сменено оборудование и cisco 2811 и HWIC-4ESW
3. поставлена перезагрузка по крону раз в сутки <- НЕ ВАРИАНТ!!!

ПОДОЗРЕНИЯ:
1. кто то ложит циску из вне! (DoS атаки)
2. не качественная работа одного из провайдеров (постоянное "дребезжание" провайдера => дерганье ip sla => переполнение какого-нибудь буфера => завал интерфейсов)

Содержание
Сообщения в этом обсуждении
"На cisco 2811 падают оба провайдера сразу!!!"
Отправлено AP , 15-Дек-10 16:40 
Конфиг покажите.
"На cisco 2811 падают оба провайдера сразу!!!"
Отправлено Gordje , 16-Дек-10 03:50 
> Конфиг покажите.

конечно:

IP провайдеров соответственно:
  IPS1 - X1.X1.X1.X1/30 (шлюз "GW ISP 1")
  IPS2 - X2.X2.X2.X2/30 (шлюз "GW ISP 2")

IP провайдеров на филиале:
  ISP 1 - F1.F1.F1.F1
  ISP 1 - F2.F2.F2.F2

IP провайдеров в центральном офисе:
  ISP 1 - C1.C1.C1.C1
  ISP 1 - C2.C2.C2.C2

Сам конфиг:


!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Gorod
!
boot-start-marker
boot-end-marker
!
logging message-counter syslog
logging buffered 10240
logging rate-limit all 10 except errors
enable secret 5 ******************************
!
no aaa new-model
clock timezone Gorod 10
clock summer-time Gorod recurring last Sun Mar 2:00 last Sun Oct 2:00
!
dot11 syslog
ip source-route
!
!
ip cef
!
!
no ip domain lookup
!
multilink bundle-name authenticated
!
!
!
!
!
!
username user01 privilege 15 secret 5 ******************************
username user02 privilege 15 secret 5 ******************************
username user03 privilege 15 secret 5 ******************************
username user04 privilege 5 secret 5 ******************************
username user04 autocommand menu Admin
archive
log config
  hidekeys
!
!
!
!
!
!
track 101 ip sla 101 reachability
!
track 102 ip sla 102 reachability
!
!
!
interface Loopback1
ip address 172.16.1.10 255.255.255.255
!
interface Tunnel350101
description to filial via ISP 1 to ISP 1
ip unnumbered Loopback1
ip mtu 1500
ip ospf database-filter all out
keepalive 10 3
tunnel source Vlan101
tunnel destination F1.F1.F1.F1
tunnel mode ipip
!
interface Tunnel350202
description to filial via ISP 2 to ISP 2
ip unnumbered FastEthernet0/1
ip ospf database-filter all out
shutdown
keepalive 10 3
tunnel source X2.X2.X2.X2
tunnel destination F2.F2.F2.F2
tunnel mode ipip
!
interface Tunnel500101
description to Center via ISP 1 to ISP 1
ip unnumbered FastEthernet0/1
ip mtu 1500
ip ospf cost 10
keepalive 10 3
tunnel source Vlan101
tunnel destination C1.C1.C1.C1
!
interface Tunnel500203
description to Center via ISP 2 to ISP 2
ip unnumbered FastEthernet0/1
ip mtu 1500
ip ospf cost 20
keepalive 10 3
tunnel source X2.X2.X2.X2
tunnel destination C2.C2.C2.C2
!
interface FastEthernet0/0
no ip address
shutdown
duplex auto
speed auto
!
interface FastEthernet0/1
description to LAN
ip address 172.16.1.1 255.255.255.248
ip nat inside
no ip virtual-reassembly
ip ospf database-filter all out
duplex auto
speed auto
!
interface FastEthernet0/0/0
switchport access vlan 101
!
interface FastEthernet0/0/1
switchport access vlan 102
!
interface FastEthernet0/0/2
shutdown
!
interface FastEthernet0/0/3
shutdown
!
interface Vlan1
no ip address
shutdown
!
interface Vlan101
description ISP 1
ip address X1.X1.X1.X1 255.255.255.252
ip nat outside
no ip virtual-reassembly
!
interface Vlan102
description ISP 2
ip address X2.X2.X2.X2 255.255.255.252
ip nat outside
no ip virtual-reassembly
!
router ospf 51
router-id 10.40.40.19
log-adjacency-changes
redistribute connected subnets route-map OSPF
redistribute static subnets route-map OSPF
network 172.16.1.0 0.0.0.7 area 0.0.0.255
distribute-list 20 in
!
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 "GW ISP 1" track 101
ip route 0.0.0.0 0.0.0.0 "GW ISP 2" 50 track 102
ip route 10.40.35.0 255.255.255.0 Tunnel350101
ip route 10.40.41.0 255.255.255.0 172.16.1.2
ip route F1.F1.F1.F1 255.255.255.255 "GW ISP 1"
ip route F2.F2.F2.F2 255.255.255.255 "GW ISP 2"
ip route C1.C1.C1.C1 255.255.255.255 "GW ISP 1"
ip route C2.C2.C2.C2 255.255.255.255 "GW ISP 2"
no ip http server
no ip http secure-server
!
!
ip nat inside source route-map 101_NAT_ISP_1 interface Vlan101 overload
ip nat inside source route-map 102_NAT_ISP_2 interface Vlan102 overload
ip nat inside source static tcp 172.16.1.2 20 X2.X2.X2.X2 20 extendable
ip nat inside source static tcp 172.16.1.2 21 X2.X2.X2.X2 21 extendable
ip nat inside source static tcp 172.16.1.2 22 X2.X2.X2.X2 22 extendable
ip nat inside source static tcp 172.16.1.2 25 X2.X2.X2.X2 25 extendable
ip nat inside source static tcp 172.16.1.2 53 X2.X2.X2.X2 53 extendable
ip nat inside source static udp 172.16.1.2 53 X2.X2.X2.X2 53 extendable
ip nat inside source static tcp 172.16.1.2 110 X2.X2.X2.X2 110 extendable
ip nat inside source static tcp 172.16.1.2 1723 X2.X2.X2.X2 1723 extendable
ip nat inside source static tcp 172.16.1.2 20 X1.X1.X1.X1 20 extendable
ip nat inside source static tcp 172.16.1.2 21 X1.X1.X1.X1 21 extendable
ip nat inside source static tcp 172.16.1.2 22 X1.X1.X1.X1 22 extendable
ip nat inside source static tcp 172.16.1.2 25 X1.X1.X1.X1 25 extendable
ip nat inside source static tcp 172.16.1.2 53 X1.X1.X1.X1 53 extendable
ip nat inside source static udp 172.16.1.2 53 X1.X1.X1.X1 53 extendable
ip nat inside source static tcp 172.16.1.2 110 X1.X1.X1.X1 110 extendable
ip nat inside source static tcp 172.16.1.2 1723 X1.X1.X1.X1 1723 extendable
!
ip sla 101
icmp-jitter "GW ISP 1" source-ip X1.X1.X1.X1 num-packets 20 interval 50
timeout 10000
threshold 10000
frequency 30
history hours-of-statistics-kept 3
ip sla schedule 101 life forever start-time now
ip sla 102
icmp-jitter "GW ISP 2" source-ip X2.X2.X2.X2 num-packets 20 interval 50
timeout 10000
threshold 10000
frequency 30
history hours-of-statistics-kept 3
ip sla schedule 102 life forever start-time now
logging facility local1
logging 10.40.50.1
logging 172.16.1.2
access-list 20 remark ============filtering incoming OSPF routes===========
access-list 20 permit 10.40.40.0
access-list 20 permit 10.40.50.0
access-list 20 permit 10.23.0.0
access-list 20 permit 10.20.0.0
access-list 20 permit 10.21.0.0
access-list 20 permit 172.16.255.0
access-list 20 permit 10.40.255.41
access-list 20 remark =====================================================
access-list 189 remark ======for OSPF redistribute & NAT via both ISPs=====
access-list 189 permit ip 172.16.1.0 0.0.0.7 any
access-list 189 permit ip 10.40.41.0 0.0.0.255 any
access-list 189 remark ====================================================
access-list 190 remark ======for OSPF redistribute=========================
access-list 190 permit ip 10.40.35.0 0.0.0.255 any
access-list 190 remark ====================================================
!
menu Admin title 
MENU

menu Admin text 1 Show Interfaces Status
menu Admin command 1 sh ip int bri
menu Admin options 1 pause
menu Admin text 2 Show routing table
menu Admin command 2 sh ip route
menu Admin options 2 pause
menu Admin text 3 Tunnel to filial (via ISP 2 to ISP 2)
menu Admin command 3 eve ma run filisp2
menu Admin text 4 Tunnel to filial (via ISP 1 to ISP 1)
menu Admin command 4 eve ma run filisp1
menu Admin text 5 Ping to ISP 1 GW ("GW ISP 1")
menu Admin command 5 ping "GW ISP 1"
menu Admin options 5 pause
menu Admin text 6 Ping to ISP 2 GW ("GW ISP 2")
menu Admin command 6 ping "GW ISP 2"
menu Admin options 6 pause
menu Admin text 7 Ping to Center ISP 1 (C1.C1.C1.C1)
menu Admin command 7 ping C1.C1.C1.C1
menu Admin options 7 pause
menu Admin text 8 Ping to Center ISP 2 (C2.C2.C2.C2)
menu Admin command 8 ping C2.C2.C2.C2
menu Admin options 8 pause
menu Admin text 9 Logoff
menu Admin command 9 exit
menu Admin clear-screen
menu Admin default 1
menu Admin single-space
!
!
!
route-map OSPF permit 10
match ip address 189 190
!
route-map 101_NAT_ISP_1 permit 10
match ip address 189
match interface Vlan101
!
route-map 102_NAT_ISP_2 permit 10
match ip address 189
match interface Vlan102
!
!
!
control-plane
!
privilege exec level 5 event manager run
privilege exec level 5 event manager
privilege exec level 5 event
privilege exec level 5 show ip route
privilege exec level 5 show ip interface brief
privilege exec level 5 show ip interface
privilege exec level 5 show ip
privilege exec level 5 show
!
line con 0
line aux 0
line vty 0 4
no motd-banner
exec-timeout 30 0
privilege level 15
login local
transport input telnet
line vty 5 15
no motd-banner
exec-timeout 30 0
privilege level 15
login local
transport input telnet
!
scheduler allocate 20000 1000
ntp server 10.40.50.4
event manager applet filisp2
event none
action 1.0 cli command "en"
action 1.1 cli command "conf t"
action 2.0 cli command "int Tunnel350202"
action 2.1 cli command "no shut"
action 2.2 cli command "exit"
action 3.0 cli command "no ip route 10.40.35.0 255.255.255.0 Tunnel350101"
action 3.1 cli command "ip route 10.40.35.0 255.255.255.0 Tunnel350202"
action 4.0 cli command "int Tunnel350101"
action 4.1 cli command "shut"
action 4.2 cli command "exit"
action 5.0 cli command "end"
event manager applet filisp1
event none
action 1.0 cli command "en"
action 1.1 cli command "conf t"
action 2.0 cli command "int Tunnel350101"
action 2.1 cli command "no shut"
action 2.2 cli command "exit"
action 3.0 cli command "no ip route 10.40.35.0 255.255.255.0 Tunnel350202"
action 3.1 cli command "ip route 10.40.35.0 255.255.255.0 Tunnel350101"
action 4.0 cli command "int Tunnel350202"
action 4.1 cli command "shut"
action 4.2 cli command "exit"
action 5.0 cli command "end"
!
end

"На cisco 2811 падают оба провайдера сразу!!!"
Отправлено Gordje , 21-Дек-10 09:29 
Судя по всему предложений нет! (
"На cisco 2811 падают оба провайдера сразу!!!"
Отправлено aZL , 21-Дек-10 09:42 
Cбросьте счетчик на туннелях и попробуйте потом посмотреть статистику потерь.
Возможно будет достаточно уменьшить mtu

"На cisco 2811 падают оба провайдера сразу!!!"
Отправлено Gordje , 23-Дек-10 03:05 
> Cбросьте счетчик на туннелях и попробуйте потом посмотреть статистику потерь.
> Возможно будет достаточно уменьшить mtu

Tunnel350101 is up, line protocol is up

...

  5 minute input rate 8000 bits/sec, 15 packets/sec
  5 minute output rate 264000 bits/sec, 23 packets/sec
     405260 packets input, 35124378 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     425323 packets output, 398216281 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

Tunnel350202 is administratively down, line protocol is down

...

  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     0 packets output, 0 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

Tunnel500101 is up, line protocol is up

...

  5 minute input rate 2000 bits/sec, 3 packets/sec
  5 minute output rate 2000 bits/sec, 2 packets/sec
     948961 packets input, 1104370134 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     462228 packets output, 58798668 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

Tunnel500203 is up, line protocol is up

...

  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     18467 packets input, 1517216 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     18743 packets output, 1541276 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out


ни каких потерь на туннелех нет вообще!!!

или я что ни так понял?

"На cisco 2811 падают оба провайдера сразу!!!"
Отправлено Gordje , 29-Дек-10 10:54 
И тишина ...
Подскажите хоть как и какие параметры помониторить на циске, но что б обязательно результаты в лог писались (отправка сообщений на syslog настроена)!