Subject: (usr-tc) arcs and ospf

From: Mike Andrews <[email-masked]>
Date: Tue, 12 Oct 1999 03:24:35 +0000

I've been having a really bizarre problem off and on the last week or two.
Occasionally some of my ARCs will suddenly stop doing OSPF right.  The
routing table loses all OSPF routes, and the ARC stops announcing its
routes via OSPF to our Ciscos.  Disabling/enabling OSPF doesn't help, but
most of the time (not all the time) rebooting the ARC will.

One thing that seemed to trigger it was rebooting some of our Ciscos...
and since that happens so infrequently (not rebooted since going to ARC
4.2 in fact) it's hard to track down.  I don't know which one exactly,
maybe our AS border router dying is what threw it off.

It also happened last night when I installed a Cisco Catalyst 2924XL
(replacing an old 10BaseT hub, finally!) which didn't involve rebooting
the Ciscos, but involved having them unreachable for a minute or two as
cables were switched and Spanning Tree did its thing...

I have another ARC with no modems on the same LAN (for testing) and it
initially did the same thing, but now I'm having trouble reproducing it.
When it was screwed up one night, it fixed itself overnight, but it
obviously took hours to do, which kinda defeats the purpose of OSPF.

I know this is really vague, but problem is, I don't really know where to
start looking for this, because I didn't document very well what my
current OSPF settings are.  (A bulk-config-to-ASCII utility would be
REALLY HELPFUL HERE :)  I could reverse-engineer HARM and try to get a
dump of that part of the config via SNMP somehow, I guess, but... yuck.
Anyway, I'm also not a real in-depth OSPF guru yet.

Has anyone seen anything like this before or know of any particular
settings that I might want to check?  This is driving me nuts (not to
mention my customers) and I'm not sure where to start.

A quick look at some of the 'show' screens -- this is with the card
actually routing correctly:

> show ospf global

GLOBAL OSPF SETTINGS
Router ID:                                 206.240.130.12
Administrative Status:                     ENABLED
OSPF Version Number:                       2
Area Border Router Status:                 DISABLED
Autonomous System Border Router Status:    ENABLED
Type-of-service (TOS) Support:             OFF
External LSDB Limit:                       -1
Multicast Extensions Support:              OFF
Exit Overflow Interval:                    0
Router's Support for OSPF Demand Routing:  OFF
OSPF Traps:                                DISABLED

> show ospf area 0.0.0.0

OSPF AREA SETTINGS
Area ID:                                   0.0.0.0
OSPF Area Type:                            TRANSIT
Area Status:                               ENABLED

> list ospf recei

OSPF Receive Policies:
 Address/Mask       Action            Routing Preference
 199.077.100.000/23 Accept            Pref1
 206.240.130.000/23 Accept            Pref1
 208.006.168.000/22 Accept            Pref1

> list ospf send

OSPF Send Policies:
Source  Address/Mask       Action
LOCAL   199.077.100.000/23 Advertise
LOCAL   206.240.130.000/23 Advertise
LOCAL   208.006.168.000/22 Advertise

> list ospf int
OSPF INTERFACE TABLE
IfIpAddr/IfInd  AreaId          IfType AdminStat IfState      Metric
206.240.130.12  0.0.0.0         BC     ENABLED   OtherDsgRtr  10

Mike Andrews (MA12) * [email-masked] * http://www.bit0.com/
VP, sysadmin, & network guy, Digital Crescent Inc, Frankfort KY
Internet services for Frankfort, Lawrenceburg, Owenton, & Shelbyville
"With sufficient thrust, pigs fly just fine." -- RFC 1925

-
 To unsubscribe to usr-tc, send an email to "[email-masked]"
 with "unsubscribe usr-tc" in the body of the message.
 For information on digests or retrieving files and old messages send
 "help" to the same address.  Do not use quotes in your message.

---

From: Jeff Mcadams <[email-masked]>
Date: Tue, 12 Oct 1999 03:38:21 +0000

Thus spake Mike Andrews

>maybe our AS border router dying is what threw it off.

Hrmm...something to check...not sure if this will have any
signficance...is the settings/values for the designated router on your
ethernet(s) when this happens.  I suspect your Cisco's were the
designated routers initially, and when they went away, one of your Arc's
got promoted to that...who knows what happened when your cisco's came
back.

When I was first playing with 4.2.29, I noticed a little bit of odd
behavior with the Arc's wrt designated routers...but then when I set up
a test lab environment for it, I couldn't reproduce it.  What I saw was
then I brought the Arc up on 4.2.29, it would take over the designated
router functionality somehow...which would be against the OSPF spec, but
possible for it to happen.  I haven't played with OSPF on the arc's
since then in any great depth...I've got one up and running on 4.2.32 at
this point, but haven't really done anything with it.

Maybe this will give you an idea of something to look at.  Switching
designated routers is definitely something you want to avoid if at all
possible because its going to cause a resyncronization of LSA databases
and a recalc of most (all?) of the shortest path first
calculations...so...this *can* cause up to a couple of minutes of your
routing being hosed as potentially every router on your network recalcs
its OSPF routes.
-- 
Jeff McAdams                            Email: [email-masked]
Head Network Administrator              Voice: (502) 966-3848
IgLou Internet Services                        (800) 436-4456

-

---

From: Mike Andrews <[email-masked]>
Date: Tue, 12 Oct 1999 20:07:39 +0000

On Tue, 12 Oct 1999, Mike Wronski wrote:

> |Mike Andrews writes...
> |>I've been having a really bizarre problem off and on the last week or two.
> |>Occasionally some of my ARCs will suddenly stop doing OSPF right.  The
> |>routing table loses all OSPF routes, and the ARC stops announcing its
> |>routes via OSPF to our Ciscos.  Disabling/enabling OSPF doesn't help, but
> |>most of the time (not all the time) rebooting the ARC will.
> |>
> |>One thing that seemed to trigger it was rebooting some of our Ciscos...
> |>and since that happens so infrequently (not rebooted since going to ARC
> |>4.2 in fact) it's hard to track down.  I don't know which one exactly,
> |>maybe our AS border router dying is what threw it off.
> |> 
> |> . . .
> |>
> |>Has anyone seen anything like this before or know of any particular
> |>settings that I might want to check?  This is driving me nuts (not to
> |>mention my customers) and I'm not sure where to start.
> |
> |Make sure your tc's aren't becoming the DR.  To check, do a "list ospf nei",
> |see if any of them are in any state other than "two way".  To fix, do a
> |"set ospf interface THE_OSPF_INTERFACE router_priority 0" on each of
> |them and reboot.
> |
> 
> That recomendation is actually in the release notes for 4.2.32...

OK, that's three votes for DR/BDR designation being the problem. :)  And
dumb me for not paying closer attention to the release notes.

Basically I had left every single router in my network set to the default
of priority 0.  I've just set my border Cisco to 50, the next fastest
Cisco to 40, all the 2500's to 20, and left the TC's and two FreeBSD
machines running gated set to 0.  I rebooted what was the DR at the time
(I know, coulda just disabled/reenabled OSPF, but the customers on it
didn't notice) and didn't have any problems with the TC's.  We'll see...

-

---

From: Jeff Mcadams <[email-masked]>
Date: Tue, 12 Oct 1999 20:07:47 +0000

Thus spake Mike Wronski

>That recomendation is actually in the release notes for 4.2.32...

There are problems with the DR/BDR code in 4.2.x then?  Hrmm...could
suck if you don't have 2 other OSPF speakers on a subnet...if they
reboot, then the Arc would become DR, and when they come back up, they
won't take over from it.  :/  Would require a reboot of possibly two
Arcs for get the DR to be something other than an Arc in that situation.
:/
-- 
Jeff McAdams                            Email: [email-masked]
Head Network Administrator              Voice: (502) 966-3848
IgLou Internet Services                        (800) 436-4456

-

---

From: Mike Wronski <[email-masked]>
Date: Tue, 12 Oct 1999 20:07:49 +0000

|-----Original Message-----
|From: [email-masked]
|[mailto:[email-masked]]On Behalf Of Aaron Nabil
|Sent: Tuesday, October 12, 1999 7:22 AM
|To: [email-masked]
|Subject: Re: (usr-tc) arcs and ospf
|
|
|Mike Andrews writes...
|>I've been having a really bizarre problem off and on the last week or two.
|>Occasionally some of my ARCs will suddenly stop doing OSPF right.  The
|>routing table loses all OSPF routes, and the ARC stops announcing its
|>routes via OSPF to our Ciscos.  Disabling/enabling OSPF doesn't help, but
|>most of the time (not all the time) rebooting the ARC will.
|>
|>One thing that seemed to trigger it was rebooting some of our Ciscos...
|>and since that happens so infrequently (not rebooted since going to ARC
|>4.2 in fact) it's hard to track down.  I don't know which one exactly,
|>maybe our AS border router dying is what threw it off.
|> 
|> . . .
|>
|>Has anyone seen anything like this before or know of any particular
|>settings that I might want to check?  This is driving me nuts (not to
|>mention my customers) and I'm not sure where to start.
|
|Make sure your tc's aren't becoming the DR.  To check, do a "list ospf nei",
|see if any of them are in any state other than "two way".  To fix, do a
|"set ospf interface THE_OSPF_INTERFACE router_priority 0" on each of
|them and reboot.
|

-M

-

---

From: Aaron Nabil <[email-masked]>
Date: Tue, 12 Oct 1999 20:08:25 +0000

Mike Andrews writes...
> . . .

Make sure your tc's aren't becoming the DR.  To check, do a "list ospf nei",
see if any of them are in any state other than "two way".  To fix, do a
"set ospf interface THE_OSPF_INTERFACE router_priority 0" on each of
them and reboot.

-- 
Aaron Nabil

-

---

From: Mike McHenry <[email-masked]>
Date: Wed, 13 Oct 1999 04:29:00 +0000

This exact problem has happened to me several times and it was NOT caused by
the ARC deciding to become the DR. I have a small network of about 13 Lucent
portmasters, 3 USR TC chassis, one Cisco 4500 and one Cisco 4700 router. ALL
of the termservers are set with OSPF priority 0, meaning they will never
become a DR or BDR. The Cisco 4500 and 4700 OSPF priorities were left as-is
leaving one to become the DR and the other to become the BDR.

If I rebooted or otherwise disturbed the DR the USR TC chassis would lose
all their OSPF routing. The Lucent equipment did not have a problem at all.
Getting the OSPF back on the USRs required a reboot of every chassis. For
now I have set one of the Ciscos to priority 0 so that it will never become
a BDR or DR. It seems as though the HARC cards have a problem recognizing a
change of a BDR to a DR while the rest of my equipment does not.

If this problem is indeed being caused by an ARC deciding to become a DR
then it would seem to be a bug. In my case this should not be happening as
all the HARCs are set to NOT participate in DR/BDR elections.

Mike McHenry
 Systems Administrator
 MinnNet Communications, Inc.

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Jeff Mcadams
Sent: Monday, October 11, 1999 8:35 PM
To: [email-masked]
Subject: Re: (usr-tc) arcs and ospf

Thus spake Mike Andrews

-

-

---

From: Jeff Mcadams <[email-masked]>
Date: Wed, 13 Oct 1999 13:58:31 +0000

Thus spake Mike McHenry
>This exact problem has happened to me several times and it was NOT
>caused by the ARC deciding to become the DR. I have a small network of
>about 13 Lucent portmasters, 3 USR TC chassis, one Cisco 4500 and one
>Cisco 4700 router. ALL of the termservers are set with OSPF priority 0,
>meaning they will never become a DR or BDR. The Cisco 4500 and 4700
>OSPF priorities were left as-is leaving one to become the DR and the
>other to become the BDR.

>If I rebooted or otherwise disturbed the DR the USR TC chassis would
>lose all their OSPF routing. The Lucent equipment did not have a
>problem at all.  Getting the OSPF back on the USRs required a reboot of
>every chassis. For now I have set one of the Ciscos to priority 0 so
>that it will never become a BDR or DR. It seems as though the HARC
>cards have a problem recognizing a change of a BDR to a DR while the
>rest of my equipment does not.

>If this problem is indeed being caused by an ARC deciding to become a
>DR then it would seem to be a bug. In my case this should not be
>happening as all the HARCs are set to NOT participate in DR/BDR
>elections.

Sounds like you've played with it considerably more than myself at
least, I'll defer to your experience with this code.  I had just played
with it enough to know that it puked mightily when something happened to
the DR...I hadn't played enough to know specifically what caused it.
Sounds like you've dug into it far enough that you know more specifics.
-- 
Jeff McAdams                            Email: [email-masked]
Head Network Administrator              Voice: (502) 966-3848
IgLou Internet Services                        (800) 436-4456

-

---

From: [email-masked]
Date: Fri, 26 May 2000 17:57:43 +0000

Can anyone tell me why a Chassis all of a sudden would simply shut
itself down?   There are no HIGH Temps or power fluctuations, it just
shuts down...

It is configured with 5 DSPs/HiperArc/NMC/ and 1 70amp PS  It is plugged
into a BCPro 850. 

  Could this be a sign of a power supply getting ready to "crap the bed"?

  Thanks in advance!

==============================================================================
Phillip Ferraro				WorldNet Access, Inc
[email-masked]	Onslow County's PREMIER InterNet Service 
Voice (910) 346-0835		    824 Gumbranch Square, Suite Q
FAX   (910) 455-1933		     Jacksonville, Nc  28540-6269
==============================================================================

-