How can I solve the problem of ARP Cache in Cisco Catalyst 6506 Core Switch?

Question

We have problem of ARP cache on Core sw 6506 . Topology for this is as per attachment.

Problem is : The pcs which are turned off for 3- 4 hour can not communicated. from this PCs some of Devices and some of Servers also can not pinging . Some of PCs have an error : Reply from x.x.x.x(Gateway):TTL expires in transit.

Now if i ping this effected PC from MSFC than its works well. Before it i have checked sh arp | i x.x.x.x ( PC Ip address ) than I can not find it. But After pinging from MSFC i can found in ARP list. So I suspect it is due to ARP cache.

I had tried to clear all arp but the problem is still exist.

This problem is in all PCs with static IPs as in PCs with DHCP - Continuous ly communicated so not isolated.

As a temp Solution we have rebooted nboth the core sw and removed one sup card from both the end.

But reboot Sw is not the solution.

So pls help me to solve the problem :

Pls give me the action plan to solve - to troubleshoot the probl

Answer 1

The "TTL expired in transit" message is not related to arp but is rather a message from the router saying it has sent your ping request through too many hops (TTL got decremented to 0).  It is actually a specific ICMP response sent typically form routers to source host to indicate that TTL has reached 0.  Check for routing loops.  For instance...

Host pings server on a different subnet...

host -> gw1 -> gw2 -> gw3 -> gw1 -> gw2 -> etc...

One thing to check is to make sure the server is actually connected to the right subnet and can ping it's own default gateway.  I can see this happening if the server is misconfigured to a subnet that none of these gateways have an interface for (virtual or otherwise) *and* you've got a look in routing due to the default gateway configs on each of these gateways pointing to each other.

One more thing I just remembered...  If you are using MPLS, I recall some potential pitfalls with regard to MPLS and IP TTL.  Lookup/play with the following two commands:

mpls ip propagate-ttl

mpls ip ttl-expiration pop

Answer 2

I don't think that this is an ARP issue, but rather that ARP is being used to diagnose a host issue. Check the VLAN's and routes on the end nodes, ensure that if the servers and clients are in different VLAN's that you have routing between the VLAN's. To me this sounds similar to having two (or more) VLAN's that are both configured on the MSFC but no default route for the switch - that would really only happen in Hybrid code if you are running that.