[CUWiN-Dev] volatile ETX & routing

Wed Oct 26 15:38:32 CDT 2005

On Wed, Oct 26, 2005 at 11:44:51AM -0500, Bill Comisky wrote:
> We've had our network users reporting that their connections have been 
> less stable since moving from OSPF to HSLSD.
> 
> First some qualtitative observations that've been reported:
> 
> - Connections will go up and down, at one moment look good (good ping
>   times, good download rates) and then bad the next.

You are talking about connections between adjacent routers?  The
variability may simply be the nature of 802.11b networking, especially at
low S/N.  Below, I ping my neighbor's house from the node at the office.
Since I ping the link-local address, routing will not affect the results.
I insert an empty line after runs of consecutive successful pings:

  # ping -n 169.254.178.63
  PING 169.254.178.63 (169.254.178.63): 56 data bytes
  64 bytes from 169.254.178.63: icmp_seq=12 ttl=255 time=6.403 ms

  64 bytes from 169.254.178.63: icmp_seq=17 ttl=255 time=4.891 ms

  64 bytes from 169.254.178.63: icmp_seq=19 ttl=255 time=4.305 ms

  64 bytes from 169.254.178.63: icmp_seq=31 ttl=255 time=32.190 ms
  64 bytes from 169.254.178.63: icmp_seq=32 ttl=255 time=17.792 ms
  64 bytes from 169.254.178.63: icmp_seq=33 ttl=255 time=8.758 ms
  64 bytes from 169.254.178.63: icmp_seq=34 ttl=255 time=15.801 ms
  64 bytes from 169.254.178.63: icmp_seq=35 ttl=255 time=14.872 ms
  64 bytes from 169.254.178.63: icmp_seq=36 ttl=255 time=30.777 ms

  64 bytes from 169.254.178.63: icmp_seq=39 ttl=255 time=12.915 ms

  64 bytes from 169.254.178.63: icmp_seq=54 ttl=255 time=19.338 ms
  64 bytes from 169.254.178.63: icmp_seq=55 ttl=255 time=5.696 ms
  64 bytes from 169.254.178.63: icmp_seq=56 ttl=255 time=7.356 ms
  64 bytes from 169.254.178.63: icmp_seq=57 ttl=255 time=8.230 ms
  64 bytes from 169.254.178.63: icmp_seq=58 ttl=255 time=9.526 ms

  64 bytes from 169.254.178.63: icmp_seq=60 ttl=255 time=5.625 ms

  64 bytes from 169.254.178.63: icmp_seq=63 ttl=255 time=12.019 ms
  64 bytes from 169.254.178.63: icmp_seq=64 ttl=255 time=15.683 ms
  64 bytes from 169.254.178.63: icmp_seq=65 ttl=255 time=20.332 ms
  64 bytes from 169.254.178.63: icmp_seq=66 ttl=255 time=21.163 ms

  64 bytes from 169.254.178.63: icmp_seq=70 ttl=255 time=18.967 ms

  64 bytes from 169.254.178.63: icmp_seq=76 ttl=255 time=6.023 ms

  64 bytes from 169.254.178.63: icmp_seq=79 ttl=255 time=5.903 ms

  64 bytes from 169.254.178.63: icmp_seq=90 ttl=255 time=17.391 ms

  64 bytes from 169.254.178.63: icmp_seq=98 ttl=255 time=8.106 ms

  64 bytes from 169.254.178.63: icmp_seq=103 ttl=255 time=14.126 ms
  64 bytes from 169.254.178.63: icmp_seq=104 ttl=255 time=18.675 ms

  64 bytes from 169.254.178.63: icmp_seq=108 ttl=255 time=4.626 ms

  64 bytes from 169.254.178.63: icmp_seq=129 ttl=255 time=25.000 ms

  64 bytes from 169.254.178.63: icmp_seq=150 ttl=255 time=23.500 ms
  64 bytes from 169.254.178.63: icmp_seq=151 ttl=255 time=5.902 ms

  64 bytes from 169.254.178.63: icmp_seq=156 ttl=255 time=8.473 ms
  64 bytes from 169.254.178.63: icmp_seq=157 ttl=255 time=12.733 ms
  64 bytes from 169.254.178.63: icmp_seq=158 ttl=255 time=20.184 ms
  64 bytes from 169.254.178.63: icmp_seq=159 ttl=255 time=28.103 ms
  64 bytes from 169.254.178.63: icmp_seq=160 ttl=255 time=17.471 ms
  64 bytes from 169.254.178.63: icmp_seq=161 ttl=255 time=12.814 ms
  64 bytes from 169.254.178.63: icmp_seq=162 ttl=255 time=16.925 ms
  64 bytes from 169.254.178.63: icmp_seq=163 ttl=255 time=33.543 ms
  64 bytes from 169.254.178.63: icmp_seq=164 ttl=255 time=14.649 ms
  64 bytes from 169.254.178.63: icmp_seq=165 ttl=255 time=20.714 ms
  64 bytes from 169.254.178.63: icmp_seq=166 ttl=255 time=4.359 ms
  64 bytes from 169.254.178.63: icmp_seq=167 ttl=255 time=6.527 ms
  64 bytes from 169.254.178.63: icmp_seq=168 ttl=255 time=9.478 ms
  64 bytes from 169.254.178.63: icmp_seq=169 ttl=255 time=4.384 ms
  64 bytes from 169.254.178.63: icmp_seq=170 ttl=255 time=11.516 ms
  64 bytes from 169.254.178.63: icmp_seq=171 ttl=255 time=15.637 ms
  64 bytes from 169.254.178.63: icmp_seq=172 ttl=255 time=8.612 ms
  64 bytes from 169.254.178.63: icmp_seq=173 ttl=255 time=24.693 ms
  64 bytes from 169.254.178.63: icmp_seq=174 ttl=255 time=15.816 ms
  64 bytes from 169.254.178.63: icmp_seq=175 ttl=255 time=25.111 ms
  64 bytes from 169.254.178.63: icmp_seq=176 ttl=255 time=10.509 ms
  64 bytes from 169.254.178.63: icmp_seq=177 ttl=255 time=18.226 ms
  64 bytes from 169.254.178.63: icmp_seq=178 ttl=255 time=7.061 ms
  ^C
  ----169.254.178.63 PING Statistics----
  179 packets transmitted, 54 packets received, 69.8% packet loss
  round-trip min/avg/max/stddev = 4.305/14.249/33.543/7.744 ms

I don't understand what's going on there.  BTW, I am seeing "ath0:
hardware error" every 67s on the node at the office, but the period of
the packet drops is different from that.  I am surprised there is even
a link at such a distance: the two buildings are on opposite sides of
the mall, and I don't think there is a line of sight.

Here is another ping on the same node, a few minutes later:

  # ping -n 169.254.178.63
  PING 169.254.178.63 (169.254.178.63): 56 data bytes
  64 bytes from 169.254.178.63: icmp_seq=0 ttl=255 time=18.988 ms
  64 bytes from 169.254.178.63: icmp_seq=2 ttl=255 time=23.512 ms
  64 bytes from 169.254.178.63: icmp_seq=3 ttl=255 time=11.300 ms
  64 bytes from 169.254.178.63: icmp_seq=4 ttl=255 time=11.165 ms
  64 bytes from 169.254.178.63: icmp_seq=7 ttl=255 time=9.555 ms
  64 bytes from 169.254.178.63: icmp_seq=8 ttl=255 time=4.380 ms
  64 bytes from 169.254.178.63: icmp_seq=9 ttl=255 time=20.053 ms
  64 bytes from 169.254.178.63: icmp_seq=10 ttl=255 time=11.609 ms
  64 bytes from 169.254.178.63: icmp_seq=11 ttl=255 time=7.376 ms
  64 bytes from 169.254.178.63: icmp_seq=13 ttl=255 time=8.689 ms
  64 bytes from 169.254.178.63: icmp_seq=15 ttl=255 time=36.113 ms
  64 bytes from 169.254.178.63: icmp_seq=16 ttl=255 time=12.917 ms
  64 bytes from 169.254.178.63: icmp_seq=17 ttl=255 time=13.072 ms
  64 bytes from 169.254.178.63: icmp_seq=18 ttl=255 time=16.374 ms
  64 bytes from 169.254.178.63: icmp_seq=19 ttl=255 time=77.142 ms
  64 bytes from 169.254.178.63: icmp_seq=20 ttl=255 time=5.688 ms
  64 bytes from 169.254.178.63: icmp_seq=21 ttl=255 time=8.083 ms
  64 bytes from 169.254.178.63: icmp_seq=22 ttl=255 time=4.744 ms
  64 bytes from 169.254.178.63: icmp_seq=23 ttl=255 time=28.662 ms
  64 bytes from 169.254.178.63: icmp_seq=24 ttl=255 time=11.586 ms
  64 bytes from 169.254.178.63: icmp_seq=25 ttl=255 time=11.562 ms
  64 bytes from 169.254.178.63: icmp_seq=27 ttl=255 time=24.554 ms
  64 bytes from 169.254.178.63: icmp_seq=28 ttl=255 time=16.510 ms
  64 bytes from 169.254.178.63: icmp_seq=29 ttl=255 time=8.067 ms
  64 bytes from 169.254.178.63: icmp_seq=31 ttl=255 time=7.084 ms
  64 bytes from 169.254.178.63: icmp_seq=32 ttl=255 time=10.460 ms
  64 bytes from 169.254.178.63: icmp_seq=33 ttl=255 time=6.726 ms
  64 bytes from 169.254.178.63: icmp_seq=34 ttl=255 time=11.337 ms
  64 bytes from 169.254.178.63: icmp_seq=35 ttl=255 time=16.442 ms
  64 bytes from 169.254.178.63: icmp_seq=36 ttl=255 time=12.544 ms
  64 bytes from 169.254.178.63: icmp_seq=37 ttl=255 time=15.146 ms
  64 bytes from 169.254.178.63: icmp_seq=39 ttl=255 time=7.462 ms
  64 bytes from 169.254.178.63: icmp_seq=40 ttl=255 time=13.644 ms
  64 bytes from 169.254.178.63: icmp_seq=41 ttl=255 time=34.386 ms
  64 bytes from 169.254.178.63: icmp_seq=42 ttl=255 time=11.520 ms
  64 bytes from 169.254.178.63: icmp_seq=43 ttl=255 time=36.073 ms
  64 bytes from 169.254.178.63: icmp_seq=44 ttl=255 time=17.437 ms
  64 bytes from 169.254.178.63: icmp_seq=45 ttl=255 time=8.336 ms
  64 bytes from 169.254.178.63: icmp_seq=46 ttl=255 time=21.451 ms
  64 bytes from 169.254.178.63: icmp_seq=47 ttl=255 time=12.516 ms
  64 bytes from 169.254.178.63: icmp_seq=48 ttl=255 time=12.389 ms
  64 bytes from 169.254.178.63: icmp_seq=49 ttl=255 time=18.869 ms
  ^C
  ----169.254.178.63 PING Statistics----
  50 packets transmitted, 42 packets received, 16.0% packet loss
  round-trip min/avg/max/stddev = 4.380/16.084/77.142/12.530 ms

The RSSI at 178.63 ranges from 1dB to 5dB; 3dB or 4dB is typical.

> - Adding a new node to the system makes everyones connections locally go
>   haywire for a bit then things tend to settle down; this seemed
>   especially apparent when booting up a local laptop node to do upgrades.
>   This may have been more prevalent when there were only a few HSLS nodes
>   up in the network.

I would like to see comprehensive ETX logs (-l etx_any) when that happens.
Do all of the nodes run up-to-date software?

> - The routing tables are very dynamic, with routes sometimes changing
>   several times in the span of (one?) minute.

This might be normal.

> - The routes chosen (often short lived) sometimes seem a poor choice,
>   routing through someone much farther away that has a poor connection to
>   everyone to get to a nearby gateway.

I would like to know more about that.

Dave

-- 
David Young             OJC Technologies
dyoung at ojctech.com      Urbana, IL * (217) 278-3933