RE: FCIP and wandering duplicates (FCoverIP)

["Douglas Otis" <dotis@sanlight.net>]

Vivek,

If you use SCTP to encapsulate FC, as example
http://search.ietf.org/internet-drafts/draft-otis-fc-sctp-ip-00.txt
which implements
http://search.ietf.org/internet-drafts/draft-ietf-sigtran-sctp-13.txt
and use the pending U-SCTP modification by Randall R. Stewart where only one
retransmission is allowed, then on a MAN, such duplication should be avoided
in all cases.
http://search.ietf.org/internet-drafts/draft-xie-stewart-usctp-00.txt
The present U-SCTP disables all retransmissions however it is to be changed
to allow a single retransmission.  This should allow a reasonable connection
suitable for FC and suitable for standard IP equipment.

Doug

> -----Original Message-----
> From: Vivek Kashyap [mailto:viv@sequent.com]
> Sent: Tuesday, September 12, 2000 3:27 PM
> To: Douglas Otis
> Cc: Raj Bhagwat; Vivek Kashyap; ipfc@standards.gadzoox.com;
> ips@ece.cmu.edu
> Subject: Re: FCIP and wandering duplicates
>
>
> Raj,
>
> I am, after reading your and Doug's replies, still unclear
> whether FC handles
> duplicates or not. It appears to me that it does not.
>
> E_D_TOV seems to be a time out that is set for retransmission between
> end points. However, it is possible to introduce transient loops in
> the fabric when a topology change (addtion of a new switch or a
> link/switch going down) occurs, due to the re-calculation of paths
> unless such changes are made atomic to the traffic. In such a case
> there could be duplicates.
>
> Of course, if the FC switches are quiesced during the switch updates
> then loops won't matter. Or is there a different mechanism or an
> algorithm that guarantees looplessnes?
>
>
> TTL retuning will at most reduce the duplicates but can't guarantee
> elimination of duplicates. So if FC somehow guarantees duplicate
> free fabrics and does not handle duplicates then it appears to me that
> FCIP always will have the danger of corrupting data due to duplicates
> inserted by IP.
>
>
> Vivek
>
> >
> > Raj,
> >
> > I do not understand what tuning TTL does to benefit any
> situation.  The path
> > between locations is not fixed so such tuning may induce lost
> packets and is
> > unrelated to time in transit.  Depending on the domain, a packet may be
> > repeated by equipment that does not necessarily decrement TTL.
> Although not
> > a desired feature, duplication may happen if the packet
> transmit status is
> > made uncertain by local signaling.
> >
> > Doug
> >
> > > -----Original Message-----
> > > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> > > Raj Bhagwat
> > > Sent: Monday, September 11, 2000 12:26 PM
> > > To: Douglas Otis; Vivek Kashyap; ipfc@standards.gadzoox.com
> > > Cc: ips@ece.cmu.edu
> > > Subject: Re: FCIP and wandering duplicates
> > >
> > >
> > > Doug,
> > >
> > > I understand how exactly TTL is used. That is why I stated
> that TTL should
> > > be set to a value larger than the number of hops between the
> source and
> > > destination. An IP end station may not know this and it may
> choose to set
> > > the TTL to the max value of 255. However, an FCIP gateway, with some
> > > intelligence, can perhaps know the # of hops to other FCIP
> > > gateways it talks
> > > to, especially in enterprise networks.
> > >
> > > It is not clear to me how there can be duplicate frames
> without the source
> > > sending twice. Could you please explain that?
> > >
> > > Thanks,
> > >
> > > Raj.
> > >
> > > ----- Original Message -----
> > > From: Douglas Otis <dotis@sanlight.net>
> > > To: Raj Bhagwat <rajb@lightsand.com>; Vivek Kashyap <viv@sequent.com>;
> > > <ipfc@standards.gadzoox.com>
> > > Cc: <ips@ece.cmu.edu>
> > > Sent: Monday, September 11, 2000 11:53 AM
> > > Subject: RE: FCIP and wandering duplicates
> > >
> > >
> > > > Raj,
> > > >
> > > > You can see duplications without the source sending twice.
> Reducing TTL
> > > > value may prevent the packet from ever arriving.  The number of
> > > times TTL
> > > is
> > > > decremented depends on the path and not time in transit.
> > > >
> > > > Doug
> > > >
> > > > > -----Original Message-----
> > > > > From: Raj Bhagwat [mailto:rajb@lightsand.com]
> > > > > Sent: Monday, September 11, 2000 11:31 AM
> > > > > To: Douglas Otis; Vivek Kashyap; ipfc@standards.gadzoox.com
> > > > > Cc: ips@ece.cmu.edu
> > > > > Subject: Re: FCIP and wandering duplicates
> > > > >
> > > > >
> > > > > Doug,
> > > > >
> > > > > Your point on TTL is well taken. It is there to prevent a
> packet from
> > > > > getting stuck in a routing loop. Duplicates of unicast packets can
> > > happen
> > > > > when the sender retransmits a packet and both the original and the
> > > > > retransmitted packets reach the destination. If the
> sender does not
> > > > > retransmit for an adequate amount of time, duplicates of
> > > unicast packets
> > > > > should not happen. This is true even when there are
> routing loops or
> > > > > topology changes in the network.
> > > > >
> > > > > As you have pointed out, there is no direct correlation between
> > > > > E_D_TOV and
> > > > > TTL. And that could be a problem. However, setting the TTL to a
> > > relatively
> > > > > smaller value (larger than the # of hops between the source and
> > > > > destination
> > > > > FCIP gateways) can reduce the possibility of duplicates when
> > > the sender
> > > of
> > > > > the FC frame waits for at least E_D_TOV before retransmitting
> > > the frame.
> > > > >
> > > > > I don't think SCTP at the FCIP gateway will solve the duplicate
> > > > > problem even
> > > > > with a single retry. There is no synchronization between
> the FC device
> > > and
> > > > > the FCIP gateway for error recovery. So, the FC device
> can retransmit
> > > > > independent of the SCTP at the FCIP gateway. One way to solve this
> > > problem
> > > > > is to fine tune the timeout values on the FC side and at the
> > > SCTP level
> > > on
> > > > > the FCIP gateway.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Raj Bhagwat
> > > > > LightSand Communications
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > From: Douglas Otis <dotis@sanlight.net>
> > > > > To: Raj Bhagwat <rajb@lightsand.com>; Vivek Kashyap
> <viv@sequent.com>;
> > > > > <ipfc@standards.gadzoox.com>
> > > > > Cc: <ips@ece.cmu.edu>
> > > > > Sent: Monday, September 11, 2000 10:14 AM
> > > > > Subject: RE: FCIP and wandering duplicates
> > > > >
> > > > >
> > > > > > Raj,
> > > > > >
> > > > > > Unless you are willing to wrap the FC frame within some
> protocol,
> > > > > duplicates
> > > > > > can happen and this has nothing to do with TTL.  TTL
> > > prevents circular
> > > > > paths
> > > > > > from forever sending a packet within a loop.  A modified
> > > SCTP protocol
> > > > > that
> > > > > > will solve both of these problems, if restricted one
> time retry.  A
> > > > > > Metro-Area-Network should induce about 5ms RTT delay such that
> > > > > a 10ms skew
> > > > > > becomes possible but is limited by the single retry.  TTL buys
> > > > > you nothing
> > > > > > with this E_D_TOV, Error_Detect Timeout.
> > > > > >
> > > > > > Doug
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: owner-ips@ece.cmu.edu
> > > [mailto:owner-ips@ece.cmu.edu]On Behalf
> > > Of
> > > > > > > Raj Bhagwat
> > > > > > > Sent: Sunday, September 10, 2000 10:57 PM
> > > > > > > To: Vivek Kashyap; ipfc@standards.gadzoox.com
> > > > > > > Cc: ips@ece.cmu.edu
> > > > > > > Subject: Re: FCIP and wandering duplicates
> > > > > > >
> > > > > > >
> > > > > > > Vivek,
> > > > > > >
> > > > > > > In a properly implemented FC-only fabric, duplicates should
> > > > > not happen.
> > > > > A
> > > > > > > frame that enters a fabric should either be delivered
> > > within E_D_TOV
> > > > > > > (typically 2 sec) or be discarded. FC devices don't
> retransmit a
> > > frame
> > > > > > > before the E_D_TOV timer expires.
> > > > > > >
> > > > > > > You do bring up an interesting point when two SAN islands are
> > > > > > > bridged across
> > > > > > > an IP network. Once an FC frame enters the IP
> network, it is hard
> > > > > > > to say how
> > > > > > > the E_D_TOV is enforced by that part of the network.
> As authors of
> > > the
> > > > > > > FCoverIP draft, we have given some thought to the
> possibility of
> > > > > > > delivering
> > > > > > > duplicate frames when both the TCP stack at the FCIP
> > > gateway and the
> > > > > > > original FC device initiate recovery/retransmission
> > > procedure in an
> > > > > > > overlapping time period. This is one of the major
> reasons why we
> > > > > initially
> > > > > > > chose to make the FCIP gateway a stateless device with no
> > > > > buffering and
> > > > > no
> > > > > > > retransmission.
> > > > > > >
> > > > > > > The good news is that the {IP encapsulated} FC frame will be
> > > protected
> > > > > by
> > > > > > > the IP TTL once it enters the IP network. The bad news is that
> > > > > > > detecting the
> > > > > > > E_D_TOV timeout while the frame is inside the IP
> network may be
> > > > > difficult.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Raj Bhagwat
> > > > > > > LightSand Communications
> > > > > > >
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > From: Vivek Kashyap <viv@sequent.com>
> > > > > > > To: <ipfc@standards.gadzoox.com>
> > > > > > > Sent: Sunday, September 10, 2000 3:17 PM
> > > > > > > Subject: FCIP and wandering duplicates
> > > > > > >
> > > > > > >
> > > > > > > > I read the FCIP document recently. It does not
> address the issue
> > > of
> > > > > > > > duplicates.
> > > > > > > >
> > > > > > > > Does fibre channel protect itself from duplicates ?
> I looked at
> > > the
> > > > > > > > header and it does not have a time to live field. Thus
> > > a duplicate
> > > > > > > > caught in a loop and then released at the 'right'
> time could be
> > > > > > > > accepted by the receiver. Even if the FC fabric path
> > > algorithms do
> > > > > > > > ensure loop free paths transient loops due to errors, switch
> > > > > > > > misconfigurations or failures are possible. Does FC handle
> > > > > such a case
> > > > > ?
> > > > > > > >
> > > > > > > > If it does not then FCIP will exacerbate the
> problem because the
> > > IP
> > > > > > > > network could have 'transient' loops. Such a 'wandering
> > > duplicate'
> > > > > > > > could be re-injected into the FC island and be accepted
> > > by a newer
> > > > > > > > incarnation of a FC inter-island exchange. Furthermore with
> > > > > no TTL one
> > > > > > > > can't really provide a timout between starting two
> > > conversations.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Vivek
> > > > > > > > --
> > > > > > > > Vivek Kashyap
> > > > > > > > IBM-NumaQ
> > > > > > > > viv@sequent.com
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
>
>
> --
> Vivek Kashyap
> Advisory Engineer
> IBM-NumaQ
> viv@sequent.com
> vivk@us.ibm.com
> 503 578 3422 (o)
>