Re: comments on draft-ietf-ips-fcovertcpip-00.txt

[Glen Turner <glen.turner@aarnet.edu.au>]

Matt Wakeley wrote:
> 
> Simply put, TCP is a byte stream, and you cannot dictate to TCP what will or
> will not be in a "TCP segment".  As far as the application layer above TCP is
> concerned, there is no such thing as a tcp segment.  And neither the Push nor
> Urgent options help.

-----------------------
NEXT GENERATION

> Section 5.2 also talks about IP fragments, which is a BAD thing to do to TCP -
> it results in performance hits.

And it makes a FCIP/TCP/IPv6 implementation impossible.

-----------------------
FRAMING

Following the discussion I had a read of the framing part
of the document.  Although the FCIP uses the TCP format
and complies with the TCP state machine it does not
comply with the TCP services.

Operating systems written to provide applications with a TCP
service will not be able to implement FCIP.  This includes
the UNIX socket API, the expression of TCP services most
dominant in the marketplace.

Operating systems will need modification to implement
FCIP.  From a quick look at the Linux kernel, the standard
path for TCP packets has to be significantly altered to
implement FCIP.  This sucks, as it means that other TCP
applications on the machine must run slower.  With RISC
CPUs it is worse -- either the TCP path or the FCIP path
must take a succession of cache hits.

You can argue that FCIP targets are always dedicated bits of hardware
(although I would contest that).

But almost all FCIP clients will be general purpose computers.
And so a web server with a gigabit ethernet connection (an
'intranet' server) with a large rear-end disk would be insane
to run FCIP -- what you gain on using FCIP you lose on the
slower HTTP path.

The choice to implement FCIP as a operating system application
(say, a UNIX daemon) or as a operating system change (say as
a Linux kernel module) should be based upon efficiency and
maintainability.  That choice should not be made by the protocol
designer, if only because the trade-offs are going to change
as technology changes over time.

-----------------------
ADAPTION LAYERS TO IMPLEMENT FRAMING

I'd also suggest that the author examine the concept of Adaption
Layers.

That is, a core FCIP protocol is defined.

Then an adaption layer is defined for TCP.  Another is defined
for STCP.  The RFC states that the TCP adaption layer MUST be
supported.

The framing bytes for TCP then become part of the adaption layer.

A similar trick can be used for iSCSI.  With iSCSI a UDP adaption
layer may also be reasonable.

-----------------------
QUALITY OF SERVICE WILL FAIL FOR MOST USERS

The document also states:

     DSCP (6 bits):  The Differentiated Service Code Points (DSCP) [6]
     shall be set to correspond to the Premium Service.  This  service
     provides "Expedited Forwarding" at each  IP hop (Per Hop Behavior
     (PHB)).

"Shall" is one of the RFC 'magic words' indicating an absolute requirement.
There is no justification for insisting upon the Premium service.

It prevents ISPs from offering a DiffServ service that best meets
the needs of FCIP users.  For example, the Premium service may have
worse latency but better bandwidth than a service explicitly tailored
for FCIP.  Similarly, the ISP may wish to implement a different
protection mechanism for an FCIP path than for a Premium path.

The requirement should be:

  FCIP targets and clients MUST allow the Differentiated Services
  Code Point to be configured.  FCIP targets and clients SHOULD
  use a DSCP of 0 until both parties are authenticated.

The second sentence allows easy policing of the use of whichever
DSCP is used for FCIP traffic.  

-----------------------
FRAGMENTATION

     IP Fragmentation:

     The Fibre Channel maximum transmittable unit (MTU) is 2148 bytes.
     A  maximum of 60 bytes of TCP header  increases the MTU of the IP
     payload  to 2208  bytes.  It  is  preferable  that  FCIP  packets
     encompass the TCP + FC MTU to avoid fragmentation of FCIP packets.
     The  resulting  packet size exceeds  the MTU of  some IP physical
     layers  (e.g.  Ethernet  MTU = 1518 bytes).  FCIP  devices should
     handle fragmentation and must handle re-assembly of FCIP packets.
     An  FCIP  device  may use  Path MTU  Discovery  (RFC 1191) or  an
     equivalent   mechanism  to  adjust  FCIP  packet  size  to  avoid
     fragmentation.   Alternatively,  the MTUs  of all FC nodes may be
     manually set to match the path MTU of the IP network.

This is much more simply put:

    A FCIP node SHOULD set the TCP PSH bit on TCP segment containing the last
    part of the encapsulated FCIP packet.

    To allow higher throughput, when sending multiple FCIP packets
    implementations SHOULD set the PSH bit when encapsulating only the
    last FCIP packet of a sequence of FCIP packets.

Then segmentation and reassembly is handled through the standard TCP
services to present FCIP with a stream of octets thay were sent in
the most efficient way possible.  Asking for the PSH bit TCP service
hints to the TCP implementation that no more data is expected to be
sent for a while.

-----------------------
TCP DYNAMICS

     10. Flow Control and Congestion Management

     <Text to be added>

I suggest you don't stuff about with this.  TCP dynamics
are an area of active development.  For example, the
ECN bit.  If you specify something here and applications
come to depend on it, then improvements to flow control
(that might actually improve performance) will not be
able to be implemented.

-----------------------
TIMEOUTS

IP is best effort and TCP avoids congestion collapse.  The implication
is that there is nothing that FC can do on its own to detect timeout faster
that will not either:
  - add to the risk of congestion collapse
  - prevent rerouting and recovery

IF some mechanism is developed, then it should be implemented
in TCP to aid *all* sessions, not just FC sessions.  That's
why the timers are usually kernel variables.

An implementation on a dedicated target may choose to make these
variables simple to set.

-- 
 Glen Turner                                 Network Engineer
 (08) 8303 3936      Australian Academic and Research Network
 glen.turner@aarnet.edu.au          http://www.aarnet.edu.au/
--
 The revolution will not be televised, it will be digitised