RE: iSCSI: Markers and Framing

["John Hufferd" <hufferd@us.ibm.com>]

Shridhar,
You input is both timely and amusing.  A number of folks have wanted input
from  ASIC Designers that were focused on 1 and especially 10 Gig and I
guess you clearly fall into that category.

You bring up a few points, that I found useful:
1. That you like the FIM approach now, and that it is useful in your design
until something REALLY better comes along.
2. That you would like "SHOULD Implement".  And you did not push for the
"MUST Implement".
3. That you, in effect, did not want to bet on the Final Framing since
there will be perhaps lots of issues and solutions still to come with iWARP
& RDMA, so don't worry about Framing now;  there is lots of experimentation
and debate that is going to occur, and who knows what will happen there.

I think I did some interpretation on point 3 so if that was wrong please
correct me.

I think I could accept the "SHOULD Implement" statement.  And perhaps that
is better then the "MUST Implement" that we have been talking about, and to
which a number of folks have objected.

There have been several folks that seem to have  reasons for not wanting
"MUST Implement".  This would give them an out if they felt that they have
a just and valid reason in their product for not doing it.  On the other
hand "SHOULD" gives the customers enough other vendors that will support
Markers, that if the customers care about it, they can easily obtain a
Multi vendor solution which exploits it.

 I can accept a "SHOULD Implement on Send", but "Optional to use", for FIM.

I also think the point you made about  iWARP, was also valid.  So I am
starting to also lean towards the new Option 6, with SHOULD, instead of
MUST.

Comments from others?

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: hufferd@us.ibm.com


"Mukund, Shridhar" <Shridhar_Mukund@adaptec.com> on 01/11/2002 05:51:12 PM

To:    John Hufferd/San Jose/IBM@IBMUS, ips@ece.cmu.edu
cc:    "Sperry, Todd" <Todd_Sperry@adaptec.com>, "Munnangi, Siva"
       <Siva_Munnangi@adaptec.com>
Subject:    RE: iSCSI: Markers and Framing




Dear Colleagues,

As an implementer of silicon solutions up to 10Gbps, my take is ...

  1. Election # 6. FIM now and some kind of framing later.

     Comments:

    a. By framing I mean what ever method iWarp effort ends up with.
       (TUF as proposed today may not be it)
    b. I strongly recommend SHOULD implement FIM on the send side.
       Implication -> Senders that do not insert markers should be
       willing to accept up to RTT*BW data drops! Headers being
       "reasonably" out-of-order is OK. Of course, senders that do not
       insert markers but are willing to pay big $$$ to the SSP will
       get their buffer/BW allocation as usual and customary :-)
    c. I think that the proponents and beholders of FIM had
       good reasons. They still hold and are even more stronger. We
       have had FIM in the iSCSI spec since version 02. Major changes
       to the iSCSI draft, at this late date, should have significant
       technical reasons.

  2. COBS is a good solution for the problem that Stuart and Mary
     originally
     set out to solve at Stanford. It's(COWS) use in the context of iSCSI
     is "misuse" at best, esp. given that the use of virgin TCP for
transport
     is mandatory(for good reasons).

     Several of you have alluded to why COWS is nasty to implement ... I
     prefer not to get there. Markers on the other hand do bring in some
     "essential" complexity but they are reasonable to implement even
     at 10Gbps. We sure could brute-force COWS, but the point is why the
     additional "incidental" complexity.

     Do not read further, unless you are open to ... :-)

-Shridhar Mukund
---------------------------------------------------------------------------

    CAUTION: MANDATORY to delete and OPTIONAL to read

  >> Wouldn't it be stupid if we had a proliferation of framing
  >> alternatives just because the "world originally seemed flat"?
  (My apology to Stephen Bailey for taking the quote out of context)

  A packetized HDLC stream(for which COBS was designed) has one whole
  space-time dimension lesser than a TCP stream. Relative to TCP
  stream, packetized HDLC stream is a flat world!

  As I see it, COBS is an alternative coding technique that makes
  delimiters explicit in an otherwise reference-less "sequence" or
  byte/word-stream. A reference-less sequence is assumed to have
  no ends or gradations. Yet the encoded sequence exposes 'handles'
  for synchronization. -> relatively synchronous after encoding.

  TCP stream on the other hand is a "time-sequence". It starts
  with a big-bang after which every byte in the sequence has an
  explicit time(sequence number) associated with it. -> It is
  absolutely synchronous even to begin with!

  The increase in entropy because of assuming a time-sequence to
  be a mere-sequence is probably :-) demonstrated by the following:

  In the Marker lingo :
 My second son was born in 99 and my first son in 94.
      Of course, 1900 AD is the marker here.
  In the COW's moo:
      My second son was born 20 blue moons after my first son.
      My first son ... I have no clue?

  Usage of COWS over TCP transport would be like loosing a needle
  in the hay stack, on purpose, and then devising a clever scheme
  to retrieve it.

  If you read on further, you may be wasting your time ...

  Are you certain that the world is round?
      If you read Stephen Hawkings or an article from Stanford in
      Scientific American 3 blue moons ago, there is scientific
      evidence of higher dimensions beyond the 4 space-time
      dimensions we perceive. In other words, round world might
      just be an illusion or a convenient definition of what we
      perceive! When I am using maps, flat world seems perfectly
      fine to me.

  Since a mere-sequence is a one dimensional space and a time-sequence
  like TCP stream is two-dimensional, COBS needs to work harder with
  packetized HDLC sequences. If COBS looks simpler than FIM, it is
  just an artifact your specific implementation. Q.E.D.

-Merlin's apprentice