RE: TCP Framing (considered helpful?)

["Rod Harrison" <rod.harrison@windriver.com>]

	I whole heartedly agree on not mandating TCP framing for
all the reasons already posted. Having worked in the kernel
team for an OSV for 8 years I can tell you there is a lot of
inertia attached to pushing an OS out of the door. Those
projects are planned a long way ahead of the ship date and
putting things in late rarely, if ever, happens. Also, they
are never shipped when they are ready, there are a couple of
windows in the year and that's it. I've seen finished
product sit on the shelf for months waiting for the
non-engineering folks to decide it's time to ship. So, in
summary, even if the OSVs we all care about have TCP framing
done and dusted now, I think it could easily be a year or
more before those stacks hit the streets. Too long in my
opinion.

	However, since it looks like TCP framing is going to happen
why don't we try and future proof ourselves and allow its
use to be negotiated? Both sides of the iSCSI session will
know if they have TCP framing or not so how about adding a
TCP parameter to the FMarker text key. As has been suggested
before we can mandate support for markers but allow their
use to be optional.

	Something like ....

	FMarker=<tcp|send|receive|send-receive|no>

	This should allow a session to use TCP markers if they are
available and markers if not.

	- Rod

-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On
Behalf Of
John Hufferd
Sent: Wednesday, May 23, 2001 6:31 AM
To: ips@ece.cmu.edu
Subject: RE: TCP Framing (considered helpful?)



Somesh,
I disagree with your statements.  You should not believe
that we can be
separate from the standard TCP/IPs.  This will not be
accepted into the
Business environment.  Likewise, if you want to wait for
Microsoft to start
shipping the function, and have that function included into
all the
Desktops and Laptops that we want to support, it is going to
be a long
wait.  This is not a negative statement about Microsoft, all
large software
projects, such as the next release of Windows, does not
quickly put new
things in that are not yet approved, or that are just
recently approved in
the standard.

We have been waiting for sometime to get all the functions
we wanted in
Winsock Direct, and the progress there is being very
measured.  This is the
way the real world works.  They will, I think add Warpness
over time, but
we do not what to wait for that.

Now for you to say that vendors such as SUN, HP, and IBM
will be more
casual about adding the function than Windows, is not the
way I read the
industry, in something as critical as TCP/IP.

Markers, are simple, straight forward to implement, and will
use very
little processor power to use.  I hope you agree with that
the
implementation is easy, and if your issue is that you
disagree on my
estimate on what processor power it takes to use,  that is
why we are
suggesting it be Optional to use.  The installation needs to
measure a
total cost performance trade off.  That is, trade off more
CPU cycles for
Desktops and Laptops (which usually has more MIPS then they
can use) vrs
cost of Hardware in Targets.

As for changing the PDU formats again, and adding padding, I
think we have
done quite enough of that, and have run down that path too
many times to
even discuss it again, so I think you called that idea
correctly.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Internet address: hufferd@us.ibm.com


"Somesh Gupta" <someshg@yahoo.com>@ece.cmu.edu on 05/22/2001
11:34:40 AM

Please respond to <someshg@yahoo.com>

Sent by:  owner-ips@ece.cmu.edu


To:   <ips@ece.cmu.edu>
cc:
Subject:  RE: TCP Framing (considered helpful?)



I think if the framing solution finds favor with the TCP
"gods",
it is a much better alternative than the markers (which I
was
a proponent of - but only because I did not believe that
a framing mechanism would be approved).

Even if the framing mechanism is not approved, an alternate
mechanism which provides for alignment of iSCSI headers at
regular intervals (at negotiate progression in TCP sequence
#s e.g. 0x12345, 0x22345, 0x32345 and so on) in the TCP
stream
is a much better solution. Unfortunately this would require
some change to the header format (for padding) which
probably
no one would like to change at this time.

The assumption that people will implement some change as big
as iSCSI will be to the OS (in an integrated way - not as
a kludgy add-on for demos) without being able to make the
relatively minor change required to the API seems to be not
a very convincing argument.

Furthermore, at least in the Unix environment, iSCSI will be
implemented in the kernel, and the accomodation for framing
won't require an external API change. And as Steph says,
the framing proposal has Microsoft participation.

The insertion and deletion of markers seems to be painful
accomodation for software based iSCSI solutions. The Unix
folks might want to speak to this, but the markers will
probably require more of a kludge in the kernel (to make
it efficient) as compared to the changes required for
framing.

Somesh

> -----Original Message-----
> From: owner-ips@ece.cmu.edu
[mailto:owner-ips@ece.cmu.edu]On Behalf Of
> Stephen Bailey
> Sent: Monday, May 21, 2001 6:51 AM
> To: ips@ece.cmu.edu
> Subject: Re: TCP Framing (considered helpful?)
>
>
> John,
>
> > I think we must depend on Markers to insure that
everything can
> operate at
> > top speed, and at the lowest cost.
>
> A key question is whether markers actually ensure that
everything
> operates at `top speed, and at the lowest cost'.
>
> Matt thinks so.  I (and, presumably those who wrote the
framing
> document) think not.
>
> My issue is not even with `lowest cost'.  I don't believe
markers will
> allow you to run at top speed.  Specifically:
>   1) I doubt the feasibility of implementing the control
required for
>      an eddy buffer (where you store data you can't place)
at 10G.
>      Admittedly, the validity of this claim can't really
be assessed
>      without actually working the implementation, so for
99% of the
>      list participants (myself included) this is a `yes it
is, no it
>      isn't' point.
>   2) an eddy buffer solution requires some substantial
speed-up in
>      both the NIC data path, and MOST IMPORTANTLY: the
host bus.  In
>      order to unload the eddy buffer while still handling
incoming
>      traffic at line rate, clearly the host bus bandwidth
must be >
>      line rate.
>
> I know of at least one general purpose framed solution
operating at
> 10G which has been available for >3 years (SGI's
GSN/ST/XIO NIC).  I'm
> sure there are others.
>
> I can't imagine there's any argument that a framed
solution would be
> voted `most likely to run fast and be cheap'.  Every
storage network
> and cluster interconnect has been designed that way since
antiquity.
>
> The key tradeoff involves the OS vendors, and I'm
wondering why we're
> speaking for them.  The question IS, how much more work is
it to
> introduce TCP framing over and above what is required to
insert iSCSI
> into their network framework.  My experience from writing
NIC and
> storage drivers for many commercial UNIX-family OSes is:
>   1) it's an easy and well defined process to insert a new
SCSI
>      transport driver into the SCSI stack.
>   2) it's hard and poorly defined process to insert
ANYTHING into the
>      network stack.
>
> Networking has historically been a user-mode activity.
Architected
> services are only provided to user mode programs.  Kernel
clients have
> been few and far between and so are handled on a
case-by-case basis.
> For example NFS.  Every OS has hacks to make NFS run fast,
but they
> are not stable interfaces for general purpose use.
>
> Even Solaris' SysV-derived STREAMS stack, which is
intended precisely
> to provide flexible, crisp interfaces for kernel network
clients, does
> not document the relevant (IP stack) intermodule
interfaces.
>
> I know that there are more and more kernel network
clients, but they
> are coming either on fluid platforms (e.g. linux), in
which case the
> argument of `it'll take too long to get OS support'
doesn't apply, or
> they are vendor-supplied, in which case a performance
iSCSI solution
> in ANY form may take a while, and the choice of framing or
markers
> isn't going to make a difference.
>
> I can't say squat about the architecture of Winsock, but
the fact that
> there is a Microsoft author of the framing proposal who
seems very
> serious about supporting framing and RDMA as quickly as
possible
> suggests that framing support should be available on
Windows very
> soon.
>
> Steph

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com