RE: iSCSI: Immediate Data

["John Hufferd" <hufferd@us.ibm.com>]

Eddy,
On a Disk Drive you are going to have probably less then 5 connections.
You can not buy memory small enough to be a problem in that environment.
Same is true with controllers with 50 connection capability.  And by the
way, in small environments, if you have to do a move, you will NOT notice
the extra overhead.

So ImmediateData=Yes is conservative for everyone.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: hufferd@us.ibm.com


"Eddy Quicksall" <EQuicksall@mediaone.net> on 10/01/2001 05:58:59 AM

To:   John Hufferd/San Jose/IBM@IBMUS, <ips@ece.cmu.edu>
cc:
Subject:  RE: iSCSI: Immediate Data



John,

I think you are missing the opposition's point. I don't think anyone is
arguing that immediate data does not give a performance boost in most cases
... rather what should the default be?

For me, I don't care because I support it. But for the others, I personally
side with them because you are telling them that it is "easy" and for
legacy
cache code, it is probably not easy (for obvious reasons).

I see a lot of "memory is cheep" but it seems like you are therefore saying
"we'll only use iSCSI in heavy weight controllers". What about disk drives,
don't we want iSCSI there too? What about bridge controllers?

I think if you took a pole, you would find that most out there are not in
the heavy weight arena that you are in. If we want iSCSI to be an
overwhelming success, we need to make it easy to port existing code bases
and just add a transport layer.

Eddy

-----Original Message-----
From: John Hufferd [mailto:hufferd@us.ibm.com]
Sent: Monday, October 01, 2001 12:05 AM
To: ips@ece.cmu.edu
Subject: iSCSI: Immediate Data


I thought it would be useful to explain why I think that Immediate Data
is
important to iSCSI, so here it is:

1. Immediate Data permits the elimination of an additional network
handshake/interaction.

2. This is important since some applications have a key sensitivity to
Latency. The most important of these is Database.

3. Not only is Database sensitive to Latency, it generally has small I/O
writes.

4. The hardware HBAs that many of you folks are building, include a
TCP/IP
Offload Engines (TOEs).  This permits support of Gigabit Line speed
without
Server Load.  However, even though many of you are attempting to
parallelize as much of the TOE processing as possible, TCP/IP processing
will still add latency to each iSCSI interaction, as compared to Fibre
Channel.

5. If we want iSCSI to be performance competitive with Fibre Channel, we
need to do something about the additional Latency left in the path even
with HW HBAs.  One important way to do this, is to simply eliminate the
extra handshake that Fibre Channel uses on the small Writes.

6. Please note that this is not a big problem with large transfers,
since
the Latency is Amortized over more data.

7. So it is the Immediate Data feature of iSCSI that will make an
important
difference in the Key Response Time Sensitive Applications, which by
luck
only transfer a small amounts of data at a time.

8. When we attempt to let iSCSI shine on the "at-distance" storage
environment, the value of Immediate Data, and Unsolicited Data are even
more valuable.  However, my concern at this time is for Immediate Data.

Now the above is the reason that I think Immediate Data is key and
important to the acceptance of iSCSI in the Enterprise Environment.  And
the following says why I think it is simple:

A. Creating a Buffer Manager that can allocate buffers that are chained
together, is kind of fundamental to the high performance environment we
are
attempting to work with.  All the processes (whether iSCSI or SCSI) will
allocate buffers (from the Buffer Manager), place data in one or more of
these buffers, and pass the pointe list to the next process.  There will
be
no copying of data.

B. This works the same whether the data comes in with the command or
separately.  When it comes in with the command, the iSCSI process will
request the Max buffers from the Buffer manager, and will be given one
or
more pointers to buffer elements.  The Command and Data will be placed
into
one of more of the buffer segments, then the buffers that are filled,
will
be queued, in some manner, to the next iSCSI or SCSI Process.  The
unused
buffer segments will be re-queued on the free queue.  (I know I am
telling
you the obvious, but for some reason this message is not getting
across.)

C. Having every thing (including the data) arrive with the command,
eliminates the code path overhead that is needed to collates data with
commands.

D. Since the only thing that is moved is pointers, it becomes a Zero
Copy
technique at its very core.

E. The only thing that needs to be managed is the overall buffer space.
And that needs to be managed anyway.  The iSCSI Window management are
the
functions that are suppose to be used to control that.

F.  The issue of having enough memory is probably not a real world
problem
except at the very extremes.  And you need to have the exact same code
to
manage the iSCSI Windows regardless of what caused the buffer space to
run
low.

G. When a Storage Controller is expected to handle a large number of
connections, the Storage Controller is expected to have a large pool of
RAM
storage, and when the Storage Controller is expected to handle only a
small
number of connections, it is hard not to have enough  memory, since the
price and availability of small RAMs is problematical.

All the above, leads me to the belief that ImmediateData=yes, is the
simplest, and most conservative option.  And it solves the Latency issue
with Enterprise Databases, thereby assuring the success of iSCSI in the
Enterprise as well as the "at-distance" computing environment.




.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: hufferd@us.ibm.com