iSCSI: Immediate Data

["John Hufferd" <hufferd@us.ibm.com>]

I thought it would be useful to explain why I think that Immediate Data is
important to iSCSI, so here it is:

1. Immediate Data permits the elimination of an additional network
handshake/interaction.

2. This is important since some applications have a key sensitivity to
Latency. The most important of these is Database.

3. Not only is Database sensitive to Latency, it generally has small I/O
writes.

4. The hardware HBAs that many of you folks are building, include a TCP/IP
Offload Engines (TOEs).  This permits support of Gigabit Line speed without
Server Load.  However, even though many of you are attempting to
parallelize as much of the TOE processing as possible, TCP/IP processing
will still add latency to each iSCSI interaction, as compared to Fibre
Channel.

5. If we want iSCSI to be performance competitive with Fibre Channel, we
need to do something about the additional Latency left in the path even
with HW HBAs.  One important way to do this, is to simply eliminate the
extra handshake that Fibre Channel uses on the small Writes.

6. Please note that this is not a big problem with large transfers, since
the Latency is Amortized over more data.

7. So it is the Immediate Data feature of iSCSI that will make an important
difference in the Key Response Time Sensitive Applications, which by luck
only transfer a small amounts of data at a time.

8. When we attempt to let iSCSI shine on the "at-distance" storage
environment, the value of Immediate Data, and Unsolicited Data are even
more valuable.  However, my concern at this time is for Immediate Data.

Now the above is the reason that I think Immediate Data is key and
important to the acceptance of iSCSI in the Enterprise Environment.  And
the following says why I think it is simple:

A. Creating a Buffer Manager that can allocate buffers that are chained
together, is kind of fundamental to the high performance environment we are
attempting to work with.  All the processes (whether iSCSI or SCSI) will
allocate buffers (from the Buffer Manager), place data in one or more of
these buffers, and pass the pointe list to the next process.  There will be
no copying of data.

B. This works the same whether the data comes in with the command or
separately.  When it comes in with the command, the iSCSI process will
request the Max buffers from the Buffer manager, and will be given one or
more pointers to buffer elements.  The Command and Data will be placed into
one of more of the buffer segments, then the buffers that are filled, will
be queued, in some manner, to the next iSCSI or SCSI Process.  The unused
buffer segments will be re-queued on the free queue.  (I know I am telling
you the obvious, but for some reason this message is not getting across.)

C. Having every thing (including the data) arrive with the command,
eliminates the code path overhead that is needed to collates data with
commands.

D. Since the only thing that is moved is pointers, it becomes a Zero Copy
technique at its very core.

E. The only thing that needs to be managed is the overall buffer space.
And that needs to be managed anyway.  The iSCSI Window management are the
functions that are suppose to be used to control that.

F.  The issue of having enough memory is probably not a real world problem
except at the very extremes.  And you need to have the exact same code to
manage the iSCSI Windows regardless of what caused the buffer space to run
low.

G. When a Storage Controller is expected to handle a large number of
connections, the Storage Controller is expected to have a large pool of RAM
storage, and when the Storage Controller is expected to handle only a small
number of connections, it is hard not to have enough  memory, since the
price and availability of small RAMs is problematical.

All the above, leads me to the belief that ImmediateData=yes, is the
simplest, and most conservative option.  And it solves the Latency issue
with Enterprise Databases, thereby assuring the success of iSCSI in the
Enterprise as well as the "at-distance" computing environment.




.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: hufferd@us.ibm.com