RE: Unsolicited Data.

["BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>]

Hi Sanjeev,

Perhaps I'm getting confused here:  From 2.3.1 below, if the F bit is set
then it indicates the immediate data that acompany the command is all the
data associated with the command.  However, if there is no immediated data,
but data is to be written the F bit can not be set to 1.  In this case the
data is either sent a unsolicited Data PDUs or as solicited data PDUs.  This
reverts back to my original point of changing the spec to state that if
unsolicited data has been negoiated then the initiator MUST honour it.

With regards to your comment: "Is it still going to help if Unsolicited data
is not chosen to be sent while IntialR2t=No has been negotiated???? Or do I
have misunderstanding with a general delivery path here."  may I can explain
my concern: If the initiator does not choose to send unsolicited data
although it has negoiated to do so, how is the data transfers insigated as
the target is expecting unsoliciated Data PDU but the initiator is expecting
an R2T?

Feel free to call me so we can discuss offline.

Thanks

Matthew Burbridge
NIS-Bristol
Hewlett Packard
Telnet: 312 7010
Tel UK (44) 117 312 7010
E-mail: matthewb@bri.hp.com



-----Original Message-----
From: Sanjeev Bhagat (TRIPACE/Zoetermeer) [mailto:sbhagat@tripace.com]
Sent: Thursday, June 07, 2001 1:09 PM
To: 'BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)'; ips@ece.cmu.edu
Cc: 'steph@cs.uchicago.edu'
Subject: RE: Unsolicited Data. 


Matthew,

among the two options you gave I would like to comment on the second

It is possible for initiator **not** to send any unsolicited data and send
the rest of data as solicited data by the use of bit 7 in Flags and Task
Attributes. This is posted here http://ips.pdl.cs.cmu.edu/mail/msg04647.html

The new 2.3.1 is:

2.3.1     Flags and Task Attributes

      The flags for a SCSI Command are:

      b7   (F) set to 1 when the immediate data that accompany the command
      are all the unsolicited data associated with this command. For a
      write, if Expected Data Transfer Length is larger than the Length the
      target may solicit additional data through R2T.

About the option no. 1 Stephen has already explained. Refer to the very
specific paragraph below of his email

In a hardware accelerate implementation, unsolicited data will be
handled differently than solicited data.  The target can chose where
the solicited data is delivered, but unsolicited data arrives through
a general-delivery path.  Therefore, if the offered unsolicited data
burst size approaches the regular burst size, an implementation will
need TWICE as much buffer memory (half in the general-delivery area
and half in the flow-controlled buffer area) to support the same
operation load.  Maybe this tradeoff is OK, but given the cost
sensitivity of storage targets, I doubt it.

I would like to comment one thing here that in the current iSCSI specs
implementation the complete data for a specific command Including (Cmd PDUS
, Data PDUs and Response PDUs) follow the same path. So the whole of data
will follow the general delivery path (same connection) whether solicited or
unsolicited. Is it still going to help if Unsolicited data is not chosen to
be sent while IntialR2t=No has been negotiated???? Or do I have
misunderstanding with a general delivery path here.

Regards,
Sanjeev

-----Original Message-----
From: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
[mailto:matthew_burbridge@hp.com]
Sent: Thursday, June 07, 2001 12:40 AM
To: ips@ece.cmu.edu
Subject: RE: Unsolicited Data. 


Hi Stephen, Sanjeev, and other interested parties,

I whole heartily agree with both of your comments and see the benefit of
unsolicited data in removing the R2T induced latency for both small writes
and large writes on systems where such latency is an issue.  However, for
large writes the unsolicited portion of the data need not be all the data in
the transfer but enough to overcome the initial latency.

My main concern is that although the initiator and target have negoiated to
use unsolicited data (InitialR2t=no), in the spec it appears to be optional.
Therefore how does the target know that unsolicited data will be sent.
Currently it does not and the target will have to set a timer that expires
if no unsolicited data is forthcoming which will incur an even more
unacceptable latency.

I see that there are two options:
1)State in the spec that if unsolicited data has been negotiated then the
initiator MUST honour it. (My preferred)
2)Have a flag in the SCSI Command PDU indicating that data is expected (No
thanks). 

Thanks

Matthew Burbridge
NIS-Bristol
Hewlett Packard
Telnet: 312 7010
E-mail: matthewb@bri.hp.com


-----Original Message-----
From: Stephen Bailey [mailto:steph@cs.uchicago.edu]
Sent: Thursday, June 07, 2001 2:20 AM
To: ips@ece.cmu.edu
Subject: Re: Unsolicited Data. 


Sanjeev,

> why would initiator choose not to send unsolicited immediate data when
> it has negotiated for IntialR2T= Yes??

I assume you mean InitialR2T=no.

I would do this because I think that the real benefit of unsolicited
data is for writes where ALL the write data can be sent unsolicited,
and I'm happy to have the target schedule the transfer of (ALL) data
for larger writes.  Of course, others would say I'm a moron.

Assuming the overwhelming majority of storage use is file access,
small writes are exclusively file system metadata writes.  Unsolicited
data is a huge win for metadata writes because the latency of these
metadata writes is frequently not covered by transferring other data
(within a single access thread).  On the other hand, the file system
(usually) generates enough concurrent demand for real data writes that
the flow control (R2T) latency is well hidden.

You might argue that if you're running a file system optimized fors
today's SANs over a link with a HUGE bandwidth * delay product
(greater than the expected demand created by the file system), sending
large amounts of unsolicited write data will substantially improve the
link utilization.  That is true, but I don't think many targets are
going to offer unsolicited data bursts remotely that large.

In a hardware accelerate implementation, unsolicited data will be
handled differently than solicited data.  The target can chose where
the solicited data is delivered, but unsolicited data arrives through
a general-delivery path.  Therefore, if the offered unsolicited data
burst size approaches the regular burst size, an implementation will
need TWICE as much buffer memory (half in the general-delivery area
and half in the flow-controlled buffer area) to support the same
operation load.  Maybe this tradeoff is OK, but given the cost
sensitivity of storage targets, I doubt it.

As always, I welcome comments from those who see things differently.

Steph