RE: ISCSI: Immediate data extending beyond a single PDU

[julian_satran@il.ibm.com]

Barry,

I think that the current draft does not allow you to accept immediate but
not unsolicited.
You can either allow unsolicited (including immediate) or request always
R2T.

And yes - if the expected length is larger that what you sent as
unsolicited (even if the unsolicited was less than the limit) the target is
supposed to send R2T (there is only one unsolicited burst per command).

The limit for immediate data is implicit - the max-PDU length and is
less-then-or-equal the first burst.

Your proposal is interesting but I doubt that it is very useful.

Unsolicited data help avoid an RTT on writes but require resources to be
allocated (that is why a target may not accept them).

Immediate data is an "optimized" form of unsolicited data in which the
command and data are collapsed to one unit. Nevertheless the immediate data
are unsolicited and require resources to be allocated ahead of time. If you
have larger amounts to send than what fits into one PDU the "collapse gain"
is minimal and does not justify the added complexity of 2 options.

Regards,
Julo



"Barry Reinhold" <bbrtrebia@mediaone.net> on 02/02/2001 21:06:29

Please respond to "Barry Reinhold" <bbrtrebia@mediaone.net>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:
Subject:  RE: ISCSI: Immediate data extending beyond a single PDU




Julian,
     I agree with the conceptual idea of immediate data, but further
refinement
is required. As a receiver, if I have allowed immediate data, but not
unsolicited data, and I get a write command with an expected data transfer
length that is greater then the amount of data in the command PDU, do I
send
an R2T or not (or from the sender's point of view does he wait for an R2T
to
complete the data transfer?) The draft is not clear on this, yet it is an
externally visible behavior that needs to be defined for interoperability.

I believe that what we need here is a refinement of two distinct concepts,
that of immediate data and that of unsolicited data. Here is my take at it:

1. Immediate data - Data that is sent within a single iSCSI SCSI command
PDU. Immediate data shall be fully contained within a single iSCSI command
PDU, and shall be limitted by the negotiated value of ImmediateData. The
size of Immediate data is not subject to the limits of FirstBurst.

2. Unsolicited data - Data that may be transfered to a target without an
explicit R2T being sent to request its transfer. Unsolicited data is sent
in
iSCSI SCSI data PDUs in which the target transfer tag is invalid. The
maximum amount of unsolicited data is negotiated between the initiator and
target through the FirstBurst key. If unsolicited data is supported by the
target, upon receipt of a write command, the target shall not send an R2T
until an iSCSI Data PDU with the Final bit set has been received. After
reception of an iSCSI Data PDU with the Final bit set R2Ts shall be sent by
the target to request transmission of any additional data associated with
this command. The initiator shall not send any additional unsolicited data
for this command.

Interaction between Immediate data and Unsolicited data -

Immediate data and Unsolicited data are features that may be selected
independently of one another. Support of either is optional.

Note: Based on the current specification of FirstBurst, support for
unsolicited data is not optional. One must select between a range of
1-65536. I think we would want to make this optional and indicate non
support for unsolicited data by setting FirstBurst to 0.


>-----Original Message-----
>From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
>julian_satran@il.ibm.com
>Sent: Friday, February 02, 2001 6:41 AM
>To: ips@ece.cmu.edu
>Subject: Re: ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Barry,
>
>There is a misunderstanding:
>
>You can send EITHER immediate data OR a sequence of unsolicited PDUs.
>
>Immediate data was conceived for short writes - for which an initiator
does
>not want to do
>two socket operations or 2 HBA operations.
>
>Larger unsolicited bursts can be sent with a sequence and then we felt
that
>you won't gain too much by having the first one glued to the command.
>
>
>Julo
>
>"Barry Reinhold - Lamprey" <bbr@lampreyNetworks.com> on 01/02/2001
22:04:28
>
>Please respond to "Barry Reinhold - Lamprey" <bbr@lampreyNetworks.com>
>
>To:   Julian Satran/Haifa/IBM@IBMIL
>cc:   "ISCSI" <ips@ece.cmu.edu>
>Subject:  ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Julian,
>
>My understanding of your email in conjunction with the current draft is:
>
>The initiator may send zero or more immediate data bytes in the CMD PDU,
>and
>may send additional unsolicited DATA PDUs with further immediate data,
with
>the
>last PDU having the F bit set. The total amount of immediate data to be
>limited
>by the value negotiated at login.
>
>There are two issues I have with this:
>
>First, the target does not know how much immediate data has being sent.
>There
>is no indication in the CMD frame to indicate that more immediate data is
>to
>follow (whether or not the CMD frame contains any immediate data). The
>initiator is free to send any amount, from zero up to the negotiated
value.
>Therefore, the target will likely send an R2T to the host at whatever
>boundry
>it believes is present - which may cause a retransmission of much or all
of
>the
>immediate data.
>
>Second, the additional unsolicited PDU's do not contain a target
>task/transfer
>tag. As such, in the mainline processing, you are perhaps forcing the
>target
>to
>look up the Initiator Task Tag in its list of active i/o's. This may exact
>a
>heavy performance penalty as optimizing this lookup maybe
cost-prohibitive.
>The
>only other scenarios in which the target had to perform this lookup was in
>low-occurance error/abort conditions.
>
>Are there negative implications to keeping immediate data to a single PDU?
>
>-- Barry
>
>
>julian_satran@il.ibm.com wrote:
>
>> Barry,
>>
>> Immediate data is "attached to the command". If your unsolicited data
>(the
>> so called initial burst) is larger that what you are ready to commit to
a
>> single PDU you can send several PDUs with the  last having the F bit.
>After
>> that the target can decide that it wants more and send an R2T or send
>> status.
>>
>> You are no supposed to send both immediate and separate PDU.
>>
>> I admit that the text is bit confusing (in 1.2.5)  and there is an error
>> also in the appendix.
>> 2.7.1 talk about unsolicited data (not immediate) (the context is data
>> PDUs).
>>
>> The new version of 1.2.5 now reads:
>>
>>    Unsolicited data can be sent as part of an iSCSI command PDU
>("immediate
>>    data") or an in separate iSCSI data PDUs.  An initiator may send
>>    unsolicited data either as immediate (up to the negotiated maximum
PDU
>>    size - DataPDULength - disconnect-reconnect mode page) or in a
>separate
>>    PDU sequence (up to the negotiated limit - FirstBurstSize -
>>    disconnect-reconnect mode page). All subsequent data have to be
>>    solicited.  The maximum size of an individual data PDU or the
>>    immediate-part of the initial unsolicited burst as well as the
initial
>>    burst size MAY be negotiated at login.
>>
>> Thanks,
>> Julo
>>
>> "Barry Reinhold - Lamprey" <bbr@lampreyNetworks.com> on 29/01/2001
>16:34:30
>>
>> Please respond to "Barry Reinhold - Lamprey" <bbr@lampreyNetworks.com>
>>
>> To:   Julian Satran/Haifa/IBM@IBMIL
>> cc:   "Jon Sreekanth" <jon.sreekanth@trebia.com>, "James Smart"
>>       <james.smart@trebia.com>
>> Subject:
>>
>> Issue:
>>
>> I am unclear on how to respond to an iSCSI SCSI command PDU when there
is
>> immediate data but not enough to meet the requirements of the IO
>operation
>> as given in the expected data transfer length field. Should an R2T be
>> issued
>> or not?
>>
>> Background:
>>
>> Clause 2.7.1 in 03 reads:
>>
>> "2.7.1 F (Final) bit
>>
>>    This bit is 1 for the last PDU of immediate data or the last PDU of a
>>    sequence answering a R2T."
>>
>> This suggests that immediate data can extend beyond the iSCSI SCSI CMD
>PDU
>> (there is no final bit on the command PDU)but the closet thing we have
to
>a
>> definition for immediate data, which is in clause 1.2.5 below, suggests
>> that
>> immediate data is limitted to the command PDU:
>>
>> "Outgoing SCSI data (initiator to target - user data or command
>>    parameters) will be sent as either solicited data or unsolicited
>>    data.  Solicited data are sent in response to Ready To Transfer (R2T)
>>    PDUs. Unsolicited data can be part of an iSCSI command PDU
>>    ("immediate data") or an iSCSI data PDU.  An initiator may send
>>    unsolicited data (immediate or in a separate PDU) up to the
>>    negotiated limit (initial burst size - mode page 02h). All subsequent
>>    data have to be solicited.  The maximum size of an individual data
>>    PDU or the immediate-part of the initial unsolicited burst MAY be
>>    negotiated at login (maximum connect size - mode page 02h)."
>>
>> Unsolicited data is bounded in a number of ways, but most significant to
>> this issue is in the description of the login key useR2t which states:
>>
>> "Note than only the first
>>    outgoing data item (either immediate data or a separate PDU) can be
>>    sent unsolicited by a R2T."
>>
>> Given the above it is unclear to me if the concept of immediate data is:
>>
>> 1. Write data that can be sent without an R2T (unsolicited write
data)and
>> starts in the command PDU.
>> 2. Write data that is entirely contained within the command PDU. (my
>> initial
>> concept of immediate data)
>> 3. The non-zero portion of data, in a possible multi-PDU write
operation,
>> that is contained in the iSCSI SCSI command PDU. Where the remaining
PDUs
>> of
>> the write operation must be solicited with an R2T.
>>
>> Given the current definition of the Final bit and the current limits on
>> unsolicited data it is unclear to me what a target should do when
>receiving
>> an iSCSI SCSI command PDU that has immediate data in it, but not
>sufficient
>> data to meet the expected data transfer length specificied in the
>command.
>> Does an R2T go out of not?
>>
>> Barry Reinhold
>> 603-659-0885
>
>Barry Reinhold
>603-659-0885
>
>
>
>