Warning: could not send message for past 4 hours

[julian_satran@il.ibm.com]

---------------------- Forwarded by Julian Satran/Haifa/IBM on 31/12/2000
14:38 ---------------------------

Mail Delivery Subsystem <MAILER-DAEMON@d12lmsgate.de.ibm.com> on 30/12/2000
22:48:16

Please respond to Mail Delivery Subsystem
      <MAILER-DAEMON@d12lmsgate.de.ibm.com>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:
Subject:  Warning: could not send message for past 4 hours




    **********************************************
    **      THIS IS A WARNING MESSAGE ONLY      **
    **  YOU DO NOT NEED TO RESEND YOUR MESSAGE  **
    **********************************************

The original message was received at Sat, 30 Dec 2000 17:43:50 +0100
from d12relay01.de.ibm.com [9.165.215.22]

   ----- The following addresses had transient non-fatal errors -----
<ips@ece.cmu.edu>

  --- The transcript of the session follows ---
<ips@ece.cmu.edu>... Deferred: A remote host did not respond within the
timeout period. with ece.cmu.edu.
Warning: message still undelivered after 4 hours
Will keep trying until message is 3 days old


Reporting-MTA: dns; d12lmsgate.de.ibm.com
Arrival-Date: Sat, 30 Dec 2000 17:43:50 +0100

Final-Recipient: RFC822; ips@ece.cmu.edu
Action: delayed
Status: 4.4.1
Remote-MTA: DNS; ece.cmu.edu
Last-Attempt-Date: Sat, 30 Dec 2000 21:48:16 +0100
Will-Retry-Until: Tue, 2 Jan 2001 17:43:50 +0100


Return-Path: <julian_satran@il.ibm.com>
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id RAA122090     for
<ips@ece.cmu.edu>; Sat, 30 Dec 2000 17:43:50 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])      by
d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id RAA182128  for
<ips@ece.cmu.edu>; Sat, 30 Dec 2000 17:43:51 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5  (863.2 5-20-1999))
id C12569C5.005BE51E ; Sat, 30 Dec 2000 17:43:44 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID: <C12569C5.005BE3E3.00@d12mta02.de.ibm.com>
Date: Sat, 30 Dec 2000 18:39:34 +0200
Subject: Re: Out of order response to R2T
Mime-Version: 1.0
Content-type: multipart/mixed;
Boundary="0__=FWeEhfndzheaTIeSWpYE0GlWip7hOAbfbjeqIwEyqP62L2X1vdpendJ0"
Content-Disposition: inline




Barry,

I can understand the rationale and the new text is strict (see attached).
However a bad digest can result in the need to redo part or the whole R2T
(or to reclaim all the data after the failure).

   If the R2T is answered with a sequence of Data PDUs the Buffer Offset
   and Length must be within the range of those
   specified by R2T, the last PDU should have the F bit set to 1, the
   Buffer Offsets and Lengths for consecutive PDUs SHOULD form a continuous
   non-overlapping range and the PDUs should be sent in increasing offset
   order.


   Regards,
   Julo

"Barry Reinhold" <bbrtrebia@mediaone.net> on 29/12/2000 23:48:30

Please respond to "Barry Reinhold" <bbrtrebia@mediaone.net>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   "Jon Sreekanth" <jon.sreekanth@trebia.com>, "James Smart"
      <james.smart@trebia.com>
Subject:  Out of order response to R2T




Julian,
     This is a bit of a delayed follow up to a conversation we had in San
Diego.
The issue has to do with how an initiator is allowed to respond to an R2T.

Right now the iSCSI draft says:

"If the R2T is
   answered with a sequence of Data PDUs the Buffer Offset and Length
   must be within the range of those
   specified by R2T and the last PDU should have the F bit set to 1; the
   Buffer Offsets and Lengths for consecutive PDUs SHOULD form a
   continuous range. "

Based on previous conversations this means that an initiator can break up
the delivery of the data into 4 segments and deliver the segments in any
order.

I would like to argue that this should be restricted such that when
responding to an R2T the data is delivered in order. I understand the logic
behind allowing the target to request data out of order based on the R2T.
This is consistent with Fibre Channel protocol and disk drive needs.
However, it is not clear to me why we should allow the iSCSI data PDUs sent
in response to the R2T to be out of order. I have three observations on
this:

1. The concept behind the target sending the R2T (based on analogy to the
FCP_XFER_RDY) is that the target is ready to receive data starting at
"offset" of a given length. This does not happen when the data is delivered
out of order forcing the end device to reassemble the information and
making
the check process more complicated.

2. This is specifically diallowed by Fibre Channel. Fibre Channel requires
that the data be delivered in one IU (sequence) and that the first frame of
the sequence must have the data offset set to the value sent in the
FCP_XFER_RDY frame. All the rest of the frames in the sequence must deliver
data in order by FC-FS rules.

For reference - FCP-2 clause 9.3

"If an FCP_XFER_RDY IU
is used to describe a data transfer and the first frame of the requested
FCP_DATA IU has a relative offset that
differs from the value in the FCP_DATA_RO field of the FCP_XFER_RDY IU, the
target shall post the error code



ôFCP_DATA Parameter mismatch with FCP_DATA_RO


ö in the FCP_RSP_INFO field of
the FCP_RSP IU."

I contacted Bob Snively (author of FCP-2) to confirm this. Bob is willing
to
discuss the semantics of the FCP_XFER_RDY data transfer with you if you
wish
to pick up that thread.

3. We have not seen this behavior in the field. If some device actually
took
advantage of this flexibility I suspect a number of implementation issues
would come up. In my experience these types of options hinder
interoperabiliy.

Also note that if a device is going to translate from iSCSI to FC, it is
going to have a difficult time with this. It must buffer up all the iSCSI
frames until it finds the first piece. It the data is sent in reverse order
and the transfer is large the buffering could be significant. Testing this
is a difficult process, and when testing is difficult interoperability
problems creep in.

In summary I would like to suggest that out of order data delivery in
response to an R2T is not a helpful feature, hinders ineroperability, is
not
compatible with Fibre Channel, and therefore makes interconnecting with
Fibre Channel legacy devices more difficult. I would like to see us adopt
the same semantics for iSCSI in this requard as Fibre Channel has.

Thanks for your time on this,

Barry Reinhold