RE: iSCSI: frame formats

[Charles Monia <cmonia@NishanSystems.com>]

Hi:

While we're on the subject of padding: Another (non-ip) transport has the
rule that padding is only allowed on the last data PDU of a command.  That
is, the initial and intermediate data transfer PDUs must not be padded.

I believe this simplifies implementations doing cut-through placement into
word-aligned buffers. 

Charles

> -----Original Message-----
> From: Barry Reinhold [mailto:bbrtrebia@mediaone.net]
> Sent: Monday, April 02, 2001 11:59 AM
> To: Rod Harrison; ips@ece.cmu.edu
> Subject: RE: iSCSI: frame formats 
> 
> 
> Rod,
> 	I'm pretty sure the we decided to go with bytes at the 
> 50th IETF meeting
> for data length. No one felt that a max data size of 16 megs 
> in an iSCSI PDU
> was an issue (remember this is not max SCSI data transfer 
> size, it is max
> data bytes in an iSCSI PDU).
> 	I do think we need to word the padding with a bit of 
> care, as I can
> envision people doing this in at least two different ways 
> that would not
> interoperate.
> 	The interoperability issues on the padding question 
> boils down to: "Where
> in the TCP stream do I put the Data Digest if the number of 
> bytes I have to
> send is not a multiple of 4?"
> 	My expectation was that the transmitter would be null 
> padding the data
> portion of the iSCSI PDU to a word boundary, then sticking on 
> the digest.
> Thus the padding is actually in the TCP stream. For example 
> if <d> = a data
> octet, <p> = pad octet, and <dd> = data digest octet, then 
> the TCP stream
> for a 6 byte data transfer would look as follows:
> <octets in the header - end modulo 4> <d> <d> <d> <d> <d> <d> 
> <p> <p> <dd>
> ....
> 	The receiver would get the value "6" in the data length 
> portion of the
> header. After pulling out the header, the receiver would pull 
> out 8 bytes of
> "data + pad" and then get the digest.
> 	The other way to do this to have a "virtual pad" such 
> that padding is
> created by the receiver when construction the iSCSI PDU from 
> the TCP stream.
> The padding is never actually in the TCP stream itself.
> 	I do not think this is as helpful, but whatever we do, 
> the spec. should
> address this minor detail so we don't trip over it.
> 
> 
> >-----Original Message-----
> >From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On 
> Behalf Of
> >Rod Harrison
> >Sent: Monday, April 02, 2001 1:44 PM
> >To: ips@ece.cmu.edu
> >Subject: RE: iSCSI: frame formats
> >
> >
> >Stephen,
> >
> >	I don't agree with your concerns about the maximum PDU
> >size. I think few targets, or indeed initiators, will be
> >interested in negotiating PDU sizes anywhere near this
> >large, so big transfers will have to be fragmented anyway.
> >
> >	However, I share your concern about the padding. I don't
> >really see why we are considering it. If one has a local
> >alignment issue it can, and I believe should, be taken care
> >of locally and not in the specification. There are several
> >easy ways of handling this sort of thing; insert and remove
> >the pad locally; separate header and payload buffers
> >allowing each to be naturally aligned, etc.
> >
> >	If we have to pad then every read of non-header data will
> >have to involve a rounding calculation on the length, and
> >then perhaps a second read to discard the pad if the
> >underlying buffer is the exact size of the data. Possibly
> >the same on send, if the data buffer is the exact size the
> >transmit code can't just 'go off the end,' it will have to
> >send the data, and then fake up some pad and make another
> >send.
> >
> >	Am I missing something here, why do we care about padding?
> >
> >	- Rod
> >
> >-----Original Message-----
> >From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On
> >Behalf Of
> >Stephen Bailey
> >Sent: Monday, April 02, 2001 3:18 PM
> >To: ips@ece.cmu.edu
> >Subject: Re: iSCSI: frame formats
> >
> >
> >Sandeep,
> >
> >> DataLen will now be max 8M/4M but then we dont wish to
> >have large
> >> iSCSI PDUs in any case.
> >
> >This max size is getting the the point where I'm sure it'll
> >be an
> >irritant.
> >
> >I would like (but, in fact, a sure way to guarantee that it
> >won't
> >happen is for me to like it :^) to view iSCSI PDUs as the
> >expected
> >grain size at which you might have software involvement in
> >an
> >otherwise hardware-driven iSCSI implementation.
> >
> >For example, when a target is returning read data for
> >multiple
> >outstanding reads, it might want to return a bit at a time
> >from each,
> >and each `bit' should be an iSCSI PDU.  Clearly sending
> >these bits
> >will be a software-level decision.  That certainly was the
> >rational
> >for allowing multiple FCP DATA PDUs per read operation, and
> >I naively
> >assumed similar logic was being applied here.
> >
> >The alternative is to say that the hardware will do iSCSI
> >PDU
> >chunking, but if that's the case, I expect that the header
> >is, well, a
> >bit bulky.
> >
> >I'm also incredibly unexcited about having data length be a
> >multiple
> >of 4 bytes (if that's still in the cards).  There operations
> >within
> >the SCSI command set which return arbitrary length data.
> >There are
> >perfectly nominal cases where you get less data than you
> >requested
> >(e.g. inquiry & request sense).  Furthermore, the SCSI
> >architecture
> >does not prohibit this, even though certain commands do, so
> >it is not
> >for iSCSI to say anything about this one way or the other.
> >
> >The problem with handling lengths that include padding comes
> >when
> >you're trying to move the data into a buffer which is a
> >non-multiple
> >length.  For example, if I ask for 22 bytes of inquiry data
> >(with a 22
> >byte buffer), what can I say, at the time a PDU arrives that
> >has 24
> >bytes?  It might have 21 or 22 bytes (or perhaps even,
> >erroneously 23
> >or 24 bytes).  A data residual coming later will tell the
> >software how
> >much was actually there, but it can't tell the hardware.
> >The typical
> >expectation of this type of transfer is that it will only
> >overwrite
> >bytes of the buffer that are actually transferred, but
> >having padded
> >lengths will not allow this.
> >
> >The completely standard solution to carrying arbitrary data
> >of
> >arbitrary length in an aligned transfer unit is to pad the
> >transfer
> >unit but report the exact (shorter) length.  Another
> >solution, used by
> >FC, is to carry a pad length.  In iSCSI, why bother---you've
> >just
> >reintroduced added the 2 bits you were trying to remove?
> >
> >The data length scenario is not comparable to IP header
> >lengths, where
> >what is being carried is not arbitrary data.
> >
> >Certainly, for iSCSI additional header segments (AHSs) you
> >could
> >arguably use this cell length technique, since we can
> >control what
> >we're carrying (AHSs that need exact byte lengths will have
> >to be
> >internally self-describing) but frankly, I still think it's
> >a bad
> >idea.
> >
> >I can't understand why we're messing around with all these
> >tricky
> >`solutions' to standard problems.  We should avoid the
> >temptation to
> >get cute, and wholesomely provide the same capabilities as
> >any other
> >SCSI transport.  Specifically:
> >  o allow long PDUs
> >  o carry exact data lengths
> >
> >Steph
> >
>