Re: iSCSI: changing MaxPDUDataLength

["Mallikarjun C." <cbm@rose.hp.com>]

> Can you suggest how this would be handled otherwise?

I earlier said I will not get into implementation details, I will 
however provide a generic answer.

Given that the changed PDU size is not specific to one command,
I see the history of "past max PDU size(s)" as a connection attribute.  
All you need on a per-task basis is really the value of one DataSN 
*where* it had changed.

We could further debate how to deal with multiple number of PDU size 
changes during the life of an I/O - but I'm reluctant to be involved in that
debate because a) it's most unlikely, and b) there's no hard requirement that 
the target must always satisfy SNACKs under extreme conditions, and finally
c) it's too implementation-specific to be discussed on the ips reflector.

Finally, I'd like to remind you that mandating "no text negotiation till quiesced"
has harmful effects on targets dealing with long writes and wanting ULPDU-
containment - whose case you may be arguing.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668 
Roseville CA 95747
cbm@rose.hp.com


----- Original Message ----- 
From: "Eddy Quicksall" <eddy_quicksall@ivivity.com>
To: "Mallikarjun C." <cbm@rose.hp.com>; "Julian Satran" <Julian_Satran@il.ibm.com>
Cc: <ips@ece.cmu.edu>
Sent: Tuesday, June 11, 2002 4:45 PM
Subject: RE: iSCSI: changing MaxPDUDataLength


> Consider the following (using Julian's suggestion):
> 
> -> cmd 1  
> -> req to change to length 2
> -> cmd 2
> 
>       <- stat 1
> <- data 2, PDU 0, Length 1
> 
>    -> cmd 3, ack 1 so change the length
> 
> <- data 2, PDU 1, Length 2
> 
> -> SNACK data 2, PDU 1
> 
> So we have to calculate the offset for Data 2, PDU 1 using old length and
> send data after that offset using new length. That means keeping track of
> PDU lengths which I would like to avoid.
> 
> Or, the target would have to force idling of commands before it gives the
> response to the change.
> 
> Can you suggest how this would be handled otherwise?
> 
> Eddy
> 
> -----Original Message-----
> From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> Sent: Tuesday, June 11, 2002 6:57 PM
> To: Eddy Quicksall; Julian Satran
> Cc: ips@ece.cmu.edu
> Subject: Re: iSCSI: changing MaxPDUDataLength
> 
> 
> Eddy,
> 
> I am not sure if you're agreeing with me or not... but let me add some
> comments.
> 
> I don't expect the change in PDU size to happen frequently - no change
> in what I earlier said.
> 
> You are making an assumption that a target must record the PDU size for
> every
> PDU  if SNACKs are to be satisfied across changed MaxRecvDataSegmentLength.
> 
> As Julian earlier said, you don't have to.  I will not go into
> implementation details, but
> I believe just the value of the DataSN from which the new size is effective
> should 
> be all that's needed.  Besides, the spec guarantees that only RunLength=0 is
> used 
> on any SNACK after the change.
> 
> You are also implying that targets can declare their changed
> MaxRecvDataSegmentLength
> anytime just for writes.  There are two problems with this - a) reads and
> writes may both
> be active in the typical case on an iSCSI connection, and b) a target can
> declare its changed
> value only if an initiator is allowed to start the Text Request (and I
> thought you were suggesting
> that we explicitly disallow that).
> 
> Hope that clears it up.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions
> Hewlett-Packard MS 5668 
> Roseville CA 95747
> cbm@rose.hp.com
> 
> 
> ----- Original Message ----- 
> From: "Eddy Quicksall" <eddy_quicksall@ivivity.com>
> To: "Mallikarjun C." <cbm@rose.hp.com>; "Julian Satran"
> <Julian_Satran@il.ibm.com>
> Cc: <ips@ece.cmu.edu>
> Sent: Tuesday, June 11, 2002 3:18 PM
> Subject: RE: iSCSI: changing MaxPDUDataLength
> 
> 
> > You are correct and that is exactly how I have had to code it. With fast
> > path and slow path code split between processors, it becomes even worse.
> > 
> > When I read your comments on why the PDU size could change (from way back
> in
> > March I think), I got did not get the impression that it would happen very
> > often and that in fact it may only happen when you are maintaining
> > allegiance to another connection.
> > 
> > If it is something that is not going to happen very often, then it would
> be
> > so much easier to have the initiator idle the commands first (all it
> really
> > needs to do is idle the READ commands). If it is going to change so often
> > that idling would be degrading, I suspect that just doing the negotiation
> > that often would itself be degrading.
> > 
> > And, be aware that the target will probably idle the R commands.
> Otherwise,
> > it will have to track PDU sizes and that would be really
> > "counter-productive" (especially when there should be no SNACKs on a good
> > connection anyway).
> > 
> > I don't see any problem with the target negotiating its size at any time.
> > And the initiator would not have to idle the WRITE or no-data commands.
> > 
> > Eddy
> > 
> > -----Original Message-----
> > From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> > Sent: Tuesday, June 11, 2002 3:35 PM
> > To: Eddy Quicksall; Julian Satran
> > Cc: ips@ece.cmu.edu
> > Subject: Re: iSCSI: changing MaxPDUDataLength
> > 
> > 
> > Eddy,
> > 
> > I am not clear on why you think the target code becomes messy on 
> > a PDU size change.  The last para in section 4.4 states the following -
> > 
> > "Parameters negotiated by a text exchange negotiation sequence become
> > effective only after the negotiation sequence is completed."
> > 
> > Since the target gets to complete a text negotiation with a Text Response
> > PDU, it can time the applicability of a changed MaxRecvDataSegmentLength.
> > 
> > Requiring that an initiator must idle the connection before changing this
> > parameter, IMHO, is counter-productive when there's a dynamic PMTU
> > degradation in the presence of long-running commands (writes in
> particular).
> > Once the initiator starts the renegotiation, target would ideally want to
> > quickly 
> > renegotiate its receive size as well to receive ULPDU-contained segments.
> > 
> > In short, seems to me that the draft is saying what you would like.
> > --
> > Mallikarjun
> > 
> > Mallikarjun Chadalapaka
> > Networked Storage Architecture
> > Network Storage Solutions
> > Hewlett-Packard MS 5668 
> > Roseville CA 95747
> > cbm@rose.hp.com
> > 
> > 
> > ----- Original Message ----- 
> > From: "Eddy Quicksall" <eddy_quicksall@ivivity.com>
> > To: "Julian Satran" <Julian_Satran@il.ibm.com>
> > Cc: <ips@ece.cmu.edu>
> > Sent: Tuesday, June 11, 2002 7:56 AM
> > Subject: RE: iSCSI: changing MaxPDUDataLength
> > 
> > 
> > > I think it should be a requirement because if it is not, all target code
> > > will have to have the messy code. Or, we could say that if it is not
> idle
> > > commands, then the target may reject it.
> > >  
> > > Eddy
> > > 
> > > -----Original Message-----
> > > From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
> > > Sent: Tuesday, June 11, 2002 9:11 AM
> > > To: Eddy Quicksall
> > > Cc: ips@ece.cmu.edu; pat_thaler@agilent.com
> > > Subject: RE: iSCSI: changing MaxPDUDataLength
> > > 
> > > 
> > > 
> > > That is a fair request - we may slip in a recommendation to that effect
> > (in
> > > chapter 11?) 
> > > 
> > > Julo 
> > > 
> > > 
> > > 
> > > Eddy Quicksall <eddy_quicksall@ivivity.com> 
> > > 
> > > 
> > > 06/11/2002 04:28 AM 
> > > Please respond to Eddy Quicksall 
> > > 
> > > 
> > >         
> > >         To:        Julian Satran/Haifa/IBM@IBMIL 
> > >         cc:        ips@ece.cmu.edu, pat_thaler@agilent.com 
> > >         Subject:        RE: iSCSI: changing MaxPDUDataLength 
> > > 
> > >        
> > > 
> > > 
> > > How about if we say that an initiator must not change the MaxPDUDataSize
> > > unless it first idles the commands (at least the ones with the R bit) or
> > if
> > > ErrorRecoveryLevel==0? 
> > >   
> > > That would simplify target code greatly and would meet the design goals;
> > and
> > > since it should be rare to change it, it would be of little impact to
> idle
> > > the commands for a split second. 
> > >   
> > >   
> > > Eddy 
> > > -----Original Message-----
> > > From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
> > > Sent: Monday, June 10, 2002 8:06 PM
> > > To: Eddy Quicksall
> > > Cc: ips@ece.cmu.edu; pat_thaler@agilent.com
> > > Subject: RE: iSCSI: changing MaxPDUDataLength
> > > 
> > > 
> > > I said only that when you change length you can ask for all PDUs after
> the
> > > ack! (no need to keep a tall - the old offsets are good up to the hole).
> 
> > > 
> > > Julo 
> > > 
> > > 
> > > Eddy Quicksall <eddy_quicksall@ivivity.com> 
> > > 
> > > 
> > > 06/11/2002 12:32 AM 
> > > Please respond to Eddy Quicksall 
> > > 
> > >         
> > >        To:        Julian Satran/Haifa/IBM@IBMIL 
> > >        cc:        ips@ece.cmu.edu, pat_thaler@agilent.com 
> > >        Subject:        RE: iSCSI: changing MaxPDUDataLength 
> > > 
> > >       
> > > 
> > > 
> > > 
> > > Julian, 
> > >  
> > > Another problem here is that the target has to calculate the offset from
> > the
> > > DataSN #. And as BegRun can be any value. E.g., BegRun=4 and RunLngth=0
> > > means starting DataSN=4, send all the data for that sequence. 
> > >  
> > > I think it would be a performance hit and waist of memory to keep a
> tally
> > of
> > > all PDU sizes just for an occasional SNACK. 
> > >  
> > > It's not that it can't be done ... it is more that it is messy and I
> think
> > > there is a solution that will satisfy the design requirements but keep
> the
> > > software simpler. 
> > >  
> > > Eddy 
> > > -----Original Message-----
> > > From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
> > > Sent: Saturday, June 08, 2002 10:16 PM
> > > To: Eddy Quicksall
> > > Cc: ips@ece.cmu.edu; owner-ips@ece.cmu.edu; pat_thaler@agilent.com
> > > Subject: RE: iSCSI: changing MaxPDUDataLength
> > > 
> > > 
> > > That is not completely accurate. 
> > > The only problem is when PDU size decreases and then SNACK must be for
> all
> > > data. 
> > > Target can also keep a mapping of numbers/to offsets (the list should be
> > > small and if it gets long ask for ack (A-bit). 
> > > 
> > > Julo 
> > > 
> > > Eddy Quicksall <eddy_quicksall@ivivity.com> 
> > > Sent by: owner-ips@ece.cmu.edu 
> > > 
> > > 
> > > 06/08/2002 10:54 PM 
> > > Please respond to Eddy Quicksall 
> > > 
> > >         
> > >       To:        pat_thaler@agilent.com 
> > >       cc:        ips@ece.cmu.edu 
> > >       Subject:        RE: iSCSI: changing MaxPDUDataLength 
> > > 
> > >      
> > > 
> > > 
> > > 
> > > 
> > > Thanks,
> > > 
> > > As a target, I won't be able to let it change until all of the
> outstanding
> > > commands are finished (running with ErrorRecoveryLevel>=1). This is
> > because
> > > I must use the PDU size to compute the offset from a SNACK's
> > > BegRun/RunLength.
> > > 
> > > So, I plan to not give the text response for a MaxPDURecvDataLength in
> FFP
> > > until I get back an ExpStatSN == next StatSN.
> > > 
> > > This will cause a delay of unknown time before the PDU size can actually
> > > change ... do you see any problem with this?
> > > 
> > > Eddy
> > > 
> > > -----Original Message-----
> > > From: pat_thaler@agilent.com [mailto:pat_thaler@agilent.com]
> > > Sent: Friday, June 07, 2002 1:13 PM
> > > To: eddy_quicksall@ivivity.com; ips@ece.cmu.edu
> > > Subject: RE: iSCSI: changing MaxPDUDataLength
> > > 
> > > 
> > > Eddy,
> > > 
> > > If one uses a message sync and steering that relys on the transport
> > segments
> > > carrying a full PDU, e.g. TCP ULP Framing Protocol (TUF), then if the
> path
> > > MTU changes one would want to change the PDU data length to fit the new
> > path
> > > MTU.
> > > 
> > > Pat
> > > 
> > > -----Original Message-----
> > > From: Eddy Quicksall [mailto:eddy_quicksall@ivivity.com]
> > > Sent: Wednesday, June 05, 2002 12:24 PM
> > > To: ips@ece.cmu.edu
> > > Subject: iSCSI: changing MaxPDUDataLength
> > > 
> > > 
> > > Does anybody know a case where it is necessary to support a new PDU data
> > > length during full feature phase?
> > > 
> > > Eddy
> > > mailto: Eddy_Quicksall@iVivity.com
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
>