RE: iSCSI: Negotiation clarifications still needed

[Bill Studenmund <wrstuden@wasabisystems.com>]

On Tue, 28 May 2002 pat_thaler@agilent.com wrote:

> Bill,
>
> My comments are inserted in line bracketed with my initials.

Thank you. I appreciate the time & attention you're putting into this.

> -----Original Message-----
> From: Bill Studenmund [mailto:wrstuden@wasabisystems.com]
> Sent: Tuesday, May 28, 2002 12:13 PM
>
> <snip>
> It's to avoid corner cases while doing what you say above, "preparing its
> strings and chopping them into PDUs."
>
> The simplest thing in my mind is to:
>
> 1) read the other side's offers into a buffer
>
> 2) parse that buffer generating responses to another buffer
>
> 3) send the output buffer
>
> Does that make sense? You don't have to agree, just does it make sense?
>
> The problem I see with what Pat described is that if you are in
> operational negotiation and have more than 1 PDU worth of data, you will
> be receiving input while you're in step 3. You have to be able to re-enter
> steps 1 & 2 before you can finish step 3.
>
> <PAT> I think that is over stating the complexity. During negotiation,
> the other side can only have one PDU outstanding at a time. That is explictly
> stated for TextNegotiation PDUs. For LoginPDUs, it isn't explicit but since
> they are immediate and the initiator can only count on the target having one
> immediate buffer for the connection it can't send another LoginRequest until
> it gets the LoginResponse without risking that a PDU will be dropped. Also
> allowing anything other than one oustanding LoginPDU at a time would open
> the possibility that both sides offered the same key. Therefore, a target
> won't get the next PDU until after it sends the response PDU (or vice-versa
> for an initiator). A PDU won't come in while a PDU is being sent from the
> outgoing negotiation buffer.

I agree. I didn't meant to imply we'd do something other than one PDU goes
each direction at a time. You're right that to do THAT would be VERY
messy.

> Once you send the PDU, there may still be some data left in the output
> buffer but it doesn't seem complex to switch at that point to waiting to
> receive and parse the next received PDU. One had better be able to switch
> from one process to another anyway as most implementations will handle
> multiple connections which can be in diffent phases. If you are willing
> to buffer all your output into a buffer and wait to send it until the other
> side has stopped sending new parameters, then all you need to do to change
> that to an implementation that sends what it has in the output buffer as
> it goes along is to have pointer into the buffer that indicates the next
> byte to be sent. <PAT>

I guess the question is where were you when you noticed that you had to
stop? How much state do you need to save to pick things up at that point.
What kind of design permits you to pick things back up at that point?
While negotiations spanning PDUs will mean there is always some state to
keep, the thing about empty PDUs is that there is no state in the parser
code that needs saving; just where are we in the buffer we are sending or
receiving.

> Letting the target answer while the initiator is trying to send an
> extended PDU adds the following complications (AFAICT):
>
> 1) initiator has to make sure a key and the '=' don't get split on a PDU
> boundry.
>
> <PAT> One doesn't have to do that anyway. We can chose the rule to not
> offering a new (non-declarative key) when one has received a partial key
> which seemed to be acceptable to a number of people. <PAT>

Maybe I'm confused. I thought I was talking about the case where the
initiator is just realizing that what it wants to send won't fit. i.e.
when we first realize, "oh, we're going to need to split this up." This is
before the other side realizes the PDU is split.

I was sloppy with my use of initiator and target. Sorry. :-(

> 3) The target has to detect a split key/value pair on in-bound, and a)
> store the received text, and b) remember not to offer said key or any
> others.
>
> <PAT> detecting an incomplete key is reasonably easy. And the target can
> chose to do the simplist thing which is not to offer new keys when a
> partial key is outstanding. <PAT>
>
> 4) The target ALSO has to make sure ITS response doesn't fill up a PDU. If
> it does, it has to remember where IT is and that it has more to send.
>
> <PAT> In your approach the target can have a buffer with more than
> one PDU worth of response in it. Once it starts sending from that buffer
> it has to remember where it is and that it has more to send. This is
> not any different. There is a buffer with pointers into for end and
> last byte sent (or next byte to send). <PAT>

The one difference is that we could end up having both a partially-full
input buffer to parse (because the info sent to us didn't fit a PDU) and
a partially-full output buffer (since our output overflowed a PDU too).
While you're right that the negotiation phase will work around this,
having both buffers open at once is just a bit more complicated.

> <PAT> I don't see much complexity difference here. If one wanted to make
> negotiation as simple as possible, one would make the Initiator the
> originator and have it solicit declarations with ?. When the group
> decided it wanted the flexibility of not offering keys if one was
> going to use the default value and letting the target offer such a key
> to force the negotiation of it, then a certain amount of complexity
> was introduced. <PAT>

To quote code, what I'm envisioning is w/ the Empty PDU idea, things like
PARAM_TEXT_PARSE_ELSE() in the intel code, and scan_input_and_process() in
the UNH code can be used as-is. Without the empty PDU idea, they need to
save more state. Maybe not much, but more.

> Truth be told, the corner cases probably don't apply at present. What we
> have (and are talking about) will work for security negotiation (we have
> to wait for the key to make it across) and for operational negotiation
> (since I don't think we have 8k worth of negotiation data). So for now the
> common case (no large vendor extentions) isn't going to tickle this set of
> problems.
>
> <PAT> I agree. This means that plug fests probably haven't hit these
> cases. <PAT>

Agreed. That's why I suggested that experiment. Just crank the PDU data
size you (and this "you" is the whole list) use in login negotiations to
something really small, like 128.  Obviously you'd never ship such a
thing, but just see what your code does, and what you think of what your
code does.

You'll find out real quick how well your code does in the case we're
talking about here.

> Does the statement of the perceived problem make sense, even if the
> opinion of the WG is to not use the empty PDUs? I would feel much
> comfortable with deciding to not use empty PDUs if folks said they
> understand the above point and they choose to address the problems
> differently.
>
> <PAT> I want to point out that there are times in the protocol when
> one will have to send an empty PDU. For instance, when getting a
> security key that is longer than a the Max PDU size (or even less
> than that - the sender isn't required to send Max size PDUs) or when
> a device has no key-value pairs to send but its partner hasn't set
> the F bit. The issue is whether to require that there be a process
> for handing the right to originate keys (or in your proposal to send
> keys at all) keys from initiator to target and back which adds
> its own complexity. I think the complexity will be
> of the same order as the existing situation. <PAT>

Well, then we have a difference of opinion. :-)

So as John puts it we have moved to the personal opinion stage. :-)

What do we want to decide?

Take care,

Bill