RE: iSCSI: Iterating long text responses

["Rod Harrison" <rod.harrison@windriver.com>]

	We have a concern over option #5, the outline below assumes
that it is safe to issue the same text command more than
once with no side effects. This does seem safe for the
current use of SendTargets, however it might not be if this
mechanism is extended for other long response text commands.

	Our preference would be for option #3 with #4 a close
second. Number 4 has the disadvantage that the initiator has
to deal with a stream of unsolicited text responses with no
idea of when they might stop.

	Perhaps #3 with the following modifications might suffice.

	Add an offset to the initiator R2T (iR2T?) allowing the
target to choose to either buffer the whole response or
regenerate it on the fly. The target should know whether the
data can be regenerated safely and so can make this
decision.

	Add a final bit to the iR2T allowing the initiator to stop
the response transfer at any time. Mandate that the
initiator must either receive all the data or send an iR2T
with final bit set to indicate no more data is required.
This prevents a target getting stuck with the response data
in a buffer forever.

	A typical transfer might be.

	I->T Text command (F bit set)
	T->I Text response (F bit clear)
	I->T iR2T incl. valid offset and length (F bit clear)
	T->I Text response (F bit clear)

	....

	Sequence either ends with

	T->I Text response (F bit set)

	or

	I->T iR2T Offset 0, length 0, (F bit set)

	Since the first text response is unsolicited all current
text command implementations can remain unchanged. Only if
the target responds with the F bit clear does the initiator
need to change.

	- Rod


-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On
Behalf Of
Mark Bakke
Sent: Friday, June 29, 2001 9:18 PM
To: IPS
Subject: iSCSI: Iterating long text responses



Initiator developers-

   Please respond to the questions at the end.

Thanks,

Mark



The current iSCSI draft allows text command and response
PDUs of up to 4096 bytes.  While we don't see any real
problems for the command PDU size, commands such as
SendTargets
can easily exceed the response size.

There are several ways we can fix this.  The first two
solutions
require no differences in the current iSCSI text command and
response; the latter three involve the use of the F bit.

1. Assume that such commands are done on a "special"
connection
   or are handled completely in software, and allow its
response
   PDU to be as large as it needs to be.

   This one appears too restrictive to be a solid solution.
It
   will also weaken any data digests done on the longer PDU.

2. Create an iterative SendTargets key (and do the same for
any
   other text commands that need this), that would allow the
   initiator to request the "next target" or "next address".

   This would work, but would require any new command that
needed
   a large response to implement an iterator.  It also means
that
   each part of the response from the iterator would have to
fit
   in the 4k PDU.

The remainder of these require that only one text command
sequence
be outstanding on a connection at a given time.  They use
the F
bit to indicate the last PDU of the sequence.  Note that
connection
allegiance is assumed for the entire sequence, and text
commands
are never retried on another connection; a new command is
issued
instead.

3. Do a text-response R2T, where the initiator controls when
the
   next partial response is sent.  This would be more
generic:

   I->T  Text Command (F bit set)
   T->I  Text Response (first PDU, F bit cleared)
   I->T  Text Command (with some indicator that we want
more)
   T->I  Text Response (next PDU, F bit cleared)

   ...

   I->T  Text Command (with indicator that we want more)
   T->I  Text Response (last PDU, F bit set)

   The main problem with this is that the target must keep
track
   of the state of its responses; if the initiator just
stops asking,
   it may have to keep a buffered response around until it
times out,
   the connection is dropped, or another text command comes
along.

4. Allow multiple response PDUs to a single text command:

   I->T  Text Command (F bit set)
   T->I  Text Response (first PDU, F bit cleared)
   T->I  Text Response (next PDU, F bit cleared)

   ...

   T->I  Text Response (last PDU, F bit set)

   Basically, we are doing (3) without the R2T.  The
initiator,
   upon sending the text command, must be prepared either to
   accept as many PDUs as come back, or throw them away if
it
   can't handle them.  This solution is a lot like #1, but
with
   the response spread across 4k PDUs.

   Also note that this (and the following scheme) avoid the
problem
   of the target keeping state; it sends ALL of the response
PDUs
   without the initiator asking for them.

5. Do #4, but allow the initiator to specify the amount of
data
   it is willing to accept:

   I->T  Text Command (F bit set, AcceptResponseLength=4096)
   T->I  Text Response (first PDU, F bit set,
TotalResponseLength=12288)

   In the above example, we have created a new text command
field:

      AcceptResponseLength

   And in the text response PDU:

      TotalResponseLength

   The initiator indicated it was willing to accept a 4k
response.
   The target returned the first 4k in the text response,
but set
   the F bit since it was at its limit.  It also returned a
   TotalResponseLength field.  Since this was greater than
the
   AcceptResponseLength, the initiator knows the amount of
buffer
   space it will need to get the full response.  It can then
choose
   whether it will re-send the command, and if so, can give
it a
   large enough response length:

   I->T  Text Command (F bit set,
AcceptResponseLength=12288)
   T->I  Text Response (first PDU, F bit clear)
   T->I  Text Response (next PDU, F bit clear)
   T->I  Text Response (last PDU, F bit set,
TotalResponseLength=12288)

   Note that most initiators will probably send an
AcceptResponseLength
   of the largest response they would normally accept, and
that most
   target responses will fit in one or a few PDUs anyway.

   #5 is really a compromise between #3 and #4; the target
has the
   benefit of being less statefull, and the initiator has
the benefit
   of controlling the amount of data it receives.

I would like to recommend either #4 or #5.  I think that #5
is
probably the safest solution, but #4 may not be a problem
for anyone.

Assuming that none of the implementors of initiators have a
problem
with #4, I would pick that.

If they do have a problem with it, we should go with #5.

Targets probably don't care much whether we do #4 or #5.


Initiator developers-

  Please indicate which solution (#4 or #5) appeals to you.

--
Mark A. Bakke
Cisco Systems
mbakke@cisco.com
763.398.1054