Re: multiple connections

["Matt Wakeley" <matt_wakeley@agilent.com>]

somesh_gupta@hp.com wrote:

> I think Matt has an important point here. Let us consider READs
> and WRITEs seperately.
>
> 1. READ - In this case the host software can post the appropriate
> read buffers to the connection on which the data is to be received
> and then post the read command to the connection on which the data
> is to be sent. By the time the target gets the command, and returns
> data, it is most likely that the NIC receiving the data will have
> accepted the read buffers.

You say "most likely", but are you willing to implement and test code to
handle the occasional times when the NIC receiving the data was delayed
(perhaps it couldn't win access to the local system bus), and the data comes
in with no place to put it? (if the data to be read was in array cache memory,
it could come back to you very fast.)

>
> Although synchronization issues are minimized, there is a
> reasonable amount of performance impact. Two different "commands"
> (in the sense of host to adapter commands - not in the SCSI sense)
> have to be posted. Two different queues have to be written to (for
> sure if they are on different NICs) and two different NICs triggered
> to accept the "commands". Then there will be two different completion
> events that will be returned by the NICs to the host - one indicating
> the acceptance/"completion"/"return" of the command buffer (indicating
> command sent successfully), and the second when the data is all
> received (along with the SCSI command status). This may also require
> interrupt handling etc which could be minimized by some clever schemes.

If you want to avoid the "occasional" race condition I described above, you
will not be able to send the SCSI command until you are sure the NIC receiving
the data is set up to receive it.

I contend that for WRITES, the same applies.  A command could be sent, and the
target could respond with a RTT to the NIC to transmit the data before the NIC
has retrieved state to know that the RTT is coming.

> So the amount of work done by the host for the host/adapter interaction
> is close to doubled. There is probably not that much impact on the
> target in this case.

I agree, probably not that much impact on the target.

> 2. WRITES - This is the really bad one in my opinion. For me, avoiding
> RTTs in iSCSI would just by itself make iSCSI a superior "transport"
> for SCSI. So assuming RTTs are not being used, the host would (changing
> the posting order from READs), first post the write command to the
> connection on which the data is to be sent, and then post the write
> buffers to the connection which is to be used for sending data.
> One case is where for whatever reason, the target gets the data before
> it gets the command, and has no clue what to do with the data.
> Let us assume that the target does get the command before it gets the
> data. The target gets a command indicating that data being written to
> whatever lun and whatever location is going to arrive on some other
> connection. First the target has taken an extra event from the adapter.
> Then, when data arrives on another NIC (and the target gets another
> event), the target goes through the list of outstanding WRITE
> commands to match the command with the data and then go about the
> business of processing the data.

Well, I'm not convinced that targets/arrays will be able to implement a
reliable write scheme that does not use RTT.  But I'll step aside to let them
raise any issues with data coming down a different path than the command.

-Matt Wakeley
Agilent Technologies


>
>
> So in this case the work has increased for the initiator as well as
> the target.
>
> Somesh