RE: RE: Re: multiple connections

[julian_satran@il.ibm.com]

Somesh,

It is hard to comment on the overhead - the terms will be hypothetical.
But I will try to do it later. On the data hold - the scheme we are talking
now
you will accept data into targets (by the device manager) only from the
commands that are in order - for which you know on what connection they are
going to arrive.
I assume that any initiator will not ship data in an order different than
that in which it ships commands - if the mode is unsolicited. If you'll use
a different queuing model
at the target then data should be shipped accordingly - or you should use
solicited mode.

Regards,
Julo

somesh_gupta@hp.com on 11/09/2000 21:31:51

Please respond to somesh_gupta@hp.com

To:   ips@ece.cmu.edu, Julian Satran/Haifa/IBM@IBMIL
cc:
Subject:  RE: RE: Re: multiple connections






> -----Original Message-----
> From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com]
> Sent: Friday, September 08, 2000 10:32 PM
> To: ips@ece.cmu.edu
> Cc: julian_satran@il.ibm.com
> Subject: FW: RE: Re: multiple connections
>
>
>
>
> Matt & Somesh point to one of the main weaknesses of the
> asymmetric scheme.
> However the initiator adapters setup is generally synchronous and the
> overhead

I would contend that the amount of overhead is fairly significant
for anything other than a iSCSI accelerator NIC - and in this
case the burden falls on the NIC. Doing two transactions is more
expensive than one and you will hit the performance knee
earlier.

> is not significant. For writes the TCP window will take care
> of the holding
> off the data

how does that happen when command and data are on different
connections (and data for different commands is intermingled)?

> that arrive before the command. The big question is - do we
> end-up better
> than
> with the symmetric scheme?
>
> Julo
>
> somesh_gupta@hp.com on 07/09/2000 03:12:05
>
> Please respond to somesh_gupta@hp.com
>
> To:   ips@ece.cmu.edu, matt_wakeley@agilent.com
> cc:    (bcc: Julian Satran/Haifa/IBM)
> Subject:  RE: Re: multiple connections
>
>
>
>
> I think Matt has an important point here. Let us consider READs
> and WRITEs seperately.
>
> 1. READ - In this case the host software can post the appropriate
> read buffers to the connection on which the data is to be received
> and then post the read command to the connection on which the data
> is to be sent. By the time the target gets the command, and returns
> data, it is most likely that the NIC receiving the data will have
> accepted the read buffers.
>
>    Although synchronization issues are minimized, there is a
> reasonable amount of performance impact. Two different "commands"
> (in the sense of host to adapter commands - not in the SCSI sense)
> have to be posted. Two different queues have to be written to (for
> sure if they are on different NICs) and two different NICs triggered
> to accept the "commands". Then there will be two different completion
> events that will be returned by the NICs to the host - one indicating
> the acceptance/"completion"/"return" of the command buffer (indicating
> command sent successfully), and the second when the data is all
> received (along with the SCSI command status). This may also require
> interrupt handling etc which could be minimized by some
> clever schemes.
>
> So the amount of work done by the host for the host/adapter
> interaction
> is close to doubled. There is probably not that much impact on the
> target in this case.
>
> 2. WRITES - This is the really bad one in my opinion. For me, avoiding
> RTTs in iSCSI would just by itself make iSCSI a superior "transport"
> for SCSI. So assuming RTTs are not being used, the host would
> (changing
> the posting order from READs), first post the write command to the
> connection on which the data is to be sent, and then post the write
> buffers to the connection which is to be used for sending data.
> One case is where for whatever reason, the target gets the data before
> it gets the command, and has no clue what to do with the data.
> Let us assume that the target does get the command before it gets the
> data. The target gets a command indicating that data being written to
> whatever lun and whatever location is going to arrive on some other
> connection. First the target has taken an extra event from
> the adapter.
> Then, when data arrives on another NIC (and the target gets another
> event), the target goes through the list of outstanding WRITE
> commands to match the command with the data and then go about the
> business of processing the data.
>
> So in this case the work has increased for the initiator as well as
> the target.
>
> Somesh
>
>
> > -----Original Message-----
> > From: matt_wakeley@agilent.com [mailto:matt_wakeley@agilent.com]
> > Sent: Wednesday, September 06, 2000 1:17 AM
> > To: ips@ece.cmu.edu
> > Subject: FW: Re: multiple connections
> >
> >
> > julian_satran@il.ibm.com wrote:
> >
> > > Dear colleagues,
> > >
> > > With all the heated debate about multiple vs. single
> > connection a request I
> > > made a while ago got no significant reply (neither for
> nor against).
> > >
> > > The request was to consider a proposal made by Kalman Meth
> > to reconsider
> > > the asymmetric model with the addition of a path selection
> > made by the
> > > initiator.
> >
> > One downside I see to this proposal is that for a host with
> > multiple iSCSI
> > nics, it will have to issue two commands (in this order): one
> > to the "data path
> > nic" to describe where the data is to be read from/written
> > to, and one to the
> > "control path nic" to send the command.  This would be
> > communication to two
> > cards instead of one.  Also, note that the communication to
> > the data path nic
> > MUST complete before the control path nic sends the command,
> > requiring some
> > kind of sync mechanism.
> >
> >
> > > This proposal allows removing the command counters - as
> > commands use a
> > > single TCP connection. The single connection can also be a shared
> > > data+control connection.
> >
> > For the case of the command connection failure and
> fail-over to a new
> > connection, I don't see how you can get away from the command
> > counters.  When a
> > fail over occurs, you will need some way of finding out what
> > commands made it
> > to the target and which didn't.  The easiest way to do this
> > is with command
> > numbering.
> >
> > >
> > >
> > > In case of multiple connection the data path to be used is
> > selected and
> > > maintained until the command ends.
> > >
> > > Thanks,
> > > Julo
> >
> > -Matt Wakeley
> > Agilent Technologies
> >
>
>
>
>