Re: Symmetric vs Asymmetric

["Matt Wakeley" <matt_wakeley@agilent.com>]

somesh_gupta@hp.com wrote:

> Matt,
>
> In your response to my message, it seemed to me that you did
> not disagree with my concerns about performance impact (only
> amplified it I think) of the assymmetric approach.

That's right.

> The impact
> will be there even when there are two connections on a single
> NIC - especially when the NIC might be a generic TCP accelerator
> kind of NIC (which will operate on a connection basis), and
> for any software based implementation.

Yes, but software implementations will be slow anyway, so who cares?

In the case where there is a iSCSI NIC implementing the iSCSI and TCP
accelerators in hardware, the host will not know whether there is one or two
TCP connections.  It will simply post the "SCSI command" and the iSCSI NIC
will take care of the rest.

> On the other hand having two connections minimizes head of line
> blocking for critical commands (but is still possible and might
> be most likely when the target is in a bad state anyhow) - so
> maybe it is not a good point.

But the normal case will be that you will be able to pump the commands to the
target without waiting for large data transfers to complete, allowing the
target to get a jump start on the command.


> Having two connections mandated would cause significant performance
> penalty in many implementations and is best avoided.

It will only impact the generic TCP (accelerated or not) implementation.  It
will improve the hardware based iSCSI implementation for the reasons of lower
latency command transfer, less expensive hardware, and less vendor testing.

-Matt Wakeley
Agilent Technologies

>
>
> Somesh
>
> > -----Original Message-----
> > From: matt_wakeley@agilent.com [mailto:matt_wakeley@agilent.com]
> > Sent: Thursday, September 07, 2000 4:56 PM
> > To: matt_wakeley@agilent.com; hufferd@us.ibm.com; ips@ece.cmu.edu
> > Subject: FW: Re: Symmetric vs Asymmetric
> >
> >
> > John Hufferd/San Jose/IBM wrote:
> >
> > > David,
> > > One of the reasons I was holding out for Symmetric over
> > Asymmetric, was for
> > > the workload balancing etc. that could be done at the iSCSI layer.
> > >
> > > I am now willing to "roll-over" but I think I owe you all
> > my reasons for
> > > deciding to support the Asymmetric approach:
> > >
> > > First, in the degenerate case, they both look the same.
> > That is, they can
> > > both support  a single Connection per Session.  So the simplest
> > > implementation is still possible with the Asymmetric
> > approach as it was
> > > with the Symmetric approach.  This means that most current iSCSI
> > > implementations can continue to exist, unchanged.
> >
> > The biggest issue with the "asymmetric" model is that it is
> > NOT "asymmetric"
> > when there is only 1 TCP connection.  When there is only one
> > TCP connection, it
> > is the "symmetric" model - both commands and data on the same
> > TCP connection.
> > Then, when there is more than one TCP connection, the
> > behavior is different.
> > It's always easier to implement something that operates that
> > same way all the
> > time, than two different behaviors.
> >
> > I propose that the asymmetric model mandate at least two TCP
> > connections
> > (implies that at least one physical connection will have at
> > least two TCP
> > connections running on it) - one for commands, the other for
> > data.  This has
> > other advantages, like commands not being flow controlled by
> > large transfers of
> > data.
> >
> > -Matt Wakeley
> > Agilent Technologies
> >
> > >
> > >
> > > But the most important reason follows:
> > >
> > > Since a session is a unique construct that identifies an
> > Initiator and a
> > > Target, I am assuming that does NOT mean each NICs has a
> > unique initiator
> > > ID but that they can, and should, share a common Initiator
> > ID.   The Target
> > > ID, on the other hand,  is either unique per Storage
> > Controller, or not,
> > > and this will depend on the type of Storage Controller.
> > This means that
> > > the Host iSCSI device driver will be able to present, to
> > the layers above
> > > it, either a single port per storage controller, or multiple ports,
> > > depending on how the Storage Controller answers Logins on
> > each of its IP
> > > connections.
> > >
> > > That means iSCSI will not be able to determine when it
> > knows enough to
> > > perform alternate path retry, and for the same reason will
> > not be able to
> > > know when or how to perform command work load balancing across the
> > > different sessions.  This is because either all connections
> > to a storage
> > > controller have either the same iSCSI session ID (SSID), or they are
> > > different -- but iSCSI will not, by itself, know which
> > sessions can be
> > > substituted for others (for balancing or retry).
> > >
> > > The only place where the symmetric approach can do workload
> > balancing and
> > > alternate path retry is where the storage controller makes
> > it look like a
> > > single target ID even though it might have multiple IP
> > addresses.  Though
> > > this might work in some IBM storage controllers (like
> > Shark) it will not
> > > work for all IBM storage controllers (like those of Mylex).
> >  This same or
> > > similar argument probably will apply to EMC's and other
> > vendors various
> > > storage controllers.  Therefore, having iSCSI do any type of command
> > > balancing or command automatic alternate path retry is problematical
> > > without outside influences (probably from upper layers) and
> > this will tend
> > > to make the iSCSI layer very thick and complicated.  It
> > will also mix the
> > > layering concepts (in general not a good idea).  Therefore,
> > I now believe
> > > that the Symmetrical approach can not be a reasonable
> > general/generic
> > > approach to Command work load balance or alternate path
> > retry, and that
> > > function is best left to the Wedge Drivers.
> > >
> > > The Asymmetric approach can not perform command balancing or command
> > > automatic path retry either, but it does not try.  It only
> > tries to balance
> > > the data across multiple connections (and that seems much
> > more reasonable).
> > >
> > > With the Asymmetric approach, data for any specific command
> > only flows on
> > > one connection.  So you can look at the data from various
> > commands as
> > > independent flows on the different connections (NICs) so
> > there should not
> > > be blocking issues, or parallel flow issues, and RDMA type
> > direct to memory
> > > functions can occur.
> > >
> > > With multiple storage controllers, there may be many
> > sessions and many data
> > > connections spread across all the various NICs in a Host.
> > >
> > > The net of all this is that the Asymmetric approach:
> > >    Probably  has good enough spread of commands and data
> > across multiple
> > >    NICs, even without Wedge Drivers.
> > >    Permits the upper layer Wedge Drivers to be left mostly
> > unchanged.
> > >    Permits the simple case with a single connection per
> > session, even with
> > >    multiple NICs.
> > >    Permits the most simple case of a single connection per
> > session with
> > >    only a single NIC.
> > >
> > >  Since I have seen no one else that made arguments for the Symmetric
> > > implementation, you maybe able to call the agreement on the
> > Asymmetric
> > > approach a consensus.
> > > .
> > > .
> > > .
> > > John L. Hufferd
> >