Re: ISCSI: Why the use of the Urgent Pointer

[Matt Wakeley <matt_wakeley@agilent.com>]

ronald lee wrote:

> Matt Wakeley wrote:
> >
> > Look, this whole discussion comes down to a simple question.  Do you want
> > iSCSI to succeed as an enterprise storage solution or not?  The intention of
> > the urgent pointer is to help meet the requirements specified in the iSCSI
> > Requirements document.
> >
> > If we review the iSCSI Requirements document it requires:
> >
> > - Low CPU utilization, equal to or better than current technology.
> >
> > This requirement cannot be met using off the shelf TCP/IP stacks running in
> > the OS, especially at 10Gbps. iSCSI solutions will have to be implemented in
> > adapters that handle the transfer of data, similar to Fibre Channel adapters.
> > Otherwise, iSCSI will eat up all the CPU cycles performing TCP/IP processing,
> > and FC will blow iSCSI away.
>
> I agree that packet processing needs to be offloaded but this might not be
> always true.  Some implementations might not care if you dedicate a server to
> a specific application such as mirroring data across LAN/WAN and you don't use
> this server for anything else.  This is up to the user.

Still, if the CPU is spinning all it's cycles in the TCP/IP stack, it won't have
much time left over for mirroring.


> > - Cost competitive with alternative storage network technologies.
> >
> > In order for iSCSI to succeed it must be able to be cost competitive with
> > other solutions (10Gig Fibre Channel, etc).  This means that it must be
> > implementable without putting gobs of (costly) memory on the adapter cards.
>
> I think same problems might exist in fibre channel if hardware loses data and
> even fibre channel has to resend the data.

Fibre channel has built in framing, thus, the data at the receiver does not have to
pile up on the FC adapter. It can be placed directly where it needs to go.

>  Its a matter of high reliable is
> the connection and the devices in the connection.  Even TCP/IP networks
> can be reliable if it was dedicated to serving I/Os only and all paths along
> the way was reserved or leased lines.
>
> > In order to avoid this memory requirement in the presence of lost frames, the
> > adapter will either have to place data in host memory were it belongs
> > *requiring a framing mechanism* or throw it away.  Fibre Channel adapters
> > today are able to operate in this manner today without a lot of memory because
> > they can place data that arrives out of order directly where it belongs in
> > host memory [because framing is provided].
>
> I think it is good to have framing, but not at the expense of all the software
> and testing that is involved in making this work.

TCP stacks should already be tested and certified to correctly process the urgent
pointer.  The only "new" software would be in the iSCSI layer itself, which is new
anyway.

>  If someone wants run at
> 10 GBit rates across WAN, they might need a little more help than just framing.
> Also, having 100Mbytes of memory is not such a big deal these days with the
> cost of memory coming down.

If cost is not an issue, why is everyone presuring vendors to reduce prices?  Of
course cost is an issue. If the extra 100MB of memory is going to add $100 to the
cost of your server, you can bet Sun will be looking around for the cheapest
solution.  Just like they want to drive every dollar out of the cost of a disk
drive.

>  Instead of having one data connection, if you had
> 100s of connections, the other connections can still operate since they did
> not lose any packets.

It wouldn't help if there was 200MB worth of data in the long haul pipe for that one
connection.

>
>
> >
> > The urgent pointer mechanism in the current draft provides the framing
> > mechanism so that data can be placed properly in the presence of dropped
> > frames.
>
> Sounds like this will help in terms of saving the memory required on the
> adapters since the command that has the lost frame still needs to wait
> for that lost frame.

Somehow, I think you missed a word somewhere.  Could you restate?


> I think overall, if you are losing packets in the network regularly, doing this
> will not help performance because TCP resends will slow you down
> anyways.  If
> the network is not losing packets, then out of order segments will just
> be momentary.

Better to resend only a few packets (the lost ones using SACK) instead of all the
data that had to be dumped on the floor because the adapter ran out of memory to
store them.

>
>
> >
> > Matt Wakeley
> > Agilent Technologies