|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] RE: iSCSI: Addition of CmdSN in Data-out PDUSome of the ordering is session-wide (commands) and some is connection (Responses) or even "tag" scoped (Data). Session imposes additional order (beyond connection) but will not change connection ordering (except for recovery). Julo Robert Snively <rsnively@Brocade.COM> Sent by: owner-ips@ece.cmu.edu 22-10-01 17:46 Please respond to Robert Snively To: ips@ece.cmu.edu, Julian Satran/Haifa/IBM@IBMIL cc: Subject: RE: iSCSI: Addition of CmdSN in Data-out PDU Weren't all the other ordering issues associated with sessions (like CSN for example)? Doesn't that interact with the requirements for ordering of immediate data? As far as I can see, the ordering becomes significantly less important at the connection level because the session level management has to take things out of order in any case. Bob > -----Original Message----- > From: Julian Satran [mailto:Julian_Satran@il.ibm.com] > Sent: Saturday, October 20, 2001 3:18 PM > To: ips@ece.cmu.edu > Subject: RE: iSCSI: Addition of CmdSN in Data-out PDU > > > > Robert, > > The ordering rule is for a connection (not for a session). It's sole > purpose is to minimize the chance for a deadlock. > > Regards, > Julo > > > > > Robert Snively > > <rsnively@broc To: "'Rod > Harrison'" > ade.com> > <rod.harrison@windriver.com>, Julian > > Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu > 16-10-01 18:02 cc: > > Please respond Subject: RE: > iSCSI: Addition of CmdSN in > to Robert Data-out PDU > > Snively > > > > > > > > > I am not quite sure what meaning "in-order" data has in > a multi-connection session. Does it mean that you actually > expect the data across multiple connections to arrive temporally > in the same order that is specified by the command sequence > numbers? Or does it mean that the data has to be arranged > in some order by the target session processing? > > Or is it better to step aside from the in-order question > for data transmission and do as Rob Harrison suggests: > > "As an initiator I would send the command PDU as soon as > I could if there were no immediate data, then queue the > DMA for the unsolicited data. Since there could be many > other DMAs queued ahead of the unsolicited data it is quite > possible that other commands, both read and write, will > be sent before the DMA completes." > > This would imply no strict ordering requirement. > > If an ordering requirement is necessary, I submit that it > is because of the problems with unsolicited data reception, > not with the actual data order. > > Bob > > > -----Original Message----- > > From: Rod Harrison [mailto:rod.harrison@windriver.com] > > Sent: Monday, October 15, 2001 4:06 AM > > To: Julian Satran; ips@ece.cmu.edu > > Subject: RE: iSCSI: Addition of CmdSN in Data-out PDU > > > > > > Julian, > > > > I think you might have misunderstood what I meant. > I was not > > advocating we change the spec in favour of relaxing the > data ordering > > requirements. I was just point out there is a problem for the > > initiator to solve in order maintain correct ordering, and that I > > believe it is not a terribly difficult one. > > > > - Rod > > > > -----Original Message----- > > From: owner-ips@ece.cmu.edu > [mailto:owner-ips@ece.cmu.edu]On Behalf Of > > Julian Satran > > Sent: Sunday, October 14, 2001 9:30 PM > > To: ips@ece.cmu.edu > > Subject: RE: Addition of CmdSN in Data-out PDU > > > > > > > > Immediate data are not mandatory (even if enabled). Having > one or ten > > DMAs > > does not change the fact that they share a TCP "pipe" and a decent > > transmitter will ship the TCP data in order - otherwise it > just throws > > his > > problems in the receiver's lap. Our additional requirement on order > > should > > make no difference. And it would be time to change the name of the > > thread. > > > > Julo > > > > > > > > > > > > "Rod Harrison" > > <rod.harrison@wind To: John > Hufferd/San > > Jose/IBM@IBMUS, "Robert > > river.com> Snively" > > <rsnively@Brocade.COM> > > Sent by: cc: > > <somesh_gupta@silverbacksystems.com>, > > owner-ips@ece.cmu. "BURBRIDGE,MATTHEW > > (HP-UnitedKingdom,ex2)" > > edu > > <matthew_burbridge@hp.com>, "'Binford, Charles'" > > <CBinford@pirus.com>, > > <ips@ece.cmu.edu> > > Subject: > RE: Addition > > of CmdSN in Data-out PDU > > 13-10-01 13:12 > > Please respond to > > "Rod Harrison" > > > > > > > > > > > > > > Even more common would be the DMA latency. As an > initiator > > I > > would > > send the command PDU as soon as I could if there were no immediate > > data, then queue the DMA for the unsolicited data. Since there could > > be many other DMAs queued ahead of the unsolicited data it is quite > > possible that other commands, both read and write, will be > sent before > > the DMA completes. > > > > You are spot one when you say the delay in obtaining the > > unsolicited > > data may be large and variable. If there are multiple DMA engines it > > does become a challenge for the initiator to ensure the correct > > ordering of the unsolicited data. Note that since there may be > > multiple unsolicited data PDUs in the same burst, and those may be > > filled with separate DMAs the ordering problem propagates down to > > within individual commands. > > > > I don't believe it is a particularly difficult > problem for > > the > > initiator to solve, but I do agree things would be easier for the > > initiator if the ordering requirement were removed. And > more so if the > > requirement to send in dataSN order were also removed, but this is > > probably too much pain for the target. > > > > - Rod > > > > -----Original Message----- > > From: owner-ips@ece.cmu.edu > [mailto:owner-ips@ece.cmu.edu]On Behalf Of > > John Hufferd > > Sent: Saturday, October 13, 2001 1:01 AM > > To: Robert Snively > > Cc: 'somesh_gupta@silverbacksystems.com'; BURBRIDGE,MATTHEW > > (HP-UnitedKingdom,ex2); 'Binford, Charles'; ips@ece.cmu.edu > > Subject: RE: Addition of CmdSN in Data-out PDU > > > > > > > > Robert, > > I think that an Initiator being able to send a waiting Read command, > > without having to wait for many large write segments -- > that are being > > sent > > (as unsolicited data) -- is very useful. And that would mean, the > > unsolicited data is waiting to be sent until the Read Commands are > > sent. > > This might be a very frequent case. > > > > . > > . > > . > > John L. Hufferd > > Senior Technical Staff Member (STSM) > > IBM/SSG San Jose Ca > > Main Office (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > > Home Office (408) 997-6136 > > Internet address: hufferd@us.ibm.com > > > > > > Robert Snively <rsnively@Brocade.COM>@ece.cmu.edu on 10/12/2001 > > 03:56:06 PM > > > > Sent by: owner-ips@ece.cmu.edu > > > > > > To: "'somesh_gupta@silverbacksystems.com'" > > <somesh_gupta@silverbacksystems.com>, "BURBRIDGE,MATTHEW > > (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>, "'Binford, > > Charles'" <CBinford@pirus.com>, ips@ece.cmu.edu > > cc: > > Subject: RE: Addition of CmdSN in Data-out PDU > > > > > > > > I have two small questions that would help me understand this. > > > > How do you know that unsolicited data is expected? By nature, > > unsolicited data is received as a "maximum surprise" which can > > only be determined after unpacking and parsing at least a > > part of the command structure, possibly even including the > > SCSI command (although there are some hints contained in the > > command header information). > > > > How can you constrain a generic system to posting the > > unsolicited data in the same order that the commands were > > emitted? In general, I would have expected the system to > > be emitting command followed immediately by the corresponding > > unsolicited data. If that is not the case, it means that there > > was a delay in obtaining the unsolicited data for transfer and > > that the delay was sufficient to allow the insertion of commands. > > If the delay is that large (and probably variable), the enforcement > > of transfer of unsolicited data in the same order as the > > commands are emitted seems to me to be a significant challenge, > > and certainly shouldn't be required as normal behavior. While it > > would make things simpler for targets (already challenged by > > unsolicited data), it seems to me that it would make things > > much more complex for initiators. > > > > Bob > > > > > -----Original Message----- > > > From: Somesh Gupta [mailto:somesh_gupta@silverbacksystems.com] > > > Sent: Friday, October 12, 2001 2:51 PM > > > To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Binford, Charles'; > > > ips@ece.cmu.edu > > > Subject: RE: Addition of CmdSN in Data-out PDU > > > > > > > > > Matthew, > > > > > > Since the unsolicited data does not follow the > > > command, you need the link list. But since the > > > unsolicited data must be sent in the same order > > > as the commands, a link list is enough. > > > > > > Let us say that you have 8 commands. The ones > > > for which we expect unsolicted data are marked > > > as Cnn(ud). And I have marked the unsolicted > > > data PDUs as UD(nn). The (nn) with data implies > > > that it is implicit and not actually carried with > > > the PDU itself. > > > > > > C01(ud) C02(ud) C03(ud) C04 C05 C06(ud) UD(01) ---> > > > --> C07(ud) UD(02) UD(03) D04 D04 D05 D05 UD(06) ---> > > > --> UD(07) C08(ud) UD(08) ---> > > > > > > After the target receives the command C01, C02, and C03 > > > for which it expects unsolicited data, it puts them in > > > a link list. It also receives C04 and C05 for which > > > unsolicited data is not expected and they don't go > > > on the list. It then receives C06 for which unsolicited > > > data is expected, and it is added to then end of the list. > > > Then an unsolicited data PDU is received. It must go > > > with the command at the head of the list which is C01. > > > Use the ITT to make sure and you can then take C01 off > > > the list and so on. > > > > > > Somesh > > > > > > > > > > -----Original Message----- > > > > From: owner-ips@ece.cmu.edu > > > [mailto:owner-ips@ece.cmu.edu]On Behalf Of > > > > BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2) > > > > Sent: Friday, October 12, 2001 7:53 AM > > > > To: 'Binford, Charles'; ips@ece.cmu.edu > > > > Subject: RE: Addition of CmdSN in Data-out PDU > > > > > > > > > > > > Charles, > > > > > > > > As you have described the spec states that "that > > > unsolicited data MUST be > > > > sent in the same order as the commands". This is not > the same as > > > > unsolicited data must follow the command associated with > > > it: For example: > > > > > > > > (Cx = SCSI Command PDU, Dx = The unsolicited data PDUs. The > > > x in all the > > > > example can be the ITT. It is not the CmdSN. > > > > > > > > This is allowed: > > > > > > > > C1 C2 C3 D1 D2 C4 D3 D4 > > > > > > > > and the target will have to use the ITT to associate the > > > data with the > > > > command. > > > > > > > > Matthew > > > > > > > > > > > > -----Original Message----- > > > > From: Binford, Charles [mailto:CBinford@pirus.com] > > > > Sent: Friday, October 12, 2001 2:50 PM > > > > To: ips@ece.cmu.edu > > > > Subject: RE: Addition of CmdSN in Data-out PDU > > > > > > > > > > > > I have not verify version 8 is still the same, but version 07-97 > > > > had a rule > > > > that unsolicited data MUST be sent in the same order as the > > > commands. > > > > > > > > Thus, there is no need for a search on the ITT. The target > > > just needs to > > > > keep of linked list of I/Os waiting on unsolicited data. > > > New commands are > > > > added to the tail, any unsolicited data *should* be associated > > > > with the I/O > > > > at the head of the list. The ITT is used as a sanity check and > > > > you're done. > > > > > > > > What am I missing? > > > > > > > > Charles Binford > > > > Pirus Networks > > > > 316.315.0382 x222 > > > > > > > > > > > > > > > > > > > > >
Home Last updated: Mon Oct 22 13:17:32 2001 7327 messages in chronological order |