RE: bridging issues -- Converging FCP-2 and iSCSI control structures

[Charles Monia <cmonia@NishanSystems.com>]

Hi Julio:

In comparing FCP SCSI control structures with iSCSI, there are a lot of
fields that are functionally identical. Therefore, as a step towards
bridging between iSCSI and FCP, as you mentioned, I went through the
exercise of rearranging the transport-independent fields within the iSCSI
control structures to reflect the equivalent FCP-2 structures. (See
ftp://ftp.t10.org/t10/drafts/fcp2/fcp2r04.pdf, for a copy of FCP-2.)

The gory details are below, including a field-by-field mapping between
corresponding iSCSI and FCP parameters.  

The conversion seems fairly straightforward and preserves iSCSI semantics.
Functionally, the main differences are:

a) Since FCP combines the task management and SCSI command blocks into a
single control structure and response, I've done the same thing with the
corresponding control blocks as mapped to iSCSI.  i.e..  In the reformatted
version, the iSCSI task management command and return status are merged with
the corresponding iSCSI command and status blocks.

b) All iSCSI-specific fields have been collected into special iSCSI headers.
I have not gone through the exercise of defining a fixed length iSCSI
preamble that fits every case however.

c) Since command ordering is handled by iSCSI and TCP/IP, the Command
Reference Number in the FCP command block is an iSCSI reserved field.

The one change in the FCP command template was the need to provide an
encoding for the ABORT TASK management function. (FCP does this by aborting
the FC-2 Exchange).

An issue for further study is how best to map the mechanism for Asynchronous
Event Notification between FCP and iSCSI.

By the way, in comparing the two kinds of command control blocks, I noticed
that iSCSI does not seem to allocate space for CDBs longer than 16 bytes
(unless that's what the "command-data" field was to be used for).  Anyhow,
assuming the command-data in iSCSI is not intended for this purpose, I
defined a separate optional extension for iSCSI to contain this information
as described in the iSCSI spec.

Issues for further study are

a) How best to map the mechanism for Asynchronous Event Notification between
FCP and iSCSI.

b) An efficient mapping between the iSCSI data transfer model and Fibre
channel.

1.  iSCSI command structure (iSCSI section 3.2)

The iSCSI Command data structure is modified to include the FCP_CMND Payload
(see table 22 of the Rev 4 FCP-2 draft standard). iSCSI information is
contained in a transport-specific header as shown below, followed by the
FCP_CMD template, and terminated with optional command-specific data.

+------------------+
| iSCSI Header     |
+------------------+
| SCSI Command     |
| Block (modeled  |
| on FCP-2)        |
+------------------+
| Additional Data  |
| (Optional)       |
+------------------+


1.1 Contents of iSCSI header:

The header up to ExpStatRn is as defined in the iSCSI spec.

	OpCode
	Length
	CmdRN
	ExpStatRn
	Initiator Tag
	Length of "Additional Data"

The "Length of Additional Data" is a 32-bit field, new to iSCSI, containing
the size of the space reserved for additional, command-dependent data.


1.2 iSCSI mapping of FCP-2 command template:

	FCP_LUN -- 8-byte Logical Unit Number
	ISCSI Definition:  Same as FCP

	FCP_CRN --  1-byte command reference number:
	iSCSI definition:	reserved field

	Task Attribute -- 3-bit field encoding Head of Queue, Simple,
	etc.
	iSCSI definition --  Same as FCP

	Task Management Flags --  8-bit field containing bit-significant
	task management flags.
		Bit 7 -- Obsolete
		Bit 6 -- Clear ACA
		Bit 5 -- Target Reset
		Bit 3 -- Reserved
		Bit 2 -- Clear Task Set
		Bit 1 -- Abort Task Set
		Bit 0 -- Reserved

	iSCSI Definition --  Bit 7 defined as "Abort Task", remaining
	bits same as FCP-2.

	Alternative iSCSI definition: Convert the task management flags
	to a 4-bit encoded field.

	[As described in what follows, the above field allows the
	reformatted command block to perform the function of the
	iSCSI task management block defined in iSCSI, section 3.7.]

	RDDATA, WRDATA --  One-bit fields indicating direction
	of data transfer
	iSCSI definition -- Same as FCP-2.

	Additional FCP Length -- Amount of space allocated
	for a CDB whose length exceeds 16 bytes.
	iSCSI definition -- Same as FCP-2.

	Additional FCP_CDB -- Contains that portion of the
	CDB which exceeds 16-bytes.
	iSCSI definition -- Same as FCP-2 (Note: this field
	does not seem to be defined in the current iSCSI proposal).

	FCP_DL -- Count of the greatest number of bytes expected
	to be transferred to or from the application client data
	buffer by the SCSI CDB.

	iSCSI definition -- Same as FCP-2.

Other FCP-2 reserved fields are mapped to corresponding reserved fields in
iSCSI.

1.3 iSCSI features not reflected in the FCP-2 data structure:

	"Additional data":  Command-specific data appended
	to SCSI command. Suggest modifying iSCSI to append
	this information to the command block and reflect its
	extent through the iSCSI header shown above.


2. SCSI Task Management command (iSCSI section 3.7)

Deleted. The function of this data structure in iSCSI is subsumed by the
revised iSCSI command block shown above.

3. SCSI Response (iSCSI section 3.3) and SCSI Task Management response
(section 3.8)

The iSCSI data structures for responding to command and task management
requests are combined to reflect the merger of the iSCSI command and task
management blocks described above.

+------------------+
| iSCSI Header     |
+------------------+
| SCSI Response    |
| Block  modeled   |
| on FCP-2         |
+------------------+
| Additional Data  |
| (Optional)       |
+------------------+


3.1 Contents of iSCSI header:

The following are as defined above.

	OpCode
	Length
	CmdRN
	ExpStatRn
	Initiator Tag
	Length of "Additional Data" (Must be zero).

3.2 iSCSI Mapping of FCP-2 response template.

	FCP_RSP, bytes 0 through 9 -- Reserved
	iSCSI mapping -- Same as FCP

	FCP_RSP, byte 10 -- Flags, contents:

	FCP_RSP_LEN_VALID --  1 = FCP_RSP_INFO field contains valid data
	iSCSI mapping -- Equivalent to FCP mapping.

	FCP_SNS_LEN_VALID -- 1 = FCP_SNS_INFO field contains valid data.
	The length of the returned sense data is contained in
	FCP_SNS_LEN.
	iSCSI mapping -- Equivalent to FCP mapping.

	FCP_RESID_OVER, FCP_RESID_UNDER -- Residual underflow or
	overflow count.
	iSCSI mapping -- Equivalent to FCP mapping.

	FCP_CONF_REQ -- 1 = Initiator must confirm receipt of status
	information unit.
	iSCSI Mapping -- Reserved.

	FCP_RESID --  if FCP_RESID_UNDER or FCP_RESID_OVER is set,
	contains count of residual data
	iSCSI Mapping -- Same as FCP.

	FCP_SNS_LEN -- if FCP_SNS_LEN_VALID, contains length of sense
	data.
	iSCSI Mapping -- Same as FCP.

	FCP_RSP_LEN --  If FCP_RSP_LEN_VALID, contains length
	of FCP_RESPONSE field. (FCP-2 limits FCP_RSP_LEN to 4 or 8).
	iSCSI Mapping -- Same as FCP

	FCP_RESP_INFO -- If FCP_RSP_LEN_VALIS = 1, this is a 4- or
	8-byte field containing the iSCSI mapping -- See 3.2.1 below.

	FCP_SNS_INFO --  If FCP_SNS_LEN_VALID =1, contains FCP_SNS_LEN
	bytes of sense data.
	iSCSI mapping -- Same as FCP-2

3.2.1.  iSCSI mapping of FCP_RSP_INFO field

	FCP_RSP_INFO is a 4- or 8-byte field containing the following:

	Bytes 0 through 2:  Reserved
	iSCSI mapping:  Same as FCP.

	Byte 3:  RSP_CODE

	Code values (hex):
		00 --  No failure or task management function complete
		04 --  Task management function not supported
		05 --  Task management function falied

		All other values reserved.

The above code values map directly to iSCSI.

	Bytes 4 through 7: Reserved. 

In the RSP_CODE field, iSCSI equivalents for the following FCP-specific
codes may be useful:

	Code values (hex):
		01 -- FCP_DATA length different than burst length.
		02 -- FCP_CMND fields invalid
		03 -- FCP_DATA RO mismatch with  FCP_XFER_RDY DATA_RO

Codes 01 and 03 indicate a mismatch between an FCP_XFR_RDY parameter and the
corresponding parameter in the actual data transfer information unit.


4.  iSCSI RTT and FCP_XFR_RDY Information Unit


+------------------+
| iSCSI RTT        |
| Header           |
+------------------+
| iSCSI RTT        |
| modeled on       |
| FCP_XFR_RDY      |
+------------------+

In iSCSI, The use of RTT is negotiated between the target and initiator. 

4.1 RTT Header contents

The following are as defined in the iSCSI spec.

	OpCode
	Length
	CmdRN
	ExpStatRn
	Initiator Tag
	Target Tag

4.2 iSCSI Mapping from FCP_XFR_RDY to RTT

	DATA_RO --  Offset of the first byte of data relative to
	the start of the initiator's buffer

	iSCSI Mapping -- Buffer offset as defined in iSCSI, section 3.9

	BURST_LEN -- On a transfer from the FCP initiator to the FCP
	target, exact number of bytes to be transferred.
	iSCSI Mapping --  Desired data transfer length as defined in
	iSCSI, section 3.9


> -----Original Message-----
> From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com]
> Sent: Sunday, August 27, 2000 8:58 AM
> To: ips@ece.cmu.edu
> Subject: bridging issues
> 
> 
> 
> 
> Dear colleagues,
> 
> Lately some iSCSI issues related to bridging to/from FC networks where
> brought to my
> attention.  I thought that it would be wise to share them 
> (and my first
> rough attempt to solve them) with you for two reasons:
> 
>    that's why we have a Working Group (don't we?)
>    there might be more of this kind and this is a good time 
> to think about
>    them and bring the with or without a solution to the WG
> 
> FC and iSCSI are SCSI mappings over a network that share a 
> "network" view
> of the
> transport infrastructure but have widely different views of 
> the network
> with regard
> to distances, reliability, flow control etc..
> 
> FCP assumes a locally controlled environment using an 
> unreliable delivery
> version of FC
> and doing recovery (a very rare event) at FCP level.
> 
> FC addressing model uses locally generated dynamic addresses 
> as part of the
> "naming"
> and its tagging.
> 
> An iSCSI initiator should be able to operate a FCP target 
> through a very
> simple and
> stateless (or almost stateless) gateway.
> 
> A designer attempting to do this (and I know many of you 
> already did or are
> in the midst
> of doing that) will find it difficult to do achieve a 
> stateless design (but
> not impossible).
> 
> The FCP recovery model from packet loses (commands, data and 
> status) is
> based on sequencing (if lost packets are within sequences) or 
> timeouts if
> those are at sequence
> boundaries.  In the FC world those are very rare events and 
> in an iSCSI+FCP
> hybrid
> could be handled by a statefull gateway (expensive) or by a stateless
> gateway and
> a more radical initiator recovery - based on TCP connection close and
> reopen.
> The later solution requires iSCSI to enforce a set of timeouts (an
> initiator requirement only)
> and provide an API for timeout setting and connection abort 
> to an upper
> layer.
> Enforcing a set of timeouts would have the good side-effect 
> of making iSCSI
> also
> tolerant to aberrant device behavior - and that is 
> unfortunately bound to
> happen more
> often as the complexity of the devices will increase.
> Please observe that the set of timeouts I am referring to are not a
> "protocol" element
> (they do not appear on the wire). Specifying the expected 
> behavior of an
> iSCSI
> initiator will simplify the task of building bridges to FCP.
> We propose to do so by adding a "host requirements" chapter.
> 
> Another bridging issue is the initiator tag-translation mechanism.
> iSCSI is requiring the initiator issue a unique tag for each 
> task but does
> not require any structure for the tag. This way an 
> implementer may choose
> the most efficient structure for his implementation (e.g.,
> use the address of it's control structure representing the 
> task as a tag).
> 
> FCP on the other hand demands some structure for the tag.
> 
> A stateless bridge would want to pass the tag unchanged from 
> initiator to
> target or translate it with a stateless rule.
> 
> If we restrict the design space to gateways that present a 
> "homogenous"
> view of the device conglomerate behind them (e.g., all 
> devices are FCP)
> and we may pass this information to the initiator - then the 
> initiator can
> use the slightly more restrictive FCP tagging mechanism.
> 
> Julo
> 
>