RE: Some Thoughts on Digests

[julian_satran@il.ibm.com]

Pat,

This is academic - but looking at your formula - and you statement that
follows it
they are inconsistent. Lacking segmentation the total length is the
block-length*nb
and the protection offered by a single digest is (approximately) nb times
lower.

Julo

pat_thaler@agilent.com on 08/12/2000 02:40:52

Please respond to pat_thaler@agilent.com

To:   jimw@giganet.com, ips@ece.cmu.edu
cc:
Subject:  RE: Some Thoughts on Digests




Jim,

Probability of an undetected error is:

  probability of error * probability an error is undetectable

The probability of an error scales roughtly with the length of the
transfer.
Therefore, this becomes:
  length (in bits) * ber * probability an error is undetectable

What Julian's equation ignored is that for a multiblock transfer the
equation should be:
  1 - probability all errors are detected
and probability all errors are detected is
  (1 - pub) ^ nb
where pub is the probability of an undetected error in a block
      nb is the number of blocks
since pub << 1, this is approximately
   nb * pub
and the probability of an undetected error in the whole transfer is
approximately
   nb *pub

So the probability of an undetected error in a transfer of nb blocks with
length per block of length/nb is

 nb * length/nb * ber * probablility an error is undetectable

and breaking the transfer into blocks with individual CRCs produced no
change (actually it produces a negligable increase in undetected errors
because of the terms of powers of pub that were discarded in the
approximation).

Regards,
Pat


-----Original Message-----
From: Jim Williams [mailto:jimw@giganet.com]

----- Original Message -----
From: <julian_satran@il.ibm.com>
> Jim,
>
> We will consider again the selection of the polynomial.
> As for your arguments for the length - the object of the digest is get
the
> probability of an undetected error down to a number acceptable for todays
> networks.
> The probability of an error going undetected is 2**-32 *
> BER*length-of-block.
> Under the general accepted practice of considering all errors as
> independent events
> the block length protected by a single CRC is limited to a specific
length
> (2 to 8k is the usual
> figure given for fiber today).

This sounds a little garbled to me.  I question whether the
probability of an undected error is linearly proportional to
block length, but assuming it is, that argues AGAINST segmenting
the block because the probability of an undected error will
of course scale linearly with the number of segments so you
are no better off.

The argument for using 2 to 8K blocks on fiber is a bit different.
If the number of bits in a block is less than 2**32, then it
requires at least 3 bits in error for an undected error to occur.
A CRC-32 will catch ALL 1 and 2 bit errors.  Therefore if
an assumption is made that bit errors are independent events
(i.e. the probability of any bit being in error is independent
of what other bits are in error) the probability of an
undected error looks like:

        ProbUndectErr = 2**-32 * (BER * blockLength) ** 3.

The cube relationship DOES in fact argue for segmenting to
get that absolute minimum undected error rate.  But no
matter how big the block is, the probability of an undected
error has an upper bound of 2**-32.

In the case of iSCSI, though, these assumptions do not hold.

1.    Many links use encoding such as 8-10 meaning that a
      single error event will corrupt 8 bits.

2.    The only errors that the iSCSI CRC will see are those
      that escaped both the link level CRC and the TCP checksum.
      These errors will have very different distribution than
      raw fiber errors.

3.    An undected error rate of 2**-32 of only those errors
      that escape both the link CRC and the TCP checksum
      is acceptable.

      I recently saw a figure that 1 in 30000 TCP segments
      has a checksum error.   Assume this is about 1 error
      if every 50MB of data transferred.  Multiply by 2**16,
      and assume that an undected (by TCP) error will occur
      once in about 3.3TB of data.  Multiply again by
      2**32 to account for a CRC-32 and the undected error
      rate is once in 1.4E19 bytes.  On a 10Gb/s link
      running continuously, this corresponds to a MTBF of 300 years.
      While not neglegable, there are certainly failure modes
      that will cause undected errors more frequently than
      this, and it does not seem justified to segment the
      message for CRC computation in a questionable attempt
      to improve reliability of the CRC.


> And yes we are probably overdesigning for
> large bursts but we have no idea at this level if there will be any
> mitigating interleaving coding underneath and long error bursts are the
> most common for of error in networks.

If long error bursts are the most common, this argues AGAINST segmenting.
The benifit of segmenting is in the case of independent bit errors,
NOT burst errors.

>
> Julo
>