RE: iSCSI read/write cost difference

["Tony Battersby" <tonyb@cybernetics.com>]

> -----Original Message-----
> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> mingz
> Sent: Thursday, April 17, 2003 12:10 PM
> To: ips@ece.cmu.edu
> Subject: iSCSI read/write cost difference
>
>
> I recently did some experiments on iSCSI benchmarking. I used
> the UNH iSCSi ref18_03 implementation and use a ram based scsi
> device for iscsi target use. so there is no any disk overhead.
>
> I used a linux kernel module to send fixed size read/write
> requests to iscsi initiator, which eventually will be filled by
> iscsi target. i use interl pro1000 gigabit nic and intel 470
> gigabit switch. and both initiator and target use same type piii
> 866 pc with 1g ram.


The "e1000" driver for the Intel gigabit NIC has some tweakable parameters
that can be adjusted to give better performance.  See
Documentation/networking/e1000.txt in the root of the kernel source for more
details.


> now the IOPS result shows that
>
> 		2k		4k		8k
> 16k		32k
> read		1975	1781	1557	1081	767
> write 	2877	2225	1588	812		533
>
> for small reuqests, read is slower than write, while for
> large requests, read is faster than write. i redo the
> experiments on another enviroment, also get similar results.
>
> can anybody explain why this happens?


I think the UNH iSCSI implementation dosen't set TCP_NODELAY on the
connection sockets, which can lead to strange performance characteristics
like this.  See the man pages for "setsockopt" and "tcp".  You can find an
example of setting TCP_NODELAY in the linux-iscsi drivers at
http://sourceforge.net/projects/linux-iscsi.  Just download the source and
grep for TCP_NODELAY.

Anthony J. Battersby
Cybernetics