[SATLUG] 10GbE configuration and performance issues

[Gregory Alan Hildstrom]

Hello. I am working on an application that needs 200MB/s sustained iSCSI throughput for high speed
data acquisition.

Server:
IBM x226
2x3GHz Xeon
4GB RAM
Intel 10GbE CX4 in PCI-X 100MHz slot (6.4Gb/s max)
RHEL5

Client:
IBM x206
1x3GHz P4
1GB RAM
Intel 10GbE CX4 in PCI-X 66MHz slot (4.2Gb/s max)
Fedora Core 6

Network:
Dell 6224 switch with 2 10GbE modules

I ran netserver on the server and "netperf -H 192.168.2.151" on the client. I got 1550Mb/s, but I
was expecting something more like 3-4Gb/s. CPU load on the slower client machine never got above
25% and I do not know how to measure PCI bus bandwidth utilization.

I tried Myricom's and Intel's tuning suggestions in /etc/sysctl.conf, but the performance was
about the same with both sets of settings. Here are Myricom's tuning parameters:
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.core.netdev_max_backlog = 250000
net.ipv4.tcp_timestamps = 0

Any suggestions? Thanks a ton. -Greg

Gregory Alan Hildstrom
Secure Systems Engineer - Trusted Computer Solutions
ghildstrom at trustedcs.com
Software Engineer - Hildstrom Engineering
hildstrom at hildstrom.com
Mobile:(210)413-6082
Home/Fax:(210)599-0469
Office:(210)340-3151x117