10G test
原本用cat5e, 果真速度上不去,換cat6, iperf 就有10G
大致流程
先用一般i7的機器測試 最高數據為
20480000000 bytes (20 GB) copied, 38.8449 s, 527 MB/s
換到storage 主機 3u 那種…最高數據為
ncat -l 60000 > /dev/nul //這邊再等著收資料
dd if=/dev/zero bs=2048 count=10000000 | ncat 192.168.5.3 60000 //這邊用dd+ncat送資料
20480000000 bytes (20 GB) copied, 25.3085 s, 809 MB/s
測試的過程用過iftop, iperf, ncat, cat, dd等工具傳送與觀察流量
細節紀錄:
第一次用 Lenovo M80 7521 的機器測試
CPU i7
Memory 4G
PCI Express 16X with Intel 10G Gigabits dual port(但只有用到一個port)
Operation System:
Linux Ubuntu 11.10
ssh server
no graphic card, no X11, text mode
3.0.0-12 generic amd64
thomas@ubuntu2:~$ cat /etc/issue
Ubuntu 11.10 \n \l
thomas@ubuntu2:~$ sudo lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 11.10
Release: 11.10
Codename: oneiric
thomas@ubuntu2:~$ uname -a
Linux ubuntu2 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
MTU:1500
lspci

ifconfig -a

[ 184.646085] ixgbe 0000:01:00.0: eth1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 184.646774] ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[ 184.953305] ixgbe 0000:01:00.0: eth1: NIC Link is Down
[ 185.100439] ixgbe 0000:01:00.0: eth1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
Machine A 192.168.5.1
Machine B 192.168.5.3
10G switch
test ping:
thomas@ubuntu:~$ ping 192.168.5.3
PING 192.168.5.3 (192.168.5.3) 56(84) bytes of data.
64 bytes from 192.168.5.3: icmp_req=1 ttl=64 time=0.240 ms
64 bytes from 192.168.5.3: icmp_req=2 ttl=64 time=0.256 ms
64 bytes from 192.168.5.3: icmp_req=3 ttl=64 time=0.252 ms
64 bytes from 192.168.5.3: icmp_req=4 ttl=64 time=0.271 ms
64 bytes from 192.168.5.3: icmp_req=5 ttl=64 time=0.269 ms
64 bytes from 192.168.5.3: icmp_req=6 ttl=64 time=0.275 ms
64 bytes from 192.168.5.3: icmp_req=7 ttl=64 time=0.259 ms
64 bytes from 192.168.5.3: icmp_req=8 ttl=64 time=0.276 ms
64 bytes from 192.168.5.3: icmp_req=9 ttl=64 time=0.279 ms
64 bytes from 192.168.5.3: icmp_req=10 ttl=64 time=0.276 ms
^C
— 192.168.5.3 ping statistics —
10 packets transmitted, 10 received, 0% packet loss, time 8997ms
rtt min/avg/max/mdev = 0.240/0.265/0.279/0.017 ms
test ncat tcp:
A. [TCP protocol]
==============================================
(1) recive firstly at B
ncat -l 60000 > /dev/null
(2) then send at A
cat /dev/zero | ncat 192.168.5.3 60000
==============================================
Summary Result:
send rate 4456937 kbits
receive rate 2844327 kbits
test ncat udp:
B. [UDP protocol]
=======================================
(1) recive firstly
ncat -u -l 60000 > /dev/null
(2) then send
cat /dev/zero | ncat -u [IP_Number] 60000
=======================================
Summary Result:
send rate 2344620 kbits
receive rate 2054655 kbits
test iperf:
thomas@ubuntu:~$ iperf -s
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
[ 4] local 192.168.5.1 port 5001 connected with 192.168.5.3 port 51698
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 4.97 GBytes 4.26 Gbits/sec
thomas@ubuntu2:~$ iperf -c 192.168.5.1
————————————————————
Client connecting to 192.168.5.1, TCP port 5001
TCP window size: 16.0 KByte (default)
————————————————————
[ 3] local 192.168.5.3 port 51698 connected with 192.168.5.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 4.97 GBytes 4.27 Gbits/sec
test chang cable 5e -> 6
換Cat6的線!
thomas@ubuntu2:~$ sudo iperf -c 192.168.5.1
————————————————————
Client connecting to 192.168.5.1, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.3 port 51700 connected with 192.168.5.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 10.9 GBytes 9.34 Gbits/sec
test netcat TCP:
cat /dev/zero | ncat –send-only 192.168.5.3 60000
send:
TX: cumm: 82.4GB peak: 3.86Gb rates: 2.97Gb 3.45Gb 2.80Gb
RX: 112MB 4.80Mb 4.53Mb 4.60Mb 4.07Mb
total: 82.5GB 3.87Gb 2.97Gb 3.46Gb 2.81Gb
ncat –recv-only -l 60000 > /dev/nul
receive:
TX: cumm: 105MB peak: 4.79Mb rates: 4.46Mb 4.56Mb 4.02Mb
RX: 76.8GB 3.85Gb 3.08Gb 3.42Gb 2.77Gb
Total: 76.9GB 3.85Gb 3.08Gb 3.42Gb 2.77Gb
test netcat UDP:
cat /dev/zero | ncat -u –send-only 192.168.5.3 60000
TX: cumm: 34.7GB peak: 2.67Gb rates: 2.67Gb 2.36Gb 2.30Gb
RX: 1.11KB 1.11Kb 0b 0b 57b
thomas@ubuntu:~$ 34.7GB 2.67Gb 2.67Gb 2.36Gb 2.30Gb
ncat -u –recv-only -l 60000 > /dev/nul
TX: cumm: 0B peak: 0b rates: 0b 0b 0b
RX: 36.4GB 3.13Gb 3.13Gb 2.54Gb 2.34Gb
thomas@ubuntu2:~$ 36.4GB 3.13Gb 3.13Gb 2.54Gb 2.34Gb
test dd| netcat :
ncat -l 60000 > /dev/nul
dd if=/dev/zero bs=512 count=1024000000 | ncat 192.168.5.3 60000
iftop result
test using dd:
128-dd.log
160000000+0 records in
160000000+0 records out
20480000000 bytes (20 GB) copied, 57.4505 s, 356 MB/s
256-dd.log
80000000+0 records in
80000000+0 records out
20480000000 bytes (20 GB) copied, 38.8449 s, 527 MB/s
512-dd.log
40000000+0 records in
40000000+0 records out
20480000000 bytes (20 GB) copied, 42.393 s, 483 MB/s
1024-dd.log
20000000+0 records in
20000000+0 records out
20480000000 bytes (20 GB) copied, 41.1297 s, 498 MB/s
2048-dd.log
0000000+0 records in
10000000+0 records out
20480000000 bytes (20 GB) copied, 40.6217 s, 504 MB/s
4096-dd.log
5000000+0 records in
5000000+0 records out
20480000000 bytes (20 GB) copied, 41.2186 s, 497 MB/s
8192-dd.log
2500000+0 records in
2500000+0 records out
20480000000 bytes (20 GB) copied, 41.0573 s, 499 MB/s
換到大的storage主機
memory 24G
raid 1+0 (20T->14T)
zero to null
dd if=/dev/zero bs=256 count=80000000
80000000+0 records in
80000000+0 records out
20480000000 bytes (20 GB) copied, 41.6271 s, 492 MB/s
dd if=/dev/zero bs=512 count=40000000
40000000+0 records in
40000000+0 records out
20480000000 bytes (20 GB) copied, 28.4953 s, 719 MB/s
dd if=/dev/zero bs=1024 count=20000000
20000000+0 records in
20000000+0 records out
20480000000 bytes (20 GB) copied, 26.1946 s, 782 MB/s
dd if=/dev/zero bs=2048 count=10000000
10000000+0 records in
10000000+0 records out
20480000000 bytes (20 GB) copied, 25.3085 s, 809 MB/s
dd if=/dev/zero bs=4096 count=5000000
5000000+0 records in
5000000+0 records out
20480000000 bytes (20 GB) copied, 25.4756 s, 804 MB/s
iftop peek =~ 6.3Gb
raw to null
dd if=raw bs=256
80000000+0 records in
80000000+0 records out
20480000000 bytes (20 GB) copied, 59.171 s, 346 MB/s
dd if=raw bs=512
40000000+0 records in
40000000+0 records out
20480000000 bytes (20 GB) copied, 29.9193 s, 685 MB/s
dd if=raw bs=1024
20000000+0 records in
20000000+0 records out
20480000000 bytes (20 GB) copied, 27.893 s, 734 MB/s
dd if=raw bs=2048
10000000+0 records in
10000000+0 records out
20480000000 bytes (20 GB) copied, 26.8301 s, 763 MB/s
dd if=raw bs=4096
5000000+0 records in
5000000+0 records out
20480000000 bytes (20 GB) copied, 27.0086 s, 758 MB/s
peek 5.8Gb
raw to raw
dd if=raw bs=256
80000000+0 records in
80000000+0 records out
20480000000 bytes (20 GB) copied, 54.2189 s, 378 MB/s
dd if=raw bs=512
40000000+0 records in
40000000+0 records out
20480000000 bytes (20 GB) copied, 48.417 s, 423 MB/s
dd if=raw bs=1024
20000000+0 records in
20000000+0 records out
20480000000 bytes (20 GB) copied, 54.0252 s, 379 MB/s
dd if=raw bs=2048
dd if=raw bs=2048
10000000+0 records in
10000000+0 records out
20480000000 bytes (20 GB) copied, 46.5879 s, 440 MB/s
Ncat: Connection refused.
dd if=raw bs=4096
5000000+0 records in
5000000+0 records out
20480000000 bytes (20 GB) copied, 46.3077 s, 442 MB/s
===================================================
some tips:
test note about lenovo M80 7521
there is only one pci express 16x port, remove original graphic card and install intel 10g net card.
boot get 5 bee warrning, skip it, it will keep booting without graphic card