Internet2 Land Speed Record
in single and multiple IPv6 TCP stream

28th, Oct.2005

15,461 km connection from Tokyo through Seattle to Tokyo Latency 177.5ms RTT

Network Configuration

treceroute from Opteron6 to Opetron1 (Tokyo - IEEAF - Seattle - IEEAF - JGN2 - WIDE - Tokyo)

[root@opteron6 ~]# traceroute6 opteron1c-v6
traceroute to opteron1c-v6 (2001:200:0:1cf6::10) from 2001:200:0:1cf8::10, 
30 hops max, 24 byte packets
1  nezu-3602-v6 (2001:200:0:1cf8::1)  0.333 ms  0.252 ms  0.251 ms
2  t-lex-sea-901-v6 (2001:200:0:1cf7::1)  89.171 ms  88.983 ms  88.97 ms
3  opteron1c-v6 (2001:200:0:1cf6::10)  177.689 ms  177.491 ms  177.49 ms

treceroute from Opteron1 to Opetron6 (Tokyo - WIDE - JGN2 - IEEAF - Seattle - IEEAF - Tokyo)

[root@opteron1 root]# traceroute6 opteron6c-v6
traceroute to opteron6c-v6 (2001:200:0:1cf8::10) from 2001:200:0:1cf6::10, 
30 hops max, 24 byte packets
1  t-lex-sea-900-v6 (2001:200:0:1cf6::2)  88.723 ms  88.687 ms  88.68 ms
2  nezu-901-v6 (2001:200:0:1cf7::2)  177.365 ms  177.361 ms  177.365 ms
3  opteron6c-v6 (2001:200:0:1cf8::10)  177.499 ms  177.492 ms  177.492 ms

Internet2 TCP Single stream Land Speed Record measurement

System

• Server configuration
• CPU: Dual AMD Opteron 248, 2.2GHz
• Mother Board: RioWorks HDAMA rev. E
• Memory: 1G bytes, Corsair Twinx CMX512RE-3200LLPT x 2, PC3200, CL=2
• OS: Linux kernel 2.6.12
• Disk Seagate IDE 80GB, 7200r.p.m. (disk speed not essential for performance)
• Network interface card
• Chelsio T110 (10GBASE-SR), TCP offload engine (TOE),
• PCI-X/133 MHz I/O bus connection
• TOE function OFF

Technical points

• (1) IPv6 packets
• (2) 9130B Jumbo frame
• (3) Use of "inter-layer coordination optimization" in TCP[1](Adjusting output rate using Chelsio T110 feature)
• (4) single stream TCP
• (5) All the trans continental connections are WAN PHY based.
• (6) IPv6 communication is almost as fast as that of IPv4.

Target Performance of single stream TCP on 15,461 km circuit

• 5.94 Gbps (TCP payload), standard frame -> 91,838Terabit meter / second (Internet2 LSR approved Single/Multi TCP Stream record)

• Internet2 Land Speed Record Page

Results

Traffic Graphs

dump of trial

These traces are submitted traial non-drop version(first 100 packets)

text format

tcpdump format

reciever side log

sender side log

Name of Institutions and persons

• The University of Tokyo
• Kei Hiraki
• Mary Inaba
• Junji Tamatsukuri
• Yutaka Sugawara
• Katsushi Inagami
• Takeshi Yoshino
• Nao Aoshima
• WIDE project
• Akira Kato
• Seiichi Yamamoto
• Yuji Sekiya
• JGN2
• Asako Toyoda
• Kazuhiko Nakamura
• Katsushi Kobayashi
• Yoshitaka Hattori
• Jin Tanaka
• Yoichi Tsukioka
• Northwest Giga Pop
• Jan Eveleth
• Bill Mar
• Beth Garrison
• Chelsio Communications
• Felix Marti
• Wael Noureddine
• Fujitsu Computer Technologies LTD
• Ryutaro Kurusu
• Masakazu Sakamoto
• Yuki Furukawa
• Yukichi Ikuta
• NTT Communications
• Katsuyuki Hasebe
• Miyuki Neo
• Yu Inada
• Mitsuo Murakami
• IEEAF

Supported by

• Special Coordination Fund for Promoting Science and Technology, Ministr y of Education, Culture, Sports, Science and Technology, Japan
• Foundry Networks,
• Alaxala Netowrks,
• Hitachi,
• CISCO systems, Inc.
• TOYO Corporation

Data Reservoir / GRAPE-DR project, University of Tokyo