Internet2 Land Speed Record in single and multiple TCP stream

24th, Dec.2004

33,979 km connection from Tokyo through Chicago, Amsterdam, NewYork to Tokyo Latency 499ms RTT

Network Configuration

treceroute from Opteron1 to Opetron3 (Tokyo - IEEAF - CAnet*4 - Surfnet - Abilene -JGN2)

[root@opteron1 root]# traceroute -q 1 opteron3c
traceroute to opteron3c (192.31.124.5), 30 hops max, 46 byte packets
1 vlan911.sl-e1200.sl.startap.net (145.146.101.1) 144.016 ms
2 vlan10.AR5.Amsterdam1.surf.net (145.146.100.193) 243.361 ms
3 PO6-0.CR1.Amsterdam2.surf.net (145.145.164.1) 260.205 ms
4 PO15-0.CR1.Amsterdam1.surf.net (145.145.160.17) 243.491 ms
5 nycmng-oc192-surfnet.abilene.ucaid.edu (198.32.8.104) 317.529 ms
6 chinng-nycmng.abilene.ucaid.edu (198.32.8.82) 337.563 ms
7 transpac-chinng.jp.apan.net (192.203.116.1) 357.193 ms
8 tpr4-transpac.jp.apan.net (203.181.248.218) 576.975 ms
9 opteron3c (192.31.124.5) 498.959 ms

treceroute from Opteron3 to Opetron1 (Tokyo - JGN2 - Alibene - Surfnet - CAnet*4 - IEEAF)

[root@opteron3 root]# traceroute -q 1 opteron1c
traceroute to opteron1c (145.146.101.4), 30 hops max, 46 byte packets
 1  vlan914.tpr4.jp.apan.net (192.31.124.1)  68.873 ms
 2  transpac-tpr4.jp.apan.net (203.181.248.217)  182.418 ms
 3  chinng-transpac.abilene.ucaid.edu (192.203.116.2)  161.681 ms
 4  nycmng-chinng.abilene.ucaid.edu (198.32.8.83)  181.687 ms
 5  po5-0.br1.Amsterdam1.surf.net (198.32.8.105)  256.001 ms
 6  PO0-0.AR5.Amsterdam1.surf.net (145.145.162.2)  256.115 ms
 7  vlan10.sl-e1200.sl.startap.net (145.146.100.194)  358.904 ms
 8  opteron1c (145.146.101.4)  498.940 ms

Internet2 TCP Single stream Land Speed Record measurement

System

• Server configuration
• CPU: Dual AMD Opteron 248, 2.2GHz
• Mother Board: RioWorks HDAMA rev. E
• Memory: 1G bytes, Corsair Twinx CMX512RE-3200LLPT x 2, PC3200, CL=2
• OS: Linux kernel 2.6.6
• Disk Seagate IDE 80GB, 7200r.p.m. (disk speed not essential for performance)
• Network interface card
• Chelsio T110 (10GBASE-SR), TCP offload engine (TOE),
• PCI-X/133 MHz I/O bus connection
• TOE function ON
• Technical points
• Traditional tuning and optimization methods congestion window size, buffer size, size of queue etc. Ethernet frame size (standard frame, jumbo frame)
• Tuning and optimization by Inter-layer coordination Fine-grained pacing by offload engine optimization at slow start phase Utilization of flow control
• Target Performance of single stream TCP on 30,000 km circuit
• 7.21 Gbps (TCP payload), standard frame -> 216,300Terabit meter / second (Internet2 LSR approved Single/Multi TCP Stream record)
• Internet2 Land Speed Record Page

Results

Traffic Graphs

• Starlight Force 10 from CAnet*4
  
• Amsterdam Force 10 ge 4/0 from Starlight
  
• Amsterdam Force10 ge 6/0 to Abilene NYCM
  
• Abilene NYCM IEEAF(OC-192) from Surfnet
  
• Abilene (NYCM) NYCM to CHIN
  
• Abilene (CHIN) NYCM to CHIN
  
• Starlight Force 10 to JGN2
  
• TransPAC/JGN2 (via Starlight F10)
  
• APAN JGN2 OC-192 (containing some smnp trouble)
  
• APAN T-LEX 10GbE (containing some smnp trouble)
  

  dump of trial

  These traces are submitted traial non-drop version(first 100 packets)

• http://data-reservoir.adm.s.u-tokyo.ac.jp/LSR/Run2-c50-XX.txt

Name of Institutions and persons

• The University of Tokyo
• Kei Hiraki
• Mary Inaba
• Makoto Nakamura
• Junji Tamatsukuri
• Nao Aoshima
• Ryo Nishimura
• WIDE project
• Akira Kato
• Seiichi Yamamoto
• Chelsio communications
• Felix Marti
• Wael Noureddine
• Fujitsu Computer Technologies LTD
• Ryutaro Kurusu
• Masakazu Sakamoto
• Yuki Furukawa
• Yukichi Ikuta
• NTT Communications
• Katsuyuki Hasebe
• Miyuki Neo
• Yu Inada
• Mitsuo Murakami
• APAN
• Kazunori Konishi
• Yoshitaka Hattori
• Jin Tanaka
• Satoshi Matsui
• JGN2
• Katsushi Kobayashi
• IEEAF
• Tyco Telecommunications
• Northwest Giga Pop
• Jan Eveleth
• CANARIE
• Rene Hatem
• Damir Pobric
• Thomas Tam
• Herve Guy
• Carleton University
• Wade Hong
• StarLight
• Linda Winkler
• SURFnet
• Kees Neggers
• Erik-Jan Bos
• Dennis Paus
• Roeland Nuijts
• SARA
• Pieter de Boer
• University of Amsterdam
• Cees de Laat
• Paola Grosso
• Freek Dijkstra

Supported by

• Special Coordination Fund for Promoting Science and Technology, Ministr y of Education, Culture, Sports, Science and Technology, Japan
• Foundry Networks, Inc.
• CISCO systems, Inc.
• NetOne Systems Co., Ltd.
• ClearSight Networks
• TOYO Corporation