Data Reservoir project
The University of Tokyo
Pittsburgh, PA USA, November 16, 2004 - Researchers from the University of Tokyo and the Japanese WIDE Project, together with engineers in Japan, Canada, the United States, the Netherlands and Switzerland completed the world's longest 10 Gigabit per second circuit ever recorded for the transmission of internet data. The high bandwidth link connected geographically dispersed servers from the University of Tokyo's Data Reservoir project stretching from the Supercomputing 2004 research exhibition in Pittsburgh Pennsylvania to the CERN research center in Geneva, Switzerland through Tokyo. The length of this fiber optic path is approximately 31,248 km, spanning 17 time zones. The link was used to perform high-speed TCP data transfers that will lead to unprecedented breakthroughs in collaborative physics and engineering experiments between dozens of research institutions worldwide without distance limitations.
In the experiment, 7.21 Gigabits-per-second TCP payload bandwidth was sustained on a single stream with standard 1500-Byte Ethernet packets between two servers connected by 31,248km network. This international cooperative project pushes the boundaries of global research and education and lays a foundation for a new array of international research opportunities.
Using 10 Gigabit networking technology that combines OC-192 Packet over SONET technology and 10 Gigabit Wide Area Networking technology, a local area network connecting computers at the Univerisity of Tokyo's SC2004 Exhibition booth in Pittsburg, PA was extended to include computers at CERN in Switzerland, connecting through SCinet, Abilene, JGN2 and APAN, Tokyo. The network from APAN to T-LEX was provided by the WIDE project. From T-LEX, the circuit was passed to Seattle using a wavelength donated by Tyco Telecommunications through the IEEAF, and cross-connected through facilities provided by Pacific Northwest Gigapop in Seattle. From Seattle the circuit was then carried across a dedicated lambda on the CA*net 4 network to the Chicago StarLight. At StarLight, the interconnect to SURFnet's Chicago-Amsterdam lambda was made, taking the connection to NetherLight in Amsterdam. Finally, between NetherLight and CERN, SURFnet's Amsterdam-Geneva lambda was used.
The data transfer is achieved between a pair of data-sharing Opteron systems from the Data Reservoir project, one server placed at SC2004 exhibition booth ofthe University of Tokyo and another at CERN, each equipped with a Chelsio T110 10 Gigabit Ethernet adapter supporting TCP/IP offload. Transfer rate of 7.21 Gbps was sustained for over 15 minutes using a single TCP stream and standard 1500-byte Ethernet frames over the 31,248 kilometer link. The combined bandwidth times distance value is a new world record at 225,298 terabit meters per second and is 80% greater than the previous Internet2 Land Speed Record of 124,935 terabit meters per second. At this transfer rate and distance, a full-length DVD can be transferred anywhere on the earth in under 5 seconds.
The Data Reservoir system also achieved a 1.6 Gbps disk-to-disk data transfer with a single quad-Opteron server with a Chelsio T110 TCP offload engine at each end of the connection. This performance figure shows that 200 Mbytes per second single box disk servers are readily available for wide range of researchers.
The demonstrations were made possible through the support of the following manufacturers, who have generously contributed their equipment and knowledge:
Foundry Networks, Nortel Networks, Cisco Systems, Bussan Networks, NTT communications and Net One Systems.
- Special Coordination Fund for Promoting Science and Technology, Ministry of Education, Culture, Sports, Science and Technology, Japan
- Foundry Networks
- Juniper Networks
- NTT Communications
- NetOne Systems
University of Tokyo, Data Reservoir /GRAPE-DR Project is a research project funded by the Special Coordination Fund for Promoting Science and Technology, MEXT, Japan. The goal of the project is to establish a global data-sharing system for scientific data and to construct a very high-speed computing engine for simulation in astronomy, physics and bio-science. GRAPE-DR project will construct 2PFLOPS computing engine and global research infrastructure that utilize multi-10Gbps networks in 2008. This experiment is performed by cooperation of the University of Tokyo and Fujitsu Computer Technologies, LTD. For more information, visit:
Contact: Kei Hiraki <firstname.lastname@example.org>
WIDE, a research consortium working on practical research and development of Internet-related technologies, was launched in 1988. The Project has made a significant contribution to development of the Internet by collaborating with many other bodies -- including 133 companies and 11 universities to carry out research in a wide range of fields, and by operating M.ROOT-SERVERS.NET, one of the DNS root servers, since 1997. WIDE Project also operates T-LEX (http://www.t-lex.net/) as an effort of stewardship for the IEEAF Pacific link in Tokyo.
Tel: +81-466-49-3618 (c/o KEIO Research Institute at SFC)
APAN (Asia-Pacific Advanced Network ) is a non-profit international consortium established on 3 June 1997. APAN is designed to be a high-performance network for research and development on advanced next generation applications and services. APAN provides an advanced networking environment for the research and education community in the Asia-Pacific region, and promotes global collaboration.
Its objectives are:
(a) to coordinate and promote network technology developments and advances in network-based applications and services;
(b) to coordinate the development of an advanced networking environment for research and education communities in the Asia-Pacific region; and
(c) to encourage and promote global cooperation to help achieve the above.
JGN 2 is an open testbed network environment for research and development, which was previously operated by JGN (Japan Gigabit Network : Gigabit network for R&D) from April 1999 to March 2004, and expanded by the National Institute of Information and Communications Technology (hereinafter NICT) as a new ultra-high-speed testbed networks for R&D collaboration between industry, academia, government with the aim of promoting a broad spectrum of research and development projects, ranging from fundamental core research and development to advanced experimental testing, in areas including the advancement of network-related technologies for the next generation and diverse range of network application technologies.
CANARIE is Canada's advanced Internet organization, a not-for-profit corporation that facilitates the development and use of next-generation research networks and the applications and services that run on them. By promoting collaboration among key sectors and by partnering with similar initiatives around the world, CANARIE stimulates innovation and growth and helps to deliver social, cultural, and economic benefits to all Canadians. CANARIE positions Canada as the global leader in advanced networking, and is supported by its members, project partners, and the Government of Canada. CANARIE developed and operates CA*net 4, Canada's national research and education network. For more information, visit:
CERN is the European Laboratory for Particle Physics, one of the world's most prestigious centers for fundamental research. The laboratory is currently building the Large Hadron Collider. The most ambitious scientific undertaking the world has yet seen, the LHC will collide tiny fragments of matter head on to unravel the fundamental laws of nature. It is due to switch on in 2007 and will be used to answer some of the most fundamental questions of science by some 7,000 scientists from universities and laboratories all around the world. For more information, visit:
Pacific Northwest Gigapop is the Northwest's Next Generation Internet, applications cooperative, testbed, point of presence; home to the Pacific Wave international peering exchange; and joint steward with WIDE of the IEEAF trans-Pacific link. PNWGP and Pacific Wave connect together high-performance international and federal research networks with universities, research organizations, and leading edge R&D and new media enterprises throughout Washington, Alaska, Idaho, Montana, Oregon, and the Pacific Rim. For more information, visit:
SURFnet operates and innovates the national research network, to which over 150 institutions in higher education and research in the Netherlands are connected. The organization is among the leading research network operators in the world. SURFnet is responsible for the realization of GigaPort Next Generation Network, a project of the Dutch government, trade and industry, educational institutions and research institutes to strengthen the national knowledge infrastructure. Research on optical and IP networking and grids are a prominent part of the project. For more information, visit:
The Internet Educational Equal Access Foundation (IEEAF) is a non-profit organization whose mission is to obtain donations of telecommunications capacity and equipment and make them available for use by the global research and education community. The IEEAF TransPacific Link is the second 10 Gbps transoceanic link provided by IEEAF through a five year IRU donated by Tyco Telecom; the first, the IEEAF TransAtlantic Link, connects New York and Groningen, The Netherlands, and has been operational since 2002. IEEAF donations currently span 17 time zones. For more information, visit:
Chelsio Communications is the established leader in 10-Gigabit Ethernet server adapters and protocol acceleration technology. Chelsio's programmable T110 10GbE Protocol Engine is the only 10Gbps Ethernet adapter available today providing full TCP/IP offload and iSCSI acceleration. The T110 has been independently verified as the highest throughput, lowest latency and most CPU efficient Ethernet adapter in the industry - all with standard 1500-byte Ethernet frames. The T110 dramatically improves application performance by offloading processor-intensive network and storage protocol stacks from overburdened processors, returning processing cycles to the application to enhance overall system performance. For detail information, visit: