1

Tsukuba WAN& the community

EGUCHI Hisashi (eguchi@maffin.ad.jp)

Tsukuba WAN Steering Committee

APAN 2003 Conference in Fukuoka Meetings and Workshops

NOC Meeting January 23, 2002

2

Tsukuba WAN is the network for research collaboration andcooperation among industry, university and government. It links with many institutes in Tsukuba area by ultrahigh-speed access ring network. As a result, they are able to use almost recourse of research institutes in Tsukuba, such as supercomputer, large-scale database, and computer simulations software.

Construction of Network began in 2000, and operation started at May 2002. The Tsukuba WAN symposium was held at May 2, 2002 as Opening ceremony. At the symposium, some program occurred such as the presentation on the results of research, which was using Tsukuba WAN, and discussion about the concept.

http://tsukuba-wan.mexttci.go.jp/

What’s the Tsukuba WAN ?　　　　　

3

Outline

High Energy Accelerator Research Organization

High Energy Accelerator Research Organization

NTT Access Network Service Systems Laboratories

NTT Access Network Service Systems Laboratories

National Research Institute for Earth Science and Disaster Prevention

National Research Institute for Earth Science and Disaster Prevention

University of TsukubaUniversity of Tsukuba

光伝

送路

Tsukuba Science City map （ Optical ring length 50km ）

Tsukuba Gigabit LaboratoryTsukuba Gigabit Laboratory

Gbit-SWRouterSuperComputer

National Institute for Materials Science

National Institute for Materials Science

National Institute for Environmental Studies

National Institute for Environmental Studies

National Institute for Land and Infrastructure Management

National Institute for Land and Infrastructure Management

National Institute of Advanced Industrial Science and Technology

National Institute of Advanced Industrial Science and Technology

MAFFINMAFFIN

The ultrahigh-speed 10G (total bandwidth is 570Gbps) access ring network for research collabolation, by the connection with supercomputer of research institutes and universities in Tsukuba.

Adoption of advanced network technology, such as Wavelength Division Multiplexing by OADM (Optical Add-Drop Multiplexer)

Research, development and field verification about technology such as interface between supercomputer and network, and security of each large scale simulations

Tsukuba Center for Institutes

Tsukuba Center for Institutes

4

Organization 　　　　　

Advisory Board(1)up-to-date technology in the computer science field(2)concept as the advanced research and information network(3)the future

Executive CommitteeIt deliberates utilization and research themes of Tsukuba WAN

Participate organization - National Research Institute for Earth Science and Disaster Prevention (NIED) - National Institute of Advanced Industrial Science and Technology (AIST) - National Institute for Environmental Studies (NIES) - National Institute for Land and Infrastructure Management (NILIM) - National Institute for Materials Science (NIMS) - Ministry of Agriculture, Forestry and Fisheries Research Network (MAFFIN) - Japan Science and Technology Corporation (JST) - Telecommunications Advancement Organization of Japan (TAO)　 Tsukuba Gigabit Laboratory - Tsukuba Center for Institutes - NTT Access Network Service Systems Laboratories - High Energy Accelerator Research Organization (KEK) [plan] - University of Tsukuba [plan]

Participate organization - National Research Institute for Earth Science and Disaster Prevention (NIED) - National Institute of Advanced Industrial Science and Technology (AIST) - National Institute for Environmental Studies (NIES) - National Institute for Land and Infrastructure Management (NILIM) - National Institute for Materials Science (NIMS) - Ministry of Agriculture, Forestry and Fisheries Research Network (MAFFIN) - Japan Science and Technology Corporation (JST) - Telecommunications Advancement Organization of Japan (TAO)　 Tsukuba Gigabit Laboratory - Tsukuba Center for Institutes - NTT Access Network Service Systems Laboratories - High Energy Accelerator Research Organization (KEK) [plan] - University of Tsukuba [plan]

Steering Committee It steers the system operation of Tsukuba WAN.

5

Rank Installation SiteManufac

turerComputer

Linpack(Gflops)

Procs

53 High Energy Accelerator Research Organization Hitachi SR8000-F1/100 917.00 100

75 University of Tsukuba Fujitsu VPP5000/80 730.00 80

86 National Institute of Advanced Industrial Science and Technology NEC Magi Cluster PIII 933MHz 654.00 1040

90 Real World Computing Partnership Self-madeSCore IIIe/PIII 933MHz 618.30 1024

120 National Institute for Environmental Studies NEC SX-6/64M8 495.20 64

133 National Institute of Advanced Industrial Science and Technology Hitachi SR8000/64 449.00 64

167 University of Tsukuba Hitachi CP-PACS/2048 368.20 2048

178 National Research Institute for Earth Science and Disaster Prevention Fujitsu VPP5000/38 351.10 38

212 National Research Institute for Earth Science and Disaster Prevention SGI ORIGIN 3000 500MHz 306.30 384

250 Meteorological Research Institute Hitachi SR8000/36 255.00 36

300 National Institute for Materials Science NEC SX-5/32H2 241.40 32

8 places and 11 systems of supercomputer in Tsukuba are rank in. TOP500 of the world.

8 places and 11 systems of supercomputer in Tsukuba are rank in. TOP500 of the world.

If they connected each other by Tsukuba WAN, it would become global area of technology and an industrial simulation field.

If they connected each other by Tsukuba WAN, it would become global area of technology and an industrial simulation field.

Tsukuba research organization in SuperComputer TOP500

TOP500 List for November 2002http://www.top500.org/list/2002/11/

6

Optical Fiber Ring 　

To each research organization

Tsukuba Center for Institutes

National Institute for Materials Science

National Institute of Advanced Industrial Science and Technology

Ministry of Agriculture, Forestry and Fisheries Research Network (MAFFIN)

National Institute for Environmental Studies

Tsukuba-NOCTAO Tsukuba Gigabit Laboratory

NTT Access Network Service Systems Laboratories

National Institute for Land and Infrastructure Management

National Research Institute for Earth Science and Disaster Prevention

NAS

OADM ADMSC

OADM ADM

SC

OADM ADM

SC

OADM ADM

SC

OADM ADM

SC

OADM ADMSC

ADMSC

ADMADM

APANIMnet

SINET

SC 　　　　 ： Super- Ｃ omputer

NAS 　　　 ： Network 　 Attached 　 Strage

OADM 　　： Optical Add-Drop Multiplexer

ADM 　　　： Add-Drop Multiplexer

POS OC-12

POS OC-192

Optical Fiber Ring

ATM/SONET OC-3, 12

JGN

7

Transmitter Receiver

DCM

DCM

O／Ｅ

Ｅ／Ｏ

SMF

DCM ： Dispersion 　 Compensating 　 Module

The method of transmitting 1.55um signals on SMF

When 1.55um signals transmits on SMF (Single Mode Fiber, 1.3um zero-dispersion optical fiber), it is necessary to do dispersion compensation, because of chromatic dispersion degrades optical signals

Op

tical

Am

plifie

r

Op

tical

Am

plifie

r

8

Variety of WDM systems

type features Wavelength allocation

1300 1400 1500 1600

DWDM

(Dense WDM)

CWDM

(Coarse WDM)

・ accommodate dozens of channels

in 1.5~1.6um

・ wavelength grid ： ~0.8nm (ITU-T G.694.1)

・ Precise wavelength control is required

・ accommodate 4~8 channels in 1.3~1.6um

・ wavelength grid ： 20nm (ITU-T G.694.2)

・ precise wavelength control is unnecessary

・ use of less costly optical components 　 is

capable

・・・・

1260 ～ 1360nm1360 ～ 1460nm1460 ～ 1530nm 1565 ～ 1625nm

O-band E-band S-band C-band L-band U-band

1530 ～ 1565nm 1625 ～ 1675nm

name

band

1300 1400 1500 1600

nm

nm

32, 64 and beyond

9

Wavelength arrangement on Tsukuba WAN

194.0 193.0 192.0199.0

f(THz)λ(nm)

1545 1550 1555 15601510

Reference frequency

191.0

1565 1570

190.0

1575 1580

189.0

1585 1590

188.0

1595 1600

187.0

1605 16 １0

195.0

15401535 1630

185.0

10Gbps-ADM Main signal (40λ)

L-bandC-band

Control Main signal (16λ) Control

10

DMUX

WDM signal （ λ1, λ2, λn ）…

TransponderSignal 1

Signal 2

Signal n

・・・・

λ1

λ2

λn

λ1

λ2

λn

MUX

Optical fiber

Outline of the WDM system

・・・・

OpticalAmplifier

OpticalAmplifier

DMUX

・・・・

λ1

λ2

λn

λ1

λ2

λn

MUX

・・・・

OpticalAmplifier

OpticalAmplifierOADM OADM

λ ｉ λ ｉ λ ｊ λ ｊ

WDM SystemWDM System

OADM SystemOADM System

Transponder

Transponder Transponder

Transponder

Transponder

Transponder

Transponder

Transponder

Transponder

Transponder

Transponder

Signal 1

Signal 2

Signal n

11

OADM system specifications

Item SpecificationsArchitecture 2 fiber UPSR ring

Nodes in ring (max.) 16 node

Ring loss budget (8 node) 88dB and more

WDM-IF

No. of wavelength 56 （ L-band ： 40 、 C-band ： 16 ）Types of Optical fiber SMF ， DSF

Line loss budget 11dB and more

Client

-IF 　Type STM-64 （ OC-192 ）， STM-16/4/1 （ OC-48/12/

3 ），Gigabit Ethernet ， Fibre Channel （ 1Gbit/s ， 2Gbit/s ）

No. of Add/Drop channel

Large node ： 40ch 　，　 Small node ： 6ch

Rack Dimensions 2100mm(H)×600mm(W)×600mm(D)

Power input DC-48V

Power Consumption (max) Large node ： 4700W 　， Small node ： 1020W

OADM: Optical Add Drop Multiplexer

12

AMP （ EDFA ）

ﾄﾗﾝｽﾎﾟﾝﾀﾞ部へ

W→E

Add/Drop filter

Add/Drop filter

Add/Drop filter

Add/Drop filter

AMP （ EDFA ）

Add/Drop filter

Add/Drop filter

Add/Drop filter

Add/Drop filter

E→W

AMP （ EDFA ）

AMP （ EDFA ）

Transponder

OADM node configurations

Small node

13

CWCCW

failure

Working channel

Protection channel

Before a failureBefore a failure After a failureAfter a failure

Large node

Small nodeA selector chooses a protection channel automatically.

A selector chooses a protection channel automatically.

To/From externalterminals

Optical Channel Protection (UPSR)

UPSR:Uni-directional Pass Switched RIng

To/From external terminals

Selector

Selector

14

`Weekly‘ Graph (30 Minute Average)Max  In:546.2 Mb/s (54.6%) Average  In:26.8 Mb/s (2.7%) Current  In:13.4 kb/s (0.0%) Max  Out:505.7 Mb/s (50.6%) Average  Out:17.1 Mb/s (1.7%) Current  Out:22.9 kb/s (0.0%)

Tsukuba WAN & APAN

NSPIXP2IMnet

APAN

KDDI-Ootemachi/Tokyo

GbEFEOC-3

OC-192 　 POS

Tsukuba-WAN

JuniperM160

JuniperM160

JuniperM160

JuniperM160

M5

60km

15

1 Joint research of ultra-high-speed access network NTT Access Network Service Systems Laboratories, etc y.mochiduki@east.ntt.co.jp http://www.ntt-east.co.jp/

2 Remote access of supercomputers through Tsukuba WAN and network extension NTT Access Network Service Systems Laboratories, etc tashiro@ansl.ntt.co.jp http://www.ansl.ntt.co.jp/

3 Network establishment of disaster condition and bulletin by remote sensing technology which mainly uses VEXCEL for supercomputer system (V

PP5000,ORIGIN3800) National Research Institute for Earth Science and Disaster Prevention (NIED) , etc ksato@bosai.go.jp http://www.bosai.go.jp/

4 Remote access of supercomputer across Tsukuba WAN and JGN. National Institute for Environmental Studies (NIES) ,etc sabe@nies.go.jp http://www.nies.go.jp/

5 Data Grid National Institute of Advanced Industrial Science and Technology (AIST) , etc s.sekiguchi@aist.go.jp http://www.aist.go.jp/aist_e/ressearch_units/research_center/grid/grid_main.html

6 Establishment of computing environment with the connection with heterogeneous computer by global network

National Institute of Advanced Industrial Science and Technology (AIST) , etc s.sekiguchi@aist.go.jp http://www.aist.go.jp/aist_e/ressearch_units/research_center/grid/grid_main.html

7 Establishment of computing environments (GridLib) National Institute of Advanced Industrial Science and Technology (AIST), s.sekiguchi@aist.go.jp http://www.aist.go.jp/aist_e/ressearch_units/research_center/grid/grid_main.html

8 Distance portal access of ITBL National Research Institute for Earth Science and Disaster Prevention (NIED), etc ksato@bosai.go.jp http://www.bosai.go.jp/

Research collaboration on Tsukuba WANContact address: suzuki5@mexttci.go.jp

16

平成１４年１２月５日独立行政法人 防災科学技術研究所

Network establishment of disaster condition and bulletin by remote sensing technology which mainly uses VEXCEL for supercomputer system (VPP5000,ORIGIN3800)

National Research Institute for Earth Science and Disaster Prevention (NIED)

17

NationalBackbone

NationalBackbone

Super SINET

Super SINET

Data Grid

SCinetNOC

3com 4924

10 Gbps

By courtesy ofForce10

SC2002, Baltimore

Indiana Univ.Cluster

Indiana Univ.Cluster

SDSCCluster

SDSCCluster

NOC

NOC

TokyoNOC

OC-12 x2

Japan

APAN/TransPAC

KEKCluster

KEKCluster

TitechCluster

TitechCluster

AISTCluster

AISTCluster

ICEPPCluster

ICEPPCluster

AISTM160

Tsukuba-U

MAFFINM160

GbE

10 GbE

GbE

NOC

GbE

NOC

AISTBooth

AISTBooth

Abiline

National Institute of Advanced Industrial Science and Technology (AIST)

Tsukuba WAN

Abiline