toward production level operation of authentication system for high performance computing ...

Toward Production Level Operation of Authentication System for High

Performance Computing Infrastructure in Japan

Eisaku Sakane and Kento AidaNational Institute of Informatics

Eisaku Sakane and Kento Aida, National Institute of Informatics

Introduction High Performance Computing Infrastructure (HPCI)

national project promoted by Ministry of Education, Culture, Sports, Science and Technology (MEXT) in Japan

distributed computing infrastructure for high performance computing “K computer”, supercomputers and high performance storage

first production level infrastructure for high performance computing in Japan

Roadmap – Mar 2011 basic design

network, authentication, user management, shared storage, testbed for advanced software

Apr – Dec 2011 detailed design Jan – Oct 2012 test operation Nov 2012 – production level operation


This talk presents pilot operations of the authentication system for HPCI.

portal

CA system

shib. SP

shib. SP

shared storage

single sign-on

apply certificate

authentication

network infrastructure

computerresource

shib. IdP

shib. IdP

shib. IdP

HPCIacct.

HPCI ID registration

review proposals

user management

certificate repository

HPCI Overview (at Nov. 2012)

Eisaku Sakane and Kento Aida, National Institute of InformaticsMore resources will be connected after 2012.

AICS, U. Tokyo

NII

HPCI Secretariat（ organized in 2011 ）

acct. registration

helpdesk

computerresource

computerresourceAICS (K-computer)

Supercomputer Centers in 9 Universities

SINET4SINET4: Science Information NETwork 4


user user user user

IX（ Tokyo)

resource provider

IX（ Osaka)

AICS LAN

usercompt. resource

storage

university university

commercial network

non-commercial network

CA

portal

university

usercompt. resource

storage

resource provider

university

usercompt. resource

storage

usercompt. resource

storage

QoS

VPN

SINET4 (cont’d) connection to 700+ academic sites IX for commercial networks

134 （ 30Gbps ） in Tokyo 22 （ 11Gbps ） in Osaka


80Gbps backbone （ planned in 2011 ）

L3VPN, L2VPN/VPLS, QoS

AICS and Supercomputer Centers in Japanese Universities

Kyushu Univ. ：PC Cluster (55Tflops, 18.8TB)SR16000 L2 (25.3Tflops, 5.5TB)PC Cluster (18.4Tflops, 3TB)

Hokkaido Univ. ：SR11000/K1(5.4Tflops, 5TB)PC Cluster (0.5Tflops, 0.64TB)

Nagoya Univ. ：FX1(30.72Tflops, 24TB)HX600(25.6Tflops, 10TB)M9000(3.84Tflops, 3TB)

Osaka Univ. ：SX-9 (16Tflops, 10TB)SX-8R (5.3Tflops, 3.3TB)PCCluster (23.3Tflops, 2.9TB)

Kyoto Univ.T2K Open Supercomputer(61.2 Tflops, 13 TB)

Tohoku Univ. ：NEC SX-9(29.4Tflops, 18TB)NEC Express5800 (1.74Tflops, 3TB)

Univ. of Tsukuba ：T2K Open Supercomputer95.4Tflops, 20TB

Univ. of Tokyo ：T2K Open Supercomputer(140 Tflops, 31.25TB)

AICS, RIKEN ：K computer (10 Pfflops, 4PB)Available in 2012

A 1 Pflops machine without accelerator will be installed by the end of 2011

Tokyo Institute of Technology ：Tsubame 2 (2.4 Pflops, 100TB)

source: Y. Ishikawa, Univ. of Tokyo Eisaku Sakane and Kento Aida, National Institute of Informatics

Hokkaido University

Tohoku University

University of Tokyo

University of Tsukuba

Tokyo Institute of TechnologyNagoya University

Kyushu University

Osaka UniversityKyoto University

AICS, RIKEN

• 12 PB+ storage • 10 PB+ storage

HPCI WEST HUB HPCI EAST HUB

Gfarm2 is used as the global shared file system

Storage

source: Y. Ishikawa, Univ. of Tokyo Eisaku Sakane and Kento Aida, National Institute of Informatics

Authentication The goal is enabling single sign-on computer resources and

shared storage in HPCI. survey of existing software technologies and operation of

grid infrastructures account management

centralized or distributed?


user

portal

HPCI acct/password

• login to computers• access to shared storage

single sign-on

% gsi-ssh host.univ.ac.jp

(1) sign-on the portal with HPCI acct.

(2) ssh login to computers without password

Shibboleth + GSI Shibboleth for account management of HPCI

HPCI account = account to sign-on HPCI federation of HPCI accounts managed in distributed way using

Shibboleth A user has a HPCI account in one supercomputer center.

Grid Security Infrastructure (GSI) for single sign-on de facto in grid communities enabling single sign-on using PKI creating proxy certificate and delegation mapping “Distinguished Name (DN)” in a client certificate and a local

account in supercomputer centers grid-mapfile


"/C=JP/O=NII/OU=CGRD/CN=Kento Aida” aida

Pilot Operations 1st phase: Apr – Dec 2011

objective: for operation organizations to get used to operate GSI and Shibboleth systems

National Institute of Informatics operating CA system and Portal building an experimental CA system including a certificate repository

– UMS provided by Shibbolized NAREGI Middleware v1.1 building an authentication portal with a proxy certificate repository

– portal provided by Shibbolized NAREGI M/W Supercomputer centers

building Shibboleth IdP setting up a GSI-enabled ssh server and client as SP



Architecture

Certificate Management

System

CA System(Shib. SP)

Portal(Shib. SP)

Proxy Cert. Repository

Shib. DS

Shib. IdP

webbrowser

GSI-SSHclient

National Institute of Informatics

Supercomputer CentersSINET 4

apply certificate

sign-on HPCI

login to compt. resources

Account DB

GSI-SSH Server

Supercomputer Centers, AICS

storage

Cert. Repository

Screenshots


Result of 1st phase We confirmed the followings

Sign-on the authentication portal with Shibboleth federation mechanism

getting a end-user certificate via the authentication portal generation a proxy certificate and downloading it to end-user’s

terminal computer logging in 9 supercomputer centers by using GSI-enabled SSH

The system works as single sign-on system. Documents for HPCI users and administrators were revised

according to feedback from participating organizations Problem

port number (22/tcp) collision between SSH and GSI-enabled SSH Administrators are reluctant to stop sshd or replace with gsi-sshd

because of security policy of supercomputer center. We will unify the port number for gsi-sshd with another port number.


Pilot Operations (cont’d) 2nd phase: Jan 2012 –

objective: evaluation of the authentication system and feedback building a production level CA system

preparing dedicated machines, HSM performing key ceremony examinations on normal or abnormal operations replacing certificates in 1st phase with new certificates issued by new CA

building an authentication portal for HPCI collaboration with the HPCI secretariat

the role of the HPCI secretariat– proposal to use HPCI (including registration of HPCI ID)– notification of review– coordination among resource providers, …

HPCI-ID is important because it connects subject DN with local account. combination examination between NII (CA), supercomputer centers

(RPs) and HPCI secretariat


Connecting Subject DN with LN Flow until subject DN and local account name (LN) are

connected A HPCI-ID is assigned to an end-user. The HPCI secretariat notifies CA and RPs of the HPCI-ID. CA manage subject DN with HPCI-ID. RP manages local account name with HPCI-ID. RP inquires the information of CA, then generates grid-mapfile.


CA

HPCIsecretariat

RP

HPCI-ID

HPCI-ID

HPCI-ID

aida (LN)

"/C=JP/O=NII/OU=CGRD/CN=Kento Aida” aida/C=JP/O=NII/OU=CGRD/CN=Kento Aida

Conclusions This talk presents an evaluation experiment of the

authentication system for HPCI.

current status and future work network

SINET4 has started production level operation in 2011. authentication

entering on 2nd phase of evaluation experiment built a production level CA system in NII and evaluated its performance starting test operation of the production level system from Feb 2012 considering when we switch hash algorithm in digital signature to SHA-2

user management still preparing to start HPCI secretariat starting test operation as soon as possible
