workshop on hpc in india challenges of garuda : the national grid computing initiative of india...

Workshop on HPC in India

Challenges of Garuda : The National Grid Computing

Initiative of India

Subrata Chattopadhyay C-DAC Knowledge Park

Bangalore, India

ATIP 1st Workshop on HPC in India @ SC-09

S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 2

Outline Introduction – C-DAC Garuda overview PoC Phase Foundation phase Current status Challenges Q & A


C-DAC - Overview Established in 1989 as a Resource Centre in

System Software for High Performance Computing (HPC)

Activities cover HPC & Grid computing, Real Time & Dependable Computing, Heritage Computing, E-Governance, Education and Training

Houses C-DAC’s TeraScale Supercomputing Facility (CTSF) as a National HPC Facility

Several turn key projects in system software, real time systems and E-Governance for the domestic and foreign markets

100-150 Technical Members


GARUDA - Overview Proof of Concept phase of project completed in

March 2008 Precursor to the National Grid Computing

Initiative Test Bed for grid technology/concepts and

applications leading to the plan for the main grid initiative

Major Deliverables on PoC Technologies, Architectures, Standards &

Research Initiatives Nation-wide high-speed communication fabric Aggregation of Grid Resources Deployment of Select applications of National

Importance Grid Strategic User Group

High-speed Networking Component in collaboration with ERNET


Garuda MPLS Network


Architecture

GARUDA Resources

•Compute, Data, Storage,

•Scientific Instruments,

•Software,..

Access Methods

• Access Portal for SOA

• Problem Solving Environments

Management, Monitoring & AccountingParyaveekshanam

•Web MDS

•GARUDA Information Service

•GARUDA AccountingSecurity Framework

• IGCA Certificates

• MyProxy

• VOMS

Resource Mgmt & Scheduling

• GridWay Meta-scheduler

• Resource Reservation

• Torque, Load Leveler

• Globus 4.x (WS Components)


Grid Infrastructure – Resources

PARAM Padma and Linux Clusters have been setup at Pune, Hyderabad & Chennai

Grid Labs have been setup at three locations

Fourteen of the partner institutions contributing resources including satellite terminals ( aggregating to more than 1500 CPUs and over 60TB storage)


GARUDA Partners Motivation

To Collaborate on Research and Engineering of Technologies, Architectures, Standards and Applications in Grid Computing

To Contribute to the aggregation of resources in the Grid

Current Participation 36 research & academic

institutions in the 17 cities

ERNET-HQ in Delhi 8 centres of C-DAC Total of 45 institutions


Virtual User Community Astrophysics High Energy Physics & Astronomy Grid Technology Disaster Management Earth Science Bioinformatics (Genome) Computational Fluid Dynamics Network Technology


Research Initiatives Objective

Create the foundation for next generation grids by addressing long term research issues in the strategic areas of

• Knowledge and data management • Programming models • Architectures • Grid Management and Monitoring• Problem solving environments • Tools and grid services

Research Initiatives in Semantic Grids Integrated Development Environment Network Simulation Mobile Agents Grid File Systems Resource Brokers


Collaborations Space Application Centre, Ahmedabad

Disaster Management Grid Middleware for Satellite Grids

Indian Institute of Science, Bangalore Climate Modeling

Madras Institute of Technology, Chennai Grid Middleware for Semantic Grid Services

Indian Institute of Technology, Mumbai Computational Fluid Dynamics Package on PARAM

Padma University of Pune

Quantum Chemistry, Materials Modeling, Bioinformatics

INDMOL Molecular Package developed EU-India Grid

BARC, C-DAC, ERNET, Saha Institute, TIFR, Univ. Pune Consortium GARR, ICTP, INFN (Italy) Cambridge University (UK)


GARUDA – POC - Overview Proof of Concept phase of project completed in

March 2008 Precursor to the National Grid Computing

Initiative Test Bed for grid technology/concepts and applications

leading to the plan for the main grid initiative Major Deliverables on PoC

Technologies, Architectures, Standards & Research Initiatives

Nation-wide high-speed communication fabric Aggregation of Grid Resources Deployment of Select applications of National

Importance Grid Strategic User Group

High-speed Networking Component in collaboration with ERNET


Highlights of GARUDA PoC Building core technical competence in grid

computing – development and deployment of Grid middleware, tools and services

Deployment of Cyberinfrastructure High speed Networking Resource aggregation (resources from various

institutes/organizations) Prototype Demonstration of Applications of

National importance Disaster Management (DMSAR) Bioinformatics

Building the Grid community Understanding research challenges in Grid

computing


Garuda–Foundation Phase:2008-09

Test interoperability of Garuda with other international grids

Monitor and operate the high speed bandwidth Garuda network and grid resources

Upgrade Garuda resources Demonstrate SOA framework that may lead to

participation from Industry Research on security and grid technologies Establish certificate authority for national grid

operation Enable selected applications on grid

environment


Achievements - Foundation Phase Explored viability of commercial applications / services

in areas of CAE and Molecular Docking on SOA GARUDA

Replaced commercial grid scheduler with OpenSource ‘Gridway’ meta scheduler

Exploring OpenSource solution for Data Grid through SRM as an alternate to SRB from Nirvana

Established the First CA in India to address security issues of grid – IGCA

Improved processing time from 6.1hrs to 54 mins for processing one set of Radar data (9GB) for the DMSAR application using the new resources of GARUDA (272 cores)

Winglet applications for processing 6000 winglets taking nearly 30 days to complete on ordinary machines were able to complete in 3 hours by harnessing the large computational power offered by GARUDA.


Objective ofOperational Phase of Garuda

To provide stable, robust, and efficient grid environment with guaranteed quality of service for various applications


Major components Architecture and Framework Middleware & Associated tools:

Development, deployment and operational support

Migration to interoperation with international grids

Resource aggregation & coordination with resources initiative

Operational pilot of applications & coordination with new application initiatives


Garuda Services

2010 -

2011 -

2012 -

2010 -

2011 -

2012 -

2010 -

2011 -

2012 -

2010 -

2011 -

2012 -

Resource Aggregation & AugmentationParticipation – Intl Commn, Stds ForumStabilize & Enhance – NKN, middleware, tools

Next Gen ServicesKnowledge DisseminationDeploy Operational Applications

-

-

-

-

-

-

Grid Enablement SupportMigration to NKNMiddleware & ToolsArchitecture & Framework


International Visibility


Eu-Indiagrid Project Goals To support the interconnection and inter-operability

of the prominent European Grid infrastructure (EGEE) with the Indian Grid infrastructure (Garuda National Grid Initiative, DAE-Grid) for the benefit of eScience applications;

To identify and aggregate research, scientific and industrial communities which may benefit from the use of Grid technology resulting in an eScience Network Community

To promote the use of advanced Grid technologies within the created Network Community relying on pilot applications in Biology and High Energy Physics Material Science, Earth and Atmospheric Sciences and specific outreach and dissemination activities

To disseminate European EGEE Grid technology achievements in India and leverage on Indian Grid experiences and skills.


ParticipantsEUROPE Italian National Institute of Nuclear Physics, INFN (project

coordinator), Metaware SpA, Italian Academic and Research Network (GARR) Cambridge University

INTERNATIONAL Abdus Salam International Centre for Theoretical Physics (ICTP)

INDIA Indian Education and Research Network (ERNET), University of Pune, SAHA Institute of Nuclear Physics, Kolkata, Centre for Development of Advanced Computing (C-DAC), Bhabha Atomic Research Centre, Mumbai TATA Institute for Fundamental Research (Mumbai)

National Centre for Biological Sciences (Bangalore)


Challenges! Heterogeneous Resources Policy across different administrative

domains Diverse requirements and priorities Funding and sustainability Killer Apps - Interoperation and standards


Thank you!

workshop on hpc in india challenges of garuda : the national grid computing initiative of india...

Documents