PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011

Cindy ZhengFor

PRAGMA Grid/Cloud TeamAnd the PRAGMA communities

Pacific Rim Application and Grid Middleware Assemblyhttp://www.pragma-grid.nethttp://goc.pragma-grid.net

PRIME 2011 Tutorial

PRAGMA Grid/Cloud Access

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Grid and Cloud

• What are they?– Collection of resources for sharing

• Differences– Grid

• Resources glued by agreed and verifiable trust– Certification– GLOBUS

• Provide system/software environment for some applications and users

– Cloud• Users setup applications on systems• Cloud provide mechanisms to allow users run

their systems as virtual machine (VM) in shared resources

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Grid Trust• Organizations and policies

– International Grid Trust Federation (IGTF)• APGridPMA• EUGridPMA• TAGPMA

– Accreditation• Software

– Certificate Authorities (CA)• Set up CA services, policies• Issue/manage certificates (user, server, services, …)

– GLOBUS• Identify users and resources in a cohesive fashion• Map user certificate to a local account• Interface local job managers

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

PRAGMA CAs

• PRAGMA-UCSD CA (https://goc.pragma-grid.net/ca)– Accredited by APGrid PMA– Included in IGTF distribution– Only issue a certificate when needed

• PRAGMA Experimental CA(http://rocks-56.sdsc.edu/exp-ca)– Only accepted by PRAGMA grid site

• PRIME 2011 students will use PRAGMA experimental CA unless your project involved with other grids

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Grids/Clouds and PRAGMA Grid/Cloud

• Grids/Clouds– Most are centrally funded

• PRAGMA grid/Cloud is unique– Grass-root

• Voluntary contribution• Open (PRAGMA member or not, pacific rim or not)• Long-term collaborative working experiment

– Heterogeneous• Funding• No uniform infrastructure management• Open to all sciences and applications• All varieties of site policies, system and network environments

– Friendly, helpful, close relationships– More challenging and rewarding

• Good for development, collaborations, integrations and testing

PRAGMA Grid/CloudPRAGMA Grid/Cloud

26 institutions in 16 countries/regions, 22 compute sites, 9VM sites (+ 6 site in preparation)

UZHSwitzerland

NECTECKUThailand

UoHydIndia

MIMOSUSMMalaysia

HKUHongKong

ASGCNCHCTaiwan

HCMUTHUTIOIT-HanoiIOIT-HCMVietnam

AISTOsakaUUTsukubaJapan

IHPC/NGONTUSingapore MU

Australia

KISTIKMUKorea

JLUChina

SDSCUSA

CICESEUNAMMexico

UChileChile

CeNAT-ITCRCosta Rica

BESTGridNew Zealand

CNICChina

LZUChina

UZHSwitzerland

LZUChina

UValleColumbia

ASTIPhilippines

IndianaUUSA

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

PRAGMA Grid Software Layershttp://goc.pragma-grid.net/pragma-doc/userguide/join.html

Local job scheduler (require one)

Globus (required)

Application Middleware

Applications

Infrastructure Middleware

SGE PBS LSF SQMS …

Ninf-G Nimrod/G Mpich-GX … Gfarm SCMSWeb MOGASCSF

Phylogenetic …FMO CSTFTSavannah MM5 AMBERSiesta

…

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011

PRAGMA Grid/Cloud Compute Resourceshttp://goc.pragma-grid.net/pragma-doc/computegrid.html

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

PRAGMA Grid/Cloud Members and Teamhttp://goc.pragma-grid.net/wiki/index.php/Site_status_and_tasks

• Team members– >240 and growing– one management contact / site– 1~3 technical support contact / site– 1~4 application drivers / application– 1~5 members / Middleware development team

• Experts– Application (various scientific domains)– Middleware (various grid middleware)– Security (grid security)– …

• Communications– Email– Mailing list – pragma-grid-team@googlegroups.com– Coordinator – zhengc@sdsc.edu

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

PRAGMA Cloud Access

• Cloud computing is new– Access procedure has not been formalized

• Contact Cindy for detail info and arrangements

• The rest of the slides shows how to access PRAGMA grid

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Obtain PRAGMA Grid Access• Find guide document at

http://goc.pragma-grid.net/wiki/images/5/51/PRIME-accounts.doc

• Be aware– When cut/paste a command line

• Microsoft word may change – A hyphen to a line character– Double-hyphen to a long line character

• Learn UNIX concepts and commands You need for your project

Know howUnderstanding (Know why)

Account application as an example Learn more online, googlehttp://mally.stanford.edu/~sr/computing/basic-unix.html Try them, hands-on

But be careful with “rm” (remove) command!

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Same Basic UNIX Commands• List files

$ ls$ ls -l (show file permission, ownership, etc.)$ ls -a (show all files, including hidden files)

• Change directory$ cd (go to my home directory)$ cd ~/.ssh (go to .ssh subdirectory under my home directory)

• Copy files locally$ cp <local-source-file> <local-destination-path>

• Copy files between local and remote systems use SSH key$ scp <local-source-file> <remote-host-name>:<remote-path>$ scp <remote-host-name>:<remote-path> <local-destination-

path>• Remember the command, but forgot how to use it

$ <command> --help$ man <command>

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Need an UNIX System on Laptop (Step 1.1)

• UNIX on laptop– MAC OS 10

HDApplicationsUtilitiesTerminal

– WindowsInstall CygwinClick

– LINUXLogin

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Create SSH Key(Step 1.2)

• Secure remote login• Generate key files

$ ssh-keygen -t rsa Use default path for ssh key files Good passphrase (hard to guess, no dictionary words) Remember the passphrase

• View SSH key files (private vs. public) From Cygwin

$ ls ~/.ssh-r--------@ 1 cindyzheng staff 951 Mar 19 01:10 id_rsa-rw-r--r--@ 1 cindyzheng staff 234 Mar 19 01:10 id_rsa.pub-rw-r--r--@ 1 cindyzheng staff 8414 Mar 19 01:10 known_hosts

From Windowsc:\Program files\Cygwin\Home\<Your-user-name-on-laptop>\.ssh

• Backup SSH key files• USB drive, take with you• Can be used thereafter• Only on secure systems• Possibly compromised key

• Immediately inform all public key holders (system administrators)

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Account Application(Step 2)

• Email to Cindy– Your full name – Institution name: UCSD– Address of the institution: UCSD– Country of the institution: USA– Your phone number– Your fax number <optional>– Your email address– The purpose of using PRAGMA grid: PRIME– Your/your institution's contribution to PRAGMA (can be work,

resources): work– Prefered login names (1st, 2nd, 3rd, but may or may not be granted)– Prefered UNIX shell (may or may not be granted) <optional>– Hostname(s) and IP address(es) of the system(s) which you launch your

applications (This is required for PRAGMA grid site systems to open their firewalls in order to allow you access.) <ask your foreign advisor for it>

– Your ssh PUBLIC key (NEVER and NEVER give anyone your private key file!! Nor your ssh password!!)

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Cindy

• Create user info pack

• Create an UNIX user account on rocks-200.sdsc.edu

• Email user– Account name on rocks-200.sdsc.edu– Certificate request license id

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Test SSH key(Step 3)

• Login to rocks-200.sdsc.edu$ ssh <username>@rocks-200.sdsc.eduEnter passphrase

Last login: Tue Apr 5 11:37:56 2011 from dyn137-110-115-232.ucsd.edu

Rocks 5.3 (Rolled Tacos)Profile built 04:25 15-Apr-2010

Kickstarted 22:21 14-Apr-2010

• In case ssh fails$ ssh –v <username>@rocks-200.sdsc.eduEmail output to Cindy

First time login on rocks system, ssh-keygen ran automaticallyAccept default ssh file pathEmpty password

• Only used internally• not recommended elsewhere

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Make a User Certificate Request(Step 3)

$ cd$ /opt/pragma-ca/bin/grid-certreq -sv ra.pragma-grid.net:pragma-exp_ra -new <license_id> -g

"SMIME user" -sou ---------(Sample output)------------- creating a certificate signing request ------------------------------------------- generate private key (size 1024 bit) ...................................................oo ....................oo ------- input user subject information -------- email can be omitted by putting a char of '.’ input ou : PRIME type exactly as this input user name : Cindy Zheng input your full name input user email : cindy@sdsc.edu input your email address, must! ------- please confirm your inputs ------------ GROUP : SMIME user SUBJECT : CN=Cindy Zheng, Email=cindy@sdsc.edu do you continue operation? (yes/no/retry)[y]: press return trying to connect RA server : ra.pragma-grid.net (11412) ... ok. request for issuing a new certificate ... ok. your request is accepted. (AcceptID=0000003) <- remember it CA operator will send an email to tell a result. save a CA certificate file : /home/cindy/.globus/cacert.pem <- note cert request files location save a private key file : /home/cindy/.globus/userkey.pem <- note cert request files location Input PASS Phrase: type a good password Verifying - Input PASS Phrase: retype the password (remember it!) • Email Cindy the acceptID

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Cindy

• Issue user certificate– Off-line CA server

• Inform user via email

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Retrieve User Certificate(Step 4)

• Login to rocks-200.sdsc.edu$ ssh <username>@rocks-200.sdsc.edu When prompt, enter your SSH password

• Retrieve user certificate$ /opt/pragma-ca/bin/grid-certreq -sv ra.pragma-

grid.net:pragma-exp_ra -em <your-email> -recv <your-acceptID>

trying to connect RA server : ra.pragma-grid.net (11412) request for exporting a certificate ... ok save a CA certificate file : /home/cindy/.globus/cacert.pem save a certificate file : /home/cindy/.globus/usercert.pem

• View and understand certificate files$ ls -l .globus-rw-r--r-- 1 cindyzheng staff 2878 Mar 23 18:26 cacert.pem-rw-r--r-- 1 cindyzheng staff 4998 Mar 23 18:26 usercert.pem-r-------- 1 cindyzheng staff 1743 Mar 23 18:26 userkey.pem

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Backup Certificate Files(Step 4)

• Copy .globus directory from rocks-200.sdsc.edu to your laptopStart Cygwin/terminal on your laptop

(Should land in your home directory)$ scp -r

<username>@rocks-200.sdsc.edu:/export/home/<username>/.globus .

• Copy c:\Program files\Cygwin\Home\<Your-user-name-on-laptop>\.globus to your USB backup disk• Take with you anywhere• Can be used usually 1 year• Only on secure systems• Possibly compromised certificate

• Immediately inform certificate issuer (Cindy)

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Test Globus Authentication(Step 5)

On rocks-200.sdsc.edu:• Create a proxy

$ grid-proxy-init Your identity: /O=grid/O=pragma/OU=PRIME/CN=Cindy Zheng

Enter GRID pass phrase for this identity: <- input passphrase Creating proxy .......................................................... Done Your proxy is valid until: Thu Apr 7 02:18:11 2011• Test globus authentication

$ globusrun -a -r rocks-200.sdsc.edu GRAM Authentication test successful• Email Cindy if successful, otherwise email output to Cindy• To create longer proxy, specify hours

$ grid-proxy-init –hours 100

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Test Job Submission(Step 5)

$ globus-job-run rocks-200.sdsc.edu/jobmanager-sge /bin/date/opt/gridengine/default/spool/rocks-65/active_jobs/330008.1/pe_hostfilecompute-0-12Tue Apr 21 13:06:18 PDT 2009rm: cannot remove `/tmp/330008.1.all.q/rsh': No such file or directory ignore this for now

Note: depending on how busy the system is, you may need to wait a bit.To find out the status of SGE queue, open/login to another shell and run$ qstat –f

queuename qtype used/tot. load_avg arch states----------------------------------------------------------------------------…----------------------------------------------------------------------------all.q@compute-0-6.local BIP 1/2 0.06 lx26-x86 328085 0.56000 data jas dr 04/15/2009 23:29:01 1 ----------------------------------------------------------------------------… ##################################################################### - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - ##################################################################### 330010 0.00000 data zhengc qw 04/21/2009 13:18:39 1

Email Cindy if successful, otherwise email output to Cindy

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Cindy

• Add your email to pragma-grid-team list

• Email pragma-grid-team@googlegroups.com ask site adminitrators to create account on their systems

• Site administrator will email your username to you and me after your account is created

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Test Access To Your Host Site(Step 6)

• Host system name/ip (where you will be launching your applications from)

• Do the same tests as you have done with rocks-200.sdsc.edu– Test SSH (Slide 20)– Test Globus authentication (Slide 29)– Test job submission (Slide 30)

• More testing tipso http://goc.pragma-grid.net/wiki/index.php/

User_Testingo Google

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Test Access to Each Site(Step 6)

• Test SSH– Find the target system name athttp://goc.pragma-grid.net/wiki/index.php/Site_status_and_tasks– Test SSH from your host system to the target host (should work)– Test SSH from your laptop to the remote host (May work)– Issues about firewall and site policies

• Test Globus authentication– From the system at your host site– To the target system

• Test job submission– From the system at your host site– To the target system

• Problem with any test– Save output– Check tips at

• http://goc.pragma-grid.net/wiki/index.php/User_Testing • Google

– Email site contact and cc zhengc@sdsc.edu

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Process At A Glance

Laptop

platform

windows

MAC

Step 1.1

Install

Cygwin

Step 1.2

Create a

SSH key

Step 2

Email application Create

account on

rocks-200

PRIME Student Cindystart

Step 3 Test SSH,

Request certificateIssue

certificateStep 4

Retrieve certificate

Step 5

Test Globus authentication and job submission

confirmation

Step 6

Test SSH, Globus and job submission to each site

sites reply

Request

to all sites

PRIME Tutorial, 4/22/2009Prime Tutorial 4/22/2011Prime Tutorial 4/22/2011

Use PRAGMA Grid

• Information onlinegoc.pragma-grid.net member sites (hostname, contacts, user guide, …) computational grid (resource table, jobmanager, cpu

architecture, …) Applications (as examples) SCMSWeb (realtime system status, software catalog, etc.)wiki.pragma-grid.net For Users

• Ask questions (email, skype) Email, skype Site contact – always cc Cindy Looking for experts – ask Cindy If Cindy is not around, pragma-grid-team@googlegroups.com