april 13 2005 global grid forum(ggf) 의 구성과 국제 표준화 활동 절차 ggf organization...
TRANSCRIPT
April 13 2005
Global Grid Forum(GGF) 의 구성과 국제 표준화 활동 절차
GGF Organization & Standardization Process
DONGGUK UNIVERSITY
Yangwoo Kim
ICAT2005
2
I. What is GGF(Global Grid Forum)?
II. Global Grid Forum (GGF) Organization
III. GGF Documents & It’s Processing Steps
IV. Case Study (Grid Information Retrieval WG)
(Demo of GIR System & P2P Distributed Crawler)
Table of Contents
3
GGF mission
Stan
dard
s
Com
munity
Operations
Leading the pervasive adoption of grid computing for research and industry
4
What Does GGF Do?
An Open Process for Development of Agreements and Specifications Grid “Recommendations” process (GFD.1)
modeled after Internet Standards Process (IETF, RFC 2026) Persistent, Reviewed Document Series
similar to RFC series; initiated October 2002 Intellectual Property policies (RAND) modeled after IETF policies
A Forum for Information Exchange and Collaboration Comparing notes on experiences, approaches Spanning vertical layers, including people who are
Doing Grid Research Designing and Building Grid software Deploying Grids Using Grids
Spanning technology areas- from directory services to scheduling to data handling to security to collaboration environments to…
5
Ed
ucatio
n
Par
tner
ship
s
Conferences, Tutorials &
Partnerships
Working Groups and Research Groups
Specifications and Best Practices
Identify where / what specs are critical
Frameworks / Architectures(how specs fit together)
Working Groups
Research and Technology Directions
Application Requirements
Research Groups and Workshops
6
Key GGF Strategies: Groups
Working Groups (WG) Crisp focus on development of a specification,
guideline, etc. Clear milestones based on delivery of drafts
Research Groups (RG) Focus can be broader than WG, but must be clear Milestones based on delivery of drafts,
organization of workshops, and workshop reports Application Focus Groups (kind of RG)
7
GGF Structure
Steering Group•GGF Management•Document Series Review
Area
Area
Area
AreaArea
Area
ADAD
ADAD
AD
ADAD
ADAD
ADAD
Editor
Document and Standards WorkSecretariat: Operations
Board of Directors(Catlett, Nitzberg, Feiereisen)
•Financial, Legal Responsibility
Conference Mgmt
Sponsor Programs
Finance/Legal
IT (Website, etc.)
Holds non-exclusive copyright for document series
GGF Corp. GGF
= working group
= research group
Chair
Grid ResearchOversight Council
•RG/WG Workshop series•Advise on Research Issues
GGF Advisory Committee
•Advise on strategic direction, industry, government•Advise on governance
Staffing & Services
AD
A-Large Committee
8
Key GGF Strategies: Meetings
3 Meetings a year: Spring, Summer, Fall Traditionally Summer meeting WG only Fall meeting full Plenary program Experimenting with “tracks” starting GGF14
Several types of sessions: Workshops: in-depth session on particular topic Tutorials: sessions to present training information Plenary: Cross-group sessions Group: Face to face sessions to advance working group activity
Meetings self-funded Try to spread roughly equally by region
Driven be attendance and host offers
9
Key GGF Strategies: Process
Open Standards Process All participants have an equal voice All meetings announced Major document revisions and decisions announced to mailing list Meeting minutes available
Meetings Frequency
Phone: Frequency is Decision of Chairs Interim: Frequency is Decision of Chairs GGF Meetings: 3x/yr, working groups strongly urged to meet
Openness All meetings must be announced to mailing list and in GridForge Interim meetings should be announced with adequate time to arrange
travel (suggested 4 weeks, minimum 2 weeks) Phone dial-in must be allowed for interim meetings
10
Key GGF Strategies: Management
Individual Groups Working Group and Research Group Chairs work with Area Dire
ctors to track progress based on charter. Area Directors (Part of GFSG)
Responsible for assisting chairs Review drafts prior to submission to editor Promote communication As a group, AD’s approve charters and advance CP and R docu
ments GFSG At-Large Subcommittee
Responsible for implementing “appeals” process for group charter approval and document advancement decisions
Oversee liaison activities with other groups Advise on long-term strategies
11
Key GGF Strategies: Advice GGF Advisory Committee (GFAC)
Advises GGF Chair and GFSG on long-term institutional strategies Evaluates performance of GGF overall and of GGF Chair Singular governing role: approves nomination, by GFSG, of GGF
Chair
Grid Research Oversight Council (GROC) Facilitates the involvement of research community in GGF Responsible for reviewing and approving workshop proposals Advises GFSG on the formation of new research groups
Grid Marketing Advisory Council (GMAC) “Anti-Hype” Working Group Advises GGF Chair and GFSG on marketing strategies for Grid
Computing Identifies and publicizes usage of Grid in industry, research, and
academia
12
How to participate in GGF Attend Meetings
We all start as tourists Join Group
Mail Lists Join list off WG webpage Archives available there too
GridForge (http://forge.ggf.org) Write Documents
Drafts can be submitted by anyone Volunteer
Group volunteer Sponsor NOMCOM
Lead Group Chair Management Committee
13
How to Join a Group
Joining a Group is free to all Join Mail Lists
Send message o “subscribe <name-wg>” to [email protected] All official decisions made on the mailing list
Allows for absences Allows for non-English speakers
GridForge (http://forge.ggf.org) Tool to help group operations Public forum for processes (public comment, document subm
ission, editorial pipeline status, etc.
14
Key GGF Strategies: Documents
Four Types of Documents Informational Experimental Community Practice Recommendations (2-state process)
Document Process and Editor/Publisher Open, Public Review Process for GGF Drafts GFSG reviews recommendations and community
practice drafts as first stage of pipeline Informational and Experimental drafts reviewed by
editor, optionally by GFSG
15
Key GGF Strategies: Documents
GWD (Grid Working Draft) Stable document for general community review Any IP issues are noted Consistent format and naming
GFD (Global Grid Forum Document) Passed through the relevant (to document type)
process including public comment and Editor review
Assigned a document number (GFD.1)
16
Document Types
Informational• Informs the community of an interesting and
useful Grid-related technology, architecture, framework, or concept
• Specifies requirements related to a particular vertical
Experimental• Informs the community of the results of Grid-
related experiments, implementations, operational experience, or to propose an experimental specification
17
Document Types
Community Practice• inform and influence the community regarding an
approach or process that is considered to be widely accepted by consensus and practice in the Grid community
Recommendations (2 stage)• Documents a particular technical specification or a
particular set of guidelines for the application of a technical specification. The recommendations documents are intended to guide interoperability and promote standard approaches.
18
Documents (Other)
Process defined by:• GFD.1.pdf
• Global Grid Forum Documents and Recommendations: Process and Requirements
http://www.ggf.org/documents/GFD/GFD-C.1.pdf
•Public Comment• http://www.ggf.org/Public_Comment_Docs/Public_Comment_Documents.htm
• Documents for download and forums for comments.
19
Information or Experiment GFD’s
x
x
DONGGUK UNIVERSITY
GridIR 시스템 및 분산 P2P크롤러의 데모
동국대학교 , 정보통신공학과네트워크 컴퓨팅 시스템 연구실
Standardization Activities with GGF Grid Info. Retrieval WG
Case Study
21
데모 순서
1. 전체 시스템 환경2. 분산 크롤러 구조3. 개발 환경4. 시나리오
1. P2P 기반 분산 크롤러 데모2. GridIR 정보검색 시스템 데모
22
전체 시스템 환경
Peers
WWW
Globus3.0Globus3.0
♣ VO : Virtual Organization
QueryProcessor
Indexer/Searcher
CollectionManager
VO
Crawler N
Crawler1
Crawler Manager
23
크롤러 매니저와 P2P 기반분산 크롤러의 구조
Admin WWW
Crawler Manager
Crawler 1
♣ G.S : Grid Service
Crawler N
Manager
G.S
DB
Register
DB
Connector
Downloader
Controller
Parser
24
그리드 정보검색 시스템 개발의 필요성 및 장점
P2P 방식 분산 크롤러의 장점 여유 컴퓨팅 자원으로 정보검색 시스템의 작업분배 가능 크롤링 기능 자체에는 시간적인 제약 사항이 약함
필요성 모든 국제표준화 활동은 시스템 개발을 통한 검증이 필수임 GIR WG 과 국제공동연구의 필요성 대두 GridIR 시스템의 장점을 검증할 수 있는 시스템 개발이 필요 대륙간 한 가상조직 (VO) 로 묶어 다양한 구성과 확장성 검증 과학계산용 응용 뿐만이 아니라 상용 응용분야로의 확대
GridIR 시스템의 장점 시스템과 사용자 레벨에서의 견고한 보안 기능 질의 대상 DB 의 변경사항을 실시간 반영 , 복잡한 질의어 처리 푸쉬 방식의 스탠딩 질의어 처리 가능 분산처리 방식으로 다양한 시스템 구성과 확장성이 용이함
25
개발 환경
VO machine OS – ReadHat Linux 9.0 Required tools
Globus Toolkit 3.0 GridIR Toolkit Amberfish – text retrieval software Crawler Manager Program Java 2 Standard Edition(J2SE) 1.4 Software Development Ki
t(SDK) Jakarta ant 1.5 Jakarta Tomcat 4.1.24 Junit 3.8.1
26
개발환경
Client machine OS – RedHat Linux 9.0 Required tools
Java 2 Standard Edition(J2SE) 1.4 Software Development Kit(SDK)
Crawler program
27
전체 시스템 데모 순서
QueryProcessor
Indexer/Searcher
CollectionManager
VO Peer
Crawler N
Crawler Manager
WWW
Crawler1
Globus3.0 CM 등록
작업할당 및 결과 반송
분산 크롤러 등록
서비스 등록
질의어 검색
Indexer 생성 Collection 생성
28
데모 - P2P Crawler
서비스 등록 CrawlerManager
$ java org.globus.ogsa.client.Create http://localhost:8080/ogsa/services/gridir/CrawlerManager crawlermanager
29
데모 - P2P Crawler
등록 Peer 는 자신의 CPU, Memory, HDD 의 정보로 크롤러 매너저에
등록한다 .
30
데모 - P2P Crawler
크롤러 매니저에 등록 Peer 가 등록되면 등록된 Peer 의 IP 를 보여준다 .
31
데모 - P2P Crawler
작업 할당 등록된 Peer 로 작업을 할당한다 .
32
데모 - P2P Crawler
작업 준비 Peer 는 크롤러 매너저로부터 작업할 URL 을 할당 받음
33
데모 - P2P Crawler
작업 완료 작업 완료 후 결과 반환 ( 텍스트 문서로 전송 ).
34
데모 - P2P Crawler
결과 문서 취합 반환된 문서들을 취합하여 Collection Manager 에게 전달 .
35
데모 - GridIR System
서비스 등록 MetaRegistiry
$ java org.globus.ogsa.client.Create http://localhost:8080/ogsa/services/gridir/MetaRegistry mr
CollectionManager $ java org.globus.ogsa.client.Create http://localhost:8080/ogs
a/services/gridir/GridIRCollectionManager cm Indexer
$ java org.globus.ogsa.client.Create http://localhost:8080/ogsa/services/gridir/GridIRIndexer idx
QueryProcessor $ java org.globus.ogsa.client.Create http://localhost:8080/ogs
a/services/gridir/GridIRQueryProcessor qp
36
데모 - GridIR System
GridIR 실행 GridIR 정보검색 시스템을 실행한다 .
37
데모 - GridIR System
등록된 서비스 확인 CollectionManager, Indexer, QueryProcessor 등록됨 .
38
데모 - GridIR System
Collection 생성 메뉴에서 ‘ New Collection’ 을 선택
39
데모 - GridIR System
생성된 Collection 확인 Name 과 Discription 을 작성하고 파일이 있는 디렉토리 설정
후 ‘ OK’ 버튼을 누르면 ‘ My Collection’ 이 생성
40
데모 - GridIR System
Indexer 생성 메뉴에서 ‘ New Index’ 를 선택
41
데모 - GridIR System
Indexer 생성 확인 Name 과 디렉토리 설정 후 ‘ OK’ 버튼을 누르면 생성
42
데모 - GridIR System
질의어 검색 생성된 인덱스에서 메뉴에 있는 ‘ New Query’ 를 선택
43
데모 - GridIR System
질의어 검색 질의어를 입력하고 ‘ OK’ 버튼을 누른다 .
44
데모 - GridIR System
검색 결과 질의에 대한 결과로 다음과 같은 정보를 보여준다 .
45
향후 활용 방안
대륙간 한 가상조직으로 묶어 다양한 구성 실험 개발결과를 GGF 에 Experimental 문서로 등록 GridIR 시스템의 장점을 검증할 수 있는 기회 다양한 시스템 운영의 결과를 Community 문서로 등록
이러한 연구 결과물을 바탕으로 국제표준화 선도 추후 시멘틱 그리드 정보검색 시스템으로의 발전가능성 타진