governance: fundamental to soa's success

Governance: Fundamental to SOA’s Success

Ari Roy 

Senior Project Manager 

DATA Inc.

Montvale , NJ

arabinda@dataincusa.com

www.datainc.biz

2

Why Governance?

“Governance is much more complex if not thought out well in the beginning”

3

SOA in the conventional enterprise

Firewall

ESB

CRM(Seibel)

ERP (SAP)

HR (PeopleSoft)

Legacy Billing system(IBM Mainframe)

1 Sales Force

2 HR

3 CRM

4 Billing

New Business Process

Symbol steps DescriptionClient OnBoardingSales Force

(Custom)

Internal Customers

External Customers

Corporate Policy

Manual

4

Evolution after one year – without Governance

Custom App

Firewall

ESB

CRM(v2)(Seibel)

ERP (v2)(SAP)

HR (PeopleSoft)

Legacy Billing system(IBM Mainframe)

1 Sales Force

2 HR

3 CRM

4 Billing

5 Custom app

New Business Process

Symbol steps DescriptionClient OnBoardingSales Force(v1.2)

(Custom)

Internal Customers

External Customers

Corporate Policy

Manual

Business Partners

PLM

SCM

Development QA Deployment Operation

Compliance Policy

Compliance Policy

5

Evolution after one year – with Governance

Development QA Deployment Operation

Design Time Policy

Run Time Policy

Management

6

SOA Governance Defined

The discipline of making SOA adoption within an enterprise consistent and aligned with overall business objectives through creation and administration of a well organized set of top‐down policies, procedures and controls.

7

Governance Roadmap ‐ 4 Long and 4 Short steps

8

Governance Roadmap ‐ 4 Long and 4 Short steps

9

Governance Roadmap ‐ 4 Long and 4 Short steps

10

Governance Roadmap ‐ 4 Long and 4 Short steps

11

Governance Roadmap ‐ 4 Long and 4 Short steps

12

Governance Roadmap ‐ 4 Long and 4 Short steps

13

Governance Roadmap ‐ 4 Long and 4 Short steps

14

Governance Roadmap ‐ 4 Long and 4 Short steps

15

Governance Model 

SOA Governance Council

PolicyEstablish Governance Process And Policies

RolesDefine Roles and Responsibilites

Processes and Procedures

Common SOA Infrastructure

owner

Domain-A

Domain-B

owner

1

2

3

16

Governance Model 

SOA Governance Council

PolicyEstablish Governance Process And Policies

RolesDefine Roles and Responsibilites

Processes and Procedures

Common SOA Infrastructure

owner

Domain-A

Domain-B

owner

1

2

3Role of the Governance Council 

• Framework for Decision Making

• Allocates Responsibility across organization

• Processes involving decision making

• Metrics for monitoring effectiveness

17

Governance Model 

SOA Governance Council

PolicyEstablish Governance Process And Policies

RolesDefine Roles and Responsibilites

Processes and Procedures

Common SOA Infrastructure

owner

Domain-A

Domain-B

owner

1

2

3Policy Management Recipe

• Definition of Policies

• Creation of Policies

• Storage of Policies

• Communication of Policies

• Feedback of Policies

18

Governance Model 

SOA Governance Council

PolicyEstablish Governance Process And Policies

RolesDefine Roles and Responsibilites

Processes and Procedures

Common SOA Infrastructure

owner

Domain-A

Domain-B

owner

1

2

3

What is a Domain ?

• A domain contains set of services that relate to same business area/context– Billing, Purchase, Client Services 

19

Governance Model 

SOA Governance Council

PolicyEstablish Governance Process And Policies

RolesDefine Roles and Responsibilites

Processes and Procedures

Common SOA Infrastructure

owner

Domain-A

Domain-B

owner

1

2

3

What is a Domain ?

• Each domain owns and manages these services – Service availability / Data and Message Format / Business Logic Encapsulation

20

How does this fit within the Enterprise ?

Corporate Governance

IT Governance

Architecture Governance

SOA Governance

<<extends>> <<extends>> <<extends>>

aligns

aligns

21

How does this fit within the Enterprise ?

Corporate Governance

IT Governance

Architecture Governance

SOA Governance

<<extends>> <<extends>> <<extends>>

aligns

aligns

22

How does this fit within the Enterprise ?

Corporate Governance

IT Governance

Architecture Governance

SOA Governance

<<extends>> <<extends>> <<extends>>

aligns

aligns

23

How does this fit within the Enterprise ?

Corporate Governance

IT Governance

Architecture Governance

SOA Governance

<<extends>> <<extends>> <<extends>>

aligns

aligns

24

How does this fit within the Enterprise ?

Corporate Governance

IT Governance

Architecture Governance

SOA Governance

<<extends>> <<extends>> <<extends>>

aligns

aligns

25

Typical Governance Framework 

26

Typical Governance Framework 

27

Typical Governance Framework 

28

Governance Process Workflow Authorized User Publishes

A new Web service(appears in registry)

Potential Consumer

discovers the Web service

1. Consumer Requests Use of Service

2. Consumer agrees on Terms of delivery

3. Consumer is Authorized4. Service is provisioned

ESB

Service Delivery is

monitored and recorded

29

Governance Requirements  scenario analysisGeneral Ledger Application

(J2EE)

Financial Reporting

Service

SOA Infrastructure

Payable/Receivable

Warehouse Application(Mainframe –COBOL/CICS)

Shipping/Receiving

Inventory Check

Customer Portal (.Net )

Online Ordering

Online Payable

Online Order Status

What is a internal control requirements?Ref :404 of Sarbanes

Oxley Act (SOX)

30

Governance Requirements  scenario analysisGeneral Ledger Application

(J2EE)

Financial Reporting

Service

SOA Infrastructure

Payable/Receivable

Warehouse Application(Mainframe –COBOL/CICS)

Shipping/Receiving

Inventory Check

Customer Portal (.Net )

Online Ordering

Online Payable

Online Order Status

Invoice amounts are properly recorded to account, amount, period

Missing Documents or incorrect information

Accurate Recording of invoices for all authorized shipments

Control PracticeRiskControl Objective

What is a internal control requirements?Ref :404 of Sarbanes

Oxley Act (SOX)

31

Governance Requirements  scenario analysisGeneral Ledger Application

(J2EE)

Financial Reporting

Service

SOA Infrastructure

Payable/Receivable

Warehouse Application(Mainframe –COBOL/CICS)

Shipping/Receiving

Inventory Check

Customer Portal (.Net )

Online Ordering

Online Payable

Online Order Status

Invoice amounts are properly recorded to account, amount, period

Missing Documents or incorrect information

Accurate Recording of invoices for all authorized shipments

Control PracticeRiskControl Objective

What is a internal control requirements?Ref :404 of Sarbanes

Oxley Act (SOX)

Many Ways to implement…

Schema Validation,Cross Referencing

32

Key components of Governance

33

SOA Governance‐Service Lifecycle 

Design Time

Upgrade Time

Run Time

Registry /Repository

34

Design Time Governance (some or all)

DesignTime

Entitlement

Notification/Approvals

Identity(?)Management

Audit Trail

Content Validation

35

Identity ManagementPurpose:

To Establish Rights and Responsibilities in the registry/repositoryMeasuring the Service usage/LoggingEnforcing Approval RequirementsEnforcing Role/Individual based Governance

Features:LDAP based, SSODigital Identity

36

Entitlements

Purpose:To grant fine grained access to registry/repository assets

Features:Ability to secure assets Ability to Classify assets and provide accessAbility to classify Policies and Assign Roles

37

Notification and Approval

Purpose:To Trigger events in response to Create, Update, Read and Delete activities

Features:Must be applied before and/or after interaction Support for different Notification models(Message based, Email)

38

Content Validation

Purpose:To scan and validate contents in Registry/Repository as per type and pre-configured compliance checks

Features:

WSDL validationSchema ValidationValidation related to Interoperability

39

Audit Trail

Purpose:To establish accountabilityTo track interaction among participants and registry/repositoryEstablish Usage pattern

Features:Format /Verbosity RequirementsArchival Policy

40

Run Time Governance (some or all)

Runtime

Service Virtualization

Message Transport

End PointManagement

Custom Management

Policy Provisioning

Version Management

ESB

41

Service VirtualizationPurpose:

To compose task-specific “virtual” services from existing services.

Features:

Ability to Consolidate one or more operations from different services into oneCreate Skeleton services from WSDLAuto generation of WSDL for new virtual service

42

Message BrokeringPurpose:

To deliver service based on business or compliance criteria

Features:Routing rules based on Content/ContextTransform Inbound request / Outbound responseLogging ,Monitoring, AlertingSLA ManagementMediate across different transport protocols (HTTP-to-JMS, JMS-to-HTTP or custom)

43

Policy provisioningPurpose:

Provisioning of Operational, Compliance policy

Features:Auto Enforcement of policies on new ServicesAuto adaptation of Client to new Policy RequirementsAuto Provisioning of policy based upon Change in service profile

44

Version ManagementPurpose:

To allow smooth evolution of production systems

Features:Publication of multiple versions of the same service simultaneouslyTransparent Rolling upgrades to published serviceBack-ward compatibilityVersion based routing

45

Custom ManagementPurpose:

Template based approach to Policy Management

Features:

Custom policy libraries for specific management needsContent, context or custom instrumentation based approach to any domain- or application-specific policyReuse of custom policies across multiple applications or SOA projects

46

End Point ManagementPurpose:

Fine grain control of the service deployed in each of the container

Features:

Managed endpoints for each serviceSpecial purpose end points based on type of usage secured/unsecured)Load Balancing/Fail Over for Highly available End points

47

Upgrade Time Considerations

¬ Understand Inter-Service relationship and dependencies

¬ Analyze the Impact of changing a Web Service in a runtime environment

¬ Complexity in Roll outing Service in Runtime Environment

¬ Service Custody Transfer

¬ Changes to existing SLA and Policies

48

Automating Governance 

Design TimeCode analysisContent Validation

Run TimeWS-I complianceUsage of Predefined schemaUsages of Specific TransportAutomated policy Discovery/provisioning

Change TimeMonitoring and Measurement of SLA metrics

(response time, availability, or throughput of service)

49

Technologies Behind Governance

50

Role of ESB in Governance

¬ Security- Ensure Privacy, Authenticity, Authorization and

Auditing of all Message exchanged

¬ Mediation - Policy based mediation (protocol/invocation)

¬ Management- Holistic view of Transactions that passes through- Intercept Service call

51

Role of Service Registry/Repository 

Where all Services are published

Implements process to publish service that matches Governance model

Contains Policies applicable to each service

52

Service Registry 

SOA Registry

Universal Description Discovery and Integration

UDDI API sets(Web service Access)

UDDI Schema (Meta Data Standard)

SOA MetaData

Business Taxonomy Policies Policy

Association

Dependencies

Service Information

Subscription

Provider Information

Configurations

53

Service Repository

SOA Repository Common Features

WSDL Libraries

Message Logs

Extensions

Reports

Blogs

Run Time Event Notification

Wikis

Dashboards

Design Time Policy Libraries

Run Time Policy Libraries Performance Info

54

Integrated Registry/Repository‐ Key Benefits

¬ Consistent view of service definition

¬ No duplication of Data 

¬ No need for data synchronization

¬ Discover both Service info and dependencies

55

Implementing SOA Governance 

56

SOA Governance Checklist ‐1

¬ Registry/Repository:Service Meta‐Data setup and Validation

Service Relationship and Dependency Management

¬ Access to Service:Workflow based Request Process 

User Configurable Policies 

57

SOA Governance Checklist ‐2

¬ Publishing ServiceWorkflow based Notification

WSDL validation and Conformance Reporting

Wizards for Publication

¬ Delivery of ServiceProvider/Consumer Binding

SLA enforcement, Versioning, Deployment 

Centralized monitoring

58

SOA Governance Checklist ‐3

¬ Delivery of Service (cont.)

Routing Management

Failover /Load Balancing

Logging and Audit Trailing

¬ Service Change Management

Service subscription management

Service Metadata subscription

59

SOA Governance Checklist ‐4

¬ Replication strategySelective synchronization /promo.Master/Slave based

¬ Enforcement of Security

Role based ACLFixed and Configurable RolesSupport for LDAP

¬ Interoperability Handling any URI data typesJava Rule Engine API

60

Analysts Comments:

• “The governance of objects and components is relatively straightforward: We create the gadget and put into a repository and fix it when we need to.”

Carl Lentz ‐ Panelist ‐ The Role of Objects in a Services‐obsessed World ‐ ACM, 10/2007

• "Enterprise governance models, early adopters are implementing organizations whose focus is to advance SOA adoption." 

Rajeev Mahajan ‐ Practice Manager ‐ The Service Integration Maturity Model: Achieving Flexibility in the Transformation to SOA ‐ IEEE, 9/2006

61

Benefits of SOA Governance¬ Greater alignment with business objectives

¬ Greater control over creation, deployment and consumption of services

¬ Centralized management of policies and regulations

¬ Can embed compliance with government and industry regulations¬ Sarbanes‐Oxley, MiFID, HIPAA, GLBA

62

Challenges of SOA Governance

¬ Multiple organizations:‐ How to create governance for service providers, infrastructure providers, and application developers? What if policies conflict?

¬ Managing exceptions:

‐ How to record and maintain sometimes necessary exceptions?

63

Challenges of SOA Governance

¬ Enforcing compliance:‐ How to make sure that policies and procedures are being followed at design time as well as runtime?‐What are the incentives for compliance?

¬ Seems counterintuitive:‐ If SOA foundation lies in loose coupling and flexibility, why do we need centralized control?

64

Case Study 

Operational Risk management in Derivative Trade Processing

65

Life Cycle of a Derivative Trade

Confirmation

Termination/Novation

Portfolio Reconciliation

Settlement

66

Process Flow

Dealer

SOA Trade ExecutionPlatform

DTCC

Clients

Trade Capture System

Trade Capture System

1

2

3

4

6

5

8

7

9

67

Implement Governance to avoid blind spots in the SOA highway

68

Resources

BEA : http://www.bea.com/framework.jsp?CNT=index.jsp&FP=/content/solutions/soa_governance

IBM :

http://www‐306.ibm.com/software/solutions/soa/entrypoints/advancing_soa_governance.html

INFOQ:http://www.infoq.com/governance/

69

Q & A