02 nf pres pc7a

7/28/2019 02 Nf Pres Pc7a

1/89

1

PowerCenter 7

Advanced: New Features

Education ServicesVersion PC7A-20040830

Informatica Corporation, 2004. All rights reserved.

7/28/2019 02 Nf Pres Pc7a

2/89

2

PowerCenter 7.1 Platforms and Connectivity

PowerCenter 7.1 Options and Upgrades

Workflow Manager: Session Editor Enhancement

Workflow Monitor Enhancements (Workflow Monitor lab)

Cross-Tool Enhancements

Designer Enhancements (Client Usability, Flat File Lookup and Union,Creating XML Definitions and Transaction-Preserving Transformations labs)

Workflow Manager: Error Logging Enhancement (Error Logging lab)

Agenda

7/28/2019 02 Nf Pres Pc7a

3/89

3

PowerCenter 7.1 Platforms and Connectivity

PowerCenter Server

64-bit AIX 64-bit HP-UX Windows NT X

AIX 4.3.3 X

Repository Server

SuSE Linux (in PowerCenter 7.1.1) PowerCenter Client

Windows NT X

Windows 98 X

PowerConnects: Web Services SAS Plus in PowerCenter 7.1.1

MSMQ

Hyperion Essbase

HTTP Most PowerConnects on Linux

Added X Discontinued

7/28/2019 02 Nf Pres Pc7a

4/89

4

PowerCenter 7.1 Options

PowerCenter

Real-Time/WebServicesZL Engine, always-on non-stop sessions, JMS

connectivity, and real-time Web Services provider

Data CleansingName and address cleansing functionality, including

directories for US and certain international countries

PartitioningData smart parallelism, pipeline and data parallelism,

partitioning

Server engine, metadata repository, unlimited designers,

workflow scheduler, all APIs and SDKs, unlimited XML and

flat file sourcing and targeting, object export to XML file,

LDAP authentication, role-based object-level security,

metadata reporter, centralized monitoring

Server group management, automatic workflow distribution

across multiple heterogeneous serversServer Grid

Profile wizards, rules definitions, profile results tables,and standard reportsData Profiling

Version control, deployment groups, configuration

management, automatic promotionTeam-Based Development

7/28/2019 02 Nf Pres Pc7a

5/89

5

PowerCenter 7.1 Upgrades

PowerMart v5 or v6 v7.1

PowerCenterv5 or v6 v7.1

PowerCenterRT v5 or v6 v7.1

Purchasableoptions

Install BaseVersion Upgrade

NewCustomers

Data Cleansing

Partitioning

Real-Time /WebServices

PowerCenter

Server Grid

Data Profiling

Team-Based Development

Note: PowerMart upgrades allowuse of global repositories but extra

repositories cost more.

7/28/2019 02 Nf Pres Pc7a

6/89

WorkFlow Manager: SessionEditor Enhancement

6

7/28/2019 02 Nf Pres Pc7a

7/89

7

v7 Session Editor

Properties and Config Object tabs havecollapsible options rather than sub-tabs

New Mapping tab consolidates Sources, Targets,Transformations and Partitions into one tab withtwo views:

Transformations view

Partitions view, with graphical display

7/28/2019 02 Nf Pres Pc7a

8/89

8

Properties Tab

Collapsible

options

7/28/2019 02 Nf Pres Pc7a

9/89

9

Config Object Tab

Collapsible

options

7/28/2019 02 Nf Pres Pc7a

10/89

10

Mapping Tab - Transformations View

7/28/2019 02 Nf Pres Pc7a

11/89

11

Mapping TabPartitions View

Graphical display shows mapping flow, partition points, partition type & number

Flag color indicates partition type

Folders for

- Partition Points

- Non-Partition Points

7/28/2019 02 Nf Pres Pc7a

12/89

Workflow MonitorEnhancements

12

7/28/2019 02 Nf Pres Pc7a

13/89

13

Workflow Monitor Enhancements

Improved Task view

Workflow run tree display

All workflows running on all servers at once

Status messages Filters menu and toolbar with more options:

Workflows that ran in a specific time frame

Sessions that ran during the last X hours

Copyright 2004 Informatica Corporation. All rights reserved.

7/28/2019 02 Nf Pres Pc7a

14/89

14

Workflow Monitor Task View

v6

v7

7/28/2019 02 Nf Pres Pc7a

15/89

15

Filter Toolbar

Display recent runs

Filter tasks by specified criteria

Select servers to filter

Select type of tasks to filter

New Filter toolbar

7/28/2019 02 Nf Pres Pc7a

16/89

16

Workflow Monitor Enhancements

Standard toolbar

Print preview

Server toolbar

Resume and recover workflow

Toggle Navigator window on/off

Toggle Output window on/off

7/28/2019 02 Nf Pres Pc7a

17/89

17

Lab NF1Workflow Monitor

7/28/2019 02 Nf Pres Pc7a

18/89


18

7/28/2019 02 Nf Pres Pc7a

19/89

19


Cool look

Validation enhancements

Object export/import

Copying and comparing objects

7/28/2019 02 Nf Pres Pc7a

20/89

20

Cool Look

Cool look (no borders to icons) default

Turn off in Tools => Customize, Toolbars tab

Many icons revised in toolbars and workspace

7/28/2019 02 Nf Pres Pc7a

21/89

21

Validation Enhancements

Invalidation

A parent object is invalidated when changes are made to its childobject

In v6, the parent object was marked invalid but the reason wasnot reported

In v7, the reason is reported in the fetch.log

Mass Validation

In v6, the user had to fetch and validate each parent objectindividually

In v7, the user can validate all the parent objects at the same

time. This is useful to identify all invalidations caused bychanging a shared child object.

Available in Repository Manager Navigator tree, List View, and(for versioned repositories) in Results View

7/28/2019 02 Nf Pres Pc7a

22/89

22

Object Export/Import

Full export/import of repository objects to/from XML

Workflows, worklets, sessions, mappings,transformations

Multiple objects in a single XML file

Automatic handling of dependent objects

Objects can span multiple folders across arepository

7/28/2019 02 Nf Pres Pc7a

23/89

23

Copying and Comparing Objects

7/28/2019 02 Nf Pres Pc7a

24/89

24

Copying and Comparing Objects

Designer and Repository Manager copy conflicts

now invoke the Copy Wizard

Copy Wizard has several enhancements Workflow Manager and Repository Manager

allow Compare Objects forworkflows and tasks

7/28/2019 02 Nf Pres Pc7a

25/89

25

Designer and Repository Manager Copy Conflict

v6

v7

Opens Copy Wizard

7/28/2019 02 Nf Pres Pc7a

26/89

26

Copy Wizard Enhancements

Simplified name resolution

Compare conflicting objects

Scope of resolution

v7

7/28/2019 02 Nf Pres Pc7a

27/89

27

Copy Wizard Enhancements contd

Compare objects before resolving a name conflict

v7

7/28/2019 02 Nf Pres Pc7a

28/89

28

Compare (Diff) Workflows and Tasks

In Workflow Manager and Repository Managerv7

7/28/2019 02 Nf Pres Pc7a

29/89

Designer Enhancements

29

7/28/2019 02 Nf Pres Pc7a

30/89

30

Designer Enhancements

Port Attribute Propagation

Lookup Transformation with Flat Files

Union Transformation

Custom Transformation

XML Enhancements

Transaction-Preserving Transformations

New Functions and Datatypes

7/28/2019 02 Nf Pres Pc7a

31/89

31


7/28/2019 02 Nf Pres Pc7a

32/89

32


When you change a port name, Designerautomatically propagates references to that portin expressions, conditions, and other ports

within the transformation Can also propagate changed port attributes

forward and backward throughout the mapping

7/28/2019 02 Nf Pres Pc7a

33/89

33

Port Attribute Propagation Steps 1-3

1. In Normal View, select oneor more ports (use Shift or Ctrl key formultiple ports). Right-click and select Propagate Attribute.

2. Dialog Box Opens

3. Select

Direction (forward / backward link path or both)

Attributes to propagate (name, data type, precision, scale)

Options implicit dependencies to include (condition and / orexpression). Disabled if Name attribute selected.

7/28/2019 02 Nf Pres Pc7a

34/89

34

Port Attribute Propagation Steps 4-5

5. Propagate updates:

I and I/Oports in forward link path

O and I/Oports in backward link path

Selected attributes for all ports in the link path

Port name in:

Dependent expressions or conditions (if options selected)

Associated port of a dynamic lookup

Custom transformations

4. Preview (best practice) shows links to affected ports ingreen, unaffected ports in red

7/28/2019 02 Nf Pres Pc7a

35/89

35

Lab NF2Client Usability

7/28/2019 02 Nf Pres Pc7a

36/89

36

Lookup Transformation with Flat Files

7/28/2019 02 Nf Pres Pc7a

37/89

37

Lookup Transformation with Flat Files 1

In v7, you can use a flat file as source for a connected or

unconnected Lookup transformation

You can use any flat file definition in the repository or you can

import it

7/28/2019 02 Nf Pres Pc7a

38/89

38

Lookup Transformation with Flat Files 2

When you import a flat file lookup source, the Designer invokes the Flat

File Wizard

7/28/2019 02 Nf Pres Pc7a

39/89

39

Lookup Transformation EditorFlat File 1

7/28/2019 02 Nf Pres Pc7a

40/89

40

Lookup Transformation EditorFlat File 2

7/28/2019 02 Nf Pres Pc7a

41/89

41

Configuring a Session for Flat File Lookup

7/28/2019 02 Nf Pres Pc7a

42/89

42


7/28/2019 02 Nf Pres Pc7a

43/89

43


Merges data from multiple pipelines into one pipeline(similar to SQL Statement UNION ALL)

Passive Transformation

Connected Mode only

Ports

Multiple input groups

Single output group

Ports in all input and output

groups must match

Usage Merging pipelines Does not remove duplicate

rows

7/28/2019 02 Nf Pres Pc7a

44/89

44

Union Transformation - Example

7/28/2019 02 Nf Pres Pc7a

45/89

45

Lab NF3Flat-File Lookup and Union

7/28/2019 02 Nf Pres Pc7a

46/89

46

Custom Transformation

7/28/2019 02 Nf Pres Pc7a

47/89

47

Custom Transformation 1

New framework for developing userdefined transformations

Uses compiler-independent APIs

C for server

C++ for client

Native transformation look and feel

Supports:

Active or passive transformations

Multiple input and output groups

Port-level metadata Transaction control

Update strategy

Partitioning

7/28/2019 02 Nf Pres Pc7a

48/89

48


Calls an active or passive procedure defined in a dynamiclinked library (DLL) or shared library

Active or Passive Transformation

Connected Mode only

Ports Mixed

Usage Perform transformation logic

outside PowerCenter

Uses Custom transformationfunctions

Sorting, Aggregation

7/28/2019 02 Nf Pres Pc7a

49/89

49


Custom transformation replaces the AdvancedExternal Procedure (active) transformation

External Procedure (passive) transformation

remains This supports Microsoft COM objects, including Java and

Visual Basic, as well as C and C++

7/28/2019 02 Nf Pres Pc7a

50/89

50

XML Enhancements

7/28/2019 02 Nf Pres Pc7a

51/89

51

XML Enhancements

XML Definition Wizard

Import from XML schemas (XML 2001 standard)

Generate XML views (groups)

XPath support

XML Editor XML workspace displays XML views and relationships graphically

Popup windows for schema details e.g. ComplexType hierarchies

Data preview

Midstream XML Parser and Generator transformations

Performance options for large XML targets

7/28/2019 02 Nf Pres Pc7a

52/89

52

Import from XML Schemas

XML schemas are much richer than DTDs:

Written in XML

Support multiple namespaces(A namespace is a schema location, e.g. URL, where a

group of related elements and attributes are defined)

Support many more datatypes(44+ simpletypes plus user-defined complextypes)

Support substitution groups e.g. alternativeroot elements

More flexible, e.g. Child elements occurring in any order

Multiple elements with the same name butdifferent content

Elements with no content

7/28/2019 02 Nf Pres Pc7a

53/89

53

Generate XML Views 1

XML definitions represent the XML hierarchy as groups,called XML views

XML Source Definition

XML Views (Groups)

7/28/2019 02 Nf Pres Pc7a

54/89

54


The XML Wizard can generate XML views from rules (entity

relationships, hierarchy relationships) or you can create

custom XML views

7/28/2019 02 Nf Pres Pc7a

55/89

55


For custom views, you can reduce metadata explosion by

several options

7/28/2019 02 Nf Pres Pc7a

56/89

56

XPath Support

XPaths list the path from the root element to an element or attributewith all intermediate components separated by /

XML Source Definition

7/28/2019 02 Nf Pres Pc7a

57/89

57

XML Editor

XML Workspace

XML MetadataNavigator

Components Pane- Properties- Actions

- Data Values, if any(shows selectedcomponent) Columns window

(shows selected view)

Double-click XML definition in workspace or Right-clickEdit XML Definition

or from Source / Targets / Transformation menusEdit XML Definition

7/28/2019 02 Nf Pres Pc7a

58/89

58

XML WorkspaceXML Views

The XML Editors workspace displays the XML views (groups)as entities connected by lines and symbols indicating the

relationships (parent/child, many:many, etc)

XML Source Definition XML Workspace

XML Views

7/28/2019 02 Nf Pres Pc7a

59/89

59

XML WorkspaceView Schema Details

XML Editor has popup windows for Edit Namespace,ComplexType Hierarchy, Data Preview, etc.

7/28/2019 02 Nf Pres Pc7a

60/89

60

Midstream XML Parser Transformation

Reads XML from a database table or message queue

In v6, had to use a mapplet with an XML Source Qualifier

7/28/2019 02 Nf Pres Pc7a

61/89

61

MidStream XML Generator Transformation

Creates XML in a database table or message queue

In v6, had to use a mapplet interface

7/28/2019 02 Nf Pres Pc7a

62/89

62

On Commit option allows user-defined commitsto flush XML data

On Commit Write to new document allows

multiple XML output files

Target cache size for XML tree (on overflowspills to disk)

Do not output empty elements avoids writingunnecessary elements

Performance Options for Large XML Targets

7/28/2019 02 Nf Pres Pc7a

63/89

63

Lab NF4Creating XML Definitions

7/28/2019 02 Nf Pres Pc7a

64/89

64


7/28/2019 02 Nf Pres Pc7a

65/89

65


In v.6, Aggregator, Rank, Joiner, and Sorter processedall input rows before emitting output rows

In v.7, these and the new Custom transformation canprocess data one transaction at a time

Benefits

Preserves transactions

Increased performance, less resource

7/28/2019 02 Nf Pres Pc7a

66/89

66

Transformation Scope

Agg, Rnk, Jnr, Srt

Transformation

Scope

Output

Transaction (added in v7) When commit encountered

All input (only v6 option) When all rows processed

Most transformations

Note: Custom transformations have whatever scopes are implemented by the developer

Row As each row is processed

7/28/2019 02 Nf Pres Pc7a

67/89

67

Example: Rank with Scope = All Input

In v6, a Rank transformation always has scope = All Input,dropping any incoming transactions

Name Salary

A1 $80K

A2 $40KA3 $50K

A4 $100K

COMMIT

A5 $30K

A6 $60K

A7 $90K

Rank on All Input

(Transactions

are dropped)

Name Salary

A4 $100K

A7 $90K

A1 $80K

A6 $60K

A3 $50K

A2 $40K

A5 $30K

7/28/2019 02 Nf Pres Pc7a

68/89

68

Example: Rank with Scope = Transaction

In v7, a Rank transformation with scope = Transactionpreserves incoming transactions

Name Salary

A1 $80K

A2 $40KA3 $50K

A4 $100K

COMMIT

A5 $30K

A6 $60K

A7 $90K

Rank on a set of

data bounded by

transactions

Name Salary

A4 $100K

A1 $80KA3 $50K

A2 $40K

COMMIT

A7 $90K

A6 $60K

A5 $30K

7/28/2019 02 Nf Pres Pc7a

69/89

69

Setting Transformation Scope

Transformation Scope

7/28/2019 02 Nf Pres Pc7a

70/89

70

Lab NF5


7/28/2019 02 Nf Pres Pc7a

71/89

71

New Functions and Datatypes

S d d M h F i

7/28/2019 02 Nf Pres Pc7a

72/89

72

Soundex and Metaphone Functions

Used in expressions

Create index based on English pronunciations, e.g.SOUNDEX(Smith) = SOUNDEX(Smyth)

Soundex

Encodes a string value into a four-character string (first inputcharacter plus 3 numbers for unique consonants)

Fast and standard

Metaphone

More accurate (but needs more computational power) Can specify length of string

Algorithm not standard

N D

7/28/2019 02 Nf Pres Pc7a

73/89

73

New Datatypes

To handle Oracle, DB2, and SQL Serverdatatypes, PowerCenter 7 supports:

blobLarge objects containing unstructuredbinary data

clobLarge objects containing single-bytefixed-width character data

nclobLarge binary objects containingsingle-byte or multiple-byte fixed-width

character data

xmltypeStructured XML data (Oracle only)

7/28/2019 02 Nf Pres Pc7a

74/89

WorkFlow Manager: Error

Logging Enhancement

74

E T

7/28/2019 02 Nf Pres Pc7a

75/89

75

Error Types

Transformation error

Data row has only passed partway through the mappingtransformation logic

An error occurs within a transformation

Data reject

Data row is fully transformed according to the mappinglogic

Due to a data issue, it cannot be written to the target A data reject can be forced by an Update Strategy

E L i Off/O

7/28/2019 02 Nf Pres Pc7a

76/89

76

Error Logging Off/On

Error Type Logging OFF (Default) Logging ON

Transformation

errors

Written to session log

then discarded

Appended to flat file or

relational tables. Only

fatal errors written to

session log.Data rejects Appended to reject file

(one .bad file per target)

Written to row error

tables or file

S tti E L O ti

7/28/2019 02 Nf Pres Pc7a

77/89

77

Setting Error Log Options

In Session task

Log Row Data

Log Source Row Data

Error Log Type

E L i Off S if i R j t Fil

7/28/2019 02 Nf Pres Pc7a

78/89

78

Error Logging OffSpecifying Reject Files

In Session task

1 file per target

E L i Off T f ti E

7/28/2019 02 Nf Pres Pc7a

79/89

79

Error Logging OffTransformation Errors

X

X

Transformation Error

Details and data are written to session log Data row is discarded

If data flows concatenated, corresponding rows in parallel

flow are also discarded

E L i Off D t R j t

7/28/2019 02 Nf Pres Pc7a

80/89

80

Error Logging OffData Rejects

Conditions causing data to be rejected include:

Target database constraint violations, out-of-space errors, log

space errors, null values not accepted

Data-driven records, containing value 3 or DD_REJECT

(the reject has been forced by an Update Strategy)

Target table properties reject truncated/overflowed rows

INSERT 0,1313,Regulator System,Air Regulators,250.00,150.00

UPDATE 1,1314,Second Stage Regulator,Air Regulators,365.00,265.00

DELETE 2,1390,First Stage Regulator,Air Regulators,170.00,70.00

REJECT 3,2341,Depth/Pressure Gauge,Small Instruments,105.00,5.00

Sample reject file

Log Ro Data

7/28/2019 02 Nf Pres Pc7a

81/89

81

Log Row Data

Logs:

Session metadata

Reader, transformation, writer and user-defined errors

For errors on input, logs row data forI and I/O ports

For errors on output, logs row data forI/O and O ports

Logging Errors to a Relational Database 1

7/28/2019 02 Nf Pres Pc7a

82/89

82


Relational DatabaseLog Settings


7/28/2019 02 Nf Pres Pc7a

83/89

83


PMERR_SESS: Stores metadata about the session runsuch as workflow name, session name, repository nameetc

PMERR_MSG: Error messages for a row of data are

logged in this table PMERR_TRANS: Metadata about the transformation such

as transformation group name, source name, port nameswith data types are logged in this table

PMERR_DATA: The row data of the error row as well asthe source row data is logged here. The row data is in astring format such as [indicator1: data1 | indicator2: data2]

Error Logging to a Flat File 1

7/28/2019 02 Nf Pres Pc7a

84/89

84

Error Logging to a Flat File 1

Flat File LogSettings (Defaults

shown)

Creates delimited Flat File with || as column delimiter

Logging Errors to a Flat File 2

7/28/2019 02 Nf Pres Pc7a

85/89

85

Logging Errors to a Flat File 2

Format: Session metadata followed by de-normalized error information

Sample session metadata**********************************************************************

Repository GID: 510e6f02-8733-11d7-9db7-00e01823c14d

Repository: RowErrorLogging

Folder: ErrorLogging

Workflow: w_unitTests

Session: s_customers

Mapping: m_customers

Workflow Run ID: 6079

Worklet Run ID: 0

Session Instance ID: 806

Session Start Time: 10/19/2003 11:24:16

Session Start Time (UTC): 1066587856

**********************************************************************

Row data format

Transformation || Transformation Mapplet Name || Transformation Group || PartitionIndex || Transformation Row ID || Error Sequence || Error Timestamp || Error UTCTime || Error Code || Error Message || Error Type || Transformation Data || SourceMapplet Name || Source Name || Source Row ID || Source Row Type || Source Data

Log Source Row Data 1

7/28/2019 02 Nf Pres Pc7a

86/89

86


Separate checkbox in session task

Logs the source row associated with the error row

Logs metadata about source, e.g. Source Qualifier,source row id, and source row type


7/28/2019 02 Nf Pres Pc7a

87/89

87


Source row loggingavailable

Source row loggingnot available

Source row logging is not available downstream of an

Aggregator, Rank, Joiner, Sorter (where output rows are notuniquely correlated with input rows)

7/28/2019 02 Nf Pres Pc7a

88/89

88

Lab NF6Error Logging

7/28/2019 02 Nf Pres Pc7a

89/89

02 nf pres pc7a

Documents