building a logical enterprise data warehouse from existing data warehouses brian beckman procter...

Building a Logical Enterprise Data Warehouse from Existing Data Warehouses

Brian BeckmanProcter & GambleSep 29, 2014

About Procter & Gamble• Countries of operation: ~70

• Countries where our brands are sold: ~180

• Consumers served by our brands: 4.8 billion (approximate)

• Last FY net sales: $83.1 billion

AgendaStarting Point

The Challenge…

…And Our Response

Keys to Success

1

2

3

4

Data Warehousing History

• Almost 20 years of global data warehousing

• Conformed internal dimensions

• Mature data warehouse environment delivering significant business value

• Constant incremental improvements, tuning, adaptation to changing business needs

Data Warehouse Evolution

• Evolved from multiple distributed Oracle DBs to consolidated multi-TB DBs on Exadata, SAP BW

• Dramatic reliability, performance improvements

• New tools being adopted (BI tools, Big Data, etc.)

However, we still have challenges• Too long, too expensive, “one-size fits all” mentality

• Multi data reporting speed and cost

• Agility gaps to enable self-serve analytics

• Data distributed across systems, internally and externally

• VUCA world in technology and business

As a result, our customers started to refer to our

warehouses as “Roach Motels”

Our ResponseAt a high level…there is no single silver bullet solution

but rather we need a:

Multi-pronged approach against an explicit architectural strategy

…addressing the primary root causes

…but built upon our successes

…yet with revolutionary changes

The Heart of our Solution

A logical enterprise data warehouse

The Logical Data Warehouse (LDW) is a new data management architecture for analytics which combines the strengths of traditional repository warehouses with alternative data management and access strategy.

LDW according to the Gartner Hype Cycle for Information Infrastructure, 2012

LDW Journey

Not a direct leap but an evolutionary journey which requires:

• A solid data foundation

• Strategy to address critical business needs first

• Treating data as an asset to be extended

• Recognition that technology is only part of the solution and technology evolution reinforces the value of a LDW

First…

Fix the “one-size-fits-all” mentality for projects and DW platforms that drove speed and cost issues

• Establish a prototyping platform

• Create rapid deployment project teams

Agile Approach to DW

• “Prototype plant” separate from standard production DW lifecycles

• Dedicated partner resources specialized in DW/BI development, ready in days

• Iterate to solutions with business partners, focusing on 80/20

Next…

Separate data from applications and address data harmonization (and thus speed of data integration) to create the logical DW foundation

• Define BI Master Data to establish a common language of golden attributes for data

• Map critical data sets to golden attributes to translate to the common language

United Nations Analogy

On-the-fly Translations

Japanese

Dutch

Spanish

UK English

US English

International English

Japanese

Dutch

Spanish

UK English

US English

Translate

One-way TranslationUK English: “BMW’s are splendid cars”

US English: “BMW’s are awesome cars”

US Boston English: “BMW’s are wicked awesome cars”

International English: “BMW’s are very good cars”

The meaning of the translation stays intact, but the exact original sentence can not be recreated

BI Master Data Concepts

• Good enough for business insights & decisions

• Not good enough to literally translate data back to the source (not intended to close the books)

• For external purposes we can translate our “International English” back to other languages, just like the UN does!

Powered by Golden Attributes

• Valid business values for key attributes across dimensions (e.g., product brands and categories, customer details, trade channels, time definitions, etc.)

• Independent of solutions and data sources

• Treat as additional master data available to further describe existing sources

MM

Ship POS MM

BI Master Data & MappingsShip

BI Tool Layer

ApplicationData Mart Layer

EDW Layer

SH + MM App

SH + MM App

Golden Attributes In Action

And then…

Consolidate all the data into one place…but since money and time are finite, do it virtually rather than physically

• Embrace a “play-it-where-it-lies” (PIWIL) mentality

• Implement data virtualization

Data Extractor

ReportingLayer

BI Tool Semantic Layer

ADW (Exadata)

Spotfire OBIEEMGRC

TemplateCust Suff Template

Info Links

Business ObjectsWebI

ReportWebI

ReportXcelsius

Xcelsius

Universe

Master Data

Product Customer

Geo Profit Center

Channel Legal Entity

Structured data sources

BW (->HANA)App Data Mart Layer

EDW Layer (Enterprise Data Model)

Unstructured data sources

Big Data

Impala, Parquet, new analytic tools, etc.

Govern

anceS

ecurityM

etadata

ShipmentsMarket

MeasuresPOS

MGRC Data Mart

Financials

FMRData Mart

MM Cubeless

Architecture

BI Master Data

Mappings

Custom Hier

Standard Standard StandardStandard

Data VirtualizationBI Tool AgnosticSemantic Layer

Demand/Consumer ViewsShare

Conformance Layer

Common Model “Harmonized Layer” - Integrated

Shipments Shares POS Financials

Ship ShareShip + Share

Ship + Share

MGRCPOS

Others

Finance OtherPOS

ProductGeo

Essbase

Data Virtualization in the BI Complex

Finally…

Determine how to insulate the DW environment from change….but since time cannot be stopped, instead:

• Pursue cloud concepts of elasticity as well as consumption based pricing for the DW

• Previous steps required for a DW cloud to be a reality

Keys to Success• Stabilize the foundation

• Embrace what has worked…

…but realize radical re-invention needed!

• Buy a sports car (fast team and environment)

• Mandate language courses (harmonize the data)

• Play golf (PIWIL)

• Bring an umbrella (cloud concepts)

Questions?