building a logical enterprise data warehouse from existing data warehouses brian beckman procter...
TRANSCRIPT
Building a Logical Enterprise Data Warehouse from Existing Data Warehouses
Brian BeckmanProcter & GambleSep 29, 2014
About Procter & Gamble• Countries of operation: ~70
• Countries where our brands are sold: ~180
• Consumers served by our brands: 4.8 billion (approximate)
• Last FY net sales: $83.1 billion
AgendaStarting Point
The Challenge…
…And Our Response
Keys to Success
1
2
3
4
Data Warehousing History
• Almost 20 years of global data warehousing
• Conformed internal dimensions
• Mature data warehouse environment delivering significant business value
• Constant incremental improvements, tuning, adaptation to changing business needs
Data Warehouse Evolution
• Evolved from multiple distributed Oracle DBs to consolidated multi-TB DBs on Exadata, SAP BW
• Dramatic reliability, performance improvements
• New tools being adopted (BI tools, Big Data, etc.)
However, we still have challenges• Too long, too expensive, “one-size fits all” mentality
• Multi data reporting speed and cost
• Agility gaps to enable self-serve analytics
• Data distributed across systems, internally and externally
• VUCA world in technology and business
As a result, our customers started to refer to our
warehouses as “Roach Motels”
Our ResponseAt a high level…there is no single silver bullet solution
but rather we need a:
Multi-pronged approach against an explicit architectural strategy
…addressing the primary root causes
…but built upon our successes
…yet with revolutionary changes
The Heart of our Solution
A logical enterprise data warehouse
The Logical Data Warehouse (LDW) is a new data management architecture for analytics which combines the strengths of traditional repository warehouses with alternative data management and access strategy.
LDW according to the Gartner Hype Cycle for Information Infrastructure, 2012
LDW Journey
Not a direct leap but an evolutionary journey which requires:
• A solid data foundation
• Strategy to address critical business needs first
• Treating data as an asset to be extended
• Recognition that technology is only part of the solution and technology evolution reinforces the value of a LDW
First…
Fix the “one-size-fits-all” mentality for projects and DW platforms that drove speed and cost issues
• Establish a prototyping platform
• Create rapid deployment project teams
Agile Approach to DW
• “Prototype plant” separate from standard production DW lifecycles
• Dedicated partner resources specialized in DW/BI development, ready in days
• Iterate to solutions with business partners, focusing on 80/20
Next…
Separate data from applications and address data harmonization (and thus speed of data integration) to create the logical DW foundation
• Define BI Master Data to establish a common language of golden attributes for data
• Map critical data sets to golden attributes to translate to the common language
United Nations Analogy
On-the-fly Translations
Japanese
Dutch
Spanish
UK English
US English
International English
Japanese
Dutch
Spanish
UK English
US English
Translate
One-way TranslationUK English: “BMW’s are splendid cars”
US English: “BMW’s are awesome cars”
US Boston English: “BMW’s are wicked awesome cars”
International English: “BMW’s are very good cars”
The meaning of the translation stays intact, but the exact original sentence can not be recreated
BI Master Data Concepts
• Good enough for business insights & decisions
• Not good enough to literally translate data back to the source (not intended to close the books)
• For external purposes we can translate our “International English” back to other languages, just like the UN does!
Powered by Golden Attributes
• Valid business values for key attributes across dimensions (e.g., product brands and categories, customer details, trade channels, time definitions, etc.)
• Independent of solutions and data sources
• Treat as additional master data available to further describe existing sources
MM
Ship POS MM
BI Master Data & MappingsShip
BI Tool Layer
ApplicationData Mart Layer
EDW Layer
SH + MM App
SH + MM App
Golden Attributes In Action
And then…
Consolidate all the data into one place…but since money and time are finite, do it virtually rather than physically
• Embrace a “play-it-where-it-lies” (PIWIL) mentality
• Implement data virtualization
Data Extractor
ReportingLayer
BI Tool Semantic Layer
ADW (Exadata)
Spotfire OBIEEMGRC
TemplateCust Suff Template
Info Links
Business ObjectsWebI
ReportWebI
ReportXcelsius
Xcelsius
Universe
Master Data
Product Customer
Geo Profit Center
Channel Legal Entity
Structured data sources
BW (->HANA)App Data Mart Layer
EDW Layer (Enterprise Data Model)
Unstructured data sources
Big Data
Impala, Parquet, new analytic tools, etc.
Govern
anceS
ecurityM
etadata
ShipmentsMarket
MeasuresPOS
MGRC Data Mart
Financials
FMRData Mart
MM Cubeless
Architecture
BI Master Data
Mappings
Custom Hier
Standard Standard StandardStandard
Data VirtualizationBI Tool AgnosticSemantic Layer
Demand/Consumer ViewsShare
Conformance Layer
Common Model “Harmonized Layer” - Integrated
Shipments Shares POS Financials
Ship ShareShip + Share
Ship + Share
MGRCPOS
Others
Finance OtherPOS
ProductGeo
Essbase
Data Virtualization in the BI Complex
Finally…
Determine how to insulate the DW environment from change….but since time cannot be stopped, instead:
• Pursue cloud concepts of elasticity as well as consumption based pricing for the DW
• Previous steps required for a DW cloud to be a reality
Keys to Success• Stabilize the foundation
• Embrace what has worked…
…but realize radical re-invention needed!
• Buy a sports car (fast team and environment)
• Mandate language courses (harmonize the data)
• Play golf (PIWIL)
• Bring an umbrella (cloud concepts)
Questions?