oracle rac 12c rel. 2 under the hood and best practices

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |

Oracle Real Applica@ons Cluster (RAC) 12c Release 2 – Under the Hood & Best Prac@ces

Markus Michalewicz Senior Director of Product Management, Oracle RAC Development

[email protected] @OracleRACpm hSp://www.linkedin.com/in/markusmichalewicz hSp://www.slideshare.net/MarkusMichalewicz


Safe Harbor Statement The following is intended to outline our general product direc@on. It is intended for informa@on purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or func@onality, and should not be relied upon in making purchasing decisions. The development, release, and @ming of any features or func@onality described for Oracle’s products remains at the sole discre@on of Oracle.

3


“Under the Hood” vs. “Best Prac@ces”

Under the Hood – You need to know Best Prac5ces – You need to do

4

MPG: 25-‐28

Front-‐wheel Dr.

178-‐hp, 2.5-‐liter I-‐4




5

MPG: 25-‐28

Front-‐wheel Dr.

178-‐hp, 2.5-‐liter I-‐4


Program Agenda

Under the Hood Installa@on Changes

Under the Hood Opera@onal Changes

Installa@on Best Prac@ces

Opera@onal Best Prac@ces

Oracle RAC Support in the Cloud

1

2

3

4

5

6


Program Agenda






1

2

3

4

5

7

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 8

Simple Flow Oracle Grid Infrastructure (GI) & Database (DB) Deployment

SoAware installa5on

1 GI Configura5on

2

3 DB Crea5on 4


Reality – mul5ple installa5on steps Oracle Grid Infrastructure (GI) Deployment

✔

4

t Base SoAware installa5on

e.g. 12.1.0.2

1

B

PSU applica5on e.g. 12.1.0.2.x

2

PSU

OneOff applica5on e.g. PatchXYZ

3

OneOff


Requirement: gold image support Oracle Grid Infrastructure (GI) Deployment

✔ Base

SoAware installa5on e.g. 12.1.0.2

1 4

t PSU applica5on e.g. 12.1.0.2.x

2


3

B PSU OneOff


Oracle Grid Infrastructure Installer 12c Release 2

•  Star@ng with Oracle Grid Infrastructure 12c Rel. 2, the Grid Installer uses a gold image instead of the tradi@onal shiphome as installa@on media

•  The zip-‐file available for download is therefore a 12.2.0.1 base gold image •  The launch script is gridSetup.sh or gridSetup.bat • User-‐created (customized) gold images are supported

– Customiza5on can include PSUs and / or addi5onal patches

•  The new deployment process has no impact on any post-‐install sodware lifecycle management opera@ons; e.g. adding nodes, cloning, patching

11

Gold image-‐based installa5on


Requirement: gold image support Oracle Grid Infrastructure (GI) Deployment

✔ Base

SoAware installa5on e.g. 12.1.0.2

1 4

t PSU applica5on e.g. 12.1.0.2.x

2


3

B PSU OneOff

Create Customized Gold Image


Oracle Database Installer will Follow

13

Future OUI enhancement

More informa@on about OUI New Features: hSps://www.slideshare.net/MarkusMichalewicz/new-‐in-‐oracle-‐universal-‐installer-‐oui


Install Steps in detail

14


•  The depreca@on of Oracle Restart has been withdrawn effec@ve July 10th 2017.

•  MOS Note 1584742.1, formerly @tled “Support Impact of the Depreca@on Announcement of Oracle Restart”, has been renamed accordingly: •  Withdrawn: Depreca@on Announcement of

Oracle Restart with Oracle Database 12c

15

Oracle Restart is Back – For Good!


•  Oracle Grid Infrastructure 12 Rel. 2 offers two cluster architectures: 1.  Standalone Cluster

•  This is the tradi@onal cluster as offered with previous versions

2.  Domain Services Cluster 1.  with Member Clusters for Databases 2.  with Member Clusters for Applica@ons

•  Also supported: •  Oracle Extended Clusters

•  hSps://www.slideshare.net/MarkusMichalewicz/oracle-‐extended-‐clusters-‐for-‐oracle-‐rac

16

More Cluster Installa@on Choices


Any New Oracle 12c Rel. 2 Cluster will be a Flex Cluster

[GRID]> crsctl get cluster name CRS-‐6724: Current cluster name is 'SolarCluster' [GRID]> crsctl get cluster class CRS-‐41008: Cluster class is 'Standalone Cluster' [GRID]> crsctl get cluster type CRS-‐6539: The cluster type is 'flex'.


Introduced during OOW 2013: hSp://www.slideshare.net/

MarkusMichalewicz/understanding-‐oracle-‐rac-‐12c-‐internals-‐oow13-‐con8806

18

Recommended during OOW 2014:

hSp://www.slideshare.net/MarkusMichalewicz/oracle-‐

rac-‐12102-‐opera@onal-‐best-‐prac@ces

The standard going forward (every Oracle 12c Rel. 2 cluster is a Flex Cluster by default, as an all-‐Hub Flex Cluster is equivalent

to the previous model )

Oracle Flex Cluster -‐ A Brief Review


•  Oracle Flex Cluster in 12c Rel. 1 required the configura@on of the GNS in general •  One GNS IP with name resolu@on

•  GNS in a Flex Cluster is required to operate Leaf Nodes as part of the cluster

•  Oracle Grid Infrastructure 12c Rel. 2 only requires a GNS when Leaf Nodes are added to the cluster •  Either during installa@on or later

19

Oracle Flex Cluster – With or Without GNS?


•  Star@ng with Oracle Grid Infrastructure (GI) 12c Rel. 2, all GI-‐related files need to be managed by Oracle ASM:

•  hSps://docs.oracle.com/database/122/CWLIN/changes-‐in-‐this-‐release-‐for-‐oracle-‐grid-‐infrastructure-‐installa@on-‐guide.htm#GUID-‐8D8FBD22-‐8744-‐4580-‐A711-‐6395B0F815CF

•  Desupport of Direct File System Placement for Oracle Cluster Registry (OCR) and Vo@ng Files

•  If your Oracle Database files are stored on a shared file system, then you can con@nue to use shared file system storage for database files, instead of moving them to Oracle ASM storage.

•  This affects NFS and Cluster File System (CFS) customers only.

•  Transi@on is smooth: •  For new installa@ons, provide file system (FS)

and installer will create files on the FS to be used for ASM disks.

•  For upgrades, manual prepara@on is required.

20

Oracle ASM is Required to Manage Clusterware Files


•  Star@ng with Oracle Grid Infrastructure (GI) 12c Rel. 2, the GIMR can be installed into a different disk group than vo@ng files and the OCR.

•  The size requirements for the GIMR have increased significantly.

21

Oracle Grid Infrastructure Management Repository (GIMR)

External Redundancy


•  GIMR is the heart of the Autonomous Health Framework (AHF)

•  In the AHF, GIMR is used as a centralized diagnos@c data repository, used by various tools

•  For example: •  Cluster Health Monitor (CHM) •  Cluster Health Advisor (CHA)

•  GIMR data can be used to op@mize the system and prevent failures at run@me using CHA-‐based analysis and for post-‐mortem failure analysis.

22

GIMR – Storage Space Well Worth Inves@ng In

Diagnos@c Data


•  Flex Diskgroups enable •  Quota Management -‐ limit the space databases

can allocate in a diskgroup and thereby improve the customers’ ability to consolidate databases into fewer DGs

•  Redundancy Change – u@lize lower redundancy for less cri@cal databases

•  Shadow Copies (“split mirrors”) to easily and dynamically create database clones for test/dev or produc@on databases

23

Oracle ASM – Flex Diskgroup and Redundancy

Flex Diskgroup

DB1 File 1 File 2 File 3

DB2 File 1 File 2 File 3 File 4

DB3 File 1 File 2 File 3

Quota DB3

File 1

File 2

File 3

hSps://www.slideshare.net/MarkusMichalewicz/whats-‐new-‐and-‐coming-‐in-‐oracle-‐asm-‐12c-‐rel-‐2-‐by-‐jim-‐williams


Program Agenda






1

2

3

4

5

24


•  Pre-‐12.2, node evic@on follows a rather “ignorant” paSern –  Example in a 2-‐node cluster: The node with the lowest node number survives.

•  Customers must not base their applica@on logic on which node survives the split brain. –  As this may(!) change in future releases

Node Evic@on Basics hhp://www.slideshare.net/MarkusMichalewicz/oracle-‐clusterware-‐node-‐management-‐and-‐vo5ng-‐disks

✔ 1 2


•  Node Weigh@ng is a new feature that considers the workload hosted in the cluster during fencing

•  The idea is to let the majority of work survive, if everything else is equal –  Example: In a 2-‐node cluster, the node hos@ng the

majority of services (at fencing @me) is meant to survive

Node Weigh@ng in Oracle RAC 12c Release 2 Idea: Everything equal, let the majority of work survive

✔ 1 2


A three node cluster will benefit from “Node Weigh@ng”, if three equally sized sub-‐clusters are built as s result of the failure, since two differently sized sub-‐clusters are

not equal.

27

Secondary failure considera5on can influence which node survives. Secondary failure considera@on will be enhanced successively.

A fallback scheme is applied if considera@ons do not lead to an ac@onable outcome.

Let’s Define “Equal”

✔

Public network card failure. “Conflict”.


CSS_CRITICAL can be set on various levels / components to mark them as

“cri@cal” so that the cluster will try to preserve them in case of a failure.

28

CSS_CRITICAL will be honored if no other technical reason prohibits survival of the node which has at least one cri@cal

component at the @me of failure.

A fallback scheme is applied if CSS_CRITICAL sewngs do not lead

to an ac@onable outcome.

CSS_CRITICAL – Fencing with Manual Override

crsctl set server css_cri5cal {YES|NO}

+ server restart

srvctl modify database -‐help |grep cri@cal

… -‐css_cri@cal {YES | NO}

Define whether the database or service is CSS cri@cal

✔ Node evic@on despite WL; WL will failover.

“Conflict”.


•  Leaf nodes require at least one Hub node in the cluster to which they can connect.

•  If a Hub node fails, all Leaf nodes connected to the failed Hub node re-‐connect to another Hub. –  Failover is transparent …

•  … on cluster level. •  … on the Leaf nodes. •  … for instances running on the Leaf nodes.

Last but Not Least – Leaf Node Failover

Earth Venus Oracle GI | HUB Oracle GI | HUB

Oracle RAC Oracle RAC

29

Mars Oracle GI | Leaf

Oracle RAC


Scalability

30


RAC Scalability is Con@nuously “Upgraded”

31

Tuning under the hood

Supercharger

Performance Exhaust System

Air Intake System

Each step alone provides marginal improvements. Together they make a (huge) difference.


Oracle RAC Scalability – A Brief Review

Oracle RAC scalability •  is independent of the

number of nodes •  does not require

applica@on changes (unlike sharding)

Oracle RAC scales •  most of the

enterprise solu@ons used today ✔

Oracle RAC scales •  Oracle Mul@tenant

Oracle RAC scales •  Oracle Database

In-‐Memory

hhp://www.slideshare.net/MarkusMichalewicz


Cache Fusion

33

A quick refresher

•  Maximum 3-‐way communica@on

•  Dynamic Resource Management (DRM) aSempts to op@mize down to 2-‐way communica@on by moving the master to the instance where the resource is frequently accessed


Small Changes Making a Huge Difference – Part 1 Introducing LMS slaves

•  Before 12c Rel. 2, LMS works on incoming consistent read requests sequen@ally –  Sessions reques@ng consistent blocks that require applying a lot of undo may cause LMS to be busy

• With Oracle RAC 12c Release 2, LMS offloads work to ‘CR slaves’ –  if the amount of UNDO to be applied exceeds a certain, dynamic threshold

•  Default is 1 slave and addi@onal slaves are spawned as needed

34


Small Changes Making a Huge Difference – Part 2 SSD-‐op5mized data retrieval

35

•  RAC determines the op@mal path to serve blocks – network or disk

•  SSDs and NVMe storage technology con@nue to drive down latency

•  e.g. flash storage may provide beSer access @mes to data than the private network under high load

•  RAC takes those sta@s@cs into account

Query

Block Block

Network conges@on


101010100001001110010100110111001010011100101001010010010100001111010100101

36

•  Using Oracle Mul@tenant, PDBs can be opened as singletons (in one database instance only), in a subset of instances or all in instances at once.

•  If certain PDBs are only opened on some instances, Pluggable Database Isola@on –  improves performance by

•  Reducing DLM opera@ons for PDBs not open in all instances.

•  Op@mizing block opera@ons based on in-‐memory block separa@on.

–  improves availability by •  Ensuring that instance failures of instances only hos@ng singleton PDBs will not impact other instances of the same RAC-‐based CDB.

Op@mized Singleton Workload Scaling Pluggable Database / Service Isola@on

NEW IN 12.2


•  Service-‐oriented Buffer Cache Access over @me determines the data (on database object level) accessed by the service. This informa@on –  Is persisted in the database. –  Is used to improve data access performance (e.g. do not manage data of a service in an instance that does not host the service).

–  Can be used to pre-‐warm an instance cache prior to a service startup (fresh start or reloca5on).

Op@mized Singleton Workload Scaling Service-‐oriented Buffer Cache Access


Availability

38


RAC Availability – Reconfigura@on Times MaSer

39

Tuning under the hood


•  Recovery Buddies •  Track block changes on buddy instance

•  Quickly iden@fy blocks requiring recovery during reconfigura@on

•  Allow rapid processing of transac@ons ader failures

Near Zero Reconfigura@on Time with Recovery Buddies A.k.a. Buddy Instances


•  Buddy Instance mapping is simple (random) –  e.g. I1 à I2, I2 à I3, I3 à I4, I4 à I1

•  Recovery buddies are assigned during startup •  RMS0 on each recovery buddy instance maintains an in-‐memory area for redo log change

•  The in-‐memory area is used during recovery –  Eliminates the need to physically read the redo

Near Zero Reconfigura@on Time with Recovery Buddies How it works under the hood

Instance I1

Instance I2

Instance I3

Instance I4

Recovery Buddy I3

Recovery Buddy I4

Recovery Buddy I1

MyCluster

Recovery Buddy I2


How Recovery Buddies Help Reducing Recovery Time

Without Recovery Buddies With Recovery Buddies

42

Detect

Evict

Elect Recovery

Read Redo

Apply Recovery

Detect

Evict

Elect Recovery

Read Redo

Apply Recovery

Up to 4x

faster


Overlooked and Underes@mated – Hang Manager

• Customers experience database hangs for a variety of reasons –  High system load, workload conten@on, network conges@on, general errors, etc.

• Before Hang Manager was introduced with Oracle RAC 11.2.0.2 –  Oracle required quite some informa@on to troubleshoot a hang -‐ e.g.:

•  System state dumps •  For RAC: global system state dumps

–  Customer usually had to reproduce “the” hang with addi@onal events to analyze it

43

Why having a Hang Manager is useful


•  Always on, as enabled by default •  Reliably detects database hangs •  Autonomically resolves them

•  Considers QoS policies for Hang Resolu@on •  Logs all detected hangs & their resolu@ons

Introduc@on to Hang Manager How it works Session

DIAG0

EVALUATE

DETECT

ANALYZE

Hung?

VERIFY

Vic5m

QoS Policy


•  Hang Manager auto-‐tunes itself by periodically collec@ng instance-‐and cluster-‐wide hang sta@s@cs

•  Metrics like Cluster Health/Instance health is tracked over a moving average

•  This moving average is considered during resolu@on

•  Holders wai@ng on SQL*Net break/reset are fast tracked

Hang Manager Op@miza@ons with Oracle RAC 12c Tuning under the hood


•  Early Warning exposed via (V$ view)

•  Sensi@vity can be set higher –  If the default level is too conserva@ve

•  Hang Manager considers QoS policies and data during the valida@on process

DBMS_HANG_MANAGER.Sensi@vity A new SQL interface to set Hang Manager sensi@vity

Hang Sensi5vity Level

Descrip5on Note

NORMAL Hang Manager uses its default internal opera@ng parameters to try to meet typical requirements for any environments.

Default

HIGH Hang Manager is more alert to sessions wai@ng in a chain than when sensi@vity is in NORMAL level.


Program Agenda






1

2

3

4

5

47




48

MPG: 25-‐28

Front-‐wheel Dr.

178-‐hp, 2.5-‐liter I-‐4




49

MPG: 25-‐28

Front-‐wheel Dr.

178-‐hp, 2.5-‐liter I-‐4


Oracle’s Goal

Best Prac5ces – You need to do Best Prac5ces – Applied for you

50


Installa@on Best Prac5ces

51


•  Oracle Grid Infrastructure 12 Rel. 2 offers two cluster architectures: 1.  Standalone Cluster

•  This is the tradi@onal cluster as offered with previous versions

2.  Domain Services Cluster 1.  with Member Clusters for Databases 2.  with Member Clusters for Applica@ons

•  Also supported: •  Oracle Extended Clusters

•  hSps://www.slideshare.net/MarkusMichalewicz/oracle-‐extended-‐clusters-‐for-‐oracle-‐rac

52

Cluster Architecture – Your Choice


Different Architectures – Different Purposes

Standalone Cluster

Tradi@onal architecture for dedicated and high performance systems; Flex Cluster support

Cluster Domain Architecture

Domain Services Cluster (DSC) & Member Clusters for large cluster estate and storage consolida@on

Extended Cluster

Site-‐awareness for Standalone Clusters and DSC;

Member Clusters not supported

53


Best Prac@ces (BP) Across Architectures

54

•  BP are designed for Standalone Cluster Deployments

•  A DSC is a Standalone Cluster that offers addi@onal services to Member Clusters •  BP for Standalone Clusters apply to DCSs

•  Member Clusters come with a simplified deployment. Basic BP apply to Member Clusters

•  Extended Clusters for RAC come with an addi@onal set of BP: •  hSps://www.slideshare.net/MarkusMichalewicz/

oracle-‐extended-‐clusters-‐for-‐oracle-‐rac

= +


How to Apply Best Prac@ces?

Read & apply Oracle Documenta@on

and My Oracle Support Notes

Run & apply Cluster Verifica@on U@lity (CVU)

and Oracle Universal Installer (OUI)

Run & follow

ORAchk

55


Each “Tool” has a Different Purpose

Establish the base system; e.g.: hardware and OS minimum requirements

Check and confirm baseline; e.g. check OS for minimum parameter requirements; supports diff comparison

Check for and then implement recommended best prac@ces

56


Recommended Usage – Automated Risk No@fica@on 1)  Schedule ORAchk to run in daemon

mode weekly or daily and email report

2)  Iden5fy ac@ons easily by viewing automated comparison of previous runs

3)  Act on recommenda@ons

57


View ORAchk Report

•  Health score •  Summary of ORAchk run

•  Table of content •  Controls for report features •  Findings •  Recommenda.ons

58


View ORAchk Findings

•  Check status •  Type of Check •  Check Message

• Where the check was run

•  Link to expand details

59


•  What to do to solve the problem

•  Links to relevant Knowledge docs •  Where recommenda@on applies

•  Where problem doesn’t apply

•  Example of data the recommenda@on is based on

View Recommenda@ons

60


Cluster Domain Installa@on Installa5on Steps

61


Step 1: Domain Services Cluster (DSC) Installa@on


Status: DSC – Installed ✔

63

Domain Services Cluster

ASM IO Service

ACFS Services

ASM Service

Trace File Analyzer (TFA) Service

Mgmt Repository (GIMR) Service

Shared ASM

Addi5onal Op5onal Services

Rapid Home Provisioning

(RHP) Service

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. 64

The Manifest File defines the characteris5cs of the Member Cluster Step 2: Member Cluster (MC) Manifest File Crea@on

$ssh <DSC_host> $cd $GRID_HOME/bin

$./crsctl create -hUsage: crsctl create policyset -file <filePath>

$./crsctl create member_cluster_configuration salesmb –file $HOME/sales_mb.xml

$scp $HOME/sales_mb.xml MC_host:/tmp

Define which DSC services the MC can use

Create MC-‐specific Manifest File

Copy Manifest File to MC installa@on node (host)

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Step 3: Member Cluster (MC) Installa@on

/tmp/sales_mb.xml MyScan



ASM IO Service

ACFS Services

ASM Service

Database Member Cluster

Uses ASM Service



Shared ASM



(RHP) Service

Private Network SAN Storage ASM Network Storage

Status: MC – Installed ✔



Cluster Domain

ASM IO Service

ACFS Services

ASM Service


Uses ASM Service


Uses ASM IO Service of DSC



Applica5on Member Cluster

GI only


Uses local ASM

Shared ASM



(RHP) Service

Private Network SAN Storage ASM Network Storage

1 2 3 4


Cluster Domain Summary

68

•  A cluster is a member cluster, if it uses the GIMR service provided by Domain Services cluster

•  The architecture requires a fresh install –  No upgrade from a 12.1 GI typical install to the Cluster Domain model; conversion planned

•  Storage configura@on types: – Member Cluster has storage that is shared amongst its nodes but local to that MC

– Member Cluster and use the shared storage on the Domain Services Cluster

Domain Services Cluster ASM IO

Service

ASM Service TFA GIMR

Shared ASM

RHP


•  GIMR is the heart of the Autonomous Health Framework (AHF)

•  In the AHF, GIMR is used as a centralized diagnos@c data repository, used by various tools

•  For example: •  Cluster Health Monitor (CHM) •  Cluster Health Advisor (CHA)

•  GIMR data can be used to op@mize the system and prevent failures at run@me using CHA-‐based analysis and for post-‐mortem failure analysis.

•  In the Cluster Domain architecture the GIMR is installed centrally on the Domain Services Cluster

69

GIMR – Centralized Diagnos@c Data in the Cluster Domain

Diagnos@c Data


Standalone Cluster Installa@on Like DSC – Consider GIMR Loca5on

70


GIMR Placement Considera@ons Try to place, not to relocate the GIMR

•  For fresh installa@ons, GIMR can be placed in dedicated diskgroups –  Default in 12.2: Vo@ng Disk diskgroup

•  For upgrades, loca@on of the GIMR is determined by the current loca@on –  Usually OCR containing diskgroup, unless the GIMR or the OCR were relocated

•  Excep@ons: –  If upgrading from a GI version that is meant to contain a GIMR, but does not, the GIMR will be put in the Vo@ng Disk containing disk group

71

External Redundancy


Program Agenda






1

2

3

4

5

72


Cloud Challenge – Scale

73

Server

Server Server Server

Server Server Server Server

Server Server Server Server Server Server Server Server

Server Server Server Server Server



Handling Scale Through Focus

74

Server

Server Server Server

Server Server Server Server

Server Server Server Server Server Server Server Server



The challenge is to iden@fy troubled servers in @me to react appropriately given human reac@on

@me.


Oracle Autonomous Health Framework

•  Integrates next genera@on tools running as components -‐ 24/7

• Discovers Poten@al Issues and No@fies or takes Correc@ve Ac@ons

•  Speeds up Issue Diagnosis and Recovery • Preserves Database and Server Availability and Performance

• Autonomously Monitors and Manages resources to maintain SLAs

75

Working for You Con5nuously


Establishes Baseline and Maintains Best Prac5ce Configura5ons Oracle Cluster Verifica@on U@lity (CVU)

•  Always on -‐ Enabled by default •  Checks O/S, GI and DB mandatory

compliance and best prac@ces with healthcheck op@on

•  Runs as daemon every 6 hours •  ASM Best Prac@ces Check •  Create Baseline collec@ons •  Add user-‐specified/disable

problem checks •  New user-‐friendly report format


Schedule ORAchk

?

Hour (0 – 23)

? ? ?

Day of month (1 – 31)

Month (1 – 12)

Day of week (0 – 6) (0 to 6 are Sunday to Saturday)

example: orachk -‐set 'AUTORUN_SCHEDULE=8,20 * * 2,5' will schedule runs on Tuesday and Friday at 08:00 & 20:00

AUTORUN_SCHEDULE =

77 77


Generates Diagnos5c Metrics View of Cluster and Databases Cluster Health Monitor (CHM)

•  Always on -‐ Enabled by default •  Provides Detailed OS Resource Metrics

•  Assists Node evic@on analysis •  Locally logs all process data •  User can define pinned processes •  Listens to CSS and GIPC events •  Categorizes processes by type •  Supports plug-‐in collectors (ex. traceroute, netstat, ping, etc.)

•  New CSV output for ease of analysis

GIMR

ologgerd (master)

osysmond

12c Grid Infrastructure Management Repository

OS Data

osysmond

osysmond

OS Data

OS Data


Introducing Oracle 12c Cluster Health Advisor (CHA)

• Real @me monitoring of Oracle RAC database systems and their hosts •  Early detec@on of impending as well as ongoing system faults • Diagnoses and iden@fies the most likely root causes • Provides correc@ve ac@ons for targeted triage. • Generates alerts and no@fica@ons for rapid recovery

79

Proac5ve Health Prognos5cs System

hSps://www.slideshare.net/MarkusMichalewicz/cluster-‐health-‐advisor-‐cha-‐deep-‐dive-‐by-‐mark-‐scardina


Cluster Health Advisor (CHA) Architecture Overview

80

OS Data

GIMR

ochad

DB Data

CHM

Node Health

Prognos5cs Engine

Database Health

Prognos5cs Engine

OS Model

DB Model

•  cha – Cluster node resource •  Single Java ochad daemon per node

•  Reads Cluster Health Monitor data directly from memory

•  Reads DB ASH data from SMR w/o DB connec@on

•  Uses OS and DB models and data to perform prognos@cs

•  Stores analysis and evidence in the GI Management Repository

•  Sends alerts to EMCC Incident Manager per target

EMCC Alert


Scalability

81


Scale Read-‐only Workload Reliably with RAC Reader Nodes

Use Read-‐Only workload (WL) on read-‐mostly Leaf node instances for ad hoc data analysis scaled across hundreds of nodes with no delay in

accessing updated data, without any impact on OLTP performance and with beSer HA*

82

* A Leaf node failure does not impact any other node.


Introduced during OOW 2013: hSp://www.slideshare.net/

MarkusMichalewicz/understanding-‐oracle-‐rac-‐12c-‐internals-‐oow13-‐con8806

83

Recommended during OOW 2014:

hSp://www.slideshare.net/MarkusMichalewicz/oracle-‐

rac-‐12102-‐opera@onal-‐best-‐prac@ces

The standard going forward (every Oracle 12c Rel. 2 cluster is a Flex Cluster by default, as an all-‐Hub Flex Cluster is equivalent

to the previous model )

Oracle Flex Cluster -‐ A Brief Review Reminder


Connect Leaf nodes to storage Leaf nodes for applica@ons do not require direct storage access;

running database instances on Leaf nodes does.

84

Install Oracle Database Home on all nodes and as needed.

If you ever want to run a database instance on a Leaf node, it needs a database home as any other node.

Extend public network to Leaf(s) For RAC Reader Nodes use case only, enable a public network connec@on on Mars by extending the network and listener resources to the leaf.

Run a Database Instance on a Leaf Node – Prepara@on


Create a Policy-‐Managed RAC DB RAC Reader Nodes require a Policy-‐Managed database. Admin-‐managed DBs cannot be extended to Leafs.

85

Create database on HUB nodes the addi@on of database instances on Leaf nodes is

dynamic and managed via command line.

Run a Database Instance on a Leaf Node – DB Crea@on

Policy management allows for an easy re-‐assignment of a Leaf nodes to other tasks.

Serverpool OLTP was pre-‐created using the oracle user.


Add a “Reader Farm” (RF) pool to the system using the “add service” command (dynamic).

86

Summary Connect

Run RAC Reader Nodes – Finaliza@on

(Re-‐)star@ng the OLTPWL Service

finalizes the DWHWL service setup.


Oracle Database In-‐Memory – ideal for RAC Reader Nodes

87

It’s easy to run Oracle Database In-‐Memory on Leaf Nodes:

alter system set

inmemory_size=100M scope=spfile sid=‘*’;

Emphasizing Leaf Node Usage

by using instance-‐specific sewngs is “work in progress”

Run Oracle Database In-‐Memory on Leaf Node Instances

The IMDB Colum Store will be ac@vated ader instance restart.

A min. 100MB Column Store size

is required.

Column Stores need to be equally sized across all instances.

select INST_ID, pool, alloc_bytes, alloc_bytes, used_bytes from GV$INMEMORY_AREA;


Availability

88


Improving Planned Maintenance

• Registered databases do not use resources •  Star@ng databases may temporarily use more resources than running databases – The same applies to stopping databases

• A system to host a maximum amount of running databases may s@ll have issues with all of them star@ng or stopping

•  Solu@on: stagger startup and shutdowns – Addi@onally: drain workload prior to stopping

89

Drain and stagger startups as well as shutdowns

Registered: •  n DB instances are defined to run on a machine (poten@ally)

Running: •  Registered databases and instances are (concurrently) running (ac@ve workload)

Star5ng: •  Registered databases and instances start •  Default is “star@ng at the same @me”.


Program Agenda






1

2

3

4

5

90


Oracle RAC is fully cer5fied in the Oracle Cloud via the Exadata

Service and as an DBaaS offering.

91

Azure provides an ability to run Oracle RAC over a min. of

three Azure IaaS instances.

AWS provides an ability to run Oracle RAC over a min. of

three EC2 IaaS instances.

Support Basics for Oracle RAC “in the Common Clouds”

hSps://www.slideshare.net/MarkusMichalewicz/how-‐to-‐use-‐oracle-‐rac-‐in-‐a-‐cloud-‐a-‐support-‐ques@on


It describes the ability to u5lize compute instances as shared storage in some way What does “ability to run RAC over 3 IaaS instances” mean?

•  Most clouds do not provide shared storage na@vely. •  “Na@vely” means “as part of their standard offer”.

•  RAC requires shared storage. Per RTCM: •  iSCSI and NFS are supported for Oracle RAC •  Restric@on: “The accessed storage must be

supported by the system and storage vendors.”

•  Using one instance as an NFS / iSCSI server, which provides shared storage to “RAC compute nodes” is a “supportable” setup following RTCM regula@on.

•  As long as the OS, network and other layers are supported, such a configura@on is supportable. •  Special ruling s@ll needs to be considered.


Strongly recommended to operate Oracle RAC in any cloud Na@vely Provided Shared Storage

•  Cloud vendors are asked by their customers to support Oracle RAC as part of their cloud offerings.

•  Other than the Oracle Cloud, no other cloud offering currently provides shared storage support as required by Oracle RAC.

•  Mul@ple non-‐na@ve solu@ons have been used by different cloud vendors to provide shared storage. •  Such solu@ons include virtual shared storage

as well as physical third party storage solu@ons.

•  As ease of support and manageability decreases with every addi@onal non-‐na@ve layer used to operate an Oracle RAC database, na@vely provided shared storage is strongly recommended to operate Oracle RAC in any cloud.


Puwng “1+1” Together – Example

Can I use the FlashGrid sodware to enable Oracle RAC in either AWS or the

Azure Cloud? Is it supported?

Determining Support: 1.   FlashGrid SoAware

•  FlashGrid Cloud Area Network provides a high-‐speed network overlay with mul@cast and QoS

•  Local drives (elas5c block storage or local SSDs) are shared between all nodes in the cluster

2.   Oracle RAC on Amazon EC2

•  AWS supports the AWS infrastructure components on which FlashGrid's solu5on relies.

•  For ques@ons about FlashGrid sodware and support capabili@es, please see the FlashGrid website…

NO, it’s not supported, as AWS and Azure as the

storage and server vendors do not support FlashGrid as a storage solu@on as required by the RTCM:


Special Ruling Affec@ng Oracle RAC Support in the Cloud

•  The “Licensing Oracle Sodware in the Cloud Compu@ng Environment” document states:

•  hSp://www.oracle.com/us/corporate/pricing/cloud-‐licensing-‐070579.pdf

•  This policy applies to cloud compuLng environments from the following vendors: Amazon Web Services – Amazon Elas.c Compute Cloud (EC2), Amazon Rela.onal Database Service (RDS) and MicrosoF Azure PlaHorm (collecLvely, the ‘Authorized Cloud Environments’) This policy applies to these Oracle programs.

•  These Oracle programs do not include Oracle RAC as well as a few other op@ons. For more details see: •  hSp://www.oracle.com/us/corporate/pricing/

authorized-‐cloud-‐environments-‐3493562.pdf


Oracle con@nues to support Oracle RAC in the Oracle Cloud via

Exadata Service and as an DBaaS offering, which is planned to scale

beyond the current 2-‐nodes.

96

General rule: As long as an IaaS Cloud meets

Oracle RAC requirements (e.g. network and storage), this cloud is in principle

“supportable” for Oracle RAC.

“Special ruling” regulates most Public Cloud and IaaS solu@ons. Check for such ruling prior to considering any solu@on.

Oracle RAC in the Cloud – Where is it Heading?


•  Virtual Machines •  Test, Development, Departmental Applica@ons

•  Oracle Apps Unlimited, PaaS •  Compute Shapes – by OCPU, Standard or High RAM

•  Block Storage – by the GB •  Up to 7.8TB database

•  Intensive Test, Development, Departmental Applica@ons

•  Custom Applica@ons •  Bare Metal Compute Shape – 2-‐node RAC DB System

•  Local SSD drives storage •  Up to 8.6TB database

97

•  Mission Cri@cal, Intensive OLTP and Decision Support

•  Oracle and Custom Apps •  ¼, ½ and Full Rack Shapes •  2/4/8-‐node Exadata DB System •  Fixed Storage and Memory by Shape

•  Up to 168TB database

Oracle RAC Database Cloud Service Infrastructure Choice Elas@c Compute Bare Metal Engineered Systems

oracle rac 12c rel. 2 under the hood and best practices

Software