ca uim log analytics - נס טכנולוגיות - מובילים בתחום ... · pdf...

CA UIM Log Analytics

Gain Full Stack Visibility With Contextual Log Insights

Mark TukhPrincipal Presale Consultant CA Division @ NESS AT

2 © 2017 CA. ALL RIGHTS RESERVED.

Analytics is the New Battleground

> 50%large organizations globally will compete using advanced analytics & proprietary algorithms

40%enterprises' net-new analytics investment will go to predictive / prescriptive

75%technology-oriented business intelligence competency centers will evolve to focus on information insight generation

2

Source: Gartner

3 © 2017 CA. ALL RIGHTS RESERVED.

Analytics In Our DNA

ANALYTICS EMBEDDED THROUGHOUT CA PORTFOLIO

AGILE MANAGEMENT AGILE OPERATIONS DEVELOPER PRODUCTS

CONTINUOUS DELIVERY MAINFRAME SECURITY

3

API analyticsRelease management dashboard Access incident responseReal-time service status

4 © 2017 CA. ALL RIGHTS RESERVED.

Why Monitor Logs?

Lot of information ONLYavailable in logs – for networks, servers as well as apps

Provide additional context to identify root cause and bottlenecks

Can provide “canary in a coal mine” messages

User information from logs & performance & fault data together can help capacity planning

Perception that “unstructured” data cannot be monitored well

6 © 2017 CA. ALL RIGHTS RESERVED.

Rear-view focused Usually Stand-AloneLack guided, situation-aware

decision making

Log Analysis tools: burden of insight discovery on users

CAPEX: Challenging value/pricing model

OPEX: Extra integration & support effort

7 © 2017 CA. ALL RIGHTS RESERVED.

Lifting the Burden

ANALYTICS-DRIVEN APPLICATIONS*

ANALYTICAL PLATFORM & SKILLS

IN-HOUSE DATA SCIENCE

IN-HOUSE DOMAIN EXPERTISE

+

+

YOUR BURDEN OUR ALTERNATIVE

*Fueled by advanced analytics, algorithms, machine learning...

8 © 2017 CA. ALL RIGHTS RESERVED.

CA’s Approach To Log Analytics Provide Contextual Analytics for Superior Experience

LOG ANALYTICS

Infrastructure Performance.

NetOps

Context

RAPID ROOT CAUSE

OPTIMIZED IT RESOURCES

ANOMALY DETECTION App Performance

Our play is “Unified Monitoring”, not a standalone tool

PREDICTIVE IDENTIFICATION

9 © 2017 CA. ALL RIGHTS RESERVED.

CA Log Analytics add-onContextual Insights for rapid issue identification

Multi-source aggregation with out of the box dashboards and reports

Search and ad-hoc analysis

Correlation and contextual alerts

Scalable, cost-effective ELK storage

Unified, template based configuration via existing tools

Capabilities In Detail

11 © 2017 CA. ALL RIGHTS RESERVED.

Log Analytics CapabilitiesDrill Down into a Log Type for Detailed Analysis

▪ Normalization, analysis and rich visualization for various log types

▪ Supported types:

– Syslog & MS Windows event log

– Apache access and error

– Tomcat access and Catalina

– Microsoft IIS

– Java log4j

– Docker

– Oracle and Microsoft SQL Server

12 © 2017 CA. ALL RIGHTS RESERVED.

Log Analytics CapabilitiesOOTB dashboards identify key events, trends to keep an eye on

▪ Compare unstructured log and event data over time to identify patterns

▪ Correlation across diverse logs and data sets and CA Unified Infrastructure Management alarms

▪ Summary insights into key events to help you focus

13 © 2017 CA. ALL RIGHTS RESERVED.

Log Analytics CapabilitiesAd-Hoc Search for Proactive Resource Optimization and root cause analysis

▪ Easy search & extensibility across different log types

▪ Save queries or policies for future use and proactive correlation

14 © 2017 CA. ALL RIGHTS RESERVED.

Log Analytics CapabilitiesConfiguration through Templates Save Time

▪ Configure and deployment monitoring through the same monitoring configuration services in CA Unified Infrastructure Management

▪ Rapid monitoring deployment through templates across groups/devices

15 © 2017 CA. ALL RIGHTS RESERVED.

Log Analytics CapabilitiesAlarms Pushed for Proactive and Rapid Issue Resolution

▪ Relevant log event alarm pushed in to CA Unified Infrastructure Management for rapid issue resolution

▪ Ability to open Log Analytics dashboards in context

▪ Tightly integrated with CA Unified Infrastructure Management workflows

Analytics Platform & CA Log Analytics Architecture

17 © 2017 CA. ALL RIGHTS RESERVED.

CA Agile Operations tools (UIM+ Spectrum+ APM)

CA log agent

or

agent-less

K

A

F

K

A

ELK Datastoreand CA Analytics Platform services - with open, rich

APIs

Windows/Linux

Application Servers

Network Devices

Cloud

Third party/IOT

Log Analytics: The Big Picture

18 © 2017 CA. ALL RIGHTS RESERVED.

Elastic Stack

o Elasticsearch:o - Schema optional document oriented database

o - Distributed and highly available search engine.

o - APIs: HTTP RESTful API and Native Java API

o - (Near) Real Time Search and Analytics

▪ LogStash:▪ - Framework for managing Events and Logs

▪ - Collect, Parse and Enrich data

▪ - Modular design with Inputs, Outputs and Filters

▪ - Enhanced with custom grok patterns for Log Analytics

19 © 2017 CA. ALL RIGHTS RESERVED.

Elastic Stack

o Kibanao - Schema optional document oriented database

o - Distributed and highly available search engine.

o - APIs: HTTP RESTful API and Native Java API

o - (Near) Real Time Search and Analytics

20 © 2017 CA. ALL RIGHTS RESERVED.

▪ Log_forwarder:

A light-weight log collection agent that reads log files on the monitored servers/devices and publishes the data on UIM

Message Bus (default subject: LOG_ANALYTICS_LOGS)

▪ Axa_log_gateway:

Receives log data from UIM by listening to subject LOG_ANALYTICS_LOGS and writes the data to AXA Kafka topic

logAnalyticsLogs for processing by Log Parser

▪ Log_monitoring_Service: Queries Elastic data at predefined schedule and will provide the following output:

• Match_Count metric for the count of matches found

• Alarm if the match count exceeds a predefined threshold

• Alarms containing sample matched logs lines (number of sample lines configurable)

New UIM probes for Log Analytics

22 © 2017 CA. ALL RIGHTS RESERVED.

Overall Picture

© 2017 CA. ALL RIGHTS RESERVED.

Unified Visibility and Reporting

Application to Infrastructure

Correlation

Continuous Operational Insight

Proactive and Predictive Analytics

AO Analytics Platform (Elastic Search)

Logs and Traces Metrics and Alarms Topology

End User(Mobile, Web, IoT)

Business KPIs(SFDC, Social,… )

OpenRESTful

APIs

Custom Data Sources

APMTransactions & Metrics

Topology

UIMMetric, Alerts, Logs,

Topology

NetworkFault, Perf, Logs

Anomaly Detection Pattern Recognition Neural Networks

Operation Analytics Applications

Use Cases

24 © 2016 CA. ALL RIGHTS RESERVED.

25 © 2017 CA. ALL RIGHTS RESERVED.

Drill Down into alarm or event

Performance Dashboard

Sample Use Case - DockerContextual drill down for rapid issue resolution

Contextual launch of Log Analytics

26 © 2017 CA. ALL RIGHTS RESERVED.

Sample Use Case – Alarm EnrichmentSpeed issue resolution to delight today’s demanding users

Product search is slow Log Analytics

User drills down to the issue

IN CONTEXT LAUNCH

OF LOG ANALYTICS

System and log alarms in CA UIM

27 © 2017 CA. ALL RIGHTS RESERVED.

28 © 2017 CA. ALL RIGHTS RESERVED.

Use Case – Syslog enrichment for CA Spectrum

Network Infrastructure

Log Analytics

Richer context with Syslog

s

y

s

l

o

g

Root cause alarm based on syslog

Can apply to other tools for syslog or other logs too

29 © 2017 CA. ALL RIGHTS RESERVED.

Use Case - Alarm/Inventory Analytics

Third-Party Open Source Tools

First Phase – CA Spectrum, CA Unified Infrastructure Management, CA Performance Management

Inventory and alarms sent to CA Agile Operations Analytics

• Alarm inventory dashboards for availability, grouping by geo, branch, agency, et al

• Correlation to other related events like syslog

CA Agile Operations Analytics

CA Agile Operations Analytics

CA Agile Operations Analytics Dashboards

30 © 2016 CA. ALL RIGHTS RESERVED.

Principal Presale Consultant

Mark.Tukh@ca.com052-6626691

Mark Tukh

www.linkedin.com/in/mark-tukh