in-memory data technologies review - sql server 2014 oltp vs safepeak dynamic caching
TRANSCRIPT
July 30 2014
The 2014 In-Memory Technologies:
SQL Server 2014 In-Memory OLTPvs.
SafePeak Dynamic Caching
SQL Server 2014 In-Memory OLTP Overview
SQL Server 2014 In-Memory OLTP Demo
SafePeak In-Memory Dynamic Caching Overview
SafePeak Product Demo
Q & A (please type questions directly into the
GoToWebinar side panel)
Agenda
2
Vladi Vexler, VP Technology
• Founder of SafePeak Technologies
• More than 15 years of experience in
software development and product
management, featuring enterprise,
web & cloud industries
• Managed R&D at SafePeak, Scepia
• Speaker on MSFT Conferences on
topics of Big Data and Scalability
• BA in Management and Economics
3
SQL Server 2014 in-memory technologies:
IN-MEMORY OLTP –
for Operational OLTP workloads
UPDATABLE CLUSTERED COLUMNSTORE INDEX –
for BI, pure data warehouse workloads
SafePeak In-Memory Dynamic Caching:For Operational apps and BI apps with repetitive queries
In-Memory Technologies
For SQL Server Apps
5
In-memory Data management
with Asynchronous disk writes
In-Memory Tables = “MEMORY OPTIMIZED” +
Native Stored Procedures = “NATIVE COMPILATION”:
Compiled and loaded as C based DLLs
Lock-free (external I/O) and Latch-free (internal):
A multi-version concurrency control mechanism
High WRITE performance,
Moderate READ improvement
SQL Server 2014 In-Memory OLTP:
What is it?
6
Embedded into the main SQL Server as an additional layer
Accessed via T-SQL or via Natively Compiled SPs (preferable)
SQL 2014 In-Memory OLTP at 10,000 feet
7
1. Add to your Database:MEMORY_OPTIMIZED_DATA filegroup
2. Create Memory Optimized TABLES: WITH (MEMORY_OPTIMIZED=ON,
DURABILITY=SCHEMA_AND_DATA)
Primary key settings, Hash and Bucket_Count can’t be altered later
3. Load data to tables + Update statistics (Memory-optimized tables do not support auto_update_statistics)
4. Create (no alter) Natively Compiled stored procedures:WITH NATIVE_COMPILATION, SCHEMABINDING, EXECUTE
AS OWNER AS
BEGIN ATOMIC WITH (TRANSACTION ISOLATION LEVEL
= SNAPSHOT, LANGUAGE= N'us_english')
SQL 2014 In-Memory OLTP at 1,000 feet:
The Quick Start Guide
8
• Effective use requires development and testing
• Prepare for required code changes: Databases design, Tables schemas, Data types & structure, Stored
procedures, Business logic, Application code
• Limitations:SQL Syntax, Schema Structure, H/A, Mirroring, Replication
SQL 2014 In-Memory OLTP at 100 feet:
Migration consideration plan
9
• Memory limitations:
• Minimum = (In-Memory Tables + Indexes Size) x 2
+ enough for mgmt of disk based tables
• Maximum = 250GB/server (for in-memory tables)
• Free Disk space= (In-Memory Tables + Indexes Size) x 2
+ enough for disk based tables
• For example:
• Your data: 200GB Data + 50GB Indexes
• You need: (200+50)x2 = 500GB RAM (and Disk)
SQL 2014 In-Memory OLTP:
Planning the Hardware
10
Not supported availability features:
• Replication (subscriber only)
• Mirroring
• Change Data Capture (CDC)
• Distributed Transactions (DTC)
• Database Snapshot
• Multiple Active Result Sets (MARS)
• Linked Servers (for access of Memory-Optimized Tables)
SQL 2014 In-Memory OLTP:
Limitations – Data Availability
11
Schema limitations / non supported features:
• Schema: ALTER, DDL Triggers, DML Triggers, Limitations with Server Triggers
Foreign Keys, Limitations with Constraints, Row size limit=8060, Index types limits
• Unsupported Data-types: Datetimeoffset, Geography, Hierarchyid, Image,
Ntext, Text, Sql_variant, Varchar(max), Xml, User data types, LOB, XML
• Unsupported Table/Column features: IDENTITY(1,1) only, UNIQUE,
RowGuidCol, Computed Columns, Data types [name], FileStream, Sparse, others
Not supported T-SQL syntax:
• Classic: OR, NOT, IN, LIKE, BETWEEN, CONTAINS, PERCENT, DISTINCT,
UNION, OUTER JOIN, MERGE, HASH, CASE WHEN, FOR XML, FOR BROWSE,
CURSORS, CTE, PARTITIONS, NEXT VALUE FOR
• Dynamic SQL (EXECUTE, EXEC), Sub Queries, Cross DB Queries
• Native SPs limitations: Access ONLY to In-Memory Tables
SQL 2014 In-Memory OLTP:
Limitations – Schema and Syntax
12
Applicability based on READ-WRITE scenarios:
• High load WRITE scenarios to a limited list of tables.
Very high performance improvement
• Specific, large data processing READS to a limited list of tables,
(not too large, memory limits).
Not all READs will get performance improvements
Applicability depending on application stage:
• Greenfield Apps: A good option to explore. Knowing the
limitations, enables building DBs and Apps based on available
features
• Live Prod. Apps, minimal changes: May be a complex option
• Commercial 3rd party apps: Probably not applicable
SQL 2014 In-Memory OLTP:
Applicability
13
SafePeak Acceleration Software
App-DB middleware or proxy
• Resolves application performance, load
scaling and load peak challenges
• By Automated Dynamic Caching and
Fast Data Delivery (~0.000,050sec)
• Caching result-sets of repetitive
SQL queries and stored procedures
• While ensuring 100% data integrity
(cache is never stale)
• No code changes
• Supports Custom and 3rd-party applications
16
Automated Dynamic Caching
Combination of cache and in-memory database:
Smart caching of Hot-data Results with real-time eviction on DMLs
Caching of any read-based queries and stored procedures
Cache is never stale. Writes commands trigger automatically:
Real-time eviction/invalidation of the right cache items
Real-time database synchronization
Efficient cache management – Keeps only “hot-data” in memory
17
Self-learning and self-adaptive:
• Learns Database objects definition and create a dependencies map
• Recognizes SQL Patterns, identifies their tables/views dependencies
and automatically uses them as rules for real time caching
Minimal configuration effort
100% ACID, 100% data integrity, 100% up-to-date database
Safe: embedded failover proxy & optional cluster deployment
Usually installed as a proxy VM
SafePeak – Intelligent and self-learning
18
Results: Large Mobile Telecom
E-Commerce Application
9X Faster DB
19Aberdeen Group
Research: E-commerce response time acceleration effect: From 6 to 5 sec.:
Sales Conversions
Customer Satisfaction
Page Views
12.5%7.5% 19%
Case Study: Tier 1 SW Vendor
SharePoint Application
SQL Server CPU Time (% )
• 1/10 CPU Load• 1/10 I/O Load• No more overload
CPU Overload
• Cache hit = 75%• Acceleration: 70% - 350%
Application Time (Sec)
20
Easy Advanced Control
Manage Cache on Database, Table and
Query Level
Automated SQL Patterns Detection
and Analysis
Real-time Deep SQL Analytics
DB speed=98msCache speed=0.15msNew Average=3.8ms
22
Summary of SafePeak Caching Features
Caching Feature More info
SQL Batch Queries Caching of any “read” query, with any types of data
SQL Stored Procedures Caching of any “read” procedure, inc. with Dynamic SQL
Automated Caching Deterministic SQL queries and Stored Procedures are automatically
activated for caching
Semi Automated Caching Non-Deterministic SQL Queries and Stored Procedures – activation by
“click-and-point” + advanced configuration
Fully Dynamic Caching Automated eviction of cache items on “writes” to relevant tables
relevant to each SQL Pattern (default setting)
Semi Dynamic Caching Caching w. reduced sensitivity of a SQL Pattern to “writes”
Advanced Settings Caching based on TTL, Scheduled Evictions, Dependencies
Configure exceptions
And override settings
Configuration of not-fully understood procedures or override its
behavior settings, for cache optimization
Cache warm-up list XML based list of queries for periodically auto-caching
Import/Export settings Save, auto-apply, export and import Database template settings
23
SafePeak vs. SQL 2014 In-Memory OLTP:
An alternative or complementary?
SAFEPEAK In-MemoryDynamic Caching
Hot Data Results
ALL Tables
READs
Medium
Yes
No
Production,Commercial 3rd party
Hours
SQL 2005-2012*2014 is in dev & testing
Virtual machine for SP
SQL Server 2014 In-Memory OLTP
Actual Table Data (x2)
In-Memory Tables
WRITEs (mostly) and READs
Very high
No (many limitations)
Yes
Greenfield (mostly)Production (w. code changes)
Weeks-Months
SQL 2014 Enterprise edition
Development efforts
Data In-Memory
In-Memory Data
Acceleration
RAM requirements
Full T-SQL support
Code Changes
Greenfield / Production /Commercial 3rd party
Deployment Time
SQL Server support
Additional Cost Factors
25
Questions?
Contact Info:Vladi Vexler
Paul Campaniello
Resources:http://www.safepeak.com
http://blog.safepeak.com
http://www.safepeak.com/download
+1 617.630.2800
27