Take a trip down memory lane

SQL Server memory management past, present and hidden gem

Po

2

Who am I ?

Carnegie Mellon University, MS in Information TechnologyBanking and Financial industry. Manage more than 4,000 database instances environment including, MSSQL, MSAS, Sybase, DB2 and Oracle. Manage the Database client application deployment to more than 100,000 desktops/Servers.Blog: http://sqlpanda.com Twitter: https://twitter.com/SQLPANDA Linkedin: http://www.linkedin.com/pub/po-chen/2/249/791

Today’s Topic

SQLOS and memory managementLarge PageDatabase Corruption

… Mostly 64-bit

NUMA WOW64Column store IndexSQL Server in Virtualization environment SQL Server 2014 (Hekaton, BP extension ..)Memory pressure

What we will cover today ? What we will not cover today ?

5

Why do we care about the memory?

6

Memory acronym

• VAS• PAE• AWE• 3GB/USERVA• MTL• LPIM• SPA• MPA• ASPA• LPA

Virtual Address SpacePhysical Address ExtensionAddress Windowing Extensions4GB TuningMemory To LeaveLock Page In MemorySingle Page AllocationMulti-Page allocation Any Size Page AllocationLarge Page Allocation

Acronym Full Name

In-complete List

7

VAS ?

8

Max Memory now means Max Memory

(Almost …)

9

Memory Manager Change in SQL 2012

/

• HeapAlloc• AllocateUserPhysicalPages• AllocateUserPhysicalPagesNum

a• VitrualAlloc

10

Performance Counter

-Memory Manager \ Reserved MemoryMemory Manager \ Reserved Memory In UseBuffer Pool \ Target (Pages)Buffer Pool \ Committed (Pages)Buffer Pool \ Stolen (Pages) + Buffer Pool \ DatabaseBuffer Pool \ Reserved (Pages)Buffer Pool \ Free (Pages)-Buffer Pool \ Stolen Potential (Pages)

Memory Manager \ Large Pages AllocatedMemory Manager \ Emergency MemoryMemory Manager \ Emergency Memory In UseMemory Manager \ Target Committed (KB)Memory Manager \ Current Committed (KB)Memory Manager \ Pages Allocated (KB)Memory Manager \ Pages Reserved (KB)Memory Manager \ Pages Free (KB)Memory Manager \ Pages In UseMemory Manager \ Page Alloc Potential (KB)

SQL 2008R2 SQL 2012

11

Memory Manager type and error log

Error Log will show the physical memory installed on the host.EX: Detected 8499 MB of RAM. This is an informational message; no user action is required. New in 2012

Conventional memory

• Default• Using

conventional memory in the memory manager.

AWE

• LPIM• “Using locked

pages in the memory manager. ” OR “Address Windowing Extensions enabled."

Large Page

• Host has RAM > 8GB

• LPIM• SQL Enterprise

Edition • “Using locked

pages in the memory manager. ” and “Large Page Allocated: 32MB”

12

SQLOS

• Sqlserver.exe -> 60MB• Sqlos.dll -> 16KB

Sqlserver.exe -> 188KBSqldk.dll -> 2MBSqlos.dll -> 26KBSqlmin.dll -> 30MBSqllang.dll -> 33MB

Pre SQL Server 2012 (2008R2) SQL Server 2012

13

SQLOS

Use Windows API to• Check LPIM privilege: AdjustTokenPrivileges()• Allocate the memory: VirtualAlloc()• Monitor the memory: QueryMemoryResourceNotification()

Lets do some debug : ( Windbg ) -> DO NOT do this on the production system

14

LPIM & AWE

Lets do some debug : ( Windbg ) -> DO NOT do this on the production system ever!!

15

LPIM & LPA

LPIM LPIM

2012: “Using locked pages in the memory manager” 2008: “Address Windowing Extensions enabled”

Host has RAM > 8GBSQL Enterprise Edition

2012: Using locked pages in the memory manager. Large Page Allocated: 32MB2008:Large Page Extensions enabled. Large Page Granularity: 2097152 Large Page Allocated: 32MB locked pages for buffer pool.

-T 834 2012: Same as before  2008:  Using large pages for buffer pool. 

DO NOT use 834 if you have column store index in SQL 2012

16

Database Corruption: 823, 824, 825, 832823: Hard IO error, Can not read page at all824: Soft IO error -> Check SUM fail during the data read. 825: IO Error, Retry success 832 error -> memory Corruption Error: 832, Severity: 24, State: 1.A page that should have been constant has changed (expected checksum: <expected value>, actual checksum: <actual value>, database <dbid>, file '<filename>', page <pageno>). This usually indicates a memory failure or other hardware or OS corruption.

Application Assisted Memory Error Recovery

JOIN US for our second annual event to get the best learning for analyzing, managing, and sharing business information and insights through the Microsoft Data Platform of technologies.

SQLSaturday Events – Nov/DecUpcoming North America Events Upcoming International Events

• Nov 16 #265 Oregon• Dec 7 #233 Washington DC

• Nov 30 #261 Moscow• Dec 7 #259 Kharkov• Dec 7 #266 Lima• Dec 13 #264 Ancona• Dec 21 #274 Slovenia

Visit www.sqlsaturday.com to register for an event near you!

Stay Involved! & Happy Holiday

• Sign up for a free membership today at sqlpass.org

• Linked In: Professional Association for SQL Server• Facebook: Professional Association for SQL Server Group• Twitter: @SQLPASS• The PASS Blog: sqlpass.org

20

Extensive reading

• How to fake the NUMA( Not soft NUMA): http://www.sqlpanda.com/2013/06/build-mssql-fake-numa-test-environment.html

• SQL Server Large Page: http://www.sqlpanda.com/2013/11/sql-server-large-page.html

• 832 error video: http://h20324.www2.hp.com/SDP/Content/ContentDetails.aspx?PortalID=1&ID=3397&booth=63&tag=687&goback=%2Egmp_1434987%2Egde_1434987_member_142079817