1

Database Administration and Performance Tuning

--Introduction

2

Course Structure• Lecturers:

– Dr. Yanghua Xiao (肖仰华 )– Email: shawyh@fudan.edu.cn– Mobile phone: 13918452801– http://gdm.fudan.edu.cn/GDMWiki/Wiki.jsp?

page=Dbtune

3

Goals of the Course

• Appreciation of DBMS architecture• Study the effect of various components on the

performance of the systems• Tune your application or system built upon DBMS• Performance criteria for choosing a DBMS• Tuning principles• Troubleshooting techniques for chasing down

performance problems

4

Text/Reference Books:• Dennis Shasha and Phillipe Bonnet: Database Tuning : Principles Experiments

and Troubleshooting Techniques. Morgan Kaufmann Publishers. 2002 (released in June 2002). TEXT.

• Dennis Shasha: Database tuning : a principled approach. Prentice Hall, 1992. REFERENCE (a good reference if cannot get the text book)

• Database Management Systems, 3rd edition. Raghu Ramakrishnan & Johannes Gehrke, McGraw-Hill, 2002.

• Hector Garcia-Molina, Jeffrey D. Ullman, and Jennifer Widom: Database Systems -- The Complete Book. Prentice Hall, 2001.

• Sitansu S.Mittra, Database Performance Tuning and Optimization, Springer,2003

• G. J. Vaidyanatha, K. Deshpande and J. Kostelac: Oracle Performance Tuning 101. Osborne/Mc-Graw-Hill. 2001. REFERENCE.

• Jim Gray (ed): The Benchmark handbook : for database and transaction processing systems. M. Kaufmann Publishers, 1991. REFERENCE.

• Richard J.Niemiec, Oracle Database 10g Performance Tuning tips and techniques, Mc Graw Hill Education,2009

• 牛新庄 , DB2 数据库性能调整与优化 , 清华大学出版社 , 2009• 胡百敬 , 等 , SQL Server 2005 　 Performance tuning, 电子工业出版社， 2008

6

Copyright:Many slides belong to the tutorial:

Database TuningPrinciples, Experiments and Troubleshooting Techniques

Dennis Shasha (shasha@cs.nyu.edu)Philippe Bonnet (bonnet@diku.dk)

And lecture notes provided byDatabase Management Systems, 3rd edition. Raghu Ramakrishnan & Johannes Gehrke

McGraw-Hill, 2002.

and some from the web …

What is Database Tuning

• Database Tuning is the activity of making a database application run more quickly.

• “More quickly” – usually means higher throughput, – though it may mean shorter response time for

time-critical applications

• A 5% improvement is significant.

7

Why Database Tuning?

• Troubleshooting:– Make managers and users happy given an

application and a DBMS

• Capacity Sizing:– Buy the right DBMS given application

requirements

• Application Programming:– Coding your application for performance

Viewpoint about DB tuning

• Easy– Need not to struggle with complicated formulas and

theorems

• Difficult– Need broad and deep understanding about

• DBMS• Application• OS• hardware

9

Why is Database Tuning hard?

The following query runs too slowly

select * from Rwhere R.a > 5;

What do you do?

PARSEROPTIMIZERPARSER

OPTIMIZER

EXECUTIONSUBSYSTEM

EXECUTIONSUBSYSTEM DISK

SYBSYSTEM DISK

SYBSYSTEM

CACHEMANAGERCACHE

MANAGER

LOGGINGSUBSYSTEMLOGGING

SUBSYSTEM

LOCKINGSUBSYSTEM LOCKING

SUBSYSTEM

NETWORKDISK/

CONTROLLERCPUMEMORY

sql commands

Where to tune?-physical

11

Where to tune? -logical

12

Optimization at each level

• Conceptual level– E.g. normalization vs denormailization

• Internal level– E.g. tablespace management, index strategy

• External level– E.g. Query optimization, optimal access plan

13

How to tune? process

• Process for DB tuning– Localize the problem by starting at the location

where the problem surfaces for the first time and trace it to the root

– Fix the root cause and test to ensure that the problem does not reappear

– Look for any adverse side effects caused by the fix

14

How to tune?Troubleshooting Methodology

Troubleshooting Methodology:– Troubleshooting (what is happening?)– Hypothesis formulation

• What is the cause of the problem?• Apply tuning principles to propose a fix

– Hypothesis verification (experiments)

15

16

Tuning Principles

• Think globally, fix locally • Partitioning breaks bottlenecks (temporal

and spatial)• Start-up costs are high; running costs are

low (disk transfer, cursors)• Render unto server what is due unto server• Be prepared for trade-offs (indexes and

inserts)

Think globally, fix locally

• Think globally to find the root case or best solution for bottlenecks– For example : High disk I/O may be caused by:

• No index resulting into table scan

• Log and data file reside in the same disk

• Fail to increase the db buffer

• …..

• Fix locally to minimize the adverse side-effect– E.g. Optimize the critical query

17

Partitioning breaks bottlenecks

• Bottleneck: the ONE part of the system limits the overall performance

• Partitioning• Reducing the load on bottlenecks either by dividing the load over

more resources or by spreading the load over time• Partitioning in space (physical resource)

– E.g. distribute account data to each branch

• Partitioning in logical space– E.g. lock contention on free list will cause the free list to be a bottleneck– Divide the free list into several pieces, each thread only lock one free list

• Partitioning in time – Long transactions may lock resource for a long time, use large portion of

buffers, slowing down short transactions– Serialize long transactions when there are little short transactions

18

Side-effect of partitioning

• Suppose we need to query data across branch of a bank

• Merging data causes extra communication cost

• Lesson– When you find a bottleneck, first try to speed up that

component; if that doesn’t work, then partition

20

Tuning Principles• Start-up costs are high; running costs are low

– Start-up costs include• Disk access

– Reading 64k segment from disk is less than twice as expensive as reading 0.5k – Frequently scanned tables should be consecutively laid out on disk – Vertical partitioning tables with hundreds of columns

• Data transfer– Latency of sending message dominate the whole costs

• Packing data and send large chunk of data rather than small ones• Query processing

– Compile often executed queries

• System calls– Opening a call is expensive– Cache connections

– Reduce the number of start-ups

21

An example:

Time = Seek Time +Rotational Delay +Transfer Time +Other

Rule of Random I/O: ExpensiveThumb Sequential I/O: Much less

• Ex: 1 KB Block» Random I/O: 20 ms.

» Sequential I/O: 1 ms.

Render onto server what is due onto Server

• Task allocation between the server and the application programs

– Factors:• Relative computing resources and load of client,

application servers and data server– Should checking be done in the middle tier?

• Where information is located?– Trigger costs less cost than Polling

• Whether the database task involves the user interaction?– Interaction with screen or user takes long time– Separate the user interaction from a long transaction

22

Be prepared for trade-offs

• Performance vs cost/resource– Adding Buffer size need more RAM

• Query performance vs storage cost for index

• Query performance vs Update performance

• ……

• Keep in mind– You want speed, how much are you willing to pay?

23

24

Tuning Mindset1. Set reasonable performance tuning goals

2. Measure and document current performance

3. Identify current system performance bottleneck

4. Identify current OS bottleneck

5. Tune the required components eg: application, DB, I/O, contention, OS etc

6. Track and exercise change-control procedures

7. Measure and document current performance

8. Repeat step 3 through 7 until the goal is met

Date Subject Literature Material1 Sep　14 Introduction　(slides　　  ) (RG　1.5-1.8,　RG　

4.2);　SP　1

2 Sep.　21 Schema　Refinement(slides) RG19-20,SP4.2 database　design　project

3 Sep.　28 Data　Storage(slides) HG　11.1-11.5

4 Sep.30 Storage　Refinement(slides) SP　2.5,　HG　11.6-11.7

5 Oct.　7 Disk　Organization(slides) HG　12

6 Oct.14 Index　Tuning-Convential　Index(slides) HG　13.1-13.2

7 Oct.21 Index　Tuning-B+tree(slides) RG　10,　HG　13.3 Reading　and　Writing

8 Oct.28 External　Sorting(slides),Index　Tuning-Hash　Index(slides),　Performance　Tuning(slides)

HG13.4,RG　11,　SP　3

9 Nov.　4 Operator　Evaluation(slides) RG14 Programming

10 Nov.　11 Query　Optimization(slides) RG15

11 Nov.　18 Query　Tuning(slides) SP　4.6-4.7 DB　Tuning　Project,　Example　1,　Example　2,　Example　3

12 Nov.　25 Overview　of　Transaction　Management(slides),　Lock　Tuning(slides)

HG　16-17,　SP　2.2

13 Dec.　2 Log　Tuning(slides) HG　18,　SP　2.3

14 Dec.9 Troubleshooting(slides) SB　7

15 Dec.　16 API　Tuning(slides) SB　5

16 Dec.　23 Exam