lecture 2 algorithm analysis arne kutzner hanyang university / seoul korea

30
Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

Upload: bethanie-mcbride

Post on 01-Jan-2016

219 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

Lecture 2

Algorithm Analysis

Arne Kutzner

Hanyang University / Seoul Korea

Page 2: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.2

Overview

• 2 algorithms for sorting of numbers are presented

• Divide-and-Conquer strategy

• Growth of functions / asymptotic notation

Page 3: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.3

Sorting of Numbers

• Input A sequence of n numbers [a1, a2,..., an]

• OutputA permutation (reordering) [a‘1, a‘2,..., a‘n] of the input sequence such that a‘1 a‘2 ... a‘n

Page 4: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.4

Sorting a hand of cards

Page 5: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.5

The insertion sort algorithm

Page 6: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.6

Correctness of insertion sort• Loop invariants – for proving that some

algorithm is correct• Three things must be showed about a loop

invariant:– Initialization: It is true prior to the first iteration of

the loop– Maintenance: If it is true before an iteration of the

loop, it remains true before the next iteration– Termination: When the loop terminates, the

invariant gives us a useful property that helps show that the algorithm is correct

Page 7: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.7

Loop invariant of insertion sort

• At the start of each iteration of the for loop of lines 1-8, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order

Page 8: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.8

Analysing algorithms

• Input size = number of items (numbers) to be sorted

• We count the number of comparisons

Page 9: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.9

Insertion sort / best-case

• In the best-case (the input sequence is already sorted) insertion sort requires n-1 comparisons

Page 10: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.10

Insertion sort / worst-case

• The input sequence is in reverse sorted order

• We need

comparisons

Page 11: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.11

Worst-case vs. average case

• Worst-case running time of an algorithm is an upper bound on the running time for any input

• For some algorithms, the worst case occurs fairly often.

• The „average case“ is often roughly as bad as the worst case.

Page 12: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

Growth of functions

asymptotic notation

Page 13: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.13

Asymptotic upper bound

Page 14: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.14

Asymptotic lower bound

Page 15: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.15

Asymptotically tight bound

Page 16: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

Merge-Sort Algorithm

Page 17: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.17

Example merge procedure

Page 18: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.18

Merge procedure

Page 19: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.19

Merging - Worst case example

• Symmetrically sized inputs (here 2 times 4 elements)

– We compare 6 with all 5 and 8 (4 comparisons)

– We compare 8 with all 7 (3 comparisons)

• Generalized for n elements:– Worst case requires n – 1 comparisons– Time complexity cn = Θ(n).

6 857 557 7

Page 20: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.20

Correctness merge procedure

• Loop invariant:

Page 21: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.21

The divide-and-conquer approach

• Divide the problem into a number of subproblems.

• Conquer the subproblems by solving them recursively. If the subproblem sizes are small enough, however, just solve the subproblems in straightforward manner.

• Combine the solutions to the subproblems into the solution for the original problem

Page 22: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.22

Merge-sort algorithm

• Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each.

• Conquer: Sort the two subsequences recursively using merge sort.

• Combine: Merge the two sorted subsequences to produce the sorted answer.

Page 23: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.23

Merge-sort algorithm

Page 24: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.24

Example merge sort

Page 25: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.25

Analysis of Mergesort regarding Comparisons

• When n ≥ 2, for mergesort steps:– Divide: Just compute q as the average of

p and r ⇒ no comparisons– Conquer: Recursively solve 2

subproblems, each of size n/2 2⇒ T (n/2).– Combine: MERGE on an n-element

subarray requires cn comparisons ⇒ cn = Θ(n).

Page 26: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.26

Analysis merge sort 2

Page 27: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.27

Analysis merge sort 3

Page 28: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.28

Mergesort recurrences

• Recurrence regarding comparisons

• Recurrence time complexity:

• Both recurrences can be solved using the master-theorem:

-1C

C

Page 29: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.29

Lower Bound for Sorting

• Is there some lower bound for the time complexity / number of comparisons with sorting?

• Answer: Yes! Ω(n log n) where n is the size of the input

• Later more about this topic ……

Page 30: Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea

09/2014 Algorithm Analysis L1.30

Bubblesort

• Further popular sorting algorithm