강원대학교 컴퓨터과학전공 문양세

강원대학교 컴퓨터과학전공 문양세강원대학교 컴퓨터과학전공 문양세

이산수학이산수학 (Discrete Mathematics) (Discrete Mathematics) 알고리즘의 복잡도알고리즘의 복잡도

(Algorithm Complexity)(Algorithm Complexity)

Discrete Mathematicsby Yang-Sae Moon

What is Algorithm Complexity?What is Algorithm Complexity?

The algorithmic complexity of a computation is some measure of how difficult it is to perform the computation.

( 문제 계산 (computation) 이 얼마나 어려운가를 나타내는 측정치이다 .)

Measures some aspect of cost of computation (in a general sense of cost). ( 계산을 위한 비용에 대한 측정치이다 .)

Common complexity measures:

• Time complexity: # of operations or steps required

• Space complexity: # of memory bits required

2.3 Algorithm Complexity


Complexity Depends on InputComplexity Depends on Input

Most algorithms have different complexities for inputs of different sizes. (E.g. searching a long list takes more time than searching a short one.)

( 대부분의 알고리즘은 입력의 크기에 따라 복잡도가 달라진다 . 당연 !)

Therefore, complexity is usually expressed as a function of input length.

( 따라서 , 복잡도는 입력의 크기 / 길이에 대한 함수로 표현한다 .)

This function usually gives the complexity for the worst-case input of any given length.

( 복잡도를 나타내는 함수는 통상 입력 크기가 최악인 경우를 고려한다 .)



Example 1: Example 1: MaxMax Algorithm Algorithm

Problem: Find the exact order of growth () of the worst-case time complexity of the max algorithm.

Assume that each line of code takes some constant time every time it is executed.


procedure max(a1, a2, …, an: integers)

v := a1 {largest element so far}

for i := 2 to n {go thru rest of elems}

if ai > v then v := ai {found bigger?}

{at this point v’s value is the same as the largest

integer in the list}

return v


Complexity Analysis of Complexity Analysis of MaxMax Algorithm Algorithm (1/2)(1/2)

What’s an expression for the exact total worst-case time? (Not its order of growth.)( 최악의 경우 , 정확한 수행 시간을 어떻게 될까 ?)



v := a1 t1

for i := 2 to n t2

if ai > v then v := ai t3

return v t4

Times for each execution of each line.

( 각 line 을 하나의 수행으로 볼 때의 시간 )


Complexity Analysis of Complexity Analysis of MaxMax Algorithm Algorithm (2/2)(2/2)

Worst case execution time:



v := a1 t1

for i := 2 to n t2

if ai > v then v := ai t3

return v t4

Times for each execution of each line.

( 각 line 을 하나의 수행으로 볼 때의 시간 )

)()()1()1()()1(

)1()1()1()1()1()1(

)()(

2

42

321

nnn

n

ttttnt

n

i

n

i


Example 2: Linear SearchExample 2: Linear Search2.3 Algorithm Complexity

procedure linear search x: integer, a1, a2, …, an: distinct integers)

i := 1 t1

while (i n x ai) t2

i := i + 1 t3

if i n then location := i t4

else location := 0 t5

return location t6

Worst case:

Best case:

Average case (if item is present):

)()()( 6541

321 nttttttntn

i

)1()( 6421 ttttnt

)()()( 654

2/

1321 nttttttnt

n

i


Example 3: Binary SearchExample 3: Binary Search2.3 Algorithm Complexity

procedure binary search (x:integer, a1, a2, …, an: distinct integers)

i := 1 j := n while i<j begin

m := (i+j)/2

if x>am then i := m+1 else j := m

end

if x = ai then location := i else location := 0

return location

(1)

(1)

(1)

Key Question: How Many Loop Iterations?


Binary Search AnalysisBinary Search Analysis

Suppose n=2k.

Original range from i=1 to j=n contains n elements.

Each iteration: Size ji+1 of range is cut in half.

( 매번 검색 범위 (j-i+1) 의 절반 (½) 씩 줄여 나간다 .)

Loop terminates when size of range is 1=20 (i=j).

( 검색 범위의 변화 : 2k 2k-1 2k-2 … 21 20, 결국 반복 횟수 = k)

Therefore, number of iterations is k = log2n

= (log2n)= (log n) ( 반복 횟수 = k 이고 , log2n = log22k 이므로… )

Even for n2k (not an integral power of 2), time complexity is still (log2n) = (log n).



Example 4: Bubble SortExample 4: Bubble Sort2.3 Algorithm Complexity

32415

23415

23415

23145

23145

23145

21345

21345

12345

12345

2nd pass1st pass

3rd pass 4th pass

Consider # of compare operations only!

(n-1) + (n-2) + … + 2 + 1 = ((n-1)n)/2 = (n2)


Example 5: Insertion SortExample 5: Insertion Sort2.3 Algorithm Complexity

Also, consider # of compare operations only!

1 + 2 + … + (n-2) + (n-1) = ((n-1)n)/2 = (n2)

32415

23415

23415

23415

23415

12345

12345

12345

Then, are all sorting algorithm’s complexities (n2)?NO! …, merge sort, heap sort, quick sort,

…


Names for Some Orders of GrowthNames for Some Orders of Growth

(1) Constant

(log n) Logarithmic

(n) Linear

(n log n) Polylogarithmic

(nc) Polynomial

(cn), c>1 Exponential

(n!) Factorial



TractableTractable versus versus IntractableIntractable

A problem or algorithm with at most polynomial time complexity is considered tractable (or feasible). P is the set of all tractable problems.(Polynomial 복잡도를 가지면 , 풀만한 ( 풀기 쉬운 ) 문제라 여기며 , 이러한 문제들의 집합을 P 라 나타낸다 .)

A problem or algorithm that has more than polynomial complexity is considered intractable (or infeasible).(Polynomial 복잡도 이상이면 풀기 어려운 문제라 여긴다 .)

But, note that

• n1,000,000 is technically tractable, but really impossible.

• nlog log log n is technically intractable, but easy.



Unsolvable ProblemsUnsolvable Problems

Alan Turing discovered in the 1930’s that there are problems unsolvable by any algorithm.

( 튜링은 “어떠한 알고리즘으로도 풀 수 없는 문제가 있음”을 밝혔다 .)

Example: the halting problem.• Given an arbitrary algorithm and its input, will that algorithm eventuall

y halt, or will it continue forever in an “infinite loop?”

( 주어진 알고리즘이 결국 정지하는지의 여부를 판단하는 문제는 결국 풀 수가 없다 .)

• We will handle the halting problem in $3.1 again.



PP versus versus NP NP (1/3)(1/3)

P is the set of all tractable problems. (i.e., the problems have at most polynomial time complexity.)( 통상적으로 polynomial complexity 를 가지면 P 이다 .)

NP is the set of problems for which there exists a tractable algorithm for checking solutions to see if they are correct.• In other words, NP is the set of decision problems for which a given p

roposed solution for a given input can be checked quickly (in polynomial time) to see if it really is a solution.

• 주어진 입력에 대해 제시된 솔루션이 바른 해인지의 여부를 빠르게 (poly-nomial time) 판단할 수 있는 알고리즘이 존재하는 문제들의 집합이 NP 이다 .

• 통상적으로 exponential/factorial complexity 를 가지면 NP 이다 .




NP-complete is the term used to describe decision problems

that are the hardest ones in NP in the sense that, if there were

a polynomial-bounded algorithm for an NP complete problem,

then there would be a polynomial-bounded algorithm for each

problem in NP.

• NP 중에서도 어려운 문제들이란 용어를 의미하며 , 하나의 NP-complete 문제가 풀리면 (P

가 되면 ), 관련된 수많은 모든 NP-complete 문제가 동시에 풀리게 된다 .

• 많은 학자들은 어려운 많은 문제들에 대해서 NP-complete 임을 증명 ( 특정 문제가 다른 NP-complete 문제로 해석 ( 변경 ) 될 수 있음을 보임 ) 하였다 .




We know PNP, but the most famous unproven conjecture in computer science is that this inclusion is proper (i.e., that PNP rather than P=NP).

Whoever first proves it will be famous!

You may cover these terms in more detail in algorithm class.



Computer Time ExamplesComputer Time Examples2.3 Algorithm Complexity

Assume time = 1 ns (109 second) per operation

#ops(n) n=10 n=106

log2 n 3.3 ns 19.9 ns

n 10 ns 1 msn log2 n

33 ns 19.9 ms

n2 100 ns 16 m 40 s

2n 1.024 s 10301,004.5

Gyrn! 3.63 ms Ouch!You should carefully design algorithms and write programs!


Homework #3Homework #3

$2.1 의 연습문제 : 4, 12, 34

$2.2 의 연습문제 : 4, 10, 20(b)

$2.3 의 연습문제 : 2, 7

Due Date:


강원대학교 컴퓨터과학전공 문양세

Documents