08 - complexity
DESCRIPTION
I used this set of slides for the lecture on Complexity I gave at the University of Zurich for the 1st year students following the course of Formale Grundlagen der Informatik.TRANSCRIPT
computerinformation information
computation
Computation requires resources
memory
bandwidth
time
...
1 public long factorial (int n) {
2 long factorial = 1;
3 int i = 1;
4 while ( i ≤ n ) {
5 factorial = factorial * i;
6 i = i + 1;
7 }
8 return factorial
9 }
1 public long factorial (int n) {
2 long factorial = 1; 1
3 int i = 1; 1
4 while ( i ≤ n ) { 4
5 factorial = factorial * i; 4
6 i = i + 1; 3
7 }
8 return factorial 1
9 }
1 public long factorial (int n) {
2 long factorial = 1; 1 1
3 int i = 1; 1 1
4 while ( i ≤ n ) { 4 n
5 factorial = factorial * i; 4 n
6 i = i + 1; 3 n
7 }
8 return factorial 1 1
9 }
1 public long factorial (int n) {
2 long factorial = 1; 1 1
3 int i = 1; 1 1
4 while ( i ≤ n ) { 4 n
5 factorial = factorial * i; 4 n
6 i = i + 1; 3 n
7 }
8 return factorial 1 1
9 }
f(n) = 1+1+4*n+4*n+3*n+1 = 11*n+3
f: N -> Ng: N -> N
f ∈ O(g) ⇔ ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≤ c*g(n))
Big O Notation
f: N -> Ng: N -> N
f ∈ O(g) ⇔ ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≤ c*g(n))
f: N % N, f(n)=11*n+3.f ∈ O(n)
Big O Notation
O(1) Constant
O(log n) Logarithmic
O(n) Linear
O(n2) Quadratic
O(nk) Polynomial
O(kn) Exponential
exponentialquadraticlinearlogarithmicconstant
!"#$%$&'()("'*$+ ,'-.$/'--'*$%01.02
O(1) Constant
O(log n) Logarithmic
O(n) Linear
O(n2) Quadratic
O(nk) Polynomial
O(kn) Exponential
exponentialquadraticlinearlogarithmicconstant
!"#$%$&'()("'*$+ ,'-.$/'--'*$%01.02O(1) ∈ O(log n) ∈ O(n) ∈ O(n2) ∈ O(nk) ∈ O(2n)
N log N N log N N2 2N N!10 3 33 100 1024 2*106
20 4 86 400 106 2*1018
100 7 664 10’000 1031 10157
1’000 10 10’000 106
10’000 13 130’000 108
100’000 17 106 1010
1’000’000 1’000 2*107 1012
N log N N log N N2 2N N!10 3 33 100 1024 2*106
20 4 86 400 106 2*1018
100 7 664 10’000 1031 10157
1’000 10 10’000 106
10’000 13 130’000 108
100’000 17 106 1010
1’000’000 1’000 2*107 1012
102 s = 1.7 minutes104 s = 2.8 hours105 s = 1.1 days106 s = 1.6 weeks107 s = 3.8 months108 s = 3.1 years109 s = 3.1 decades1010 s = 3.1 centuries1011 s = never :)
O(c*f) = O(f)
O(f) + O(g) = O(f+g) = max{O(f),O(g)}
O(f*g) = O(f)*O(g)
O(c*f) = O(f)
O(f) + O(g) = O(f+g) = max{O(f),O(g)}
O(f*g) = O(f)*O(g)
O(f) ⊆ O(g) ⇔ f ∈ O(g)O(f) = O(g) ⇔ (O(f) ⊆ O(g)) ∧ (O(g) ⊆ O(f))O(f) ⊂ O(g) ⇔ (O(f) ⊆ O(g)) ∧ (O(g) ≠ O(f))
a b c d e f gabcdefg
0 2 3 0 0 0 00 0 1 0 0 0 00 0 0 2 0 0 00 0 0 0 5 4 00 0 0 0 0 0 30 0 0 0 0 0 30 0 0 0 0 0 0
1
3
22
5 3
34b
a
c d
e
f
g
a b c d e f gabcdefg
0 2 3 0 0 0 00 0 1 0 0 0 00 0 0 2 0 0 00 0 0 0 5 4 00 0 0 0 0 0 30 0 0 0 0 0 30 0 0 0 0 0 0
1
3
22
5 3
34b
a
c d
e
f
g
procedure FloydWarshall () for k := 1 to n for i := 1 to n for j := 1 to n path[i][j] = min ( path[i][j], path[i][k]+path[k][j] );
O(2*n-1) = O(n)
O(n*(n+1)/2) = O(n2)
O(log n2) = O(log n)
O((3*n2 + 6*n+9)*log(1+2*n)) = O(n2log(n))
O(2*n-1) = O(n)
O(n*(n+1)/2) = O(n2)
O(log n2) = O(log n)
O((3*n2 + 6*n+9)*log(1+2*n)) = O(n2log(n))
Example: Linear search
4 2 5 1 6 3
4 2 5 1 6 3
Example: Linear search
4 2 5 1 6 3
Example: Linear search
4 2 5 1 6 3
Worst case: n stepsBest case: 1 stepAverage case: n/2 steps
Example: Linear search
f: N -> Ng: N -> N
f ∈ Ω(g) ⇔ ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))
Best case estimation
f: N -> Ng: N -> N
f ∈ Ω(g) ⇔ ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))
Best case estimation
(3n2+6n+9)*log(1+2n) ≤ 36 n2 * log n(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)
f: N -> Ng: N -> N
f ∈ Ω(g) ⇔ ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))
Best case estimation
(3n2+6n+9)*log(1+2n) ≤ 36 n2 * log n(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)
(3n2+6n+9)*log(1+2n) ≥ n2 * log n(3n2+6n+9)*log(1+2n) ∈ Ω(n2 * log n)
f: N -> Ng: N -> N
f ∈ θ(g) ⇔ ∃ c1, c2, n0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))
Average case estimation
f: N -> Ng: N -> N
f ∈ θ(g) ⇔ ∃ c1, c2, n0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))
Average case estimation
(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)
f: N -> Ng: N -> N
f ∈ θ(g) ⇔ ∃ c1, c2, n0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))
Average case estimation
(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)
(3n2+6n+9)*log(1+2n) ∈ Ω(n2 * log n)
f: N -> Ng: N -> N
f ∈ θ(g) ⇔ ∃ c1, c2, n0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))
Average case estimation
(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)
(3n2+6n+9)*log(1+2n) ∈ Ω(n2 * log n)
(3n2+6n+9)*log(1+2n) ∈ θ(n2 * log n)
boolean f ( int[][] a , int n ) { for ( int i = 0 ; i < n ; i++ ) { for ( int j = i + 1 ; j < n ; j++ ) { if ( a[i][j] == 0 ) {return false;} } return true; }}
Example
boolean f ( int[][] a , int n ) { for ( int i = 0 ; i < n ; i++ ) { for ( int j = i + 1 ; j < n ; j++ ) { if ( a[i][j] == 0 ) {return false;} } return true; }}
f ∈ O(n2)f ∈ Ω(1)f ∈ θ(n2)
Example
1
2
3
4
5
6
Example: Binary search
1
2
3
4
5
6
Example: Binary search
f ∈ O(log n)f ∈ Ω(1)f ∈ θ(log n)
solvablein O(nk)
Psolvablein O(nk)
P NPsolvablein O(nk)
P NPsolvablein O(nk)
verifiablein O(nk)
P NP⊂solvablein O(nk)
verifiablein O(nk)
P NP=?solvablein O(nk)
verifiablein O(nk)
NP-completeNP, and one cannot do better
NP-complete problem:
Hamiltonian path
b
a
c d
e
f
g
Hamiltonian path
NP-complete problem:
Traveling salesman
NP-complete problem:
Graph coloring
NP-complete problem:
Subset sum
S={x1, … , xn}t ∈ N∃ {y1, … , yk} ⊆ S : Σyi=t ?
NP-complete problem:
Subset sum
S={x1, … , xn}t ∈ N∃ {y1, … , yk} ⊆ S : Σyi=t ?
S= {8, 11, 16, 29, 37}t = 37
{11, 16}{8, 29}{37}
The halting problem
is undecidable
Will this program terminate?
Tudor Gîrbawww.tudorgirba.com
creativecommons.org/licenses/by/3.0/