![Page 1: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/1.jpg)
1
Design and Analysis of
Algorithms
演算法設計與分析
Lecture 1
February 24, 2016
洪國寶
![Page 2: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/2.jpg)
2
Outline
• Course information
• Motivation
• Outline of the course
• Introduction to design, analysis, algorithms
![Page 3: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/3.jpg)
3
Course information (1/6)
• Instructor: Professor Gwoboa Horng
• Office hours:
– Mon, Wed 6:00 – 6:30 pm
– By appointment
• Basic assumption
It is assumed that students in this course have had some exposure to algorithms and their analysis.
![Page 4: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/4.jpg)
4
Course information (2/6)
• Textbook
Introduction to algorithms, 2nd ed. (2001) or
Introduction to algorithms, 3rd ed. (2009)
by Corman, Leiserson, Rivest, and Stein,
The MIT Press
• Course web page:
• http://ailab.cs.nchu.edu.tw/course/Algorithms/
• e-campus
• Solutions to exercises and problems:
http://mitpress.mit.edu/algorithms/
![Page 5: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/5.jpg)
5
Course information (3/6)
• Textbook
Introduction to algorithms, 2nd ed. (2001) or
Introduction to algorithms, 3rd ed. (2009)
by Corman, Leiserson, Rivest, and Stein,
The MIT Press
• 2nd vs 3rd
• Pseudo-code convensions
• New chapters
• Price
![Page 6: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/6.jpg)
6
Course information (4/6)
• The goal of this class is to
present fundamental techniques,
for
designing efficient computer algorithms,
proving their correctness, and
analyzing their performance.
![Page 7: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/7.jpg)
7
Course information (5/6)
• This class is
– Not a lab or programming course
– Not a math course, either
![Page 8: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/8.jpg)
8
Course information (6/6)
• Grading (Tentative)
Homework 25%
(You may collaborate when solving the homework, however when writing up the solutions you must do
so on your own. Handwritten only.)
Midterm exam 25% (Open book and notes)
Final exam 25% (Open book and notes)
Class participation 25%
![Page 9: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/9.jpg)
9
Outline
• Course information
• Motivation
• Outline of the course
• Introduction to design, analysis, algorithms
![Page 10: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/10.jpg)
10
Motivation (1/6)
• Why study algorithms?
Computers are used to solve problems.
Now: Computer = Tool = Passive device
Need to tell it how to do
Algorithms
Future: Computer = Assistant = Active device
Need to tell it what to do
Artificial Intelligence
![Page 11: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/11.jpg)
11
Motivation (2/6)
• Problems: concrete vs. abstract
– The human genome project
– Internet related problem (Search engine etc)
– Electronic commerce (Cryptography etc)
– Manufacturing and other commercial settings
(scheduling)
• Will focus on abstract problems
![Page 12: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/12.jpg)
12
Motivation (3/6)
• Why abstract problems?
– One algorithm many applications
Matrix chain multiplication, optimal polygon triangulation
Optimal merge pattern, Huffman code
– Applications to be found later (kill time)
FFT (signal analysis)
Cutting rectangular (VLSI)
![Page 13: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/13.jpg)
13
Motivation (4/6)
• From a research point of view, what do
they imply?
RE-SEARCH
李家同教授
重視基礎科學也做無目的的學問
【2000/07/02/聯合報民意論壇】
![Page 14: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/14.jpg)
14
Motivation (5/6)
• The importance of efficient algorithms:
– Deep Blue (Chess)
– Factorization (RSA)
Current record: 768-bit RSA modulus
Progress in Factorization
![Page 15: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/15.jpg)
15
Motivation (6/6)
• The importance of efficient algorithms
Total system performance depends on
choosing efficient algorithms as much as
on choosing fast hardware.
A faster algorithm running on a slower
computer will always win for sufficiently
large instances.
![Page 16: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/16.jpg)
16
Outline
• Course information
• Motivation
• Outline of the course
• Introduction to design, analysis, algorithms
![Page 17: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/17.jpg)
17
Outline of the course
• Introduction (1-4)
• Data structures (10-14)
• Dynamic programming (15)
• Greedy methods (16)
• Amortized analysis (17)
• Advanced data structures (6, 19-21)
• Graph algorithms (22-25)
• NP-completeness (34-35)
• Other topics (5, 27, 31)
![Page 18: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/18.jpg)
18
Outline
• Course information
• Motivation
• Outline of the course
• Introduction to design, analysis, algorithms
![Page 19: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/19.jpg)
19
Introduction to
Design, Analysis, Algorithms
• Introduction to Algorithms
• Introduction to Analysis
• Introduction to Design (next lecture)
![Page 20: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/20.jpg)
20
Introduction to Algorithms
• Definition of algorithms
An algorithm is any well-defined
computational procedure that takes some
value, or set of values, as input and
produces some value, or set of values, as
output. (p. 5)
![Page 21: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/21.jpg)
21
Introduction to Algorithms
Properties of algorithms:
‧ Input. An algorithm has input values from a specified set.
‧ Output. For each set of input values an algorithm produces output values
from a specified set. The output values are the solution to the problem.
‧ Definiteness. The steps of an algorithm must be defined precisely.
‧ Correctness. An algorithm should produce the correct output
values for each set of the input values.
‧ Finiteness. An algorithm should produce the desired output values after a
finite number of steps for any input in the set.
‧ Effectiveness. It must be possible to perform each step of an algorithm
exactly and in a finite amount of time.
‧ Generality. The procedure should be applicable for all problems of the
desired form, not just for a particular set of input values.
![Page 22: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/22.jpg)
22
Introduction to Algorithms
• We can also view an algorithm as a tool for
solving a well-specified computational
problem. (p.5)
(design)
• A program is an expression of an algorithm
in a programming language.
(implementation)
![Page 23: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/23.jpg)
23
Introduction to Algorithms
• Example of a well-specified computational
problem: sorting problem (p. 5)
Input: sequence a1, a2, …, an of numbers.
Output: permutation a'1, a'2, …, a'n such
that a'1 a'2 … a'n .
![Page 24: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/24.jpg)
24
Introduction to Algorithms
• An instance of a sorting problem:
Input: 31, 26, 52, 41, 58, 55
Output: 26, 31, 41, 52, 55, 58
![Page 25: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/25.jpg)
25
Introduction to Algorithms
• A simple “algorithm” for sorting
i=1
while i < n
swap ai and ai+1
i=i+2
• Does it produce correct output for the previous
instance?
• Is it a correct algorithm for the sorting problem?
![Page 26: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/26.jpg)
26
Introduction to Algorithms
• An algorithm is said to be correct if for
every input instance, it halts with correct
output.
• Correctness is not obvious. (see slides 34 – 40)
![Page 27: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/27.jpg)
27
Introduction to Algorithms
• How to describe algorithms?
– Flow charts
– Pseudo-codes like Pascal
– Programs
![Page 28: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/28.jpg)
28
Introduction to Algorithms
• Pseudo-code conventions (pp. 20 - 22)– Indentation indicates block structure.
– The looping constructs while, for, and repeat and the conditional constructs if, then, and else have the same interpretation as in Pascal.
– The symbol (//) indicates that the remainder of the line is a comment.
– A multiple assignment of the form i ← j ← e assigns to both variables i and j the value of expression e; it should be treated as equivalent to the assignment j ← efollowed by the assignment i ← j.
– A return statement immediately transfers control back to the point of call in the calling procedure.
![Page 29: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/29.jpg)
29
Introduction to Algorithms
• Pseudo-code conventions (pp. 20 - 22)– Variables (such as i, j, and key) are local to the given procedure.
We shall not use global variables without explicit indication.
– Array elements are accessed by specifying the array name followed by the index in square brackets.
– Compound data are typically organized into objects, which are comprised of attributes or fields. A particular field is accessed using the field name followed by the name of its object in square brackets.
– Parameters are passed to a procedure by value: the called procedure receives its own copy of the parameters, and if it assigns a value to a parameter, the change is not seen by the calling routine. When objects are passed, the pointer to the data representing the object is copied, but the object's fields are not.
– The boolean operators and and or are short circuiting.
![Page 30: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/30.jpg)
30
Introduction to Algorithms
• Pseudo-code conventions 2nd vs 3rd
– Comments
– Assignment
– Compound data
![Page 31: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/31.jpg)
31
Introduction to Algorithms
• More about sorting
Why don’t CS professors ever stop talking
about sorting?
![Page 32: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/32.jpg)
32
Introduction to Algorithms
• Correctness is not obvious (1/7)
![Page 33: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/33.jpg)
33
Introduction to Algorithms
• Correctness is not obvious (2/7) (nearest neighbor)
![Page 34: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/34.jpg)
34
Introduction to Algorithms
• Correctness is not obvious (3/7)
![Page 35: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/35.jpg)
35
Introduction to Algorithms
• Correctness is not obvious (4/7) (closest pair)
Q: There is a problem with the above algorithm. What is it?
![Page 36: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/36.jpg)
36
Introduction to Algorithms
• Correctness is not obvious (5/7)
![Page 37: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/37.jpg)
37
Introduction to Algorithms
• Correctness is not obvious (6/7) A correct algorithm
![Page 38: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/38.jpg)
38
Introduction to Algorithms
• Correctness is not obvious (7/7)
To show an algorithm is incorrect is much easier:
Just give an instance that the algorithm produces
incorrect output. (consider previous sorting “algorithm”)
![Page 39: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/39.jpg)
39
Introduction to Algorithms
• A correct sorting algorithm: insertion sort
![Page 40: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/40.jpg)
40
Introduction to Algorithms
• To sort n elements:
– Return if n = 1
– Sort the first n-1 elements
Insert the n-th element in the right place
( insert an after a’n-1 if a’n-1 < an ; or
insert an between a’i and a’i+1 if a’i < an < =a’i+1 ; or
insert an before a’1 if an < a’1 )
![Page 41: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/41.jpg)
41
Introduction to Algorithms
• To sort n elements:
– Return if n = 1
– Sort the first n-1 elements
Insert the n-th element in the right place
Insertion-sort(A,n)
if n=1 then return A[n]
else Insertion-sort(A,n-1)
Insert(A, n-1, A[n])
![Page 42: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/42.jpg)
42
Introduction to Algorithms
Insert the n-th element in the right place
2 3 4 8 9 6
Insert(A, j, key)
i= j ;
while i > 0 and A[i] > key
A[i+1] = A[i]
i = i - 1
A[i+1] = key
n=6
A[n]=6
j=n-1=5
key=6
Insert(A, n-1, A[n])
![Page 43: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/43.jpg)
43
Introduction to Algorithms
Insertion sort (recursive version)Insertion-sort(A,n)
if n=1 then return A[n]
else Insertion-sort(A,n-1)
Insert(A, n-1, A[n])
Insert(A, j, key)
i= j ;while i > 0 and A[i] > key
A[i+1] = A[i]i = i - 1
A[i+1] = key
![Page 44: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/44.jpg)
44
Running Insertion-sort(A,n)
Insertion-sort(A,n)
Insertion-sort(A,n-1)
Insertion-sort(A,n-2)
Insertion-sort(A,4)
Insertion-sort(A,3)
Insertion-sort(A,2)
Insertion-sort(A,1)
return A[1]Insert(A, 1, A[2])
Insert(A, 2, A[3])
Insert(A, 3, A[4])
Insert(A, n-2, A[n-1])
Insert(A, n-1, A[n])
Insertion-sort(A,n)
if n=1 then return A[n]
else Insertion-sort(A,n-1)
Insert(A, n-1, A[n])
![Page 45: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/45.jpg)
45
Running Insertion-sort(A,n)
return A[1]Insert(A, 1, A[2])
Insert(A, 2, A[3])
.
.
Insert(A, n-3, A[n-2])
Insert(A, n-2, A[n-1])
Insert(A, n-1, A[n])
A[j] From recursion to iteration
![Page 46: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/46.jpg)
46
Insertion sort (iterative version)
![Page 47: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/47.jpg)
47
Think twice
• We have explained
– problems, instances
– algorithms, pseudo-code conventions
– recursion, iteration
– correctness
• If you have problems understanding the above
discussion, please consider drop this course.
![Page 48: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/48.jpg)
48
Example of insertion sort8 2 4 9 3 6 j=2
![Page 49: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/49.jpg)
49
Example of insertion sort8 2 4 9 3 6
![Page 50: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/50.jpg)
50
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6 j=3
![Page 51: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/51.jpg)
51
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
![Page 52: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/52.jpg)
52
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6 j=4
![Page 53: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/53.jpg)
53
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
![Page 54: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/54.jpg)
54
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6 j=5
![Page 55: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/55.jpg)
55
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
![Page 56: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/56.jpg)
56
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6 j=6
![Page 57: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/57.jpg)
57
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
![Page 58: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/58.jpg)
58
Example of insertion sort8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
![Page 59: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/59.jpg)
59
Introduction to Algorithms
• Correctness of insertion sort
– By induction
– By loop invariant
![Page 60: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/60.jpg)
60
Correctness of insertion sort
• By induction
Use blackboard.
![Page 61: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/61.jpg)
61
Loop invariant
• A Method for Proving the Correctness of
Iterative Algorithms (that is, algorithms
contain loop structures)– Similar to mathematical induction
• A loop invariant is a statement about a
condition of program variables that is true
just before a loop begins executing and is
also true after each iteration of the loop.
![Page 62: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/62.jpg)
• A loop invariant states a condition that is true immediately before every iteration of a loop.
• The invariant should be true also immediately after the last iteration.
Loop Invariant
Time (# of computation steps)
It. 1 It. 2 It. 3 It. 4 · · · It. k
![Page 63: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/63.jpg)
• Initialization:
• Maintenance:
• Termination:
The invariant holds here.
If the invariant holds here… …it holds here.
If the invariant holds here, the algorithm is correct
(does what we want).
Loop invariant
![Page 64: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/64.jpg)
A Loop Invariant for Insertion Sort
• Invariant: Before the iteration of the outer loop
where j = k + 1, array A[1..k] stores the k
elements initially in A[1..k] in sorted order.
Array A[k+1..n] is unaltered.
An improper loop invariant: i < j
![Page 65: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/65.jpg)
65
Example of insertion sort
8 2 4 9 3 6
A[1] A[2] A[3] A[4] A[5] A[6]
![Page 66: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/66.jpg)
66
Example of insertion sort
8 2 4 9 3 6
Before the iteration, we have j=2
A[1] A[2] A[3] A[4] A[5] A[6]
A[1..j-1] in sorted order
Loop starts with j=2
A[j..n] unaltered
Initialization
![Page 67: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/67.jpg)
67
Example of insertion sort
2 8 4 9 3 6
Now we have j=3
A[1] A[2] A[3] A[4] A[5] A[6]
A[1..2] are in sorted order.
That is, A[1..j-1] in sorted order
End of j=2 iteration
A[j..n] unaltered
Maintenance
![Page 68: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/68.jpg)
68
Example of insertion sort
2 4 8 9 3 6
Now we have j=4
A[1] A[2] A[3] A[4] A[5] A[6]
A[1..3] are in sorted order.
That is, A[1..j-1] in sorted order
End of j=3 iteration
A[j..n] unaltered
Maintenance
![Page 69: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/69.jpg)
69
Example of insertion sort
2 4 8 9 3 6
Now we have j=4
A[1] A[2] A[3] A[4] A[5] A[6]
Therefore, A[1..j-1] in sorted order
is a loop invariant for the for loop.
A[j..n] unaltered
End of j=3 iteration
Invariant: Before the iteration of the outer loop where j = k + 1,
array A[1..k] stores the k elements initially in A[1..k] in sorted order.
Array A[k+1..n] is unaltered.
Maintenance
![Page 70: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/70.jpg)
70
Example of insertion sort
2 3 4 6 8 9
A[1] A[2] A[3] A[4] A[5] A[6]
A[1..j-1] = A[1..6] is sorted.
Loop ends when j=7 Termination
![Page 71: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/71.jpg)
71
Correctness of insertion sort
• By loop Invariant
Use blackboard.
![Page 72: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/72.jpg)
Recap
• For “designing efficient algorithms”
problems, you need to
1. outline your idea, and describe your
algorithm,
2. give an example,
3. prove correctness, and
4. analyze time complexity
![Page 73: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/73.jpg)
73
Recap
• A correct sorting algorithm: insertion sort
• For “designing efficient algorithms” problems,
1. outline your idea, and describe your algorithm,
2. give an example,
3. prove correctness, and
4. analyze time complexity
![Page 74: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/74.jpg)
74
Recap
Insertion sort (recursive version)Insertion-sort(A,n)
if n=1 then return A[n]
else Insertion-sort(A,n-1)
Insert(A, n-1, A[n])
Insert(A, j, key)
i= j ;while i > 0 and A[i] > key
A[i+1] = A[i]i = i - 1
A[i+1] = key
• For “designing efficient algorithms” problems,
1. outline your idea, and describe your algorithm,
2. give an example,
3. prove correctness, and
4. analyze time complexity
![Page 75: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/75.jpg)
75
Recap 8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
• For “designing efficient algorithms” problems,
1. outline your idea, and describe your algorithm,
2. give an example,
3. prove correctness, and
4. analyze time complexity
![Page 76: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/76.jpg)
76
Recap
• Correctness of insertion sort
– By induction, or
– By loop invariant
• For “designing efficient algorithms” problems,
1. outline your idea, and describe your algorithm,
2. give an example,
3. prove correctness, and
4. analyze time complexity
![Page 77: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/77.jpg)
77
Analysis
• Exact analysis of insertion sort
• Sorting Algorithm Animations (for different tj)
http://www.sorting-algorithms.com/
• For “designing efficient algorithms” problems,
1. outline your idea, and describe your algorithm,
2. give an example,
3. prove correctness, and
4. analyze time complexity
![Page 78: Design and Analysis of Algorithms - ailab.cs.nchu.edu.twailab.cs.nchu.edu.tw/course/Algorithms/104-2/AL01.pdf · from a specified set. The output values are the solution to the problem](https://reader031.vdocuments.pub/reader031/viewer/2022030400/5a71fef37f8b9aa2538d4e28/html5/thumbnails/78.jpg)
78
Questions?