アルゴリズムの設計と解析

黄 潤和、佐藤 温（TA） 2013.4～

Contents (L14 – Review shortest path algorithms) Weighted Graph Shortest path problem Single-source shortest path algorithm Dijkstra’s algorithm

All-pairs shortest paths algorithms Matrix-based algorithms

Matrix-multiplication algorithm Floyd-Warshall algorithm

Minimal spanning tree Prim’s algorithm Kruskal’s algorithm

Travel salesman problem

2

3

Weighted Graph 重み付きグラフ

In a weighted graph, each edge has an associated numerical value, called the weight of the edge 重み付きグラフでは、各枝に重みと呼ばれる数値をもっている。 Edge weights may represent, distances, costs, etc. 枝の重みは距離やコストなどを表している。 Example: In a flight route graph, the weight of an edge represents the distance in

miles between the endpoint airports：各空港間の距離（マイル）

ORD PVD

MIA DFW

SFO

LAX

LGA

HNL

4

Shortest Path Problem 最短経路問題

Given a weighted graph and two vertices u and v, we want to find a path of minimum total weight between u and v 二つの頂点uとvからの重みの合計が最小となる経路を考える Applications Flight reservations(飛行機予約) Driving directions（ナビ、道案内） Internet packet routing (インターネットパケットルーチン)

Example: Shortest path between Providence and Honolulu（プロビデンスーホノルル間）

ORD PVD

MIA DFW

SFO

LAX

LGA

HNL

5

Shortest Path Properties 最短経路の特徴

Property 1: A subpath of a shortest path is itself a shortest path

最短経路の部分経路はそれ自体が最短経路である Property 2: There is a tree of shortest paths from a start vertex to all the other

vertices スタート地点から全ての点への最短経路の木 Example: Tree of shortest paths from Providence プロビデンスからの最短経路の木

ORD PVD

MIA DFW

SFO

LAX

LGA

HNL

6

Problem: From a given source vertex s ∈ V, find the shortest-path weights δ(s, v) for all v ∈ V. If all edge weights w(u, v) are nonnegative, all shortest-path weights must exist. IDEA: Greedy. 1.Maintain a set S of vertices whose shortest-path

distances from s are known. 2.At each step add to S the vertex v ∈ V – S whose

distance estimate from s is minimal. 3.Update the distance estimates of vertices adjacent to v.

Single-source shortest paths Problem

7

Regarding Dijkstra’s Algorithm (1) compute the distances of all the vertices from a given start vertex s ダイクストラのアルゴリズムは スタート地点sから全ての頂点へ の距離を計算する。 (2) grow a “cloud” of vertices, beginning with s and eventually covering all the vertices 頂点のクラウド(雲)をスタート地点から始め、徐々にすべての頂点へと広げていく。

(3) store each vertex v a label d(v) representing the distance of v from s in the subgraph consisting of the cloud and its adjacent vertices クラウド内のサブグラフとその隣接点はそれぞれスタート地点s からの距離を表すラベルd(v) を持っている。 (4) The distance of a vertex v from a vertex s is the length of a shortest path between s and v 頂点sからvへの最短経路の長さを距離とする

Work in class： Please your version of Dijkstra’s algorithm (擬似コード)

Dijkstra’s Algorithm (pseudo code)

8

Work in class： Please make comments to the algorithm

9

Example

C B

A

E

D

F

0

4 2 8

∞ ∞

4 8

7 1

2 5

2

3 9

C B

A

E

D

F

0

3 2 8

5 11

4 8

7 1

2 5

2

3 9

C B

A

E

D

F

0

3 2 8

5 8

4 8

7 1

2 5

2

3 9

C B

A

E

D

F

0

3 2 7

5 8

4 8

7 1

2 5

2

3 9

10

Example (cont.)

C B

A

E

D

F

0

3 2 7

5 8

4 8

7 1

2 5

2

3 9

C B

A

E

D

F

0

3 2 7

5 8

4 8

7 1

2 5

2

3 9

11

Dijkstra’s algorithm in Java http://www.algolist.com/code/java/Dijkstra's_algorithm

poll() メソッドは、キューの先頭を削除および返します

Exercises 9-2

12

Exercises 9-1

Please add comments to the Dijkstra’s algorithm in Java. (source_code)

ダイクストラアルゴリズムを用いてグラフGのノードuを根とする最短経路木を見つけなさい。アルゴリズムの各段階において各頂点の重みと名前を記すこと。

13

• General (including negative weight) � Input: Digraph G = (V, E), where V = {1, 2, …, n}, with edge-weight function w : E → R. Output: n × n matrix of shortest-path lengths δ(i, j) for all i, j ∈ V. • Time Complexity?

• Single source shortest paths in general � Bellman-Ford: O(VE)

• all-pairs shortest paths? • IDEA: • Run Bellman-Ford once from each vertex. • Time = O(V2E).

All-pairs shortest paths using Bellman-Ford

14

All-pairs shortest path problem description

15

Proof of Claim

Notice that

16

Compute matrix multiplication (1)

?

similar?

17

Compute matrix multiplication (2)

Is it good?

n times of matrix multiplication, right?

?

18

Review D&C algorithm for multiplication

Improved multiplication

19

How to detect negative-weight cycle?

the diagonal: its value is 0, ? its is negative value, ?

negative-weight cycle

20

dynamic programming is a method for solving complex problems by breaking them down into simpler sub-problems. It is applicable to problems exhibiting the properties of overlapping sub-problems which are only slightly smaller and optimal substructure (described below). When applicable, the method takes far less time than naive methods. e. g. Recursive algorithm Divide and conquer Dijkstra algorithm

Floyd-Warshall algorithm Also dynamic programming, but faster!

21

Input representation: We assume that we have a weight matrix W= (wij) (i,j) in E wij= 0 if i=j wij= w(i,j) if i≠j and (i,j) in E (has edge from i to j)

wij= ∞ if i≠j and (i,j) not in E (no edge from i to j)

Input and output

By Andreas Klappenecker

Output representation: If the graph has n vertices, we return a distance matrix (dij), Where, dij the length of the path from i to j.

Intermediate Vertices Without loss of generality, we will assume that V={1,2,…,n}, i.e., that the vertices of the graph are numbered from 1 to n. Given a path p=(v1, v2,…, vm) in the graph, we will call the vertices vk with index k in {2,…,m-1} the intermediate vertices of p.

22

Key Definition The key to the Floyd-Warshall algorithm is the following definition: Let dij

(k) denote the length of the shortest path from i to j such that all intermediate vertices are contained in the set {1,…,k}. We have the following remark

23

dij(1) dij

(2) dij(k) dij

(…)

Thus, the shortest path δ(i, j) = dij

(n) Also, dij

(0) = aij .

The Floyd-Warshall Algorithm Floyd-Warshall(W) n = # of rows of W; D(0) = W; for k = 1 to n do for i = 1 to n do for j = 1 to n do dij

(k) = min{dij(k-1) , dik

(k-1) + dkj(k-1)};

end-do; end-do; end-do; return D(n);

24

By Andreas Klappenecker

do if dij(k-1)>dik

(k-1)+djk(k-1)

then dij

(k-1)dij dik(k-1)+djk

(k-1)

relaxation

Time and Space Requirements • The running time is obviously O(n3). • However, in this version, the space

requirements are high. • One can reduce the space from O(n3) to

O(n2) by using a single array d.

25 By Andreas Klappenecker

26

27

Exercises 10

EX 10.2 Please add source code for printing out the resulting matrix (printDK(int k)) to the Floyd-Warshall Algorithm in Java (in my homepage). Ex10.3 Read the following text http://homepage3.nifty.com/asagaya_avenue/trial/discussion/nishikawa_net.pdf

EX 10.１ What does the matrix below used in the shortest-paths algorithms correspond to in regular matrix multiplication? What do the symbols, 0 and ∞, stand for?

What are spanning trees?

28

Spanning trees from BFS and DFS

They are not same!

From SSSP to MST

30

(From Dijkstra’s to Prim’s algorithms) Change Dijkstra’s algorithm’s (selecting a shortest path) minDistance(f) + length(f,n)

to keeps adding smallest edge to expand its path

目的：shortest path

目的：minimal edges

Q4. How about their results? Are their results same?

31

An example

Prim’s algorithm

32

{ T = φ; U = { 1 }; while (U≠V) { let (u, v) be the lowest cost edge such that u∈U and v∈V - U; T = T ∪{(u, v)} U = U ∪{v} } }

Complexity = O(|E|log(|E|))

http://www.deqnotes.net/acmicpc/prim/ Reference in Japanese

33

C B

A

E

D

F

0

4 2 8

∞ ∞

4 8

7 1

2 5

2

3 9

C B

A

E

D

F

0

3 2 7

5 8

4 8

7 1

2 5

2

3 9 by Dijkstra

C B

A

E

D

F

0

4 2 8

∞ ∞

4 8

7 1

2 5

2

3 9

by Prim’s

The results are the same.

Q4. Which is the minimal spanning tree? (what differences?)

34

Dijkstra Prim’s

35

Work in class: draw the result of Prim’s algorithm

36

37

Kruskal's algorithm (basic part)

(Sort the edges in an increasing order) A:={} while E is not empty do { take an edge (u, v) that is shortest in E and delete it from E if u and v are in different components then add (u, v) to A }

Kruskal’s v. s. Prim’s algorithms

38

• In Kruskal's algorithm, – The set A is a forest. – The safe edge added to A is always a

least-weight edge in the graph that connects two distinct components.

• In Prim's algorithm, – The set A forms a single tree. – The safe edge added to A is always a

least-weight edge connecting the tree to a vertex not in the tree.

39

Kruskal's algorithm (disjoint set) MST_KRUSKAL(G,w) 1 A:={} 2 for each vertex v in V[G] 3 do MAKE_SET(v) 4 sort the edges of E by non-decreasing weight w 5 for each edge (u,v) in E, in order by non-decreasing weight 6 do if FIND_SET(u) != FIND_SET(v) 7 then A:=A∪{(u,v)} 8 UNION(u,v) 9 return A

Animation of Kruskal’s algorithm http://f.hatena.ne.jp/mickey24/20090614234556

Walk-Through Consider an undirected, weight graph

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10

Sort the edges by increasing edge weight edge dv (D,E) 1

(D,G) 2

(E,G) 3

(C,D) 3

(G,H) 3

(C,F) 3

(B,C) 4

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2

(E,G) 3

(C,D) 3

(G,H) 3

(C,F) 3

(B,C) 4

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3

(C,D) 3

(G,H) 3

(C,F) 3

(B,C) 4

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3

(G,H) 3

(C,F) 3

(B,C) 4

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Accepting edge (E,G) would create a cycle

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3

(C,F) 3

(B,C) 4

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3

(B,C) 4

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Work in class: Please continue to finish this algorithm and draw the resulting tree.

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3 √

(B,C) 4

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3 √

(B,C) 4 √

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4

(B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3 √

(B,C) 4 √

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4 χ (B,F) 4

(B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3 √

(B,C) 4 √

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4 χ (B,F) 4 χ (B,H) 4

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3 √

(B,C) 4 √

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4 χ (B,F) 4 χ (B,H) 4 χ

(A,H) 5

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3 √

(B,C) 4 √

5

1

A

H B

F

E

D

C

G 3

2

4

6

3 4

3

4 8

4

3

10 edge dv (B,E) 4 χ (B,F) 4 χ (B,H) 4 χ

(A,H) 5 √

(D,F) 6

(A,B) 8

(A,F) 10

Select first |V|–1 edges which do not generate a cycle

edge dv (D,E) 1 √

(D,G) 2 √

(E,G) 3 χ (C,D) 3 √

(G,H) 3 √

(C,F) 3 √

(B,C) 4 √

5

1

A

H B

F

E

D

C

G 2

3

3

3

edge dv (B,E) 4 χ (B,F) 4 χ (B,H) 4 χ

(A,H) 5 √

(D,F) 6

(A,B) 8

(A,F) 10

Done

Total Cost = Σ dv = 21

4

} not considered

What is travel salesman problem?

54

TSP:

A Salesman wishes to travel around a given set of cities, and return to the beginning, covering the smallest total distance. Create a TSP Tour around all cities

(1) Return to the beginning

(2) No condition to return to the beginning.

It can still be regarded as a TSP by connecting the beginning city and the end city to a ‘dummy’ city at zero distance

A B

Two classifications of TSP

55

A route returning to the beginning is known as a

Hamiltonian Circuit

A route not returning to the beginning is known as a

Hamiltonian Path

The smallest total distance?

56

Sweden 24978 Cities 2004 Applegate, Bixby, Chvátal, Cook and Helsgaun

Solutions: (1) Try every possibility? (n-1)!, 24977 possibilities (2) Optimizing Methods take very long time (3) Heuristic Methods may not optimal

The Nearest Neighbor Method (Heuristic) - A Greedy method

57

1. Start Anywhere

2. Go to Nearest Unvisited City

3. Continue until all Cities visited

4. Return to Beginning

58

A 42-City Problem The Nearest Neighbour Method (Starting at City 1)

1

21

20

19

29

7

30 28

31 37

32 36

11

9

34

10

39

38

35

12

33

8

27 26

6 24

25

14

15 23

22

2 13

16

3 17 4

18

42

40

41

5

59

The Nearest Neighbour Method (Starting at City 1)

1

21

20

19

29

7

30 28

31 37

32 36

11

9

34

10

39

38

35

12

33

8

27 26

6 24

25

14

15 23

22

2 13

16

3 17 4

18

42

40

41

5

60

The Nearest Neighbour Method (Starting at City 1) Length 1498

1

21

20

19

29

7

30 28

31 37

32 36

11

9

34

10

39

38

35

12

33

8

27 26

6 24

25

14

15 23

22

2 13

16

3 17 4

18

42

40

41

5

61

29

Remove Crossovers

1

21

20

19

7

30 28

31 37

32 36

11

9

34

10

39

38

35

12

33

8

27 26

6 24

25

14

15 23

22

2 13

16

3 17 4

18

42

40

41

5

62

Remove Crossovers

1

21

20

19

29

7

30 28

31 37

32 36

11

9

34

10

39

38

35

12

33

8

27 26

6 24

25

14

15 23

22

2 13

16

3 17 4

18

42

40

41

5

63

Remove Crossovers Length 1453

1

21

20

19

29

7

30 28

31 37

32 36

11

9

34

10

39

38

35

12

33

8

27 26

6 24

25

14

15 23

22

2 13

16

3 17 4

18

42

40

41

5

64

(1) compute a MST (minimum spanning tree) whose weight is a lower bound on the length of an optimal TSP tour. (2) use MST to build a tour whose cost is no more than twice that of MST's weight as long as the cost function satisfies triangle inequality. http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/AproxAlgor/TSP/tsp.htm

Outline of an APPROX-TSP-TOUR

R. B. Muhammad

65

Operation of APPROX-TSP-TOUR

Let root r be a in following given set of points (graph)

MST-PRIM Preorder

List vertices visited in preorder walk. L = {a, b, c, h, d, e, f, g}

Hamiltonian cycle optimal TSP tour

R. B. Muhammad

66

Exercises 13

EX 13.2 (1) What is a TSP problem? (2) Refer the city connection graph in Fig 1, start from the node A, - to draw a MST - to an optimal TSP tour

EX 13.１ Modify the Dijkstra’s algorithm to Prim’s algorithm.

Fig1. City connection graph