partitioning divide-and-conquer examples …cs451/lectures/l4_partitioning.pdfpartitioning...
TRANSCRIPT
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Partitioning and Divide and Conquer
Strategies
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Partitioning• Partitioning Strategies
– Data partitioning• aka domain decomposition
– Functional decomposition
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.1
• For nuclear reactor simulation, what type of partitioning would be most effective?
Data decomposition and functional decomposition are both effective for different parts of the simulation.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Partitioning• Partitioning Strategies
– Example: Adding Numbers• divide sequence of n numbers for m processors• each process adds up n/m numbers• m partial sums are added for a total• operation – Master-Slave
– broadcast numbers using MPI_Scatter– compute local sums– compute sum on master using MPI_Reduce
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Partitioning• Divide and Conquer
– partitioning continued on smaller and smaller problems
– recursive definitions– M-ary trees, e.g., binary trees
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Partitioning• Divide and Conquer
– Example: Adding Numbers• problem division
– divide sequence of n number in two to create two processes with half of the numbers each
– recurse until enough processes for processors• addition
– add up the numbers in each process• problem combination
– odd processes pass values to even processes– even processes add communicated value to local sum– logically renumber processes and repeat combining
step until one process left.
How would this compare to our earlier addition
example?
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Partitioning• M-ary Divide and Conquer
– Same as divide and conquer except that we divide into more pieces at each step.
•quadtrees
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.2
• How does the divide and conquer approach used in the previous example address load balancing given that the regions are of such widely varying sizes?
By dividing space so that the numberof points in each region is about the same.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Bucket Sort – sequential
buckets
merge listssort
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.3
• Will each bucket have the same number of elements? Why or why not?
No. The number of elements willonly be approximately the sameif the values are uniformly distributed in the interval.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Bucket Sort – parallel VERSION 1
buckets
merge listssort
processors
unsorted
sorted
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.4
• What is the major problem with the parallel bucket sort just presented?
All processes examine every data element and then only process the ones in their sub-interval.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Bucket Sort – parallel VERSION 2
buckets
merge listssort
processors
mini buckets
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.5
• Which of the two parallel bucket sorts requires more communication to set up the buckets for sorting?
Version 1 requires that each process get a copy of all data: n*m.
Version 2 requires each process getn/m elements and sends n/m in the worstcase: 2*n.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.6
• Which of the two parallel bucket sorts will have faster communication to set up the buckets for sorting?
It depends on the machine.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.7
• Which of the two parallel bucket sorts will have faster computation to set up the buckets for sorting?
Version 2 will be faster in setting upbuckets (for large problems) because it will use parallelism to put elementsin buckets.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Numerical Integration
– integrate f(x) from a to b– i.e., compute area under curve f(x)– divide the area so each process computes
the area for one region– the area under the curve is sum of the areas
computed by all of the processes
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Numerical Integration
– midpoint of rectangular regions
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.8
• How could we test whether we are using enough rectangles for the integration?
Do the evaluation for r rectangles and for 2*r rectangles. If the difference is smallenough, then there are enough rectangles.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Numerical Integration
– trapezoid for regions
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Numerical Integration
– adaptive quadrature
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.9
• How can you address the load imbalance issue in adaptive quadrature?
Create a work list of regions to be computed. (work load)
Create an initial subdivision with many more pieces than processor and assign multiple pieces to each processor from different areas. (randomized)
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.10
• In addressing the load imbalance issue in adaptive quadrature with a work list, what issues arise?
We now have a shared work list that will cause contention.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.11
• In addressing the load imbalance issue in adaptive quadrature with many subdivisions, what issues arise?
We can still end up with a processor that has to do many more subdivisions than other processors and therefore has much more work to do.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.12• Can you see any convergence issues that
might be possible with adaptive quadrature?
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.13• Are the convergence issues with
adaptive quadrature any different than with the other approximation methods we discussed?
No, something similar can happen with all of them.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• N-body Problem
– typically determine the effects of forces between bodies
– Gravitational N-body problem• find the positions and movements of bodies in
space subject to gravitational forces from other bodies using Newton’s laws of physics.
• forces between each pair of bodies is proportional to 1/r2, where r is the distance between bodies
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.14
• In an N-body simulation, what communication problem arises for parallelization and why?
Since every body’s position is a functionof every other one on every time stepor iteration, a straightforward implementation requires all-to-all communication on every time step.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples
• Gravitational N-body parallelization– partition the bodies in 3d space and assign
a process to each region of space– pass messages for each pair of bodies that
captures the force between the bodies
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.15
• Name two problems that arise with the spatial partitioning of bodies and direct communication of individual forces.
Spatial partitioning may cause a large imbalance in work.
Individual force communication will cause a large communication overhead.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples
• Gravitational N-body parallelization– partition the bodies in 3d space and assign
a process to each region of space– pass messages for each distant body
cluster that captures the force between the cluster of bodies and a single body
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples• Barnes-Hut (N-body) parallelization
– Start with the 3D space. – Partition using an octtree.– For any region that has too many particles
• Recursively partition using an octtree.– compute the total mass and center of mass
of each cubic region– The force on each body can be obtained
by traversing the tree starting at the root and stopping when the clustering approximation is valid.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Divide-and-Conquer Examples
• Orthogonal recursive bisection– more general than octtree
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Quiz 4.16
• How has the Barnes-Hut approach addressed a parallelization problem for N-body simulations?
It subdivides the bodies so that each processor will have the same (approximate) amount of work.
Partitioning
Divide-and-Conquer Examples
Summary
CS/IT 451 & CS 551 – Parallel Processing
Lecture 4 ― Partitioning and Divide-and-Conquer Strategies
Summary• Partitioning• Divide and Conquer• Tree constructions• Bucket sort• Numerical Integration• N-body problem