A cache-aware algorithm for PDEs on hierarchical data

structures

Frank GüntherJune 2004

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

3

Motivation

MultigridMultigridEfficiency in memoryEfficiency in memory

ParallelizationParallelization

Amount of dataAmount of data AdaptivityAdaptivity

Hierarchical data structures

Hierarchical data structures

Solver forPDEs

Solver forPDEs

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

4

Space-filling curves• G. Cantor: cardinality of manifolds• Is there a continuous and bijective ?• E. Netto: No!• Search for continous and surjective mappings• Peano, Hilbert, Sierpinski, Moore, Lebesgue

2]1,0[]1,0[: f

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

5

Mathematical ingredients and demands

• Finite-Element-Discretization

• Squares on Cartesian grids

• Multigrid• Hierarchical

generating system• (a-priori-) adaptivity

• Embedding of arbitrary geometries in the unit square

• Poisson’s equation• Stokes’ equation• Dirichlet-boundary-

conditions

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

6

Peano’s curve and stacks

• Formation of “lines of points”, which are processed monotonously

• Lines are conserved for all depths of recursion• Stacks as data structure• “Coloring” depends on basis:

nodal basis on finest level: 2 colors, 2 stacks hierarchical generating system: 4 colors, 8 stacks

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

7

The Algorithm I

• Develop “rule sets” for stack access– Deterministic– No unnecessary access– All kinds of points (inner, outer, on boundary,

hanging) must be covered

• Efficient programming of stacks and stack access

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

8

The Algorithm II

• Input: linearized geometry-based tree

• Recursive cell-oriented programming

• Optimizations for cell types

• “OO by hand”

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

9

Resolution Variables % reg. grid Iter

27 x 27 298 40,05 38

81 x 81 1.140 15,95 38

243 x 243 3.800 5,78 38

729 x 729 13.840 2,3 38

2187 x 2187 36.369 0,67 38

6561 x 6561 109.952 0,23 38

Regular grids

Resolution Variables Iter

27 x 27 744 39

81 x 81 7.144 39

243 x 243 65.708 39

729 x 729 595.692 39

2187 x 2187 5.374.288 39

6561 x 6561 48.407.888 39

Adaptive grids

Results for Poisson’s equation

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

10

Cache hit rate in L2-Cachebeyond 99,0%

Performance• Simulation

– Prediction– “What happens, and where does it happen?”

• Measuring– Hardware Performance Counter– Confirmation of prediction

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

11

Nearly always the same!

What are the costs of a variable?• Within regular or adaptive grid?• Within an embedded geometry?

A cache-aware algorithm for PDEs on hierarchical data structures Frank Günther

12

Conclusion and outlook

Efficiency by methodology

Efficiency inhardware

Numerical efficiency

Fluid structureinteraction

Full adaptivityadv. treatment of

boundaries

More equations

Parallelization

3D

Technology with high potential

PDE-solver with space-filling

curves