This presentation has been put together as a common effort of Urs Ziegler, Anne Greet Bittermann, MathiasHoechli. Many pages are copied from Internet web pages or from presentations given by Leica, Zeiss andother companies. Please browse the internet to learn interactively all about optics. For questions &registration please contact www.zmb.unizh.ch .

Basic Introduction to

Image Processing

Presentation of multidimensional data

3D-data has to be presented in in a 2D-fashion for publication

on paper. The data set might be represented i.e. as image

gallery, top+side view, projection. A virtual light source and/or

shadows on a virtual projection plane are helping to recognize

spatial relations.

Interactive models, movies and animations can be “published”

on web-pages or into power point-presentations.

Image processing and analysis

After registration on the microscope the digital images are loaded to image

processing software for further processing. The data includes information about

pseudo color, pixel dimensions, time scale etc.

First image data get adjusted by background subtraction, contrast enhancement,

etc. Colors might be assigned; subvolumes selected; z-mismatchs corrected by

pixel-shifts.

The softwares offer different options to look at the multidimensional data sets.

i.e. slice viewer, gallery view, section view, projections, full 3D volume

representations, surface models, time bar, color coded overlays of several

channels, transparencies, ...

The software offers analytical tools for measurement and quantification: automated

counting of features, measurements of areas and volumes, tracing of filaments,

measuring of distances, evaluation of colocalization, ...

Automated Multidimensional

Data Processing

Dimensions:

xy = 2D

xyz = 3D

xyzt = 4Dxyzt! = 5D

micrograph processing softwares:

Imaris (Bitplane)*

Volocity (Improvision)NIH image **

BioImageXD **

•Campus-Lizenz an der Uni Zürich**scientific freeware on the internet

2-dimensional distribution of image Points (Pixel)

Digital images:

x

y

Detectors record a limited amount of image points (pixel number)

within a xy grid. Each image point has its own grey level (dynamic

range).

Increasing the amount of image points as well as the number of

grey levels leads to bigger image files and longer calculation

times.

256 grey levels are coded by 8 bit. 256 grey levels are presented

by a computer monitor.

Today, detectors are pushed to discriminate 1024, 4096 or more

grey levels. The human eye can discriminate about 60 gray levels

(6 bit).

Digital resolution

x

y

z

) ! z) ! z) ! z) ! z

3D Data set

The information within

the optical sections along

the z-axis can be used to

reconstruct a 3-dimensional image.

x

y

z

) ! z) ! z) ! z) ! z

) ! z) ! z) ! z) ! z

4D Data set

t1 t2

3-D stacks recorded along the time course

x

y

z

) ! z) ! z) ! z) ! z

) ! z) ! z) ! z) ! z

5D Data set

t1, t2, t3, ...

Wavelenghts adding another dimension of fluorescent data. Time laps of

multi-channel 3D stacks generate a 5D data set. Wavelenght information

is displayed as pseudo-colors.

) ! z) ! z) ! z) ! z

A voxel (= volume element) is the 3D-equivalent of the

2D-pixel. It is the smallest unit of a sampled volume.

The given maximal lateral (x,y) resolution of 0.2 µm

and the axial (z) resolution of 0.4 µm of a voxel

results in an elongated shape (point spread function).

Voxels

3D

-> each Voxel has

6 neighbor voxels

For the calculation and visualisation are the neighbor voxels

of great importance.

2D

-> each Pixel has

4 neighbor pixels

Neighbours

Presentation and effort:

* Simple presentations (fast, allows 2D-publishing):

gallery view, section view, projections

* Intense calculations (time consuming, for analysis):

full 3D volume representation, surface rendering,

shadowing, stereo view

* Animations (time consuming, analysis & presentation):

rotating 3D models, time sequences of 3Dvolumes

Galleries ofimages are the

most simple

data presen-

tation.

for xyz

xyt xy! ...

Image Gallery

Optical section

through a cube

containing fibers

Projecting the structures

of all sections to the ground

level („Extended Focus“)

Projecting optical sections to one plane

Average Projection:Simple to very complex mathematical procedures. Summing up the grey

values of all voxels with identical xy-coordinates along the z-stack,

divided by the numbers of optical sections.

Maximal Intensity Projection (MIP):Only the voxel in the z-stack, which has the highest grey value, will be

projected.

Background signal gets projected too and might cause noise/blur.

Suppress background first!!

Projection types

x1

y1 Z 1

Z 2

Z 3

Z 5

Projektion

y2

x2

x3

y3

MaximalIntensityPointProjection-> sharp image

Z 4

x1

y1 Z 1

Z 2

Z 3

Z 5

y2

x2

x3

y3

Z 4

Projektion Averagingmay lead toenlarged structures and background

Gallery presentation of a neuron

1 2 3 4

5 6 7 8

9 10

Maximal

intensity

projection of the

optical

sections

of the

neuron

Maximumintensity

projection

with one

sided

illuminationand

shadow.

(“easy3D”)

x

y z

stack of images

Image of the section

x

z

Section throughthe stack

gallery of images

Section through the stack along the y-axis

x

y z

y

z

Sectioning through

a stack of images- perpendicular

{

Y - Z

Computer

representation

of section levels

in XY, XZ, YZ

{

X - Z

X - Y{{ {

Intense calculation

for 3-D representations

1. Volume rendering

Ray tracing

2. Surface rendering

Segmentation of z-stacks

Depth encoding of voxelsShadowing

3. Animations

time course

rotations zooms etc.

Volume rendering

Even if fog (background) limits the visibility, we get an

idea of the structure of the trees.

Volume rendering

Ray Tracing

A virtual ray passing the volume accumulates the grey levels of the

voxels, normalizes the summed value and presents it on the screen.

ScreenVolume

Virtual ray

Volume rendering with adjustments of the grey values

Adjustment of the greylevel according to thedistance between voxel

and screen.

Adjustment of the greyvalue according to the greyvalue of the voxel just

passed. screen

Voxels hit by the virtual ray

Volume rendering - example

3D representation of a multifluorsescent cellmonolayer(4 channels)

Surface rendering

Creating objects with solid surfaces.

Surface rendering: Iso-Surface modelling

1st Step: Segmentation of the z-stacks. Identification of Voxels belonging to an object.The criteria for the identification is the grey

value of the voxel.

All voxels, whose grey value are higher (brighter) than the chosenthreshold belong to the object, the others belong to the backgroundand will be discriminated. This threshold value is chosen by thescientist.

(Neighborhood rule: If a voxel belongs to theobject, but one of its 6 neighbor voxels doesnot belong to the object, it will be defined as asurface voxel.)

2nd Step: Depth encoding of the Voxels.

The previously identified surface voxels have all the same greyvalue and would result as a non structured evenly grey image on thescreen of the monitor. Therefore, in a second step, the grey values ofthe voxels are adjusted according to the distance of the surface voxelsto the screen.

z

x

y

All voxels have the same grey value

Depth dependentadjustment of the grey

values.

distance (depth)

Surface rendering: Iso-Surface modelling

3rd step: Shadowing

The topology can be accentuated using a one sided shadowing effect.To do that, neighboring surface voxels are connected to form a polygon.The grey values of the surface voxels are adjusted dependent on the an-gle between the viewing direction and the normal of the polygon surface.

Viewing direction

and incident lightThe normal

to the poly-

gon and the

viewing

direction

include the

angle " .

Sur-face voxels define polygons

!

Surface rendering: Iso-Surface modelling

Thresholds 110 (red) und 60 (white) + Transparency

Representation of several surfaces

Surface models of the same dendrites using different threshold values

Which model shows the real surface ?

Threshold 68 Threshold 138

Surface modeling: setting the threshhold

Adequate Filament Imaging

Stereo-Representation

The 3D impression can be achieved squinting the eyes or using special stereo

viewers (or crossing the eyes).

The depth feeling can be simulated by calculating two separate slightly tilted

3D-models of the same scene as if they were viewed by the left eye and the right

eye. The final stereo pair can be observed using different techniques.

Stereo-

Represent

ation II

The 2 pictures

of the stereo

pair

are colored in

red & green and

superimposed.

The 3D im-

pression can be

achieved using

bicolor goggels.

Surface view combined

with the visualization of

internal structures

Surface view

Looking inside

Mo l,stained with acridine orange - 20 optical sections

Gallery view of 20 optical sections

3D-representation: x-y, x-z, y-z

x-y y-z

x-z

Section view of 20 sections

Looking inside

Transparency & slicer tool

Looking inside

... by using transparency

Animations - fly through

Volume and surfacerendering allow you toturn and zoom the dataset. Extreme Zoomallows you to virtuallyenter the sample.

Measurements

i.e.:

- Automated data segmentation

- Particle counting

- Size regognition

- Distance measuerments

- Filament tracking

- Movement tracing

" Results are visualized in the 3D model

" Results are listed as numbers in Exel-sheets

Colocalisation

The relation of the intensity values

from 2 channels are presented in

a two dimensional histogram.

In case of colocalization, the inten-

sity clouds of both channels are

overlapping.

Colocalization is not an absolute

fact but allways relate to voxel size

and resolution.

AnimationsAnimations are series of single images put together into a movie. The images might be a

volume view, a projection, a slice, a time point. The animation is done by just playing the

sequential data set, or by rotating 3D models or volume representations, by zoom-in & fly-

through motions, changing of surfaces and transparencies, etc.

Today#s computer allow to calculate and

represent animated sequences reasonably

fast. Movie files can be published i.e in

power point or on the web. Also interactive

file formats are possible.

Animation in time

Changes of a 3D-volume with time might be

presented as a gallery of projection views -or as a movie. Animation and stereo view

facilitate the recognition of spheric relations

in this context.

t1

t2

t3

t4

Analysis&Animation

Particlesrecognition

and tracing

in time

Deconvolution

What is to be gained?

• Increase in resolution x, y, z

• Noise is reduced

• The image formation process is optimized

(astigmatism, point spread function, ...)

Widefield fluorescent data can be improved a lot by decon-

volution.

Confocal data show less z-distortions, less out-of-focus blur,...

-> deconvolution shows only very little effect.

Fluorescent bead with a diameter of 0,1 µm

Convolution - Theory

•Measure object of known size, but smaller than the

resolution of the microscope (i.e. 100nm fluorescent

beads)

•Compare the microscope image with the

ideal/theoretical representation of the object.

•Determine the difference of the measured and the

real object.

•Correct unknown objects with the determined

difference.

Deconvolution procedure

measured

„real“

Deconvolution effect

3D, 4D, 5D- data reconstruction is time consuming!!!

=>Only correctly recorded images are worth to spend the time to deal

with the 3D presentation!!!

=> Keep your data small:

° Reduce image resolution (512 x 512 pixel = 262 kB).

° Crop images so that they containing only the most important structural details.

° Work with as less channels as possible.

° stay with 8 bit

=>Keep the coffee pot hot in order to wait patiently until the calculations are finished.

=>Use classical image processing tools to improve the quality of the images.

and: Don$t expect to much of a 3-D presentation.