21st international workshop on“single molecule spectroscopy and super-resolution microscopy in the...

Filling the usability gap: Bioinformatics solutions for Image-Scanning Microscopy,

Stochastic Optical Fluctuation Imaging, and Surface Single Molecule Experiments

Dirk HähnelIII. Institute of Physics – BiophysicsGeorg-August-University Göttingen

21st International Workshop on“Single Molecule Spectroscopy and Super-resolution

Microscopy in the Life Sciences”

Berlin 4th September 2015

2

Making it easy and user-friendly?


6. price < 10thd. USD

1. physicist

2. chemicist

3. artifacts

4. image stacks

5. dynamical biosystems

CSDISMRequirements Palm Storm

SIM SSIM

Sted Tirf SOFI

3

Image scanning microscopy


Schulz, O., Pieper, C., Enderlein, J. et.al. Resolution doubling in fluorescence microscopy with confocal spinning-disk image scanning microscopy. PNAS vol.110 no. 52 21000-21005 (2013).

Müller, C. B. & Enderlein, J. Image Scanning Microscopy. Phys. Rev. Lett. 104, 1–4 (2010).

4

Critical timing requirements


spin-disc

laser

camera

5

CSDISM micromanager plugin


micromanager • calibration • acquisition parameter setting

real time trigger• laser• ccd

6

Fast CSDISM


Imaging of dynamical biological systems:acquisition time > image reconstruction

real time trigger• laser• ccd• live image reconstruction

micromanager • calibration • acquisition parameter setting

7

Calibration


nonlinear LVM fit

coordinates (z,y,x) forimage reconstruction

few seconds later

8

Fast image reconstruction routing


benchmark:image reconstructionlogic Circuit ~5MHz

theoretically up to 45 parallel cameras possible

Or

frame n-1

frame n

frame n+1

intermediate image iteration with 90Mhz/Pixel• no memory swapping• virtual circuit routing

new data n

intermediate image n-1

Intermediate image nvery low key FPGA frame grabber + logic hardware

new data n+1

becomes

9

Integration requirements


2015 coming soon:• beta software• setup recipe• double resolution• complexity blackbox

10

Stochastic optical fluctuation imaging


Dertinger, T., Enderlein, J., et.al. Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI) Vol. 106 no. 52 PNAS (2009).

physical challenges:

1. subpixel resolution

2. linearize brightness

implementation challenges

3. timing / speed

4. memory allocation

5. cumulants computation

11

Physical: sub pixel resolution challenge


avg. of movie SOFI fSOFI (4x)

Rat hippoc. Neuron(Neuro-transmitter receptor subunit GABABR1) blinking QDot525

real spacefourier space real space

12

Physical: brightness challenge


linearized fSOFI

deconvolution

2nd order fSOFI image

(4x pixel)

averageimage

Rat hippoc. neuron, neuro-transmitter receptor subunits GABABR1 : QD525 (green), GABABR2: QD625(red)

13

Implementation: timing challenge


acquisition

image reconstruction

final SOFI image

acquisition image reconstruction

Final SOFI Image

Imaging today: no dynamics Imaging dynamical biological processes

live imagingreal time reconstruction

14

Implementation: memory challenge


subpixel = more gates subpixel = more timecumulants => data swappinglinearization => very complex

start

input frame

SCMOS input:3 Gpixel => SOFI image1,5 GByte => SOFI image

SCMOS subpixel:12 Gpixel => SOFI image6 GByte => SOFI image

…

end

15

Implementation: cumulants computation challenge


challenge:memory space < data 3D stack

cumulants are moments corrected by lower order moments

Tremendous matrix operations

16

Integration and development roadmap


camera link

FFTcumulants

linearizationmicromanagerintegration

coming soon

integration• standard FPGA card • beta software coming soon• setup recipe• complexity blackbox

17

Manually: single molecule measurements on surfaces


physical challenge:

• drift

• bleaching

implementation challenges:

• drift compensation

• photon economy

• repositioning

stage

coverslide

objective lense

PMC AOTF Laser

APD 1

APD 3

AP

D 2

AP

D 4

Z(r) Y(r)

X(r)

position

signal

sample

18

Objective: single molecule sample vs. urban environment


Copyright Berliner Zeitung

19

Automated: single molecule measurements on surfaces


measurement:• photon counting• orientation• imaging• …

positioning: autofocus

excitation: • intensity• polarization• exposure• …

TCSPC defocused image

multiple

events

20

Implementation: drift challenge


time (minutes)

drift

(um

)

lateralaxial

max error:lateral 6,7nmaxial 7,8 nm

mean:lateral 0,6 nmaxial 0,1 nm

21

Implementation: photon economy and repositioning challenge


reproducible measurements:

identical orientation pattern

on a coordinate for different

measurement events.

180nm 180nm

180nm 180nm

180nm 180nm 180nm

1…30

22

Acknowledgements
