an effective & interactive approach to particle tracking for dna melting curve analysis...
TRANSCRIPT
An Effective & Interactive Approach to Particle Tracking for DNA Melting Curve Analysis李穎忠DEPARTMENT OF COMPUTER SC IENCE & INFORMAT ION ENGINEER INGNAT IONAL TA IWAN UNIVERS ITY
DNA Melting Curve Analysis Used for the detection of DNA sequence variants
DNA Melting Analysis in Temperature-Gradient Micro-channel
Temperature-Gradient Micro-channel
Heater
Carrier (Bead/Droplet
) ThermometerSubstrate
2/54
DNA Melting Curve Analysis
Temperature
Fluo
resc
ent I
nten
sity
Melting Temperature
3/54
DNA Melting Curve Analysis
4/54
MotivationPeople label each particles (carrier) frame by frame
That is time-consuming
We design an annotation tool to reduce human effort
5/54
Related WorkParticle tracking
ParticleTracker: An ImageJ plugin for multiple particle detection and tracking [Sbalzarini et al., Journal of structural biology 2005]
u-track [Jaqaman et al., Nature Methods 2008]
Interactive video annotationTracking with active learning [Vondrick et al., NIPS 2011]Interactive object detection [Yao et al., CVPR 2012]
6/54
Proposed System
Userannotation
Detection of bounding circle of
the particle
Acquisition of labels at other frames by
tracking the particle
User correctionUpdate of tracker & labels
Acquisition of all correct labels
7/54
Detecting Bounding Circle of a Particle
Median filter
Otsu's method
Edgedetection
Least-squares fitting
Dilation
Erosion
8/54
Least-Squares Fitting of Bounding CircleAssume the coordinates of the detected edge are
Let and denote the center and the radius of circle respectively
{ (𝑥𝑐−𝑥1 )2+( 𝑦𝑐−𝑦 1 )2=𝑟 2
⋮
(𝑥𝑐−𝑥𝑁 )2+( 𝑦𝑐−𝑦𝑁 )2=𝑟2
⇒ { 2 𝑥1𝑥𝑐+2 𝑦 𝑐𝑦1+(𝑟2−𝑥𝑐2− 𝑦𝑐
2 )=𝑥12+𝑦1
2
⋮2 𝑥𝑁 𝑥𝑐+2 𝑦𝑁 𝑦1+(𝑟 2−𝑥𝑐
2−𝑦 𝑐2 )=𝑥𝑁
2 +𝑦𝑁2
9/54
Least-Squares Fitting of Bounding Circle
[ 2𝑥1 2 𝑦1 1⋮ ⋮ ⋮
2𝑥𝑁 2 𝑦𝑁 1] [ 𝑥𝑐
𝑦 𝑐
𝑟2−𝑥𝑐2− 𝑦𝑐
2 ]=[ 𝑥12+𝑦1
2
⋮𝑥𝑁
2 +𝑦𝑁2 ]
𝒛=(𝑨T 𝑨 )−1𝑨T𝑩
𝑨𝒛=𝑩
10/54
Possible Choices of TrackersLinear interpolation
Correlation filter based tracker [Zhang et al., ECCV 2014]
Normalized cross-correlation matching
11/54
Linear Interpolation
1 2 3 4 5 6 7 8 9 10 11 12
12/54
Linear Interpolation: User Correction
1 2 3 4 5 6 7 8 9 10 11 12
13/54
Linear Interpolation: Update of Labels
14
5 6 7 8 9 10 11 122 3
14/54
Linear Interpolation: Update of Labels
14 5 6 7 8 9 10 11 122 3
15/54
Linear Interpolation: User Correction
1 2 3 4 5 6 7 8 9 10 11 12
16/54
Correlation Filter Based Tracker
[Zhang et al., ECCV 2014]𝐺=h⊗ 𝑓
Input image
h=ℱ− 1(ℱ (𝐺 )ℱ ( 𝑓 ) )
¿ℱ−1(ℱ (𝑒−‖𝒙− 𝒙∗
𝛼 ‖)ℱ ( 𝑓 ) )
Correlation Filter
17/54
Online Update of FilterFrame 1
𝐻1=h1
18/54
Online Update of FilterFrame 2
𝐻1⊗𝐹
𝐻2=(1− 𝜌 ) 𝐻1+𝜌 h2
19/54
12
One-Way Method
20/54
12
One-Way Method
21/54
12
One-Way Method3
22/54
12
One-Way Method3 Re-train the filter
23/54
12 3 4 5 6 7 8
9 10 1112
1314
Two-Way Method
24/54
12 3 5 6 7 8
9 10 1112
1314
Two-Way Method
4
25/54
15 6 7 8
9 10 1112
1314
Two-Way Method4
2 3
26/54
15 6 7 8
9 10 1112
1314
Two-Way Method4
23
27/54
1
7 89
14
Two-Way Method4
23 5 6
10 1112
13
28/54
1
7 89
10 1112
1314
Two-Way Method4
23 5 6
29/54
12
3 4 5 6 78 9 10 11
1213
14
Two-Way Method
30/54
Normalized Cross-Correlation MatchingGiven a image f and template t, normalized cross-correlation (NCC) measures the similarity between each part of f and t:
𝛾 (𝑢 ,𝑣 )=∑𝑥 ,𝑦
( 𝑓 (𝑥 , 𝑦 )− 𝑓 𝑢 ,𝑣 ) (𝑡 (𝑥−𝑢 , 𝑦−𝑣 )−𝑡 )
√∑𝑥, 𝑦
( 𝑓 (𝑥 , 𝑦 )− 𝑓 𝑢 ,𝑣 )2∑𝑥 , 𝑦
(𝑡 (𝑥−𝑢 , 𝑦−𝑣 )−𝑡 )2
Template Input image Output NCC
31/54
Normalized Cross-Correlation Matching
Template
Frame 1
32/54
Normalized Cross-Correlation Matching
Frame 2
33/54
12
One-Way Method
34/54
12
One-Way Method
35/54
12
One-Way Method3
36/54
12
One-Way Method3
Update the template
37/54
12
3 4 5 6 78 9 10 11
1213
14
Two-Way Method
38/54
Failure in Tracking with Normalized Cross-Correlation
Template of particle 1
12
39/54
Combining NCC & Extrapolation
Frame t-2 Frame t-1 Frame t
12 1 2 1 2x x x
where
𝛿 (𝑢 ,𝑣 )=𝑒−‖(𝑢−𝑥 ′ ,𝑣− 𝑦 ′ )
𝜎 ∙ 𝑙 ‖2
40/54
Combining NCC & Extrapolation
NCC Score of predicted location
Combined score
41/54
ExperimentsEvaluate how much human effort our system can reduce
Simulate the process of annotating video with our system
Evaluation metricNumber of manual annotation
Count a tracked bounding box as a correct label if the distance between the centers of it and the ground-truth bounding box is not more than 10 pixels
42/54
MethodsInterp
CF-1way
CF-2way
NCC-1way
NCC-2way
NCC-Extrap-1way
NCC-Extrap-2way
43/54
The Order of LabelingFor those methods not restricting the order of labeling
Always correct the label with maximum center location error
For other methodsSame as the video display order
44/54
Video DatasetName # frames # particles # annotations
Droplet1 1203 15 635
Droplet2 637 53 4192
Bead 420 5 727
Video Droplet 1 is for parameter tuning which is performed using brutal force search
45/54
Parameter Tuning for CF-1way
Ground-truth correlation
46/54
Parameter Tuning for CF-1way
𝐻𝑡=(1−𝜌 ) 𝐻𝑡 −1+𝜌 h𝑡
47/54
Parameter Tuning for NCC-Extrap-1way
𝛿 (𝑢 ,𝑣 )=𝑒−‖(𝑢−𝑥 ′ ,𝑣− 𝑦 ′ )
𝜎 ∙ 𝑙 ‖2
48/54
Parameter Tuning for NCC-Extrap-1way
𝜙 (𝑢 ,𝑣 )=𝑤×𝛾 (𝑢 ,𝑣 )+ (1−𝑤 )×𝛿 (𝑢 ,𝑣 )
49/54
ResultDroplet2
(# annotations = 4192)Bead
(# annotations = 727)Interp 457 (10.90%) 88 (12.10%)
CF-1way 1475 (35.19%) 79 (10.89%)
CF-2way 1973 (47.07%) 112 (15.41%)
NCC-1way 56 (1.34%) 11 (1.51%)
NCC-2way 129 (3.08%) 21 (2.89%)
NCC-Extrap-1way 53 (1.26%) 9 (1.24%)
NCC-Extrap-2way 115 (2.74%) 20 (2.75%)
50/54
Error Analysis for NCC-Extrap-1way
51/54
Error Analysis for NCC-Extrap-1way
52/54
Error Analysis for NCC-Extrap-1way
53/54
Target
Error
ConclusionsWe designed a system for particle annotation in video sequences
Our system can reduce human effort in annotation
Combining NCC and extrapolation achieves the best result
It is better to annotate video in its display order
Future workUse polynomial curve fitting to predict the location of particle in the next
frame
54/54
Thank you for listening