1. 1. Recovering Inner Slices of Translucent Objects by Multi-frequency Illumination Kenichiro Tanaka1,2, Yasuhiro Mukaigawa2, Hiroyuki Kubo2, Yasuyuki Matsushita1, Yasushi Yagi1 1Osaka University, 2Nara Institute of Science and Technology (NAIST)
2. 2. Goal 2 Scene Near Infrared Photo Inner layer recovery Result
3. 3. Related work 3 Layered scene Scattering sceneClear scene Single surface scene Already clear Narasimhan et al. 2005 Nayar et al. 2006 etc. Szeliski et al. 2000 Li et al. 2014 Our goal etc.
4. 4. Overview 4 Multiple high-frequency pattern projection Direct components Upper layer Inner layer Optimization
5. 5. Appearance of layered objects 5 Upper layer Inner layer Depth dependent PSFs Blurred and superposed observation Complex appearance Summation of all layers Blurry Deeper layer is more blurred by scattering
6. 6. Direct reflection Scattering High-frequency illumination 6 [Nayar et al.] Normal observation Direct components
7. 7. What is direct component?
8. 8. HFI separation with PSF model 8 Blurry scene Direct component Extraction around the center Pattern pitch Brightness Blurry scene (Spread PSF) Depends on pitch of pattern
9. 9. HFI separation with PSF model 9 Direct componentNon-blurry scene Blurry scene (Spread PSF) Non-blurry scene (Sharp PSF) Depends on shape of PSF Brightness Pattern pitch
10. 10. HFI separation for Layered scene 10 Upper layer Inner layer Normal observation Direct component Pattern pitch Brightness Different PSFs
11. 11. Multi-frequency illumination Different brightness of direct components 11 = + Smaller pitch Larger pitch = + Direct components
12. 12. Multi-frequency illumination Different brightness of direct components 12 (1, 1) (1, 2) (2, 1) (2, 2) = + = + Smaller pitch Larger pitch
13. 13. In matrix form 13 = (1, 1) (1, 2) (2, 1) (2, 2) Estimate via optimization
14. 14. Informative layers exist sparsely along to depth Optimization Estimate informative layers 14 (1, 1) (1, ) ( , 1) ( , ) Uninformative layers 1 Uninformative layer goes to zero 2 2 + 1 0arg min subject to From many candidate PSFs Direct components Brightness Clear layers
15. 15. Experimental Setup 15 Cooled CCD camera Target object Beam splitter Projector (with NIR light) Pair of same lenses
16. 16. Results 16 Normal NIR photo Inner layer Scene Upper layer Recovery
17. 17. Results Recovery of the painters signature 17 Upper layer Inner layerPart of paint
18. 18. Color image extension Mural paint covered by white mold 18 Scene Recovered original paint
19. 19. Applications Arts / History Oil paints Mural paints, ancient documents Forensic Evidence recovery Medical Skin Oil paint Ancient document 19 Skin layers Oil paint Disguised secret
20. 20. Summary Goal Recovery of clear inner layer Method 20 Multiple high-frequency pattern projection Direct components Upper layer Inner layer Optimization
21. 21. FAQ Frequently Asked Questions
22. 22. How many patterns do we use? We use about 15 pitches of patterns, and each of them, we shift the pattern for one-third of the pitch, so total projection is about 270 patterns. 22
23. 23. How the real world phenomena relates to this work? Its important thing. High frequency illumination method separates diffuse reflection and subsurface scattering. However, diffuse reflection is a kind of subsurface scattering in a small scale view. So, how to define them? They are defined by some scale threshold and in high frequency illumination, this threshold is determined by the pitch of the pattern. This is our interpretation. 23
24. 24. What type of projector do we use? We use a projector development kit, named Lightcommander, manufactured by Texas Instrument. FYI Camera: Aspen CG-6, Apogee Optical components: OptoSigma, Edmund Optics 24
25. 25. How many layers can be recovered? We recovered 3 layers in a stacked translucent paper scene. We expect our method can recover more layers. 25
26. 26. Why the result of oil paint is gray-scale? Because we use infrared light as a light source. Inner layer cannot be seen using visible lights, so the observation is not a superposition of layers. On the other hand, Using near infrared, inner layer can be slightly seen, so we use near infrared for observing oil paints. 26
27. 27. Did we apply to any real items? No, not yet. However, Now we are discussing with a museum researcher in Nara, a Japanese historical city. 27