Download - Tango[VivekKumar_CS-C_6Sem_MIT]
Vivek Kumar
• Are our devices location aware ?• In global respect only• GPS, Radio navigation, WLAN• What about local arena ?• Tango comes into play• Uses computer vision to enable mobile devices, such as smartphones
and tablets, to detect their position relative to the world around them without using GPS or other external signals.
Location Awareness
Introduction
• Development being done at Google under the aegis of ATAP - Advanced Technology and Projects wing.
• Johnny Chung Lee is the technical leader for the project.• Working with universities, research labs, and industrial partners
spanning nine countries around the world to build on the last decade of research in robotics and computer vision, concentrating that technology into a unique mobile device.
Collaborators
3D Motion and Depth Sensing
• Devices contain customized hardware and software designed to track the full 3D motion of the device, while simultaneously creating a map of the environment.
• About 2,50,000 3D measurements per second.• Sets of localised data combined to get a 3D path.• Depth sensing is achieved with special sensors.
Motivation
• Provides mobile devices a human-scale understanding of space and motion.
• Smartphones, Tablets will be able to SEE.• Physical world as a UI to computers.• Visual SLAM : Simultaneous Localization and Mapping.• Phones that don’t limit their boundaries to touchscreen.
Tango Developer Kit
Specs contd ...
• 7’’ Screen• NVIDIA Tegra K1 processor• 4 GB RAM• 128 GB Storage• Integrated Depth Sensing• Motion Tracking Camera with fish eyed lens• WiFi, BTLE, 4G LTE• Available only for developers
Specs contd ...
• Echo locator sensors, gyroscope.• Uses centralised sensors to figure out where the device you’re
holding is within a known environment.• Present tech allows us to put so much compute power and so many
sensors in a device that it can figure out it’s own environment.• Different algorithms are the basis of 3D rendering and modelling in
real time.• APIs available in C, C++, Java and Unity game engine.
Core Functionalities
• Motion Tracking
• Area Learning
• Depth Perception
Motion Tracking
• Project Tango’s core functionality is measuring movement through space and understanding the area moved through.
• The Project Tango APIs provide the position and orientation of the user's device in full six degrees of freedom, referred to as its pose .
6 degrees of freedom
Area Learning
• Project Tango allows devices to use visual cues to navigate and understand the world around them.
• Visual features of the area it is moving through is recognized when it sees those features again. These features can be saved in an Area Description File (ADF) to use again later.
Possible Use Cases
• Device's coordinate frame can be intentionally aligned with a pre-existing coordinate frame so that content in a game or app always appears in the same physical location.
• Another advantage is for multi-player experiences, where two users can localize to the same coordinate frame, shared through a cloud service. This allows multiple people to interact in the same physical space where all of their relative positions are known.
• Location-aware shopping experience.
Depth Perception
• Project Tango devices are equipped with integrated 3D sensors that measure the distance from a device to objects in the real world.
• The Project Tango APIs provide a function to get depth data in the form of a point cloud. This format gives (x, y, z) coordinates for as many points in the scene as are possible to calculate.
Use Cases
• A game might want to detect when the user is approaching a wall or other object in the area and have that be part of the gameplay.
• Combining depth perception with motion tracking, distances can be measured between points in an area that aren't in the same frame.
• Exact elevation data can be used in apps, games to embed intentional effects.• Point cloud can be filtered to get customized effects.
Applications
• Mapping the physical world• Indoor mapping - home, malls, workplace.• Visual annotation - Add direction overlays, see the plan for a play your team is
executing on the helmet screen, highlight "dangerous"(weaving, too fast, whatnot) drivers on the car dashboard.
• Integrate sensor data to extend human perception - Add an IR overlay. Sample sound across the room, do a volumetric display of noise levels. "see" the strength of your WiFi signal.
Applications contd ...
• Image post processing• You have a 3d map of an area, plus pictures of all textures - rearrange to your
hearts content - Interior Design.• 3D context aware mobile applications - travelling, meeting.• Gamification of local environment.• Role Playing Games can be played in local environment.
Applications contd ...
• Physical training manager - immediate feedback.• Relief & Rescue - 3D plan of house.• Application already developed :• Sensopia's Magic Plan - Interior Mapping• Developer Limbic's - Zombie Gunship IR game• 3D Printing
Applications Contd ...
• NASA being already using at ISS.• Only limited to imagination.
References
● Google ATAP - https://www.google.com/atap/project-tango/● Resource-Aware Large-Scale Cooperative 3D Mapping from Multiple Cell
Phones● Chao X. Guo, Ryan C. DuToit, Kourosh Sartipi, Georgios Georgiou, Ruipeng
Li, John O’Leary, Esha D. Nerurkar, Joel A. Hesch and Stergios I. Roumeliotis● Google unveils 'Project Tango' 3D smartphone platform
http://phys.org/news/2014-02-google-unveils-tango-3d-smartphone.html● Hollister, S. Google's Project Tango Is Now Sized For Smartphones.
http://www.gizmodo.com.au/2015/06/
References contd ..
● hands-on-googles-project-tango-is-now-sized-for-smartphones/ hands-on ● http://www.eweek.com/innovation/google-working-on-tablets-with-enhanced-visi
on-capabilities.html● CHISEL: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device
using Spatially-Hashed Signed Distance Fields● Matthew Klingensmith Carnegie Mellon Robotics Institute Google Project Tango
A Convenient 3D Modeling Device ● Deval Keralia, Khyati K.Vyas and Khushali Deulkar Bell, L. Google Project
Tango and Intel RealSense smartphone