chemnitz dec2014
Post on 21-Jul-2015
32 Views
Preview:
TRANSCRIPT
Visual Analytics as a Cognitive Science
Brian Fisher
SFU School of Interactive Arts & Technology and Program in Cognitive Science
UBC Media & Graphics Interdisciplinary Centre (MAGIC)
My Background• UG Biology, Medical Biophysics tech at CWRU Med • Scientific Programmer, Varian • Ph.D Experimental Psychology, UCSC • UWO & Rutgers Centres for Cognitive Science • Human-Information Interaction AKA “Cyberpsychology”
• Institute for Robotics and Intelligent Systems Networks of Centres of Excellence “a Cognitive Basis for the Design of Intelligent User Interfaces to Complex Systems” (SFU-UWO)
• 1999- Associate Director, UBC Media And Graphics Interdisciplinary Centre, Computer Science, Psychology, Institute for Computing, Information and Cognitive Systems
• 2004- SFU School of Interactive Arts and Technology and Program in Cognitive Science
My double life
• Ph.D in Experimental Psych
• Postdoc w Cognitive
Science society founder &
president
• Psychonomics Fellow
• VIS-related symposia at
Cogsci, & APS, papers on
cogsci of interaction
• Fuzzy-logic/Bayes models
• Postdoc funded by Inst. for
Robotics and AI
• VAST SC, VEC, VACCINE
• Led Dagstuhl “Interaction with
Information for Visual
Reasoning”, Cogsci-based
papers at VIS, CHI, BELIV.
Participants
“Big Data”: Volume, Velocity & Variety
• In 2011, data expected to be about 1.8 zettabytes (1021).
• In 2013, Internet traffic to reach 667 exabytes (1018)/yr.
• Comparison: US Library of Congress is ~10 petabytes (1015).
• “By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.” (McKinsey Global Institute 2011)
Challenges for computational approaches
• “3 Vs” challenge • Relevance, validity, reliability of data uncertain • Insufficient time to reach solution
• Model challenges • Multiple models to chose from • Assumptions may or may not hold
• “Wicked problems” challenge (Rittel) • Lack criteria to evaluate solution • Each problem is unique (no population) • Problem is not understood until solution is found
The information needed to understand the problem depends upon one’s idea for solving it. -- Rittel & Webber 1973
The ultimate challengeIn business, government & the professions specific people are responsible for the decisions that are made and how they are executed. These people are derelict in their duties if they only accept what the model tells them. Either we re-engineer society to accept a computer model as the ultimate authority or we find a way that human decision makers can exercise due diligence for computational as well as informational aspects of the problem.
An prehistory of visual analytics
Visualization history
• NSF meeting: “Visualization in Scientific Computing” • Nov. 1987 Computer
Graphics • First IEEE Visualization (now
SciVis) conference in 1990
“The purpose of [scientific] computing is insight, not numbers.” Richard Hamming
“Visualization is a method of computing.” Authors of report
Information visualization • 1990 Conference
on diagrammatic reasoning
• 1995 InfoVis Conference • “Information
Visualization: Wings for the Mind” Keynote by Stuart Card
Stuart Card’s view• Increase the memory & processing
resources available to users •Reduce the search for information by using
visual representations to enhance the detection of patterns
•Engage perceptual inference operations •Use perceptual attention mechanisms for
monitoring •Support manipulation of information
Infographics represent information (data, knowledge, opinions, etc.) in context to support understanding
Visual thinking: Infographics
Visualization literacy
• Build a “language” of collections of images that support thinking
• Diagrammatic reasoning science of how we understand complex diagrams
••iThinking as “smart seeing and Projecting”
Actively looking at external representations and projecting onto them makes us more powerful thinkers than thinking in our heads alone.
DDavid Kirsh example
Design Approach: Bertin
6
Bertin Semiologie Graphique (1967)
Cartographer, built description of how data should be represented visually
Jock Mackinlay, Stanford Ph.D. dissertation
Tableau software, used for our work with Boeing etc.
Design Approach: Edward Tufte
Big Data Example (Amaral)Metabolism
Experts are overwhelmed by sheer volume and complexity of data
Cartographic Representation (Amaral)
Guimera & Amaral, Nature 433, 895 (2005)
http://www.visual-literacy.org/periodic_table/periodic_table.html
Role of psychology in VIS
• Design based on theories (but no effective eval of those theories)
• Adaptation of methods from cogsci (but original methods not well understood)
• Rarely, ongoing collaboration with cognitive scientists
On the Death of Visualization (Lorensen 2004)
Can It Survive Without Customers?
• Visualization, alone, is not a solution.
• Visualization is a critical part of many applications.
• Visualization, the Community, lacks application domain knowledge.
• Visualization has become a commodity.
• Visualization is not having an impact in applications.
Visual analytics origins
Battelle PNNL R&D Agenda Panel
• In US, Panel meeting in 2004 •Brown, GMU, Georgia Tech, OSU, Penn State, Purdue,
SFU , Stanford, UC, UI, UM, UNC, UU, WPI •Boeing, Microsoft, PARC, Sandia Labs •CIA, DHS, FBI, NIST, NSA, unspecified
• Gave rise to • Industry Consortium •DHS Centre of Excellence
• Ccicada (Rutgers DIMACS) • VACCINE (Purdue et al)
• In Europe, EU Vismaster Coordination action • DFG Scalable Visual Analytics Priority program
Visual analytics
“This science must be built on integrated perceptual and cognitive theories that embrace the dynamic interaction between cognition, perception, and action. It must provide insight on fundamental cognitive concepts such as attention and memory. It must build basic knowledge about the psychological foundations of concepts such as ‘meaning,’ ‘flow,’ ‘confidence,’ and ‘abstraction.’ “ “Illuminating the Path” (IEEE Press)
“The science of analytical reasoning facilitated by interactive visual interfaces”
How are VA Information systems different?
• Development based on understanding of expert cognition in situ • Informed by current cognitive & social science • Engagement with community of experts • Emergent cognitive science of expert reasoning
• Clear support for analytical processes-- reasoning, collaboration & interaction • Graphical analog for analytic processes • Support “Human-information discourse” • Integrated across roles in the community
Visual analytics
Pg. 4 in Thomas, J., Cook, K., Institute of Electrical and Electronics Engineers, Dept of Homeland Security, & United States. (2008). Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Press. Retrieved from http://www.osti.gov/energycitations/product.biblio.jsp?osti_id=912515
Cognitive & Perceptual Sciences
Visual Information Systems
Graphic & Interaction Design
Mathematical & Statistical Methods
Effective Situated
R&D
http://www.vacommunity.org/HomePage
Visual analytics as a translational cognitive science
How to bridge informatics & psychology?
• VIS offers: • Implementations • Funding • Awesome
Research Questions
• Psych offers • Methodology • Theory • Phenomena • Cheap talent
Challenges: defining boundary objects, culture clash, publication venues, academic jobs for Cogs grads…
My approach: start in the middle!• Develop bridging
“Cyberpsychology” theory & methods
• Hope is that… • They are building
from each shore • Somehow we will be
able to align things
http://www.magic.ubc.cahttp://www.icics.ubc.ca
http://interaction-science.iat.sfu.ca
Bridging ideas from D-Cog
• Visualization literacy is a form of “Smart seeing and projecting” w external representation
• We propose 2 additional D-cog perspectives: • Agent-machine coupling: coordination of thought and
action in dynamic artificial environments •Cognitive architecture modeling and characterizing
“personal equation” of individual differences (e.g. perceptual expertise)
• Socially-distributed cognition •Grounded theory in Clark’s Joint Activity Theory
framework analysis of pair/group collaborative decision making
Agent-machine coupling in air traffic control
• Cognitive architecture from psychology • Extend to expert human performance
• Cognitive expertise • Visual expertise • Visuomotor expertise • Multimodality & modularity
• Test human capabilities in dynamic display environments
Controller/display systems in air traffic control
• NextGen ATC “fishtank” projection
• Change camera position for better view
• How will global motion affect tracking?
Liu, G. Austen, E. L., Booth, K.S. Fisher, B., Argue, R. Rempel, M.I., & Enns, J. (2005) Multiple Object Tracking Is Based On Scene, Not Retinal, Coordinates. Journal of Experimental Psychology: Human Perception and Performance. 31(2), Apr 2005, 235-247.
http://www.youtube.com/watch?v=tKJVB4id_TY
FINST theory of spatial indexing
Multiple object tracking (Pylyshyn)
Fit human tracking function (Lui)
... Then add display motion
Tracking vs object speed
Tracking in warped space
Tracking in warped space
Conclusion: We track in allocentric space
• Retinal speed of targets does not determine performance
• Motion of targets relative to each other does • But only if motion preserves good metric
characteristics of space • Explanation is at the level of a human -
display cognitive system
• “Pair analytics” sessions • Student visual analyst & trained
domain expert collaborate on analytic task
• Student “drives”, expert “navigates”
• Video session & capture screen
Social D-Cog in safety analysis
Joint Activity Theory (Clark)
• Language is an essentially collaborative activity, like playing duet or paddling canoe
• We work to build common ground so as to communicate effectively and efficiently
• Clark’s theory: • Defines kinds of common ground • Formalizes the notion of activity as a “joint action” • Describes the processes by which common ground is
developed through joint action
Bird Strikes
• http://www.youtube.com/watch?feature=player_detailpage&v=5jN0bqL9cM0
Tableau Bird Strike Pair Analysis
\IN-SPIRE Bird Strike Pair Analysis
Wade Internship•Video recorded and screen captured over 10
Paired Analysis sessions using both Tableau and IN-SPIRE
• Influenced design decisions on: • 777 •P8-A • 787 • 747-8
•Changes to pilot training manual
Wade Internship• Presented work to:
• 787 Engineers • Aviation Safety Community of Practice • Aerodynamics, Performance, Stability and Control
flight data recorder analysis group • Advanced Analytics group • UW Aeronautics and Astronautics students • Boeing Educational Network webcast (400+)
• 500+ people exposed to Visual Analytics, Paired Analysis for Aviation Safety
Joint activity analysis• Observe joint attention management by
project markers • Multimodal markers — mouse gesture & verbal
(content & prosody) • Vertical and horizontal markers
• We define event structure for analysis based on markers & actions
• Social constraints seen in self-talk incidents, provides effective verbal protocols for protocol analysis methods
Metrics include:
• Fluidity of the process.
• Management of joint attention.
• Situational awareness.
• Cognitive economics.
Publications• Kaastra, L. T. and Fisher, B. (2014) Tracking Joint Activities in Visual Analyses.
Proceedings of the Psychonomics Society meeting
• Kaastra, L. T. and Fisher, B. (2014) Pair Analytics as a Field Experiment Methodology.. In Proceedings of the 2014 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization
• Kaastra, L. T., Arias-Hernandez, R., & Fisher, B. (2012). Evaluating analytic performance. In Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization
• Arias-Hernandez, R., Green, T. M., & Fisher, B. (2012). From Cognitive Amplifiers to Cognitive Prostheses: Understandings of the Material Basis of Cognition in Visual Analytics. In Interdisciplinary Science Reviews, Vol. 37 No. 1.
• Arias-Hernandez, R., Kaastra, L. T., & Fisher, B. (2011). Joint action theory and pair analytics: In- vivo studies of cognition and social interaction in collaborative visual analytics. In Proceedings of the 33rd Annual Conference of the Cognitive Science Society
• Arias-Hernandez, R., Kaastra, L. T., Green, T. M., & Fisher, B. (2011). Pair Analytics: Capturing Reasoning Processes in Collaborative Visual Analytics. In (HICSS), 2011 44th Hawaii International Conference On System Sciences.
Cognition Perceptual
Science Methods
Social Science Methods
Computation and Visualization
Methods
Graphic & Interaction
Design Methods
Analysis “in the wild”
What we need
How to do it• Institutes/centres/programs that combine:
• Application experts • VIS people • Cognitive scientists • Designers
• Publication venues for translational research • Two-part conference approach
• Outreach to application venues • Return to VIS community
Thanks to SCIENCElab!• Dr Richard Arias-
Hernández • Dr. Nathalie Prevost • Dr. Linda Kaastra • Dr. Payam Rahmdel • Samar Al-Hajj • Nadya Calderón • Tera Marie Green • Ethan Soutar-Rau • Ali Khalili
• Barry Po • Aaron Smith • Andrew Wade • Doug Mackenzie
Cogs/IAT 885 Visually Enabled Reasoning
• Cognitive theory • Psychology of human reasoning • Modelling causality • Alternative (modal/hybrid) logics
• Analytic practice • Problems and datasets from SEMVAST, Boeing • Teams w paper, IN-SPIRE, Tableau Jigsaw
• Outcomes • Learn reflective analytic practice • Prepare for internship as analyst
Textbooks
• How We Reason. Philip Johnson-Laird, Oxford Press
• Causal Models: How People Think about the World and Its Alternatives. S. Sloman, Oxford Press R
• Human Reasoning and Cognitive Science. Keith Stenning & Michiel van Lambalgen, MIT Press
Cogsci society logo
top related