ic analysts & the aquaint program dr. john d. prange aquaint program manager [email protected]...

IC Analysts &The AQUAINT Program

Dr. John D. PrangeAQUAINT Program Manager

[email protected]

http://www.ic-arda.org

2

• For ARDA and AQUAINT they are:– Intelligence Community (IC) Analysts

• For the Non-IC World there is (potentially) a similar type of Target Audience:– “Professional Information Analysts”– For Example:

• Investigative / “CNN-type” Reporters• Financial Industry Analysts / Investors• Historians / Biographers • Lawyers / Law Clerks• Law Enforcement Detectives• And Others

Target Audience for AQUAINT:Who are They?

3

Taking a Closer Look

What Do We See When We FocusDirectly In On OurIC Analysts?

IC Analysts

4

• First: There are different level of intelligence within the IC -- Strategic, Operational, Tactical.

– ARDA is focusing on Strategic Level IA’s

• Second: There is no stereotypical analyst even within our Strategic Level Intelligence Agencies.

– Clear, significant differences exist:: • Across the national IC agencies as well as within individual agencies• Across Single Discipline “INT” Analysts• Between Single “INT” Analysts and All Source Analysts

– These differences are greatly accentuated by thebreadth of all IC reporting requirements.

• Third: There are significant skill level differences among IA’s

– Yes, the most seniors IA’s are exceptional– But the junior IA’s aren’t bad either

On the One Hand:Major Differences Do Exist Among Our IC Analysts

IC Analysts

5

• We believe that these similarities are significant and strong enough that:

– Taken collectively they highlight key differences between Intelligence Analysts and the “Casual” Consumers of Information that is being fueled by the Information Revolution and targeted by the commercial world

– A common set of critically important Info-X problems for the IC can be identified and articulated

– Multi-agency R&D programs against these common Info-X problems can be developed to the benefit of all IC Agencies

IC Analysts

VS.

“Casual” Consumers Of Information

But If We Change Our Focus:Universal Similarities Can Be Identified

6

• They are far more than just casual users of information

• They work in an information rich environment where they have access to large quantities of heterogeneous data

• They are almost always subject matter experts within their assigned task areas

• They track and follow a given event, scenario, problem, or situation for an extended period of time

• They frequently have extensive collaboration with other analysts

• They are focused on their assigned task or mission and will do whatever it takes to accomplish it

• The end product that results from their analysis is often judged against the standards of:

Timeliness Accuracy UsabilityCompleteness Relevance

Intelligence Community Analysts:What do They have in Common?

IC Analysts

7

Information Exploitation (Info-X)

Presentation and Visualization

Information Discovery

AnalyticKnowledge

Information Retrieval

InformationUnderstanding

Assessmentand

Interpretation

Content Data Mark-up

Content Data Transformation

Synthesis and Fusion

IC Analysts

Data Filtering& Selection

Reporting and Dissemination

What Functions Does It Include?

Info-X is Focused on Content & Its Meaning!

8

Intelligence Community & The Intelligence Cycle

7Dissemination

2Planning &Direction

4Processing

3Collection

5Analysis

6Reporting

1Intelligence

Requirements

Focus of Info-X

9

“Information Spectrum”

Information Exploitation (Info-X)Its Role Within the Intelligence Cycle

Collection Processing Analysis Reporting

Part of the “INTELLIGENCE CYCLE

Raw Data

BITS

SIGNALS

SENSOR

PHYSI

CAL

DATA

OBJECTS

IM …

SIG …

HUM …

MAS …

OS ...

10

Collection:Five Basic Intelligence Sources or Disciplines

• Imagery Intelligence (IMINT)– Includes both overhead and ground imagery of all types– NIMA is IC functional manager for IMINT

• Signals Intelligence (SIGINT)– Information derived from intercepted communications, radar, and

telemetry– Includes subdisciplines of COMINT (COM=Communication), ELINT

(EL=Electronic), FISINT (FIS=Foreign Instrumentation Signals)– NSA is IC functional manager for SIGINT

• Measurement & Signature Intelligence (MASINT)– Technically derived intelligence data other than imagery and SIGINT– Employs nuclear, optical, radio frequency, acoustics, seismic and

material sciences– e.g. Distinctive radar signature of a specific aircraft system or

chemical composition of air and water samples– Central MASINT Organization (component of DIA) is focus for all

MASINT activities

11

• Human-source Intelligence (HUMINT)– Involves clandestine and overt collection techniques used mainly by

CIA, FBI, and the Defense & State Departments– Principal types of HUMINT collection:

• Clandestine source acquisition of information• Overt data collection by civilian and military personnel assigned to

US diplomatic and consular posts• Debriefing of foreign nations and of US citizens who travel abroad

and who access to foreign information• Official contacts/exchanges with foreign governments

• Open-source Intelligence (OSINT)– Publicly available information appearing in print or electronic form– Includes radio, television, newspapers, journals, internet, commercial

databases, videos, graphics, drawings– Major collectors are FBIS (Foreign Broadcast Information Service) &

NAIC (National Air Intelligence Center)

Collection: Five Basic Intelligence Sources or Disciplines (continued)

12





Presentation and Visualization

Information Discovery

AnalyticKnowledge

Information Retrieval

InformationUnderstanding

Assessmentand

InterpretationContent Data

Mark-up

Content Data Transformation

Synthesis and Fusion

Data Filtering& Selection

Reporting and Dissemination

IC Analysts

InformationRaw Data

BITS

SIGNALS

SENSOR

PHYSI

CAL

DATA

OBJECTS

IM …

SIG …

HUM …

MAS …

OS ...

13





Raw Data

BITS

SIGNALS

SENSOR

PHYSI

CAL

DATA

OBJECTS

IM …

SIG …

HUM …

MAS …

OS ...

Information

Single Discipline Analysis

All Source Analysis

… INT

IntelReport

IC Customers

Intelligence

Types of Intelligence• Current• Estimative• Warning• Scientific & Technical• Research

14

Types of Intelligence

• Current Intelligence– Addresses day-to-day events– Seeks to:

• Apprise consumers of new developments and related background• Assess significance• Warn of near-term consequences• Signal potentially dangerous situations in the future

– Presented in regular publications (daily, weekly etc.), in ad hoc memorandums/messages and oral briefings to senior officials

• Estimative Intelligence– Deals with what might be or what might happen; Starts with

analysis of available facts then migrates into the unknown or even unknowable

– Goal: Help policy makers / decision makers to navigate the gaps between available facts by suggesting alternative, plausible patterns that fit available facts and to provide informed assessments of the range and likelihood of possible outcomes

– Flagship reporting vehicle is National Intelligence Estimate

15

Types of Intelligence (continued)

• Warning Intelligence– Sounds an alarm or gives time critical notice; connotes urgency

– Often involves situations that might involve US military forces or might cause their deployment, threats to US facilities and personnel overseas, sudden and deleterious effects on US foreign policy (coups, ethnic violence, etc.)

– Warning analysis involves exploring alternative futures and low probability / high impact scenarios

• Scientific and Technical Intelligence– Information on Technical Developments and characteristics,

performance, and capabilities of foreign technologies including weapon systems or subsystems.

– Generally includes detailed technical measurements

– Derived from analysis of all-source data

– Generally produced in response to specific national requirements derived from weapons acquisition programs, arms control negotiations, or military operations

16

Types of Intelligence (continued)

• Research Intelligence

– Monographs & other in-depth studies

– Two specialized subcategories:

• Basic Intelligence

– Geographic, demographic, social, military and political data on foreign countries

• Intelligence of Operational Support

– Reports of all types that are tailored, focused, and rapidly produced for planners and operators

– Example: DIA support to military forces & its operation of JWICS (Joint Worldwide Intelligence Communication System)

17





Raw Data

BITS

SIGNALS

SENSOR

PHYSI

CAL

DATA

OBJECTS

IM …

SIG …

HUM …

MAS …

OS ...

“Machine Readable”

“Interpretable& Actionable”

“Human Understandable”

Info-X is FOCUSED ON CONTENT & IT’S MEANING

“Artificial Languages”Dominate

““Natural Languages”Dominate

Information

Single Discipline Analysis

All Source Analysis

… INT

IntelReport

IC Customers

Intelligence

Types of Intelligence• Current• Estimative• Warning• Scientific & Technical• Research

18

Natural vs. Artificial Languages

• Media formats, encodings, protocols, and other externals are heavily based upon “Artificial Languages”; that is:– Closed or formally controlled symbol sets and lexicons

– Well-defined syntax; Precise semantics

– Strict requirements for data integrity; Extensive use of error correction

– Readability and Understanding often depends upon access to keys, proper synchronization, knowledge of protocols and encodings being used, setup variables, etc.

• The understanding and interpretation of the content and meaning of data is heavily based on “Natural Languages”:– Open symbol sets, lexicons and syntactic/semantic rules

– Unconstrained in its construction; Includes frequent exceptions and irregular constructions

– Inherently, even intentionally, ambiguous

– Heavy reliance on unbounded context and external world knowledge

– No “silver bullet” solution to understanding. Increasing depth, breadth and accuracy of understanding is an Incremental Process.

19

IC Info-X Requirements:Pushing the Envelop Along Every Dimension

Each Dimension Consists of Multiple ComponentsAll are important to IC Analyst Community • Data Characteristics

– Quantity– Quality– Types / Quantity / Source of Errors– Source of Creation Control

• Data Coverage– Topics/Domains– Media– Types– Languages– Time Frame and Span of Interest– Emphasis on “Recall” as well as “Precision”

• Depth of Info-X– Number of Details Tracked Simultaneously– Human cognitive skills that must be emulated– Depth of Understanding / Interpretation– Amount of Interpretation / Judgment– Importance of Predictions / I&W– Tenets of Intelligence always apply (Accuracy; Timeliness; Usability;

Completeness; Relevance)

DataCharacteristics

DataCoverage

Depth ofInfo-X

CommercialSystems

State-of-the-ArtResearch Systems

IC Info-XRequirements

Space of Info-X

Systems

20

The Heart & Soul of Info-X:

• Current data volumes pose significant difficulties; And vast increases in data volumes of all types are projected for the future

• Number of analysts will almost certainly continue to decrease.

• Growing requirement to analyze language data in foreign languages and in native character scripts; but fewer analysts per language

• For all of our R&D efforts, Image understanding and interpretation continues to remain well within the purview of our IC analysts

• Growing requirements to fuse information across multiple media types, across multiple data objects, across time and space; Current system architectures make this difficult, if not impossible

• Broader, more complex reporting requirements; Reporting timelines are shorter; expectation of consumers of Intelligence products are higher

Currently Our Highest Quality Info-X “Systems” are our IC Analysts BUT . . .

21

In a Word: Intelligence Production is “Analyst Bound”

Providing Info-X Systems that Significantly Assist, Augment, and Complement the Work of our IC Analysts is Clearly the Answer . . .

But that’s easier said than done!

22

AQUAINT: Attacking the Data Chasm

Future

Fully Intersected;Automatically

Generated;Variable Structure/Format;

Full Context Responses

Full Context-Based

QuestionScenario

Level III

Full Context-Based

QuestionScenario

Fully Intersected;Automatically

Generated;Variable Structure/Format;

Full Context Responses

Level II

Variable NarrativeSummary;

Multi-Media Presentations;

Simple InterpretedResults

Cross MediaCross Document

Simple Judgement

Level I

Fixed Templatesor

Tabular Lists

Mulit-ValuedFactual QuestionsQuestions

Answers

Today

50/250 BytePassage from

Single TextDocument

SingleFactualIsolated

Questions

Data Chasm

Missing Data

MANY Heterogeneous Data Sources;

All Types, Sizes, Locations

IncreasingVolumes

(Petabyte & up)

Synthesis Across“Documents”/Media

ContradictoryDataReliability

of DataMultiple

Perspectives

23

AQUAINT: Focusing on QA Scenarios

• Requires handling a Full Range of Complexity & Continuity of Questions

• Need to understand & track the analysts’ line of reasoning and flow of argument

• QA System requires significantly greater insight into knowledge, desires, past experiences, likes and dislikes of “Questioner”

• Place much higher value on recognizing and capturing “background” information

• Questioner/System dialogue is now more than just a means for clarification

FactoidQuestion?

WhyQuestions

?

InterpretiveQuestions?

Judgement Questions

?

OtherQuestions?

Information Analysts

Predictive Questions

?

Overarching Context / Operational Requirement

24

• A Different Paradigm may be useful when handling QA Scenarios:

• Current Analytic Paradigm:

AQUAINT: Fostering a Different Paradigm

– Sequentially “Filter Down” to the

final result

Processing & Analysis

Data

Results

– Works when QA’s are independent, isolated activities

– Cast a “wider net” while searching

for “golden nuggets” (Answers)

AnswersSpace of Data Objects and Sources

How Wide to Cast the “Net”?

What Info to Retain? In what form?

For how long?

– Automatically Extract, Represent,

and Preserve “closely related”

background information within

context of the QA Scenario

Background

Discarded

25

Continuing . . .

• First Your Questions and Comments

• Then More on IC Analyst Functions and Methods and their relationship to the AQUAINT R&D Program

– Kelcy Allwein, DIA

– Frank Hughes, JMIC DIA

• Followed by Breakout Sessions with Brief Backs tomorrow morning