Authoring and Critiquing COA’s in SHAKEN:

A Storyboard showing work in progress

UT

May 2002

Disclaimers

• SHAKEN cannot be used to author some aspects of COA’s, e.g.– More sophisticated spatial knowledge (e.g. “Attack the

enemy’s eastern flank”; “Direct the fires deep within the battlespace”; “The 3rd Brigade Overtakes the Cavalry”)

• SHAKEN cannot simulate some aspects of COA’s, e.g.– Non-normative outcomes (e.g. “The attack might fail”)

Other components of the E2E system – such as the Umass Simulator and NuSketch – should contribute to these areas.

Our Focus at UT1. Authoring COA’s: We’re expanding the component

library with the Events, Entities, Roles, and relations that are required for describing BS situations and COA’s. We’re anticipating that spatial information will come from NuSketch.

2. Critiquing COA’s: Several sources can contribute here, including KANAL’s check of pre-req’s and expected affects, as well as Umass’s non-normative simulations. We’re planning to contribute pattern recognition methods which are applied to declarative descriptions of COA’s to identify their strengths and weaknesses. Also, we’re planning to contribute to methods for SME’s to author knowledge to improve these critiques.

Let’s discuss each of these in turn …

1. Authoring Battlespace Descriptions

• We’ve identified several areas of expansion for the component library– Goals

• things an agent intends to accomplish/prevent/etc.• has-goal, objective, Goal, Intentional (cliché), Unauthorized (cliché)

– Resources• things used/consumed/output/etc. by Actions• resource, by-product, relationship to participant slots

– Views• user-defined collections of participants• viewpoint, View (Aggregate)

Library Expansion (cont.)

– Time• Event time, Entity “time”, relative time, Temporal Roles• start-time, end-time, time-of-existence, duration, rate, frequency, etc.

– Space• Spatial frames (e.g. “geospace”), Spatial Roles, Granularity• Conduit, Corridor, Landmark, Obstacle, Axis, Hide, Cross, Approach, etc.

– Communication• Information, Carrier, Medium, Express, Interpret, Transmit, etc.• Encode, Decode, Language, Learn, Confirm, Monitor, Discover, Deceive, etc.

– Aggression• Trespass, Invade, Attack, Shoot, Impair, Damage• Protect, Defend, Die/Kill, Live/Survive, etc.

Library Expansion (cont.)

– “Meta-Events”• Assist, Maintain, Begin, Stimulate, Stop, Control, Motivate, Oppose, Delay• re-, counter-, • Reciprocal, Intentional, Unauthorized

http://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentations/models.ppt

`

An Example COA in SHAKEN• We’ve encoded General Otstott’s COA#1 for

Objective Jayhawk. A CMAP for it appears on the next slide.

• The CMAP is a good start, but it’s incomplete in two ways:– We could not well express some parts of the COA.

We’re continuing to expand the component library to add this expressiveness.

– We omitted all spatial aspects of the COA, anticipating that this information will come from NuSketch.

• Nevertheless, this COA is fully coded in SHAKEN (i.e. it’s not a mock-up) and it gives us a useful laboratory for further work.

Put the Cmap for the COA here

Enemy

Ally

A Brief Aside:Enabling SME’s to “speak loosely” while

authoring knowledge in SHAKEN

• Loose Speak (LS): the phenomenon that human listeners are able to correctly interpret a speaker's imprecise utterance.

• Speaking loosely while authoring knowledge requires the correct interpretation of a SME's imprecise utterance.

• The next couple of slides show a few examples of loose speak in authoring knowledge in shaken.

• For more detail please visit:• http://www.cs.utexas.edu/users/mfkb/RKF/MayWorksh

op02/Presentations/Pete-slides.ppt

Example 1 of LSGiven input: Following their attack, attack aviation will refuel and rearm

SME encodes the following CMAP:

Note: Attack-Aviation is encoded as the object of Refuel and Rearm, but the object of Refuel and Rearm should be Military-Unit

Example 1 of LS (Continued)

• The Loose-Speak mechanism recognizes the inconsistency, and asks:

If the user clicks on the “(See CMAP)” link …

The Updated Version of CMAP

Example 2 of LSGiven input: Fire support starts with the attackSME encodes the following CMAP:

Note: The temporally-starts slot is incorrectly used to link the two events directly.

Example 2 of LS (Continued)• The LS mechanism links the temporally-starts slot to the

time-intervals of Fire-Support and Attack, and asks the user to “Accept” or “Cancel” the new version.

Example 3 of LSGiven Input: Holding Attack

SME enters the new concept:

Example 3 of LS (Continued)

• The LS mechanism attempts to interpret the new concept:

If the user clicks on the “(See CMAP)” link …

Example 3 of LS (Continued)

2. UT’s approach to enabling SHAKEN to critique COA’s

• Knowledge for critiquing takes the form of patterns to match against COA’s. Each pattern has associated conclusions to draw or actions to take when the match is successful.

• For example, General Otstott’s critique of COA#1 included this observation for fire support:

• We envision that SHAKEN will generate this observation by matching the patterns on the next two slides with our COA#1 CMAP.

COA#1 makes effective use of attack aviation assets by attacking deep to neutralize the enemy artillery early. Attack aviation is available to assist attacking ground elements by the time the attack is joined.

An Example of a Pattern

Attack

Neutralize

Artillery-BatteryAviation-Battalion

Attack

Time-Interval

Time-Interval

causes

object objectagent

before

time

time

Enemy

plays

An attack by air assets that is able to penetrate and neutralize the enemy artillery before the start of the main attack offers good fire support.

“Main Attack”

element-of

object

Penetratecauses

Another Example of a Pattern

AttackFire-Support

Military-Unit

Time-Interval

Army-Division

Time-Interval

time

supports agent-of

timetemporally-starts

A military unit that supports an army division when it begins its attack offers good fire support.

agent

Pattern Matching• The critiquing patterns are unlikely to match a

COA exactly – flexible matching is required.• We’re compiling knowledge to enable flexible

matching. Some is sound, some is heuristic.• Matching knowledge is of the form X Y. X

might match with the KB and Y with the user's input or vice versa.

• The next couple of slides show the matching knowledge that we have compiled so far.

• For more detail please visit– http://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentati

ons/Project-talk-final.ppt

Matching Knowledge• Some slots are transitive, symmetric, and/or reflexive. For example,

has-part is transitive, so

• Some slots ascend through parts. For example, object ascends through is-part-of, so

• We have compiled extensive (sound) knowledge of the form:

For example,

More Matching Knowledge

• We have also compiled knowledge which encodes complex relationships. For example:

• Our matching knowledge also includes heuristic rules, such as:

Matching COA#1

through transitivityof causes

Matching COA#1 (cont.)

An Aside: Using the Pattern Matcher to Improve

SHAKEN’s Q/A system• The Q/A system does not look far for an answer to

a query. The knowledge we’re compiling for the flexible matcher can broaden its search.

• For example, a SME might ask of COA #1 “What attacks did the 23rd Battalion make?” This is expressed in KM as: (the Attack agent-of of 23rd)

• The current SHAKEN will return nothing, even though an answer exists. SHAKEN just needs to look a little harder…

Improving SHAKEN’s Q/A

“23rd”

Attack

agent

The flexible matcher can find an answer by applying the matching knowledge we have compiled.

“What attacks did the 23rd Battalion make?”

End of Aside.Now back to the main theme of generating a critique…

Generating the Report

Attack

Neutralize

Artillery-BatteryAviation-Battalion

Attack

Time-Interval

Time-Interval

causes

object objectagent

before

time

time

Enemy

plays

“Main Attack”

element-of

object

Penetratecauses

Whenever SHAKEN matches a pattern with a COA, it adds an entry to the final report. Associated with each pattern is knowledge about the dimension of COA critiquing (e.g. Simplicity) that the entry contributes to, and the strengthof that contribution (these are somehow tallied to score theCOA on the {--, -, +, ++} scale). In SHAKEN, this might be:

Pattern

Feasibility

Strength-Value*high

pattern

contributes-tostrength

The pattern itself (rooted at “Attack”) might be enclosed in a

big node

Generating the Entry in the Report

Attack

Neutralize

Artillery-BatteryAviation-Battalion

Attack

Time-Interval

Time-Interval

causes

object objectagent

before

time

time

Enemy

plays

“Main Attack”

element-of

object

Penetratecauses

We could have “canned text” associated with each pattern.A nice alternative would be to have a text template associated with each pattern, then to instantiate the template using the particulars of the COA that got matched.Here’s an example template:

Attack by Aviation-Battalion is expected to penetrate and neutralize Artillery-Battery before the start of Attack, thereby offering good fire support.

Note: our current text generator would be used to generate phrases for the

colored nodes. Those phrases wouldplug-in for the colored text.

Wouldn’t it be nice to have

A Tool for Editing Patterns

• The SME’s main task this summer will be authoring knowledge to critique COA’s. “Basic SHAKEN” (i.e. component library, CMAPs) provides most of the functionality required for SME’s to enter patterns.

• A “Pattern Editor” would provide a bit more. It would help a SME to design a pattern that successfully matches a COA, then to generalize it.

CMAP

Pattern

SHAKEN proposes correspondences between a CMAP and a Pattern. The SME is allowed to revise the Pattern, generalizingnodes, editing correspondences, etc.