8/16/2019 97P F-16 DATAB

1/18

F-16 DATABASE TERRAIN CUEING

AN INVESTIGATION O F DISPLAY HANDLING QUALITIES

Maj Kevin Christensen, USAF M)

Capt Gregory Weber, USAF

Mr. Michael Seelos, USAF

416th Flight Test Squadron

Edwards AFB, CA

Mr. Sean G illen

Lockheed Martin Tactical Aircraft Systems

Ft. Worth, TX

Figure

1, F-16 Block SOD

Background

In response to several mishaps in which perfectly good

F-16s

were flown

into the ground, the

F-16

ystem Program Office decided

to

integrate the Digital

Terrain System (DTS ) into all

F-16s.

Since DTS was an “off-the-shelf’ purchase,

this integration effort was considered a

low

cost way to reduce controlled flight

into terrain mishaps, especially when compared with the cost of such accidents.

Although not the primary function

of

DTS for reducing controlled flight into

terrain mishaps, Database Terrain Cueing DBTC), aided pilot awareness of the

surrounding terrain by providing a cue in the head up display (HUD).

118

8/16/2019 97P F-16 DATAB

2/18

The F-16 Combined Test Force at Edwards Air Force Base, California

recently completed Phase 1 flight testing

of

the DTS. During this testing, the DTS

team identified and corrected several handling qualities problems. This paper will

not only share the results of our DBTC testing, but more importantly offer

valuable lesson s for other flight test programs to use.

Figure 2 DTS

Data Transfer

Cartridge

Test Article Description

The heart of DTS was a portable black box called the Data Transfer

Cartridge (DTC) built by Fairchild Industries (Figure 2). The improved

DTS/DTC was a form, fit, and function replacement for the standard F-16 DTC.

Th e D T S D T C differed from a standard DTC in that

it

had a new mass memory

for storing a terrain database and a new microprocessor to drive the DTS. The

actual DTS software was contained in TERPROM (TM) software developed by

British A erospace Sy stems and Equipment Com pany.

119

8/16/2019 97P F-16 DATAB

3/18

Th e terrain database used by the F-16 DTS cam e from the Level 1 Digital

Terrain Elevation Data (DTED) developed by the National Imagery and Mapping

Agency (Figure

3).

The raw DTED represented the terrain by storing elevations

on “posts” along a grid

in

latitude and longitude. In the F-16 DT S, these terrain

da tab ase posts w ere spaced apart by about

500-600

feet

i n our

test area. The

terrain database also contained

a

Vertical Obstruction Database (VOD), which

stored the location and height of manually entered man-made obstacles such s

towers.

Figure

3

Digital FlightMap Database

Th e DTS consisted of five functions: terrain referenced navigation (T RN ),

predictive ground collision avoidance system (PGCAS), database terrain cueing

(DBTC), obstacle warning and cueing (QW/C), and passive ranging (PR).

Only

two of the five functions, TR N and D BTC, will be discussed further.

Terrain Refere nced Navigation

The primary purpose of the terrain referenced navigation (TRN) function

of the DTS was to provide accurate registration of the aircrzft within the terrain

database. TR N de term ined aircraft position relative to the terrain database by

using radar altimeter and inertial navigation

unit

NU) data

to

generate a terrain

profile of the actual terrain under the aircraft (Figure

4 .

This terrain profile was

then matched to the terrain database to determine estimates of corrections to INU

latitude , longitude, and elevation. As part of the TR N function, a Kalman filter

computed various uncertainties and error states associated with its estimates of

position corrections.

120

8/16/2019 97P F-16 DATAB

4/18

Figure 4 Terrain Referenced Navigation

Database Terrain Cueing

Database terrain cueing (DBTC) is a vertical steering guidance function

that aids the pilot in flying low level at a selected terrain clearance height (TCH).

DBTC improved a pilot’s awareness of

t he

terrain by providing a cue

in the

head

up display. Using the TRN position relative to the database, the DB TC function

generated a worst case terrain profile by simulating a sensor scanning the terrain

ahead of the aircraft (Figure 5). This scan area was adjusted to account for

uncertainties

in

TRN position corrections. Additionally, the scan was shifted into

a

turn based on turn rate. With this scan area of the database defined, the final

worst case profile w as reduced to a two-dimensional representation

of

the terrain

contained

in

the scan area (Figure

6).

The DBTC function then computed the normal acceleration or “g

command” necessary to clear the terrain in the worst case profile by the pilot-

selected terrain clearance height (TCH) within the limits of

2.0

to -0.9

incremental g’s. The end product of this DBTC calculation was a box displayed

to the pilot

i n the head up display (Figure

7).

Th e displacement of the DB TC box

from the flight path marker was proportional to the change

in

g needed

to

f ly

at

the TC H. Figure 7 shows an example in which the pilot must push about 0.5 g in

order to f ly the flight path marker into the DBTC box. Aircraft g was displayed

in

t he

upper left corner

of

the HUD.

121

8/16/2019 97P F-16 DATAB

5/18

Figure

5,

Simulated

Scan

Pattern

Figure 6 Two-D imensional Worst Case Profile

122

8/16/2019 97P F-16 DATAB

6/18

Terrain Cue Flight Path Marker

Current g

25

mils =

1.0

g

Figure 7, Head U p Display DBTC Presentation

Test Methods

The DBTC function was first tested by two different ground simulations.

Lockheed-Martin Tactical Aircraft Systems (LMTAS) used an unmanned, off-line

simulation to characterize the DBTC algorithm and compare DBTC performance

to that of terrain following systems. This simulation used a pilot model and did

not address the issues involved with displaying the cue to the pilot.

LMTAS

further tested DBTC on the ground by using their handling qualities simulator

HQ Sim). Th is simulator was a man-in-the-loop simulation which used actual

avionics and flight control hardware and software. Th e aircraft was modeled by a

six degree of freedom high fidelity model tha t had been used during F-16 Low

Altitude Navigation and Targeting Infrared for Night (LANTIRN) development

and flight test. This provided the first opportunity to assess the presentation of the

DBTC cue itself. Initial HQ Sim results highlighted the potential for a pilot-

induced oscillation to develop while following the DBTC cue at high

groundspeeds.

Initial DBTC flight test consisted of flying low level routes over isolated

peaks, rolling terrain, and rough terrain. Test points were chosen across the F-16

speed envelope, from 340 to 590 knots groundspeed. Th e DB TC function was

exercised at all TC Hs , from 100 to

1,000

feet. Finally, two aircraft loadings, on e

123

8/16/2019 97P F-16 DATAB

7/18

light and one heavy, were used to characterize how changes

in

aircraft

performance and mass properties would affect DBTC performance (Figure

8).

I

I

Figure 8, Two A ircraft Loadings Flown

In addition to flight testing

across

the aircraft envelope, two separate

techniques for following the DBTC cue were initially chosen to aid

in evaluating

the DB TC function. Th e D TC characterizatioii runs were accomplished to

validate the unmanned simulation performance results through a comparison with

actual flight test data. Pilots were tasked to follow the DBTC box as smoothly as

possible. This technique required the pilot to act as a compensator to smooth out

any abrupt movements of the DBTC box.

O n

the contrary, the DBTC

performance assessment runs were accomplished

to

allow pilots and engineers to

evaluate the acceptability of the DBTC cue under

a

more operationally realistic

scenario. Pilots were tasked to follow the DB TC box as closely as possible, as if

it were

a

manual terrain following cue.

Initial Test Results

The initial DBTC flight test effort took place from April through June of

1996. A total of 15 sorties and 27 hours were flown on a single Block 50 F-16D

test jet. This testing was a subset of the Phase

1

DTS flight test effort which

also

evaluated the other four DT S functions.

Figure 9 is a two-dimensional representation of initial DBTC performance

while flying over rough terrain. Th e x-axis shows distance along the low level

route in nautical miles. The upper curve show s aircraft pressure altitude, while

the lower curv e shows terrain elevation. For reference, the middle curv e shows

the 500 foot terrain clearance height added

to

ground elevation.

In

comparing the

top and middle curves, one can see that the aircraft flew well above the selected

TC H. In fact the second peak crossing was nearly

500

feet high. Additionally, as

the aircraft approached the peak, a late commanded climb resulted in a large

positive flight path angle at peak crossing. Th is

in

turn resulted in the aircraft

continuing to climb for several hundred feet more after crossing the peak, known

124

8/16/2019 97P F-16 DATAB

8/18

as “ballooning.” From a military

utility

standpoint, these DBTC performance

problems could degrade the pilot’s ability to terrain mask a t a specified TCH.

Over Roueh Terrain at 500 ft T C H

9000

8

-

-0

7

6

a

5

4000

1 2 3 4 5 6 7 8 9 1

Dis tance

NM)

Figure9, Initial DBTC Performance

In addition to these DBTC performance issues, pilots found the DBTC cue

to be jittery and too sensitive when they attempted to follow the cue as closely as

possible. From an operational perspective, the test team did not think that pilots

would try to f ly the box as closely as the test pilots were. How ever, the jerky

movement of the box was a big distraction which could divert the pilot’s attention

away from the actual terrain. Th e pilot would be able to do a better jo b of terrain

masking by visually referencing the actual terrain.

Most

significantly, if a pilot

did try to follow the box closely, flight safety could be jeopardized by a divergent

pilot-induced oscillation

PIO)

that developed at various airspeed and terrain

combinations. Th e test team determined that the potential

P I 0

should be further

investigated and characterized.

Handling Qualities Evaluation

In order to investigate the handling qualities .(HQ) of the terrain cue, the

DTS team developed an HQ evaluation with a tight tracking task

to

force high

pilot gains. Th is HQ evaluation allowed the team to quantify HQ trends with

125

8/16/2019 97P F-16 DATAB

9/18

flight conditions using the Cooper-Harper H Q rating sca le and to identify any H Q

“cliffs”. Additionally, the team hoped that this HQ evaluation would generate

pilot comments in o rder to isolate and correct the specific HQ problems.

Th e defined HQ tracking task w as to keep the flight path marker inside the

DBT C cue “box” while flying wings level at a particular airspeed. Each test point

specified an airspeed and a terrain route. Terrain routes were selected to sam ple

isolated peak , rolling terrain, and rough terrain types. T o limit the scope of this

HQ evaluation, all test points were flown at 500 feet TCH. Th e

desired criteria

for this HQ evaluation was defined as following the cue with at least half of the

flight path marker (FPM)

in

the DBTC box fo r the entire route with no more than

five excursions per minute. This represented the pilot following the DB TC cue to

within

I-

0.2

g. The

adequate criteria

was defined as following the cue w ith the

flight path marker at least touching the DBTC box for the entire route with no

more than five excursions per minute. This represented the pilot following the

cue to within /- 0.4 g. Since PIOs were anticipated, pilots were told that

overshoots while correcting back to the box w ere not to be counted as excursions.

A thorough postflight video review was used to determine task performance.

Following each HQ test point, pilots were asked to give a quantitative

Cooper-Harper rating. These ratings were based on the tracking performan ce that

was attained and the amount of pilot compensation required to meet this level

of

tracking performance. Sinc e some of the HQ deficiencies seemed related to

turbulence, pilots were also asked to give turbulence ratings following each run

based

on

the definitions

from

the Department

of

Defense

Flight

Information

Handbook. In general, pilots rated the severity of turbulence,

s

light, moderate,

o r severe, based upon. its effect on aircraft attitude and altitude. Turbulence w as

also rated as either intermittent or continuous based on its frequency. Perhaps

even more important than the quantitative HQ ratings, the pilot comments that

these HQ evaluations inspired helped considerably in isolating specific HQ

problems.

Handling Qualities Evaluation Test Results

Th e D BTC HQ evaluation consisted

of 5

more sorties flown in June and

July

of

1996.

In

analyzing the results

of

this evaluation, the test team found

a

corre lation in three areas: turbulence, terrain roughness, and groundspeed.

The left plot in Figure 10 shows the correlation between Cooper-Harper

ratings and turbulence while flying over an isolated peak at 590 knots.

Unfortunately, even the Air Force Flight Test Center couldn’t control turbulence,

so there were not many data points. However, these points did show that handling

126

8/16/2019 97P F-16 DATAB

10/18

qualities degraded to Level 3 when moderate turbulence was encountered during

this test point. Pilots commented that as turbulence increased, so did jitter

in

the

DB TC box. On some test points, this jitter was so bad that it was nearly

impossible to track the box. In order to compensate for this jitter, pilots would f ly

the “average” of the box position.

Figure

10,

Initial Handling Qualities Results

Th e center plot in Figure

I O

shows the correlation between C ooper-Harper

ratings and terrain roughness while flying at 590 knots. Tes t points flown in

rough terrain resulted

in

Level

3

HQ ratings

for

all four pilots sampled. Th e pilots

commented that in rough terrain, the box moved quickly, sometimes in discrete

jumps, making it nearly impossible to follow. To compensate for this, pilots

anticipated box movements based on the terrain ahead.

The right plot

in

Figure

I O

shows the correlation between Cooper-Harper

ratings and groundspeed while flying over rough terrain. At the highest

groundspeed sampled, all four pilots agreed that DBTC handling qualities were

clearly Level 3. Th e pilots commented that at the high speed, any inputs to correct

back to the box caused

i t

to swap sides, resulting in a PIO. T o compensate for this

highly sensitive DBTC box, pilots had to lower their gains. If the P I 0 became

divergent, pilots had to completely back out

of

the loop before attempting to track

the box again.

Handling Qualities Improvements

One step

in

improving the DBTC handling qualities was to reduce

or

eliminate discrete jumps in the DBTC cue. The Lockheed-Martin engineers

determined that one possible source might be two states

of

the TRN Kalman filter.

127

8/16/2019 97P F-16 DATAB

11/18

First, the TRN map error state was added to the height of terrain within the worst

case profile used by DBT C. Additionally,

the

TRN map error uncertainty was

added into the calculation of the aircraft worst case height. During a single

iteration of the DBTC function, the worst case profile may jump

in

altitude

s

indicated by the dashed terrain profile in Figure 1 1 Similarly, the TRN estimate

of

worst case height may jum p as indicated by the white aircraft. After these

calculations are made, the DB TC function would generate a new g-command. In

the example

i n

Figure

I

DBTC commanded the white aircraft back to the preset

T C H over the dashed terrain by generating a large positive discrete ju m p in the

DBT C box. By removing these two calculations, Lockheed-Martin engineers

hoped that more consistent worst case profiles and worst case aircraft heights

would redu ce the magnitude and occurrence of discrete jum ps

in

the D BT C box.

.....-._._.

.

ap h r Un c e r t a in t yJ

,.

..

_._.

_.. ....

Map

ErrorState Jlrmp

Figure 11, Discrete Jump Example

Another step in improving the DBTC handling qualities was to reduce the

jitter in the DB TC cue. Figure 12 shows a pilotized block diagram of the DBTC

control loop. The very center of this and all things

in

the world is the pilot

himself. The pilot directly responded to

the

HUD cue and produced an input to

the aircraft. The top box represents the DB TC algorithm generating the required g

com mands. The bottom row represents the feedback of aircraft g.

This signal

was smoothed out by filtering high frequency noise. Th e difference between

comm anded and actual g was then used to generate the displacement of the DB TC

box in the HUD .

128

8/16/2019 97P F-16 DATAB

12/18

Figure 12 Simplified DBTC Control Diagram

In analyzing the DBTC jitter and P I 0 problems, Lockheed-Martin

engineers looked closely at time slices taken from different paths

of

the block

diagram . Th e time slices

in

Figure

13

were taken from a

590

knot test point over

rough terrain. Th e top plot shows the DBTC steering com mand in the HUD. The

second curve shows the feedback

loop

signal, actual aircraft

g

smoothed

out

by

the low pass filter. Finally, the bottom curve shows the pilot’s stick force

in

response to the HUD display. This was one of many examples of

PI0

seen during

flight testing. Looking at the plot of stick force, it was apparent that these inputs

had

a

big effect on actual aircraft

g.

The feedback of this aircraft g to DBTC

steering caused the box

to

rapidly move opposite of the pilot’s input, resulting i n a

PIO.

This can clearly be seen by comparing the top and second curves. This led

LMTAS engineers to target the feedback loop rather than the DBTC algorithm in

their quest to reduce the

P I 0

potential.

Since the P I 0 tendency had a definite correlation to increased

groundspeed, engineers added a velocity-dependent lag filter to the feedback loop

(Figure

14 .

Additionally, the existing low pass noise filter was modified

in

an

attempt to reduce the jitter problem coming from turbulence.

129

8/16/2019 97P F-16 DATAB

13/18

1 2 3 4 5

6

1 2 4 5 6

Time

sec)

Figure 13 Pilot Induced Oscillation Time Slice

Figure

14

Improved DBTC Control Diagram

130

8/16/2019 97P F-16 DATAB

14/18

The left graph in Figure 15 shows how parameters in the low pass filter

were changed to reduce the magnitude of high frequency noise. The parameters

of interest were damping ratio 6 ) and natural frequency 0,).he two curves

show the change from 0.7 to

1 1

damping ratio. Th e right graph in Figure

15

show s the velocity-dependent lag filter. At low airspeeds, the time lag T)was

minimal, but at higher airspeeds, this time lag increased to slow down the

response of the DBTC box to changes

i n

aircraft g. Thus, the inherent pitch

sensitivity of the F-16 at higher airspeeds was compensated by the lag

in

the

feedback loop.

Th e first step

in

optimizing the feedback filter parameters was to

f ly

them

i n

the HQ simulator. This allowed engineers to bound the parameters, thus

limiting

the

number

of

.sorties required to come up with the optimum

configuration.

Figure 15, Noise Filter Bode Diagram and Velocity-Dependent Lag Parameter

Handling Qualities Improvement Flight Test

On each flight test sortie, the pilot was given three DTSDTCs, one

containing the baseline configuration

for

the flight, and the other two varying on e

or

two parameters from that baseline. The pilots were not told the parameters on

any DTC until the optimization process was complete. Thus, by direct “blind”

comparisons of the three DTSDTCs flown by a single pilot on a single sortie, a

“best” configuration was chosen. Th e parameters that changed from

the

baseline

to

the “best” configuration were evaluated by the engineers. The configurations

flown in the next sortie were chosen based on the current “best” configuration and

the engineers’ predictions of which parameters to alter further. This process w as

131

8/16/2019 97P F-16 DATAB

15/18

repcated

until

a final sct of threc DTSDTCs was flown to pick the optimum

configuration.

Handling Qualities Improvem ent Test Results

The additional six DBTC sorties flown in follow-up flight test was money

well spent by the F-16 program office. Figure 16 dcm onstrates that the

optimization process really worked. Th e final “best” set of parameters improved

handling qua lities across the board. Although the final evaluations were only

flown in light intermittent turbulence, the jitter in the box was significantly

reduced and handling qualities while flying over an isolated peak at

590

knots

moved up to Level I . Furthermore, the sensitivity of the DBTC box at high

speeds was reduced, with no significant P I 0 tendencies noted. The third plot

in

Figure 16 shows that at our most demanding test point, 590 knots over rough

terrain, handling qualities improved from

Level

3 to Level 2. Th e problem which

kept this “best” set

of

parameters from having Level

1

handling qualities was that

the DBTC box still moved too fast for

a

pilot to follow. Th is is best illustrated by

the center plot

i n

Figure 16 which shows the correlation between Cooper-Harper

rating and terrain while flying at

590

knots. Over rough terrain, only adequa te

tracking criteria could be attained.

Figure

16

Final Handling Qualities

Results

132

8/16/2019 97P F-16 DATAB

16/18

Over

R o w h

Terrain at 500

ft TCH

9000

8

I

3

7

al

g 6

a

a

5

4000

1 2 3 4 5 6 7 8 9 10

Distance

(NM)

Figure 17, Improved DBTC Performance

The changes in DBTC also improved overall DBTC performance (Figure

17).

The new DBTC configuration commanded a climb earlier than the original.

This resulted in a near level flight path at peak crossing, allowing the pilot to fly

much closer

to

the preset TCH.

Future Testing

Phase

2

of DTS testing is currently underway at Edwards in

1997.

In this

phase, the test team hopes to find other ways to improve this system. Specifically,

the team plans to extend the DBTC HQ investigation down to lower altitudes to

see if there is any correlation between H Q and TCH. W e also hope to expand the

types of turbulence encountered during our future HQ investigations.

The test team is also planning on doing a military utility evaluation of

DBTC while flying with night vision goggles (NVGs) (Figure

18).

Th e test team

thinks that this is the area that pilots will find DBTC m ost useful. In high moon

illumination conditions, pilots can easily see and avoid terrain while flying at low

altitude with NVGs. However, current generation NV Gs lack the depth

perception that pilots are accustomed to in the daytime. Th is makes it more

difficult

to

jud ge altitude above the ground. Th e DBT C system should

compliment NVGs well, since rather than having to look at and interpret the

digital radar altimeter readout, pilots will be able to cross-check the DBTC

1

33

8/16/2019 97P F-16 DATAB

17/18

symbol to determine where they are in relation to their desired TC H. This

increased terrain awareness should reduce pilot workload while flying at low

altitudes with NV Gs.

Figure 18, F-16 Night Vision Goggles

Lessons Learned

This flight test effort not only provided an excuse to have fun flying low

and fast, but i t also generated numerous lessons for the test team to pass on to ou r

fellow flight testers.

Our first big lesson was that even “avionics” flight test may call for a

handling qualities evaluation. This is especially true for new displays designed as

flight directors to help reduce pilot workload. Another excellent example is the

flight testing

of

helmet mounted displays, especially if used for sensor slaving.

Our test team found that the HQ evaluation using the Coopei-Harper rating scale

was an excellent tool. Not only did i t allow us to correlate HQ with flight

conditions, but the pilot comments generated by our

HQ

evaluation allowed us

to

isolate and fix specific H Q problems. Of course this method has proven valuable

for classical flying qualities evaluations over the years, but i t also worked well to

wring out the HQ problems of our DBTC display.

1

34

8/16/2019 97P F-16 DATAB

18/18

In

the

flight test community, after watching several spectacular crash

videos, we have been programmed to th ink of a divergent PI0 as a catastrophic

event equal to loss of aircraft control. This is not always the case . Although we

saw divergent PIOs in our flight testing, aircraft control was never in question.

Stopping

t he

P I 0 was as easy as looking away from the DBT C box. Another big

misconception is that PIOs are caused by “high gain” pilots who jus t don’t have

the right “feel” for their aircraft. The test pilots were humbled because they

caused a P I 0 while trying to follow the box without having to try hard at all. Th e

test pilots found that at the most severe flight condition,

if

they even looked at the

box,

they

were in a PIO. Even our “low gain” pilot found himself in a divergent

PIO. The big lesson from this is that when humbled by a PIO, you can always

blame the engineers. Another big lesson is to sample a cross section of pilot

gains,

or

you may never uncover that hidden P I 0 tendency.

Our test effort proved once again that PIOs can and will happen during

flight

test, so

h a v e

a

plan to deal

with

t h e m .

O u r

encounters with PIOs called

for

some creative test planning which stressed the flexibility of our flight test effort to

its limit. Luckily our program had time in the schedule and could afford to add

more sorties without impacting any production releases. Other programs may not

have that luxury.

In

fixing our HQ problems, we came up with som e other lessons. First,

test teams need to refrain from jumping to hasty conclusions that may delay

finding the real fix. In ou r case , when we first discovered our HQ problems, our

initial reaction was

to

blame the basic DBTC algorithm. With thorough data

review, however, our engineers were able to

fix

the problem with filters in the

feedback loop.

A lesson relearned was the importance of ground simulation. This enabled

our team to not only check the

HQ

fixes, but also to optimize them before we go t

in to the air. The team’s ability to flight test several DBTC options on a single

sortie was another big

key

to our success, enabling pilots to conduct a direct blind

comparison of the new configurations. Other test teams should also look for ways

to flight test different options on the same sortie, significantly improving the fly-

fix-fly test approach.

Finally, teamwork between pilots and engineers was essential

in

correcting

the problems seen

in

flight test. Overa ll, through our teamw ork, we were able to

take this “off-the-shelf” piece of equipment and turn it in to something useful to

the pilot in the field.

135