7/30/2019 e38443a1b957c33306c13ec9e2ba95c1

1/2

Grid adaptive interpolation filters

Chee Sun Won

A general framework for image interpolation in a uniformly spaced

image grid is presented. The proposed formulation is suitable for repre-

senting fractional (or sub-pixel) pixels as well as integer pixel interpo-

lations. Also, by fitting an interpolation kernel to the grid formulation,

finite impulse response filter coefficients can be readily determined for

a given sampling interval and filter length. As an example, a four-tap

lowpass filter is derived by fitting the Lagrange interpolation kernel

to a row (or column) expansion by 2 on integer pixel locations.

Introduction: Image interpolation is a popular technique for various

image processing problems including colour restoration for missing

pixels [1] and motion compensation for video compression [2, 3].

Image interpolations are performed on integer pixel locations as in [ 1]

or on fractional sub-pixels to increase the accuracy of the motion esti-

mation as in [2, 3]. Noting thatfinite impulse response (FIR) filter coef-

ficients can be determined by sampling (or approximating) a desirable

sampling kernel such as the continuous-time sinc function, it would

be useful if we have a formula to adjust a sampling kernel to the

layout and the scale of the image grid. Motivated by this requirement,

this Letter: (i) provides a general form of uniform interpolations for

both integer and sub-pixel locations in terms of the sampling interval

andfilter length, (ii) derives a grid adaptive four-tap filter from the pro-

posed formulation.

General formula for uniform interpolations: As shown in Fig. 1a, the

original image has a uniform grid with KvKh pixels. Here, the ver-

tical and horizontal distances between the two nearest known (exist-

ing) pixels are Tv and Th, respectively. Then, our goal is to insert

and interpolate Nv and Nh pixels within the intervals of Tv and Th,

respectively. So, after the interpolation we will have a total of

[Kv(1 +Nv) Nv] [Kh(1 +Nh) Nh] pixels and the sampling inter-

vals for horizontal and vertical directions will be changed to

Dh = Th/(Nh + 1) and Dv = Tv/(Nv + 1), respectively. For example,

Fig. 1b shows the case of Nv = Nh = 1 with Dv = Tv/2 and

Dh = Th/2.

Tv

Th

Th

h

v

Tv

a b

Fig. 1 Uniformly spaced interpolation

a Before interpolation with known (bold square) pixelsb Interpolation for missing (dotted square) pixels

All necessary interpolation parameters include {Kv,Kh,Nv,Nh,Dv,

Dh, Tv, Th} . Having de

fined all the interpolation parameters, a generalformula for interpolating the missing value y(kvTv + nvDv, khTh +

nhDh) with a 2D FIRfilter can be written as follows

y(kvTv + nvDv, khTh + nhDh) =

Mv

mv=Mv+1

Mh

mh=Mh+1

y((kv + mv)Tv, (kh + mh)Th)

P(mvTv nvDv, mhTh nhDh)

(1)

where 0 kv Kv 1 , 0 kh Kh 1 , 1 nv Nv , 1 nh Nh ,

and the support area of the 2D filter covers 2Mv 2Mh.

Owing to the computational complexity, a separable filter is fre-

quently adopted such that P(mvTv nvDv, mhTh nhDh)=Pv(mvTv

nvDv)Ph(mhTh nhDh) . Then (1) is equivalent to applying a 1D

FIR filter Pv with 2Mv-tap for vertical direction and then a 1D FIRfilter Ph with 2Mh-tap for horizontal direction. In this case, the

expression in (1) can be simplified as a 1D FIR filter for each

image row and column. So, for notational convenience, if we drop

subscript v and h, we then have a 1D FIR interpolation formula

as follows:

y(kT+ nD) =M

m=M+1

y(kT+ mT)P(mT nD) (2)

where n denotes the pixel location to be interpolated within two refer-

ence (known) samples, y(kT) and y(kT+ T). So we need to evaluate

(2) for all n = 1, ,N.

The merit of expression (2) is that it covers both fractional and integer

pixel interpolations. That is, pixels to be interpolated can be located

either in the fractional pixel positions (i.e. D , 1) or in the integer

pixel locations.In H.264/AVC a six-tap filter (i.e. M= 3) is used for the interpolation

of half pixels in motion compensation [2]. This corresponds to the

form of (2) with N= 1 and its symmetric filter coefficients are

P(D) = P(T D) = 20/32 , P(T D) = P(2T D) = 5/32 ,

P(2T D) = P(3T D) = 1/32 . Similarly, the four-tap filter of

the AVS (audio video system) [3] has the filter coefficients as

P(D) = P(T D) = 5/8 and P(T D) = P(2T D) = 1/8 .

Grid adaptive four-tap filter from Lagrange interpolation kernel: From

a mathematical point of view, the Lagrange interpolation is to generalise

the linear interpolation by approximating the sinc function [4]. The

Lagrange interpolation kernel is an Lth-order polynomial function deter-

mined by L + 1 sample values as follows:

P(t+ mT nD) = QmTnD(t)QmTnD(mT nD)

(3)

where QmTnD(t) =

k

(t (kT nD))/(t (mT nD)).

For any sampling grid layout and scale we can calculate the filter coef-

ficients by fitting (3) to the grid. For example, by setting the parameters

in (2) as T= 2, N= 1, M= 2 andD = 1, we have

y(2k+ 1) =2

m=1

y(2k+ 2m)P(2m 1)

= y(2k 2)P(3) +y(2k)P(1) +y(2k+ 2)P(1)

+y(2k+ 4)P(3)

(4)

Here, the filter coefficients P( 3), P( 1), P(1), and P(3) are samples

of (3) when thefi

lter function P is located in the middle of the interpo-lating pixel (i.e. t= 0). This leads us to a four-tap filter by fitting the

Lagrange interpolation filter kernel to ourfilter formulation with T= 2

andD = 1, and we have the coefficients as follows:

P(3) =Q3(0)

Q3(3)=

1

16= P(3), P(1) =

Q1(0)

Q1(1)=

9

16

= P(1) (5)

Experiments: We compared the performance for the linear averaging

filter (denoted as Lin-2 or linear-2), the H.264/AVC six-tap filter [2]

(denoted as H.264-6) with coefficients (1, 5 20, 20, 5, 1)/32, the

AVS four-tap filter [3] (denoted as AVS-4) with coefficients ( 1, 5,

5, 1)/8 and the proposed four-tap filter (denoted as proposed-4) with

coeffi

cients (

1, 9, 9,

1)/16. First, the magnitude responses of thefilters are shown in Fig. 2. For the pass zone performance, linear-2

has the largest attenuation, while AVS-4 has the largest amplification,

both of which may cause excessive smoothing and distortions. On the

other hand, the H.264-6 and the proposed-4 filters have relatively flat

responses in the pass zone. For the transition band characteristics,

H.264-6 and the AVS-4 have the most rapid transitions from the pass

zone to the stop zone and their pass zones extend beyond the normalised

frequency of 0.5 (see Fig. 2). This extended pass zone can cause an

aliasing artefact.

Peak-signal-to-noise ratios (PSNRs) are compared with well-known

test images in Table 1. All test images are expanded as the same

sampling layout and scale of the proposed four-tap filter (i.e. doubling

the number of rows and columns with T= 2, N= 1, M= 2, and

D = 1). As shown in the Table, although the filter length is shorter

than H.264-6, the proposed four-tap filter yields the highest averagePSNRs. This is because our filter coefficients (prop-4) are adaptively

determined such that the sampling layout and scale in the experiments

performed in Table 1 match exactly with those of the Lagrange inter-

polation kernel.

ELECTRONICS LETTERS 31st January 2013 Vol. 49 No. 3

7/30/2019 e38443a1b957c33306c13ec9e2ba95c1

2/2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.00

0.2

0.4

0.6

0.8

1.0

1.2

1.4

normalised frequency

magnituderesponse

linear-2

H.264/AVC-6

AVS-4

proposed-4

Fig. 2 Comparison of magnitude responses

Table 1: PSNR comparisons (first 12 images are from http://sipi.usc.edu/DATABASE/ and remaining 15 images arefrom http://www.imagecompression.info/)

F il e n ame Li n- 2 H 2 64 -6 AV S- 4 P ro p- 4 F il e n ame L in -2 H 2 64 -6 AVS -4 P ro p- 4

airplane 32.62 33.37 33.26 33.46 big_tree 38.33 39.16 39.01 39.37

akiyal #1 34.15 34.24 34.32 34.62 bridge 35.96 36.65 36.53 36.84

baboon 23.16 22.39 22.45 22.93 cathedral 37.92 38.61 38.5 38.85

barbara 26.19 25.01 25.17 25.85 deer 34.14 33.11 33.27 33.84

f or em an 3 3.2 2 3 3. 25 3 3. 25 3 3. 6 fireworks 36.56 39.79 39.45 38.79

f ru it _m ix ed 3 1. 4 3 1. 41 3 1. 42 3 1. 76 flow er _f ove on 4 6.1 4 7. 48 4 7. 33 4 7. 58

goldhill 31.65 31.11 31.15 31.62 hdr 44.36 45.79 45.63 45.82

h ou se 3 0.4 4 3 0. 53 3 0. 54 3 0. 83 l eave s_ is o_ 20 0 3 2.5 1 3 4.7 1 3 4. 53 3 4. 19

l ena 3 3.3 1 3 3. 72 3 3. 67 3 3. 93 l eaves _i so _1 60 0 3 1.8 9 3 3.4 2 3 3. 33 3 3. 24

ma n 3 1.3 8 3 1. 45 3 1. 44 3 1. 72 n ig ht sh ot _i so _1 00 4 1.5 8 4 4.2 3 4 3.8 9 4 3. 86

peppers 32.6 32.45 32.41 32.79 nihgtshot_iso_1600 37.12 37.05 37.08 37.42

tree 27.56 27.67 27.67 27.97 spider_web 47.26 51.08 50.66 50.57

artficial 36.32 36.92 36.88 37.02 zone_palte 8.36 7.34 7.44 7.91

big_building 35.08 36.82 36.62 36.37 AVERAGE 33.75 34.399 34.33 34.55

Conclusions: We have established a general form of uniform inter-

polation formula in this Letter. The formulation is useful to determine

the filter coefficients adaptively for a given image sampling layout

and scale. To demonstrate the power of our grid adaptive filter,

four-tap FIRfilter coefficients are derived by fitting the Lagrange inter-

polation kernel to the sampling scale of doubling the number of rows

and columns at integer pixel locations. In our experiments the grid adap-

tive four-tap filter yields the highest average PSNR values (even higher

than the six-tap filter).

Acknowledgments: This work was supported by the Basic Science

Research Program through the National Research Foundation of

Korea (NRF) funded by the Ministry of Education, Science and

Technology (20110025770).

The Institution of Engineering and Technology 2013

17 July 2012

doi: 10.1049/el.2012.2481

One or more of the Figures in this Letter are available in colour online.

Chee Sun Won (Department of Electronic and Electrical Engineering,

Dongguk University-Seoul, Seoul, 100-715, Republic of Korea)

E-mail: cswon@dongguk.edu

References

1 Su, C.-Y., Chang, M.-K., and Hong, C.-M.: Optimal integer FIRfilter-ing for colour interpolation, Electron. Lett., 2010, 46, (20),

pp. 1376

13772 Wiegand, T., Sullivan, G.J., Gjontegaard, G., and Luthra, A., Overview

of the H.264/AVC video coding standard, IEEE Trans. Circuits Syst.Video Technol., 2003, 13, pp. 560576

3 Wang, R., Huang, C., Li, J., and Shen, Y., Sub-pixel motion compen-sation interpolation filter in AVS. Proc. of IEEE ICME, Taipei,Taiwan, 2004, pp. 9396

4 Lehmann, T.M., Gonner, C., and Spitzer, K., Survey: interpolationmethods in medical image processing, IEEE Trans. Med. Imag., 1999,18, (11), 1999, pp. 10491075

ELECTRONICS LETTERS 31st January 2013 Vol. 49 No. 3