7/27/2019 ho1_HVS_2013

1/32

Human Visual System

4c8 Handout 2

7/27/2019 ho1_HVS_2013

2/32

7/27/2019 ho1_HVS_2013

3/32

Vision : The Human Visual System

(HVS) Light is focused onto the

retina

Retina consist of twotypes of cell

cones sensitive to colourand luminance, locatednear the centre of theretina

rods located near theperiphery of the retina,

much more sensitive tolight, luminance only, moresensitive to motion, lessresolution

Lens

PupilRetina

Optic NerveBlind Spot

7/27/2019 ho1_HVS_2013

4/32

Vision : The Human Visual System

(HVS)

Electrical Impulses from

the retina are chanelled

by the optic nerve to

the Visual Cortex The Visual Cortex does

a whole bunch of smart

things including

filtering, object

recognition, edge

detection.

Lens

PupilRetina

Optic NerveBlind Spot

7/27/2019 ho1_HVS_2013

5/32

Intensity Sensitivity of HVS

=

just noticeabledifference in

intensitya constant defining

a quantum for

perceived intensity

Given a background Intensity I,

the user is asked to increase

the intensity of the circle until

it is barely visible.

This experiment demonstrates

a phenomenon known as

Webers Law

intensity is the power of

the incident visible light

7/27/2019 ho1_HVS_2013

6/32

Intensity Sensitivity of HVS

We can express this as a differential equation

The solution is

p defines a perceptual intensity scale. Our perception of intensity is

linear wrt p. When we talk about intensity values in images we arereferring to this scale.

256 levels are sufficient and hence 8-bits numbers are commonlyused to define intensity ranges in images.

=

= log

7/27/2019 ho1_HVS_2013

7/32

Colour Sensitivity

Cone Cells in the eyes convert wavelengths of life

into 3 values known as a tri-stimulus

The tri stimulus values

encode the relativestrengths of each of the

3 colour basis.

Different colourscorrespond to different

mixtures of tri-stimulus

values.

7/27/2019 ho1_HVS_2013

8/32

The RGB Colour Space

attempts to mimic HVS requires the definition of 3 colour primaries

CIE RGB red = 700 nm, green = 546.1 nm, blue = 435.8 nm

must determine tristimulus (ie. RGB) values for amono-chromatic light source as a function of itswavelength. (perceptual studies)

These functions are known as colour

matching functions and can be used

to estimate RGB for anycombination of colours.

Webers Law also

applies

7/27/2019 ho1_HVS_2013

9/32

YUV and related colour spaces

By convention colour spaces for TV broadcast use a tristimulus of 1luminance (Y) and 2 chrominance values (U and V) to representcolour.

YUV was used so that Colour TV signals would be backwards

compatible on Black and White TV sets.

the luminance of a pel (Y) in the YUV space is approximately

= 0.3 + 0.6 + 0.1

Note: exact values of weights vary

the higher weight for green reflects the increased sensitivity of theHVS to luminance in wavelengths corresponding to the colourgreen.

7/27/2019 ho1_HVS_2013

10/32

YUV contd.

U and V values are defined below for PAL

= 0.5 = 0.625

Hence conversion between RGB and YUV is linear

=C

, where C =0.3 0.6 0.1

0.15 0.3 0.450.4375 0.375 0.0625

RGB values can be found from YUV values bycalculating the matrix inverse of C.

7/27/2019 ho1_HVS_2013

11/32

Examples of Conversions

Black (rgb = [0 0 0]) has yuv = [0 0 0]

White (rgb = [255 255 255]) has yuv = [255 0 0]

Shade of Gray (rgb = [x x x]) has yuv = [x 0 0]

Red (rgb = 255 0 0) has yuv = [76.5 -38.3 111.6]

Green (rgb = [0 255 0]) has yuv = [153 -76.5 -95.6]

Note: It is common to scale the U and V components sothat it fits inside the range 0 to 255 (add 128 to both

values)

7/27/2019 ho1_HVS_2013

12/32

7/27/2019 ho1_HVS_2013

13/32

RGB v YUV

RGB

YUV

7/27/2019 ho1_HVS_2013

14/32

HSV Colour Space

Often used in forimage analysis

H hue = the shade of

a colour (red, green,purple etc.)

S saturation = colourdepth (from washed

out/grey to vivid) V Value = brightness

of the colour

7/27/2019 ho1_HVS_2013

15/32

HSV Colour Space

Conversion from RGB is non-linear

= max(,,)

= min , ,

=

6 =

2 + 6 = 4 + 6 =

7/27/2019 ho1_HVS_2013

16/32

RGB v HSV

HSV

Hue Saturation Value

RGB

7/27/2019 ho1_HVS_2013

17/32

Colour Spaces for Compression

JPEG/MPEG etc. use the YUV (YCbCr) colour space because

spatial frequency sensitivity of the HVS can be exploited

Spatial frequency is measured in

cycles per degree. It can be measured

at any orientation.

N cycles

Spatial Frequency =

tan() = Cycle Period (metres)Viewing Distance

7/27/2019 ho1_HVS_2013

18/32

Spatial Frequency Sensitivity (Horizontal)

Grating increases in freq. Left to Right

Intensity decreases vertically.

Sensitivity is given by the perceived

height of the columns.

7/27/2019 ho1_HVS_2013

19/32

Spatial Frequency Sensitivity

HVS less sensitive to

chrominance than luminance

chrominance frequencies > 10

cycles/degree are not perceived

nominal max for luminance is100 cycles per degree

Max sensitivity is at about 5

degrees/cycle

Vertical Frequency Sensitivityis similar but HVS is less

sensitive to lower frequencies

7/27/2019 ho1_HVS_2013

20/32

140 160 180 200 220 24020

30

40

50

60

70

80

90

100

110

120

Greyscale(-)andSensitivity(--)

Column

50 100 150 200 250 300 350 400 450 500

Spatial Freq. Response

Mach Banding

Bands appear to be brighter on the left than on the right. This is due to spatial

filtering in the visual cortex. This phenomenon is simulated with simple filtering of

an image row using a low pass filter with a symmetric impulse response.

7/27/2019 ho1_HVS_2013

21/32

Original at full colour resolution

Consequences of Colour Sensitivity

512 x 512 x 3

= 0.64 MB

Original Image

7/27/2019 ho1_HVS_2013

22/32

Subsampling Colour Planes

2:1 in both

directions

Keep Discard

Downsample the U and

V chrominancechannels and leave the

Y channel alone

7/27/2019 ho1_HVS_2013

23/32

4:1 Colour

DownsamplingOK

512 x 512 + 256 x 256 x 2 = 0.31 MB (1/2 bandwidth of original)

Downsample the U andV chrominance

channels and leave the

Y channel alone

7/27/2019 ho1_HVS_2013

24/32

16:1 Colour

DownsamplingStill OK

512 x 512 + 128 x 128 x 2 = 0.24 MB (1/3 bandwidth of original)

Downsample the U andV chrominance

channels and leave the

Y channel alone

7/27/2019 ho1_HVS_2013

25/32

16:1 Luminance

DownsamplingNot good

128 x 128 x 3 = 0.04 MB (1/16 bandwidth of original)

Latex

Downsample all 3

channels evenly

7/27/2019 ho1_HVS_2013

26/32

Chrominance Downsampling

You will often see ratios in the description of

codecs

4:2:0 means 4:1 Chrominance downsampling

(2:1 along rows and columns)

4:2:2 2:1 Chrominance downsampling only along

the rows. ie. half the colour samples are kept

4:1:1 4:1 Chrominance downsampling alongrows. No downsampling along rows.

4:4:4 no Chrominance downsampling

7/27/2019 ho1_HVS_2013

27/32

Consequences of Spatial Frequency SelectivityActivity Masking

Noise harder to see in Textured areas due to reduction

in contrast sensitivity at higher spatial frequencies.

7/27/2019 ho1_HVS_2013

28/32

Measuring Picture Quality

Objective Measures Mean Squared Error

=1 ()

is the image, is the ground truth/reference image and N isthe number of pixels in the image

Peak Signal-to-Noise Ratio (in dB)

= 10log255

7/27/2019 ho1_HVS_2013

29/32

Measuring Picture Quality

Objective Measures of Quality do not in general align well withthe HVS

A 100 x 100 block of noise has been added to each image at two locations. Because of

activity masking it is much less visible in right image. Hence perceived quality of the

right image should be higher.

7/27/2019 ho1_HVS_2013

30/32

Measuring Picture Quality

However, the MSE and PSNR for both images will be the same because the variance ofthe noise is the same in both images.

These images show the difference between the corrupted images and the original.

MSE

3.7

7/27/2019 ho1_HVS_2013

31/32

7/27/2019 ho1_HVS_2013

32/32

Summary

We discussed HVS factors that influencecompression

human contrast sensitivity depends drops as

spatial frequency increases contrast sensitivity is less for chrominance than

luminance

We discussed ways of measuring image

quality necessary to quantify levels of degradation in

compressed images.