Retinal Image Enhancement Using Laplacian Pyramidal Multi-scaling

Sharifuzzaman Khan

School of Technology & Health, KTH, Sweden leon7900@gmail.com

Uvais Qidwai KINDI Center for

Computational Research Dept. of Computer Science

& Engineering, Qatar University

uqidwai@qu.edu.qa

Hamed Muhammad School of Technology &

Health, KTH, Sweden hamed@sth.kth.se

Umair Qidwai Consultant Ophthalmologist

Isra Medical University Pakistan

drumair216@yahoo.com

Abstract—Early detection of retinal diseases is important to avoid complications and permanent vision loss. In this paper retinal neovascularization and molecular degeneration has been emphasized. Neovascularization is in form of randomly disoriented micro vessels in retina. So image enhancement techniques are excellent way to extract the vessels, find out blood leakages, determine direction of growth and estimate the growth rate with vessel localization. A comparative study has been done on prior retinal image enhancement algorithms. In this project multi-scale image analysis is used as main image enhancement technique with the help of Laplacian Pyramid. The target is achieved by translating an image into several image scales and reconstructing with enhancement tools available in MATLAB image processing toolbox. Results are evaluated with object background contrast ratio, contrast- noise -ratio and 2-D contour plot. The enhanced images appear as a better source for edge detection and vessel extraction compare with the primary image. For this project normal fundus images from publicly available database are chosen.

Keywords- Multi-scale Analysis, Laplacian Pyramid, Retinal image enhancement, Blind deconvolution, Contrast enhancement

I. INTRODUCTION Early detection of abnormal new vessels in diabetic

retinopathy, age-related molecular degeneration and hypertension or after intra ocular lens implant can prevent a person from acute or permanent vision loss. Most often retinal diseases are associated with vessels of the retina and retinal neovascularization (new vessel generation). It initializes form the tail end of the normal blood vessels with a disoriented penetration. Sometimes retinal images suffer from low-contrast micro vessels, non-uniform background, vessel intensity drifting and addition of noise. In semi-automatic retinal imaging process lighting artifacts, motion blurring and defocusing are very common. Advanced algorithms are introduced for detection of blood vessels, abnormal molecular generations and enhancement of vessel edges. Among them pattern recognition techniques, matched ltering, vessel tracking/tracing, mathematical morphology, multi-scale approaches, model based approaches and parallel/hardware based approaches are used in most algorithms [1]. Blind deconvolution approach using Maximum Likelihood Estimation technique delivers promising results for blurred

images including post-processing steps, image color space conversions, thresholding, region growing and edge detection [2]. Comparison has been done between histogram equalization, local normalization, linear unsharp masking, wavelet transform and contourlet transform. Images with contourlet have shown much improved results [3]. Another multi-scale approach is done using non-subsampled contourlet transform for retinal images considering directional feature detection [4].

In our proposed algorithm ‘multi-scale’ image analysis is used as main image enhancement technique following the principle of Laplacian Pyramid [5]. The target is achieved by translating an image into several image scales and reconstructing with enhancement tools available in MATLAB image processing toolbox. Results are evaluated with object background contrast ratio, contrast-noise-ratio and 2-D contour plot. The enhanced images appear as a better source for edge detection and vessel extraction compare with the primary image. Usually, vessel extraction is often the first stage of retinal image processing before pathological diagnosis algorithm is applied.

II. MULTI-SCALING It is obvious that images contain information at different

scales. With Fourier analysis different frequencies in higher levels can be observed. To extract that information from an image, normally filtering operation is applied and which have shown tremendous success in recent decades. In this paper, investigation is done with multi-resolutional analysis on gray scale images. Images have been constructed with changing frequency pattern (left to right) in the target image as shown in Figure 1. Different image scale has been introduced in blurring technique with the help of average filtering. The gray value differences are prominent with changing average filter radius. Image intensity profile algorithm is applied on each image with a scale of a pixel distance in the form of frequency distribution.

With the blurring process the curve line pattern in the image gets faded and blurred as well as the line frequency of the intensity profile decreases. The more the radius increase of the averaging filter the more frequency decreases. It’s very much clear that in different resolution of the image has shown different pattern segments. It is also clear that detection of

2014 IEEE Region 10 Symposium

978-1-4799-2027-3/14/$31.00 ©2014 IEEE 141

image details is optimal to a certain scale. an image into several scales assists deep imfeature extraction of that image.

Figure 1. Changing frequency pattern accordimage scales.

III. LAPLACIAN PYRAMIDAL DECOM

When an image is transfer to another imaor gains some information. So, It is importainformation for image analysis otherwise iimage reconstruction. In [5] Burt and Adimages can be divided into sub bands woperator in a pyramidal structure. In LapDecomposition (LPD) one image is filteredlow pass Gaussian filter and certainly it proof band pass-filtered images. The subsamfollow the sampling theorem, and it is decomposition all structures that are sampltimes per wavelength are suppressed bysmoothing filter [6]. From level to levdecreases or increases with a factor of 2. process one image is down sampled with a lead to a coarse level suppress some informasize of the image reduces to half. Each pyrammatching scales and to extract the features sbetween two immediate pyramid levels withcoarser one.

L(p) = G(p) - 2 G(p+1)

Let consider L (p) as the last imag

decomposition. G (p+1) is the low pass filterand half of its size. So, L (P) is the coarser stequation (1) [6]. Similarly the reconstructinverse of previous equation.

G (p-1) = L (p-1) + 2 G (p)

Decomposition of mage analysis and

ding to the changing

MPOSITION age scale it losses ant to identify that it hampers proper

delson shown that with a Laplacian placian Pyramidal d with a series of oduces a sequence

mple images must proven that for

led less than four y an appropriate

vel the resolution In decomposition factor of two and

ation as well as the mid level contains

subtraction is done h up sampling the

(1)

ge of Laplacian red image of G (p) tructure of G (P) in ion process is the

(2)

G (p-1) is the previous scale o× N matrix a pyramid level of decomposition step the smallesfor reconstruction till the origin

IV. PROPOS

A. Structure In our proposed algorithm orcertain level. Then the desirextracted with highpass filterinimages and enhanced with unimages are stacked into a set extracted frequencies are addedthe blood vessels. Along wadditional pre-processing stepshave been shown in Figure 2.

Figure 2. Block Diagram for th

B. RGB to Gray Conversion Output from fundus came

image. Every single pixel in Rthree colors- red, green and bluit is converted to gray level whvalue within 256 different grayblack and brightest will be winteger of gray scale is common

C. Green Channel Selection Previous research has show

channel brings the most contraimage has been processed throuin Figure 3.

•Gra•Tex•Con

Image Pre-Processing

•FreqDecomposition

•Freq•Uns•MulStak

Reconstruction

of G (p) in equation (2). For an N f 1D N+1 can be constructed. In st image can be of one pixel and nal image.

SED ALGORITHM

riginal image is sub sampled to red frequency components are ng process from the decomposed nsharp masking. The resultant of bandpass image. Finally the d with the main image to fortify

with this technique there are s inclued in our algorithm which

he Proposed Algorithm.

era is in color image or RGB RGB contains a combination of ue. To simplify image processing ere every pixel shows a possible y shades. Darkest shade will be

white. In image processing 8-bit nly used.

wn that [7] selection of green ast features of an image. Retinal ugh different channels as shown

ay Conversionxture Evaluationntrast- Enhancement

quency Scaling

quency Extractionsharp Maskingltiscale Frequency king

2014 IEEE Region 10 Symposium

978-1-4799-2027-3/14/$31.00 ©2014 IEEE 142

Figure 3. Channel separation in color retinal image. Blue plane (a), green plane (b), red plane (c).

And it is clear that red and blue channel contains noises with brighter and darker intensity respectively

D. Image Selection Before image processing it should be ensured that the

images are appropriate to go through image reconstruction. As there are latest and upgraded versions of fundus machines but sometimes it has been reported to have image artifacts mostly from machine user interface. Means the artifacts are the portion of the image which comes from the image acquisition process not from the retina [8].

E. Texture Evaluation Texture analysis can be a good way for image selection.

Usually user machine artifacts in fundus images come from motion blurring and defocusing [8]. An image texture explains the spatial variation pattern (pixel based) in image and provides information about the superficial roughness of the image. In motion blurring and defocusing, pixel intensity variation in image decreases as well as the texture.

Figure 4. Image texture evaluation. Main image (a), Motion blurred (b), Gaussian blurred (c).

Entropy calculation is done to measure the texture of the image. In Figure 4, first image (original) entropy value is 6.0318, in second and third image it increases to 6.1398 and 6.1752 respectively. It is observed that entropy value increases with the decreasing of texture and it can help to choose the suitable image from an image set.

F. Contrast Enhancement Evaluation Image contrast enhancement is one of the major parts of all

image processing. It doesn’t increase or decrease overall brightness of the image rather it defines the brightness of image object by which the object is separable from the image background in gray image. For multi-scale analysis it is very important to make a sharp difference between object and image background. Sometimes a certain level of contrast difference may be present between vessels and background but below the threshold of human perception [9].

Figure 5. Contrast enhancement techniques. (a) CLAHE, (b) De-correlation Stretch, (c) Histogram Equalization, and (d) Unsharp Masking.

The ‘Unsharp Masking’ [10] is good for the edges but for low contrast edges this tool is not adequate enough. Another algorithm is ‘De-correlation Stretch’ which focuses on the visual enhancement and improves color differences in image. In respect to noise, ‘Histogram Equalization’ is not suitable and neither immune to, but it is capable of preserve small details. Among others, Contrast Limited Adaptive Histogram Equalization (CLAHE) gives better result that we have used for our algorithm, proposed by Zuiderveld [11]. Here are some of the finest contrast enhancing method are discussed available in the MATLAB Image Processing Toolbox which are also be distinguished with naked eye observation as shown in Figure 5. But the process should be optimal because there are chances of noise addition and other image artifacts.

V. EXPERIMENT AND RESULTS The images we have chosen for the project are from

publicly available retinal image database [12]. Images are in compressed JPEG format captured with Canon CR non-mydriatic 3-CCD camera with a 45 degree FOV. Each image contains 768 X 584 pixels. As the contrast enhancement is done in pre-processing step measuring SNR will not be adequate enough to evaluate the result because noise is not uniformly distributed for retinal images. Rather, Contrast-Noise-Ratio (CNR) is useful method for assessing the ability of an imaging procedure to generate clinically useful image contrast. We have used only one image for our proposed method.

A. Image Scale Analysis and Enhancement Main advantage of multi-resolutional analysis is that the

image features can be decomposed into different resolutional pattern which are stored in different level of sub band images. Analysis can be done on each sub band independently with

2014 IEEE Region 10 Symposium

978-1-4799-2027-3/14/$31.00 ©2014 IEEE 143

enhancement and modification. Reconstruction is the inverse process by adding the scaled images. Our sample image is decomposed into three sublevels with Lapacian operator to get the detail image and gradually down sampled with a factor of 2. Each detailed image contains information of main image

in a form of lower scale (down sampled) image. In Figure 6 the first image in row A and B is ¼ th size and middle is ½ th size of main image but last one is as same as original one because it is the immediate detail image of 1st decomposition level. As per our main technique, filtering operation has been done on the coarser images to extract the vessel information. ‘Unsharp masking’ is applied to focus on the minor edge details of the coarse images as because the information of tiny micro vessels is stored in those high frequency components. This procedure is conducted on each sublevel in decomposition.

In reconstruction every sub bands are up sampled with 2 factor. In each up sampling process, pixels with thick vessels information are interpolated from every direction. So, the resultant image contains additive information (which actually is erroneous) in a way of thick and prominent blood vessels. On the other hand micro vessel components get distorted and it occurs on each up sampling step. In Error! Reference source not found. 6 the first image of row A has more red pixels (high intensity value) representing thick borders and edges compare to image in row B. So, to eliminate the false information filtering and sharpening are used. Because image intensity deviation from blood vessel wall to background is reduced in filtering process in each reconstruction step in row B images.

A

B

Figure 6. Up sampled coarse images without filtering (Row A) and images with filtering (Row B).

Sets of band pass images are formed in reconstruction

process shown in Error! Reference source not found. 6. And row B contains the high frequency components and minor details of the sample image. Frequency bands (vessel information) at these sub levels stack one over another to make a batch of extracted blood vessels as appeared in Figure 7. Finally, the batch is added to the original image as shown in Figure 8.

Figure 7. Image stack of different layers.

Figure 8. Resultant Enhanced Image.

B. Resultant Image Evaluation We have three sets of images in Figure 9 displayed in gray

level and also in pseudo color matrix scaling. In first column is our sample image which is enhanced only with the image ‘contrast’ technique available in MATLAB; in second column the contrast enhanced image has gone through frequency scaling with Palladian decomposition and reconstruction procedure but without frequency been extracted from sub levels, neither sharpened.

The last column is the resultant enhanced image with filtered and sharpened edges with proper reconstruction as our method proposed. It is visible that, the vessel edges hold a strong ground and the object background difference is prominent in third image. The image background also seems to be more uniformly illuminated for the enhanced image. For edges and object contour detection, 2-D contour plot is used to visualize the isolines of the images with 4 contour levels as shown in Figure 10. It is shown that blood vessel orientation is well defined and prominently visible in enhanced image where as other images contain dominating background over micro blood vessels and edges.

2014 IEEE Region 10 Symposium

978-1-4799-2027-3/14/$31.00 ©2014 IEEE 144

Figure 9. Comparison between images- Original image with only contrast enhancement (left), Laplacian construction without filtering (middle) and proposed method with filtering (right).

Figure 10. 2-D Contour plot of images.

C. Quantative Evaluations With naked eye observation from the above Figure 9 it

seems that the contrast level is higher in without filtered image (left) than the image with filter (right). So, it is worth to go for an Object Background Contrast Ratio (OBCR) with a simple algorithm as equation 3.

(3)

For this purpose same ROI (Region of interest) has been selected from each image.

The method is applied on the original image, image without

filtering and image with filtering enhancement. From the results from Table 1 it is clear that the contrast level is higher in the filtered image compare to the non-filtered images.

Table 1. Comparison in different contrast modalities.

Image OBCR Contrast-Noise- Ratio

Original 2.92 7.862 Non-filtered 4.860 9.2274 Enhanced 6.408 9.8802

Promising results (Table 1) has been shown after calculating contrast-noise-ratio (CNR) [13] of the images following

equation (4). Same pixel area has been chosen from the vessel section and nearest background as shown in Figure 11.

(4)

Figure 11. Local contrast ratio measurement.

VI. CONCLUSION AND DISCUSSION The concept of multi-scale image analysis has been

emphasized in this paper with the help of Laplacian Pyramidal decomposition and reconstruction. It is an efficient approach as the result of final modified image is better than the original image in a way of edge and curve detection. Along with edge enhancement the overall image quality is not very much disturbed, rather it improves image CNR at the end. It can be said that, this method is quite successful for retinal vessel extraction whereas vessel extraction is the first stage of retinal image processing before diabetic screening program. The method may also be synchronized with real-life applications of automated retinal vessel segmentation in healthcare technology. Multi-resolutional analysis is basically a junction point from where the image can be reconstructed with different tools in different directions. We have tried to maintain the design and description simple. Theory and methods are demonstrated more with pictorial representation rather than symbolic equations.

ACKNOWLEDGMENT The effort and contribution of our project members are

highly appreciated. We would like to thank Mr. Helder André, St. Erik Eye Hospital, Stockholm for his valuable advice. Also grateful for the co-operation we got from The Royal Institute of Technology and Karolinska Institute, Sweden.

REFERENCES [1]. M. Fraz, P. Remagnino, A. Hoppea, B. Uyyanonvarab

and S. Barman, "Blood vessel segmentation methodologies in retinal images – A survey," Computer

2014 IEEE Region 10 Symposium

978-1-4799-2027-3/14/$31.00 ©2014 IEEE 145

Methods and Programs in Biomedicine, vol. 108, no. 1, p. 407–433, 2012.

[2]. U. Qidwai and U. Qidwai, "Blind Deconvolution and Retinal Abnormality Detection in Blurred Retinal Images," Journal of Medical Imaging and Health Informatics, vol. 1, pp. 1-10, 2011.

[3]. Peng Feng,Yingjun Pan, Biao Wei, Deling Jin "Enhancing retinal image by the Contourlet transform," Pattern Recognition Letters, vol. 18, no. 4, p. 516–522, 2007.

[4]. C.-C. Lee, C.-Y. Shih, S.-K. Lee and W.-T. Hong, "Enhancement of blood vessels in retinal imaging using the non-subsampled contourlet transform," Multidimensional Systems and Signal Processing, vol. 23, no. 4, pp. 423-436, 2012.

[5]. P. J. Burt and E. H. Adelson, "The Laplacian Pyramid as a Compact Image Code," vol. 31, no. 4, pp. 532 - 540, 1983.

[6]. D. B. Jähne, Digital Image Processing -Multiscale Representation, 2005, pp. 135-153.

[7]. L. L. Simon, K. A. Oucherif, Z. K. Nagy and K. Hungerbuhler, "Bulk video imaging based multivariate image analysis, process control chart and acoustic signal

assisted nucleation detection," Chemical Engineering Science, vol. 65, no. 17, p. 4983–4995, 2010.

[8]. T. M. Clark, Ophthalmic Photography- Fundus photography instrumentation and technique, twin chimney, 2002, pp. 14-82.

[9]. H. Roehrig, M. Sundareshan and T. Ji, "Adaptive image contrast enhancement based on human visual properties," vol. 13, p. 573–586, 1996.

[10]. A. Polesel, G. Ramponi and V. J. Mathews, "Image enhancement via adaptive unsharp masking," vol. 9, p. 505–510, 2000.

[11]. K. Zuiderveld, Contrast limited adaptive histogram equalization, vol. Graphics gems IV, Academic Press Professional, Inc. San Diego, CA, USA ©1994, 1994, pp. 474-485.

[12]. J. Staal, M. Abramoff, M. Niemeijer, M. Viergever and B. v. Ginneken, "Ridge based vessel segmentation in color images of the retina," http://www.isi.uu.nl/Research/Databases/DRIVE/, vol. 23, pp. 501-509, 2004.

[13]. SNR and noise measurements for medical imaging: I. A practical approach based on statistical decision theory." Physics in medicine and biology 38.1 (1993): 71.

2014 IEEE Region 10 Symposium

978-1-4799-2027-3/14/$31.00 ©2014 IEEE 146