Neuromorhpic Retinal

Prosthesis

eingereichterHAUPTSEMINAR BERICHT

von

Julian Till Sprung

geb. am 03.03.1988wohnhaft in:Hessstr. 77

80797 MunchenTel.: 0160 97331476

Lehrstuhl furSTEUERUNGS- und REGELUNGSTECHNIK

Technische Universitat Munchen

Univ.-Prof. Dr.-Ing./Univ. Tokio Martin BussUniv.-Prof. Dr.-Ing. Sandra Hirche

Dr. Jorg Conradt

Betreuer: M.Sc. Indar SugiartoBeginn: 02.11.2012Abgabe: 18.01.2013

Abstract

Visual prostheses that employ the electrical stimulation of nerves of the visual sys-tem have evolved within the last roughly 30 years. A great variety of differentapproaches to create a meaningful visual perception for the patient are developed,still ongoing research is facing a multitude of challenges and drawbacks. Previousclinical experiments on implanted visual prostheses have shown that the in vivoperformance is still very limited on simple tasks such as single letter recognitionor shape recognition under perfect illumination conditions. One solution regard-ing the experienced limitations is the approach of using neuromorphic devices andstrategies. In this work a state-of-the-art overview on retinal prostheses is given andfurthermore the usage of a neuromorphic asynchronous event based dynamic visionsensor for applications in retinal prostheses is investigated. It is shown that the lowlatency of this visual sensor helps sustaining the critical temporal accuracy and thatits event type information flow can be integrated into a spike generation algorithmanalog to the human retina physiology.

Zusammenfassung

Sehprothesen, welche die elektrische Neurostimulation von Nerven des visuellen Sys-tems verwenden haben sich in den letzten 30 Jahren stark weiterentwickelt. EineVielzahl an verschiedenen Herangehensweisen mit dem selben Ziel, der Erzeugungeines bedeutungsvollen Sinneseindrucks beim Patienten, wurden entwickelt, dennochsteht die Forschung einer großen Menge an Herausforderungen und Ruckschlagengegenuber. Bisherige klinische Studien zu implantierten Sehprothesen haben gezeigt,dass deren in vivo Funktionalitat noch sehr limitiert ist auf einfachste Aufgabenwie das Erkennen einzelner Buchstaben oder von schlichten Formen unter perfek-ten Beleuchtungsbedingungen. Ein weiterer Losungsansatz zur Bewaltigung dieserBeschrankungen beinhaltet die Verwendung von neuromorphen Bauteilen und Strate-gien. In dieser Arbeit wird ein Uberblick uber den Stand der Technik von Retina-Implantaten beschrieben, sowie die Untersuchung eines neuromorphen, eventbasieren-den, asynchronen, dynamischen Visuellen Sensors fur den Einsatz im Bereich vonRetina-Implantaten. Es wird gezeigt, dass mit Hilfe der geringen Latenzzeit des Sen-sors die entscheidende zeitliche Genauigkeit erhalten und der eventartige Informa-tionsfluss in einen Algorithmus zur Erzeugung von Impulsen, anlog der neuronalenAktionspotentiale in der menschlichen Netzhaut, integriert werden kann.

2

CONTENTS 3

Contents

1 Introduction 5

2 State-Of-The-Art Overview 72.1 Visual Prostheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 Retinal Prostheses . . . . . . . . . . . . . . . . . . . . . . . . 82.1.2 Challenges and Drawbacks . . . . . . . . . . . . . . . . . . . . 10

2.2 The Silicon Retina (DVS) . . . . . . . . . . . . . . . . . . . . . . . . 102.2.1 Principle Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2.2 Technical Details . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.3 Exemplary Applications . . . . . . . . . . . . . . . . . . . . . 12

3 Neuromorphic Prosthesis Approaches 133.1 DVS in Retinal Prostheses . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1.2 Experimental . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Summary and Outlook 17

Appendix 19

List of Figures 25

Bibliography 27

4 CONTENTS

5

Chapter 1

Introduction

Over 0.16 million (0.2%) people in Germany and 39 million (0.58%) poeple in theworld are considered blind. In industrial nations, the major cause for blindness(about 40 - 50% of the cases) relies on retinits pigmentosa (RP), a mostly inheriteddegeneration of the photoreceptors in the retina, and most notably age-related mac-ular degenerartion (AMD) which leads to retina damage in the center of the visualfield, the macula, with increasing age. Due to the demographic change in Germanyand most other western nations the number of AMD patients will increase in thefuture. [FBWH12, WHO12, WHO10]

Yet, post-mortem examinations of the retina of blind patients have shown that 80%of the inner nuclear layer and approximately 30% of the ganglion cell layer remainintact even after years of blindness [Dow08]. Inspired by the succes of cochlearimplants that electrically simulate the cochlear nerve to evoke hearing perceptionsafter damage or degeneration of the hair nerve cells in the inner ear, the remainingretinal ganglion cells (RGCs) in blind people can be stimulated using a retinalprosthesis.

The field of visual prostheses has further evolved and contains a great variability ofdifferent approaches for stimulation location and technique. In spite of ongoing re-search a high number of challenges and a high degree of complexity is faced that cannot be managed yet. A more recent approach includes the usage of neuromorphiccomponents and algorithms. In this context neuromorphic refers to a bioinspiredtechnology meaning that the biological (most of the time) sensory information re-trieval and processing are being reproduced using electronic circuit components andmicroprocessors that function in the style of the corresponding biological body part.Such a device is the asynchronous dynamic vision sensor (DVS) that is presented inthis work and its employment for visual prostheses is investigated.

The human visual system is a part of the central nervous system and consists of (withincreasing level of information processing) the eyes (most importantly the retina),the optic nerve, the optic chiasma, the optic tract, the lateral geniculate body, theoptic radiation, the visual cortex and the visual association cortex. An illustrationof the visual system is shown in figure 2in the appendix with the corresponding steps

6 CHAPTER 1. INTRODUCTION

Figure 1.1: Schematic of the anatomy of the eye and magnification of the retinaincluding photoreceptors and nerve layers. Light to signal conversion occurs in therod and cone cells. Then the nerve layers identify simple shapes, such as brightpoints surrounded by dark points, edges, and movement. Taken from [Wik13b].

of visual information processing and representation.In more detail, a cross section of the anatomy of the eye is shown in figure 1.1and will not be described further in this work. For additional information see forinstance [WLH05]. More of note for this work is the physiology and the anatomyof the retina, latter consists of a multitude of different cell layers and is locatedat the posterior wall of the eye attached to the retinal epithelium, as shown inthe magnification in figure 1.1. Summarized from posterior to anterior, the retinaconsists of a layer of photoreceptors (rods and cones) that transduce light into releaseof neurotransmitters in synapses to bipolar and horizontal cells of the next celllayers. These cells are connected to the next layer of amacrine cells which on theother hand are connected to the retinal ganglion cells that send the resulting visualinformation as spike pulses through their axons along the optic nerve towards thevisual cortex (where the incoming image is represented in the consciousness). Fromphototransdution to spike generation in the RGC first feature extractions steps likelocal edge detection are performed due to different interneuron connections and theexcitatory and inhibitory responses of horizontal and bipolar cells. Furthermore,these small neuron networks act as adders, differentiators and integrators among alittle number of photorecptor inputs with a variety of possible overall responses todifferent photoreceptor stimuli. Exemplary an On and Off-center RGC is shown infigure 3 in the appendix where the RGC spikes whenever the retinal stimulus of thecenter are is high compared to the surrounding area and vice versa, respectively.

7

Chapter 2

State-Of-The-Art Overview

This chapter covers the state of the art development in the areas of electrical retinalprostheses, and a description of a neuromorphic visual sensor device that is integralto the strategy described in chapter 3.

2.1 Visual Prostheses

The working principle of a visual prosthesis is straight forward the transductionfrom light into electrical current pulses for neurostimulation somewhere along thevisual pathway to create a meaningful visual sensation for the patient. Yet theimplementation into a functioning prosthesis reveals many obstacles and a highdegree of complexity.The first accidental discovery of electro-cortical stimulation evoked visual percep-tions of small light spots that are often referred to as ’phosphenes’ was made by theFrench physician Charles le Roy in 1755 (even before the discovery of the electro-physical action in nerve cells by Galvani)[HS07].The first designed experiments to elicit visual sensations through electrical stim-ulation of the occipital cortex can be traced back to the German neurologist andneurosurgeon Forster in 1929 [MFF12]. Further research also experimented withstimulation at different locations along the visual pathway ranging from extraocularstimulation at the occipital cortex, subcortical and the optic nerve to intraocularstimulation directly at the retina.The implementation at the stage of the retina implies certain advantages over otherlocations. Due to its position at the beginning at the visual pathway it requires thelowest amount of information preprocessing. Also, through the advance of modernsurgical techniques the implanting at the retina has become comparatively ordinarydue to easier surgical access compared to extraocular areas inside the skull. Retinalstimulation also takes advantage of the known spatial ganglion cell position andits correspondence to the visual field making it therefore simpler to convert spatialimage information into spatial stimulation distribution. Although retinal implantsare only applicable in cases of photoreceptor degeneration with remaining (at least

8 CHAPTER 2. STATE-OF-THE-ART OVERVIEW

Figure 2.1: Retinal anatomy and retinal prosthesis implant locations. Epiretinalimplants are designed to interact with the retina from the vitreous side and arein closest proximity to ganglion cells. Subretinal implants are placed between theretina and the choroid, in the place normally occupied by photoreceptors. Takenfrom [WCI07]

partially) intact ganglion cell layer this work focuses mainly on retinal devices. Incases of damage to the RGCs or the optic nerve subcortical or cortical prosthesesare required.

2.1.1 Retinal Prostheses

For retinal devices two general approaches evolved based on the implant position:subretinal where the implant is situated between the retinal pigment epitheliumand the retinal cell layer directly replacing the degenerated photoreceptors andepiretinal where the device is implanted on top of the retinal ganglion cell layerinside the vitreous humor, see figure 2.1.Subretinal devices hold the advantage that solely the function of the photorecep-tors needs to be reproduced technically since the bipolar and amacrine cell layerare in most cases of PR and AMD still intact. Therefore, the first retinal implantswere subretinal implants that consisted solely of a light to current converting multi-photodiode array (MPDA) as proposed by the Australian Researcher Tassiker in1956 [Dow08]. But it has proven in real implementations that the ambient lightreaching the photodiodes is insufficient for effective stimulation due to the low pho-todiode efficiency compared to biological photoreceptors. Most recent subretinalMPDA contain an additional power source for stimuli amplification. As an ad-vanced example, the schematic of the subretinal MPDA system of the Universityof Stanford is illustrated in figure 2.2. This system processes the data stream com-ing from the video camera on a portable microprocessor. The computed image isdisplayed on the video goggles that contain a LCD microdisplay and illuminated

2.1. VISUAL PROSTHESES 9

Figure 2.2: Subretinal prosthesis system of the Daniel Palanker group of the StanfordUniversity. Taken from [Pal13]

with a pulsed near-infrared light. This way projecting the desired imaged on theretina where it is received by the photovoltaic pixels of the subretinal implant. Thepixels convert the infrared light into proportional biphasic electric currents whichstimulate the proximal retinal neurons that transmit the signal towards the opticnerve fibers.

Currently, the most renowned groups that are working with subretinal implantsare Optobionics (Chicago, IL, USA), the Palanker group at Stanford University(Stanford, CA, USA) and Retina Implant AG (Tubingen/Reutlingen, Germany).

An epiretinal prosthesis consists in general of four main parts: a camera mountedat the patients head, a processing unit, a power supply (and transfer system in caseof wireless connection) and the implant inside the eye that directly stimulates RGCsabove whom its located. Due to this RGC stimulation the preprocessing steps usu-ally performed by the retinal cell layers are missing and need to be compromised forby image processing (performed on an external portable microprocessor). Advan-tageous in comparison to subretinal implants are the possibilities to solely implantstimulating electrodes on the retina, thus allowing for smaller implants, since allother components can be located outside of the eye (or at parts that are simpler toaccess). This also allows updates on the image processing or stimulation algorithmwithout additional surgery. Exemplary the epiretinal Argus 2 system is shown infigure 2.3 and the EPIRET3 system, that uses unique chip storage in an artificallense, in figure 4 in the appendix.

Currently researching on epiretinal prosthesis are Second Sight (Sylmar, CA, USA),Epiret (Giessen/Aachen, Germany), the Australian Vision Prosthesis Group, Intel-

10 CHAPTER 2. STATE-OF-THE-ART OVERVIEW

(a) (b)

Figure 2.3: (a)The external equipment includes glasses, a video processing unit(VPU) and a cable. (b)The implant is an epiretinal prosthesis surgically implantedin and on the eye that includes an antenna, an electronics case, and an electrodearray. Taken from [Sec13]

ligent Medical Implants (Bonn, Germany) and the Boston Implant Project (Boston,MA, USA). [MFF12]Clinical studies including human implants of retinal prostheses have been conductedby Second Sight and Retina Implant AG [Dow08].

2.1.2 Challenges and Drawbacks

Although the advancement of normal state of the art visual prostheses among thelast decades some of the primarily faced challenges include limitations to the num-ber of electrodes due to crosstalk, biocompatibility of stimulating electrodes, cablesand encapsulation material, electrode degradation caused by physiological immuneresponse, power requirements, signal and image processing on portable devices, in-terference with remaining residual vision, objective functional evaluation that is notsolely based on subjective patient feedback and training after implementation andrecovery from the surgery until removal of the implant (typically after a period ofsome days to a few months). [Dow08]In addition, the multidimensional image processing features of the human visualsystem and the encoding into spike patterns are less exact known and more complexas for instance the auditory system, therefore making it more complex to reproduceits function technically.

2.2 The Silicon Retina (DVS)

2.2.1 Principle Idea

Almost all current camera devices function in a static frame based, light intensity ac-cumulating way, thereby producing high amounts of data, especially in combination

2.2. THE SILICON RETINA (DVS) 11

(a) (b)

Figure 2.4: (a) Working principle and realization as basic electric circuit of the DVS[ETH13]. (b) Signal conversion of the DVS [LBM12].

with high temporal resolution.The asynchronous embedded dynamic vision sensor (eDVS or DVS), which is alsocalled Silicon Retina by its inventors at ETH Zurich, is constructed to mimic retinaanalog signals creation and processing.As the human visual system is not frame based but rather event based. Regularcamera devices lack the temporal information (higher than the frame rate) of anevent occurring in each single pixel. The DVS functions event driven where everysingle pixel works independently. The human visual system also works based onlocal light intensity contrasts rather than with absolute intensity values enabling itto function over the range of 120 dB of illumination intensity producing contrastperception. The DVS mimics that behavior in a way that each pixel solely reportsan intensity change of a certain threshold for increasing intensity (ON event) ordecreasing intensity (OFF event) other than reporting about the absolute brightnesswhich is illustrated in figure 2.4 (b).

2.2.2 Technical Details

The most important technical specifications include 128×128 pixel resolution, 120dBdynamic illumination range, 23mW power consumption, 2.1%-contrast thresholdmismatch and 15 µs maximum temproal resolution.The analog circuit of a single DVS pixel is shown in figure 2.4 (a). Its workingprinciple is described by its inventors as: The pixel uses a continuous-time front endphotoreceptor,(inspired from the adaptive photoreceptor), followed by a precision self-timed switched-capacitor differentiator (inspired by the column amplifier used in thepulsed bipolar imager). The most novel aspects of this pixel are the idea of self-timing the switch-cap differentiation and self-biasing the photoreceptor. This pixel

12 CHAPTER 2. STATE-OF-THE-ART OVERVIEW

does a data-driven AD conversion (like biology, but very different than the usual ADCarchitecture). Local capacitor ratio matching gives the differencing circuit a preciselydefined gain for changes in log intensity, thus reducing the effective imprecisionof the comparators that detect positive and negative changes in log intensity. Thepixel is drawn to use quad mirror symmetry to isolate the analog and digital parts.Most of the pixel area is capacitance. The periphery uses the Boahen lab’s AERcircuits. The chip includes a fully programmable bias current generator that makesthe chip’s operation largely independent of temperature and process variations; alldozen chips we have built up into boards behave indistinguishably with identical digitalbias settings. [ETH13]

2.2.3 Exemplary Applications

A variety of different applications is created with the usage of the DVS that takesadvantages of the low latency and the comparatively low amount of produced data.Among these are object tracking, low latency pole balancing (that is so far notpossible with regular camera systems due to the short time constants of the balancingversus the amount of data to be processed of frame based systems in order to extractpole position,angle and motion) and fast motion analysis. [ETH13]

13

Chapter 3

Neuromorphic ProsthesisApproaches

As mentioned before, neuromorphic algorithms, for instance saliency-based imageprocessing [PIW10], and devices have the capability to tremendously increase theperformance of recent and future prostheses devices. This chapter deals with theinvestigation of a DVS integration into a retinal prostheses.

3.1 DVS in Retinal Prostheses

3.1.1 Introduction

The function of the retina is not limited solely to the light into signal transductionbut performs first processing steps of the incoming visual information such as hor-izontal or vertical movements. This information processing is not only very precisein the spatial aspect but also regarding the temporal domain. In this context amillisecond resolution is crucial as experiments have shown that the typical maxi-mum photocurrent of the retinas photoreceptors is reached after approximately 30ms but small changes in the picoampere range that can change a cell membranepotential can already be measured after few milliseconds [LBM12]. A reliable mil-lisecond temporal resolution is not reached by regular small camera devices that areapplicable for visual prosthesis systems that have usually a frame rate in the rangeof 30 Hz (33 ms).The DVS with its reliable millisecond temporal resolution (up to 15 µs for goodillumination conditions) due to its asynchronous event based principle can fulfillthis requirement.

3.1.2 Experimental

A French group around Henri Lorach investigates the usage of the DVS for visual in-formation processing in combination with bioinspired algorithms to reproduce RGC

14 CHAPTER 3. NEUROMORPHIC PROSTHESIS APPROACHES

PR

HC

ON-BC OFF-BC

GC

AC

exci tation

inhibi t i on Θ

(a) (b)

Figure 3.1: Architecture of the computational model as an adaption of the retinalstructure. Taken from [LBM12]

style spike outputs such as required for neural stimulation in a retinal prosthesis.This section is mostly based on their work the work of [LBM12].

Based on electrophysiological data of rabbit retina experiments the retinal simula-tion model is build accordingly. Thus, containing ten different types of ganglioncells (that are characterized by their elctrophysiological response as well as theirmorphological parameters). Further, based on the experimental data for each typeof ganglion cell four different components of response to stimuli are established andinvestigated: ON-driven excitatory response, ON-driven inhibitory response, OFF-driven excitatory response and OFF-driven inhibitory response, see also figure 3 forthese four response types. For each type of ganglion cell and each type of responsea descriptive temporal and spatial filter function is established as shown in equa-tions 3.1 and 3.2, respectively, where an event is represented as e = (x, y, p, t) withposition of occurrence x and y, polarity of the event p (ON or OFF) and timestampt.

3.1. DVS IN RETINAL PROSTHESES 15

Figure 3.2: Reconstruction of the ON-Beta ganglion cell excitatory current in re-sponse to a 600 µm square of light . Taken from [LBM12].

hspatial(x, y) =3∑

i=1

αie−

(x−x0i)2+(y−y0

i)2

r2i (3.1)

htemporal(t) =2∑

i=1

βi(t− t0i )e−

(t−t0i)

τi (3.2)

The 15 introduced free parameters(αi, βi, ri, x0i , ...) for each combination of the 10

types of ganglion cells and the four types of responses are fitted based on electro-physiological data, its values can be found in table 1 of [LBM12]. See also figure 3.2for measured and fitted temporal and spatial response example.The chosen algorithm employs an iterative implementation. The data from the DVSis transferred via USB cable to the computer. The events are binned into 1 ms slotssince the chosen iterative time step is 1 ms regardless of incoming events to ensurea 1 ms temporal resolution. In each iterative step a 128 by 128 cells matrix I thatrepresents the currents at each pixel or cell, each cell is updated according to theevolution of the alpha function, described in equation 3.3 and 3.4. The algorithmprinciple is illustrated in figure 3.3.

fi(t+ dt) = dt e−

dtτi g(t) + e

−

dtτi f(t) (3.3)

16 CHAPTER 3. NEUROMORPHIC PROSTHESIS APPROACHES

Figure 3.3: The principle of the algorithm taken from [LBM12]

gi(t+ dt) = e−

dtτi gi(t) + hspatial (3.4)

The impact of an incoming event is accounted for by adding the spatial kernelhspatial,i as given in equation 3.4.From the cell values in I spikes are generated each time the difference of inhibitory anexcitatory currents reach a threshold or its integer multiple (representing excitationregression). The threshold is set to lead to a maximum fire rate of 200 Hz of onecell according to biological data.

3.1.3 Results

The investigation of temporal precision of the artificial retina shows that due to itslow latency it is sufficient for reproduction of mammalian retina phototransduction.Nevertheless, the events produced by the DVS disaccord to biological photorecep-tors and in terms of their ON-OFF characteristic opposing to the variable flow ofneurotransmitters (the processing in the retina can be considered analogous untilspike generation in the ganglion cells). The employed retina model therefore con-verts from spikes (ON and OFF) into analog currents and back into spike generationwhich might seem long-winded but delivers a proper working solution in combina-tion with the events of the DVS. Each output of the implemented retina modelcan be considered though as the globally input into a ganglion cell (summing upall contributing interneurons of underlying cell layers) other than a representationof all retinal interneurons. Still the retinal model creates satisfactory results withcomparable spike timing and count variability and the responses with the fitted pa-rameters shows high coefficients of determination (R2 = 0.93) compared with resultsfrom electrophysiological measurements of ganglion cell produced currents in tempo-ral and spatial regards for the 10 types of ganglion cells and the four response typesused. The feasibility of employment of the DVS for retina prostheses is succesfullyshown. Furthermore, there is so far no competing small visual sensor device witha comparable temporal resolution. The proposed algorithm can be considered asfirst step towards fully event-based algorithms or implementations as pure anlogeelectrical circuits.

17

Chapter 4

Summary and Outlook

Asynchronous neuromorphic sensors are well suited for applications in visual pros-theses as shown with DVS. It can handle similar range of different illumination asthe eye (around 120 dB). The event type information opposed to frame based allowsthe crucial high temporal precision, is flexible and produces low data rates whichis important for real-time requirements of a visual prostheses. In contrast, a highspeed camera that would provide suitable temporal resolution is in comparison tothe DVS big, heavy, needs cooling, has high energy consumption, needs high illu-mination levels and produces huge data rates and is therefore not suited for this(embedded) application. Yet, real-time is not reached in real world scenarios wherethe computational cost accounting for incoming events is about 10 times higher thanrequirement (Note that Matlab is used for computation) [LBM12].For the future, further investigation of event based algorithms (for embedded de-vices) needs to be done. First steps towards real-time ability can be done by im-plementing the algorithms in C, C++ or assembler. Further speed increase will bepossible through the use of dedicated hardware such as asynchronous convolutionchips which are under development and analog or mixed signal circuits especiallydesigned for this scenario.The produced outputs of the employed retinal model, as seen in figure 1, shouldalso be test in clinical studies to determine which output increases the helpful visualperception of the patient the most in different scenarios such as reading letters orface recognition.The employment of event based visuell sensors and algorithms in this field can beconsidered superiour over regular frame based devices. Further research could alsoallow for a variety of applications in robotics, object recognition or autonomousvehicle control.

18 CHAPTER 4. SUMMARY AND OUTLOOK

19

Appendix

20 CHAPTER 4. SUMMARY AND OUTLOOK

Figure 1: Parallel coding of a complex stimulus showing the different cell typeoutputs. Taken from [LBM12]

21

Figure 2: The human visual system, also referred to as visual pathway, and thecorresponding levels of information processing representation. Taken from [Wik13c]

22 CHAPTER 4. SUMMARY AND OUTLOOK

Figure 3: On-centers and off-centers of the retina. Taken from [Wik13b]

23

Figure 4: The EPIRET3 system of Epiret. Stimulation chip and receiver coil areplaced in an artifical lense after surgical removal of the natural lense.

24 CHAPTER 4. SUMMARY AND OUTLOOK

LIST OF FIGURES 25

List of Figures

1.1 Schematic of the anatomy of the eye and magnification of the retinaincluding photoreceptors and nerve layers. Light to signal conversionoccurs in the rod and cone cells. Then the nerve layers identify simpleshapes, such as bright points surrounded by dark points, edges, andmovement. Taken from [Wik13b]. . . . . . . . . . . . . . . . . . . . . 6

2.1 Retinal anatomy and retinal prosthesis implant locations. Epiretinalimplants are designed to interact with the retina from the vitreousside and are in closest proximity to ganglion cells. Subretinal implantsare placed between the retina and the choroid, in the place normallyoccupied by photoreceptors. Taken from [WCI07] . . . . . . . . . . . 8

2.2 Subretinal prosthesis system of the Daniel Palanker group of the Stan-ford University. Taken from [Pal13] . . . . . . . . . . . . . . . . . . . 9

2.3 (a)The external equipment includes glasses, a video processing unit(VPU) and a cable. (b)The implant is an epiretinal prosthesis sur-gically implanted in and on the eye that includes an antenna, anelectronics case, and an electrode array. Taken from [Sec13] . . . . . . 10

2.4 (a) Working principle and realization as basic electric circuit of theDVS [ETH13]. (b) Signal conversion of the DVS [LBM12]. . . . . . . 11

3.1 Architecture of the computational model as an adaption of the retinalstructure. Taken from [LBM12] . . . . . . . . . . . . . . . . . . . . . 14

3.2 Reconstruction of the ON-Beta ganglion cell excitatory current inresponse to a 600 µm square of light . Taken from [LBM12]. . . . . . 15

3.3 The principle of the algorithm taken from [LBM12] . . . . . . . . . . 16

1 Parallel coding of a complex stimulus showing the different cell typeoutputs. Taken from [LBM12] . . . . . . . . . . . . . . . . . . . . . . 20

2 The human visual system, also referred to as visual pathway, and thecorresponding levels of information processing representation. Takenfrom [Wik13c] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3 On-centers and off-centers of the retina. Taken from [Wik13b] . . . . 22

26 LIST OF FIGURES

4 The EPIRET3 system of Epiret. Stimulation chip and receiver coilare placed in an artifical lense after surgical removal of the naturallense. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

BIBLIOGRAPHY 27

Bibliography

[CBC+09] Jorg Conradt, Raphael Berner, Matthew Cook, Tobi Delbruck, andCh Zurich. An Embedded AER Dynamic Vision Sensor for Low-LatencyPole Balancing. Management, pages 780–785, 2009.

[Del08] Tobi Delbruck. Frame-free dynamic digital vision. Quality, pages 21–26,2008.

[Dow08] J Dowling. Current and future prospects for optoelectronic retinal pros-theses. Eye, 23(10):1999–2005, 2008.

[ETH13] ETH Zurich. http://siliconretina.ini.uzh.ch, 2013.

[FBWH12] Robert P Finger, Bernd Bertram, Christian Wolfram, and Frank G Holz.Blindness and Visual Impairment in Germany. Deutsches Arzteblatt In-ternational, 2006:484–489, 2012.

[FNY+06] Tetsuo Furumiya, David C Ng, Koutaro Yasuoka, Keiichiro Kagawa,Takashi Tokuda, Masahiro Nunoshita, and Jun Ohta. Functional ver-ification of pulse frequency modulation-based image sensor for retinalprosthesis by in vitro electrophysiological experiments using frog retina.Biosensors and Bioelectronics, 21:1059–1068, 2006.

[FYK+06] T Furumiya, S Yamamoto, K Kagawa, T Tokuda, M Nunoshita, andJ Ohta. Electrical stimulus experiments of the frog retina using a pulsefrequency modulation image sensor for retinal prosthesis. InternationalCongress Series, 1291:57 – 60, 2006.

[HS07] Noninvasive Human and Brain Stimulation. A Brief Summary of the His-tory of Noninvasive Brain Stimulation Alvaro Pascual-Leone and TimothyWagner. (Figure 1):1–7, 2007.

[LBM12] Henri Lorach, Ryad Benosman, and Olivier Marre. Artificial retina :the multichannel processing of the mammalian retina achieved with aneuromorphic asynchronous light acquisition device. 066004, 2012.

[LD10] Shih-chii Liu and Tobi Delbruck. Neuromorphic sensory systems. CurrentOpinion in Neurobiology, pages 1–8, 2010.

28 BIBLIOGRAPHY

[MC] Georg R Muller and Jorg Conradt. Self-Calibrating Marker Tracking in3D with Event-Based Vision Sensors 2 Event Based Stimuli Tracking inSensor Coordinate Frame. Event (London).

[MEM13] MEMS Journal. http://memsjournal.typepad.com/.a/6a00d8345225f869e2015432e8bd81970c-800wi, 2013.

[MFF12] Jong Min, Ong Frcophth, and Cruz Franzco. Review The bionic eye :a review PROSTHESIS. Clinical and Experimental Ophthalmology, (De-cember 2010):6–17, 2012.

[Pal13] Daniel Palanker. http://www.stanford.edu/˜palanker/lab/retinalpros.html,2013.

[PHV+] Daniel V Palanker, Philip Huie, Alexander Vankov, Yev Freyvert, Har-vey Fishman, Michael F Marmor, and Mark S Blumenkranz. Attractingretinal cells to electrodes for high-resolution stimulation. Cell.

[PIW10] N Parikh, L Itti, and J Weiland. Saliency-based image processing for.016006, 2010.

[Ret13] Retina Implant AG. http://retina-implant.de/de/patients/technology/default.aspx, 2013.

[RPE+04] Serge Resnikoff, Donatella Pascolini, Daniel Etya, Ivo Kocur, Ramachan-dra Pararajasegaram, Gopal P Pokharel, and Silvio P Mariotti. Policyand Practice Global data on visual impairment in the year 2002. Bulletinof the World Health Organization, 012831(04), 2004.

[Sci04] Clinical Sciences. The Artificial Silicon Retina Microchip for the Treat-ment of Vision Loss From Retinitis Pigmentosa. 122, 2004.

[Sec13] Second Sight. http://2-sight.eu/en/system-overview-en, 2013.

[UPV12] Technische Universit, Daniel Polterauer, and Matthias Vobl. VisuelleProthesen. Elektronik, 2012.

[WCI07] Jessica O Winter, Stuart F Cogan, and Joseph F Rizzo Iii. CO. ReviewLiterature And Arts Of The Americas, 1:1– 25, 2007.

[WHO10] WHO. Global Data on Visual Impairments. World Health OrganisationReport 2010, 2010.

[WHO12] WHO. Visual impairment and blindness: Fact Sheet 282, 2012.

[Wik13a] Wikipedia: Eye. http://en.wikipedia.org/wiki/Eye, 2013.

[Wik13b] Wikipedia: Retina. http://en.wikipedia.org/wiki/Retina, 2013.

BIBLIOGRAPHY 29

[Wik13c] Wikipedia: Visual System. http://en.wikipedia.org/wiki/Visual system,2013.

[WLH05] James D Weiland, Wentai Liu, and Mark S Humayun. Retinal Prosthesis.Review Literature And Arts Of The Americas, pages 361–401, 2005.

[ZZB06] Kareem A Zaghloul, Kareem A Zaghloul, and Kwabena Boahen. A SiliconRetina that Reproduces Signals in the Optic Nerve A silicon retina thatreproduces signals in the optic nerve. 3(3):257–267, 2006.