semiotcs

29
 1/29 Semiotics is the study of the social, cultural, and historical processes through which signs such as photographs acquire and circulate meaning. It is a useful critical approach with which to challenge simplistic beliefs in the realism of the photographic image; and as a critique of humanist and modernist concepts of artistic expression which place the photographer in a central position; where the work's success is measured against the author's intentions, and understanding these intentions means understanding the work. In this view, meaning is created by individuals and communicated using a transparent language. By contrast, a semiotic approach emphasizes how in all signs the relationship between signified (the meaning or content) and signifier (the form of the message) is arbitrary, based on social and cultural consensus; and how all meaning is context determined. Meaning does not pre- exist language, as the author's ideas or feelings, but is created by and through it. Ferdinand de Saussure (1857-1913), who developed structural linguistics in the early 20th century, first proposed that meaning is generated by the difference between signs within a system, rather than by the correspondence between signs and a pre-existing reality. His ideas were then extended to the study of all types of sign systems, including visual ones, and refined by including psychoanalytical ideas and issues relating to reception. Barthes analysed how realist sign systems, such as photography, disguise their own conventionality by presenting themselves as natural. This ‘naturalness’ is then used by dominant cultural groups to make their own values and ideologies appear natural, thereby imposing them as ‘common sense’ on marginalized or disempowered groups. He also argued that the meaning of any given work is not fixed, but a process only temporarily arrested by the reader. Different contexts and audiences bring to a work a different network of intertextualities (references to, or knowledge of, other texts, images, and ideas) and thereby change its meaning. Other writers, such as Rosalind Krauss, have re-evaluated the early model of semiotics proposed by the American philosopher Charles Sanders Peirce (1839-1914), who categorized signs as iconic, indexical, or symbolic, as being more useful to understanding visual signs. Read more: http://www.answers.com/topic/semiotics-and-photography#ixzz1Eb7OGpD0  SIGNS AND LANGUAGE Human beings recognize patterns of information and organize them to generate meaning. Collections of these organized patterns form the languages that humans use when they communicate. We use certain "signs" among ourselves that do not point to anything in our actual surroundings. Instead of announcers of things, they are reminders ... they take the place of things that we have perceived in the past, or even things that we can merely imagine by combining memories, things that might be in the past or future experience. They serve to let us develop a characteristic attitude toward objects in absentia , which is called "thinking of" or "referring to" what is not here. --Suzzane Langer 1016 Human beings possess the ability to notice patterns in their environments. When the perception of these patterns leads to the interpretation of new information in the context of previous knowledge, we might say that meaning occurs. The notion of meaning, or the making sense out of one's information, is an important aspect of human communication. There is little agreement as to how the term "meaning" should be defined, nor is there agreement as to how meaning is created, preserved and destroyed in the midst of the communication process. However, attempts to reconcile these disagreements have led to the development of a number of differing points-of-view. Important among these are the following: that meaning is contained in the patterns themselves, that meaning is created entirely in the minds of the individual senders and receivers, that meaning arises from the social interactions of the communicators. One widely used approach to the study of the relationships among patterns of perception and meaning is called semiotics. Central to semiotics is the notion of the sign.

Upload: ihllhm

Post on 07-Apr-2018

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 129

129

Semiotics is the study of the social cultural and historical processes through which signs such asphotographs acquire and circulate meaning It is a useful critical approach with which to challengesimplistic beliefs in the realism of the photographic image and as a critique of humanist and modernistconcepts of artistic expression which place the photographer in a central position where the workssuccess is measured against the authors intentions and understanding these intentions meansunderstanding the work In this view meaning is created by individuals and communicated using a

transparent language By contrast a semiotic approach emphasizes how in all signs the relationshipbetween signified (the meaning or content) and signifier (the form of the message) is arbitrary basedon social and cultural consensus and how all meaning is context determined Meaning does not pre-exist language as the authors ideas or feelings but is created by and through it Ferdinand deSaussure (1857-1913) who developed structural linguistics in the early 20th century first proposedthat meaning is generated by the difference between signs within a system rather than by thecorrespondence between signs and a pre-existing reality His ideas were then extended to the study ofall types of sign systems including visual ones and refined by including psychoanalytical ideas andissues relating to reception Barthes analysed how realist sign systems such as photography disguisetheir own conventionality by presenting themselves as natural This lsquonaturalnessrsquo is then used bydominant cultural groups to make their own values and ideologies appear natural thereby imposingthem as lsquocommon sensersquo on marginalized or disempowered groups He also argued that the meaningof any given work is not fixed but a process only temporarily arrested by the reader Different contexts

and audiences bring to a work a different network of intertextualities (references to or knowledge ofother texts images and ideas) and thereby change its meaning Other writers such as RosalindKrauss have re-evaluated the early model of semiotics proposed by the American philosopherCharles Sanders Peirce (1839-1914) who categorized signs as iconic indexical or symbolic as beingmore useful to understanding visual signs

Read more httpwwwanswerscomtopicsemiotics-and-photographyixzz1Eb7OGpD0

SIGNS AND LANGUAGE

Human beings recognize patterns of information and organize them to generate meaning Collectionsof these organized patterns form the languages that humans use when they communicate

We use certain signs among ourselves that do not point to anything in our actual surroundingsInstead of announcers of things they are reminders they take the place of things that we haveperceived in the past or even things that we can merely imagine by combining memories things thatmight be in the past or future experience They serve to let us develop a characteristic attitude towardobjects in absentia which is called thinking of or referring to what is not here --Suzzane Langer 1016

Human beings possess the ability to notice patterns in their environments When the perception ofthese patterns leads to the interpretation of new information in the context of previous knowledge we

might say that meaning occurs The notion of meaning or the making sense out of ones informationis an important aspect of human communication

There is little agreement as to how the term meaning should be defined nor is there agreement as tohow meaning is created preserved and destroyed in the midst of the communication processHowever attempts to reconcile these disagreements have led to the development of a number ofdiffering points-of-view Important among these are the following

bull that meaning is contained in the patterns themselvesbull that meaning is created entirely in the minds of the individual senders and receiversbull that meaning arises from the social interactions of the communicators

One widely used approach to the study of the relationships among patterns of perception and meaningis called semiotics Central to semiotics is the notion of the sign

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 229

229

Such a statement as The word cat stand for a certain small mammal is not either true or false Itstruth depends upon agreement between the speakers that it be true In terms of such agreement theyunderstand each other or where disagreement occurs they will meet with misunderstanding --Gregory Bateson

A sign is a pattern of data which when perceived brings to mind something other than itself

Although this definition appears simple on the surface it has complex implications Please pause tolook at Figure 1 for a moment or two

Figure 1

Now briefly and to yourself describe the thoughts that Figure 1 brought to your mind It may help ifyou write these down

waiting

please look at the picture and form your

thoughts before you continue

This situation illustrates the three fundamental building blocks which together with the rules thatdescribe how they relate to one another will be used to construct the Semiotic Model ofCommunication

bull The first of these building blocks is the data or the perceived pattern of dark-on-light that toan observer is Figure 1 This will be called the sign

bull The second building block is the real-world animal that Figure 1 resembles This will be calledthe object In the terminology of the semiotic model the sign is said to refer to its object --similarly the object is sometimes called the referent of the sign

bull The third building block is the thought that forms in the mind of a reader as he or she gazes at

Figure 1 This will be called the concept

These three elements relate to one another as a semiotic system

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 329

329

At the beginning of the exercise did this sign bring to mind a large African or Indiananimal Or memories of a trip to the zoo Or images recalled from a favorite book read as a child or a

television show or a movie Perhaps it brings to mind an American political party or perhaps thenotion of memory (as in a large animal with a trunk and big ears that never forgets ) Notice thatwhatever the sign brings to mind the concept is related to the readers past experience with theobject

This is always the case with signs and one of the advantages of the semiotic model lies in its ability tohighlight relationships among the sign the concepts the sign brings to mind and the experience of thereader The next picture illustrates this relationship

ICON INDEX AND SYMBOL

Why does a particular sign bring to mind a particular concept For example why does bring

to mind an animal while does not In this case the connection lies in theresemblance of the sign to the object 1004

It might be that one day during a trip to the zoo the reader saw a large animal -- and so later when heor she sees a printed image that resembles the animal that earlier experience is brought to mind

Connection-by-resemblance is one of the three fundamental ways that signs concepts andexperiences relate This particular kind of sign is called an icon If a sign is a perception that refers to

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 429

429

or brings to mind something other than itself an icon is a type of sign that resembles the thing that it

refers to Thus is an icon because it resembles the animal that it brings to mind 1020

You may have noticed that in our discussion of we have carefully refrained from using theword elephant The reason for this is that the word elephant is itself a sign though a different kind of

sign than

Note what do the s mean in the last paragraph

This second type of sign is called a symbol Symbols and the objects that they bring to mind arerelated in an arbitrary manner This means that there is no known reason why the symbol and theobject are related For example there is no reason why the large animal under discussion might not

be tagged by a different word -- nordnet for example or frindlemat or perhaps barracudaElephant is used simply because over the years it has come to be used -- no one knows why

Elephant Used As A Symbolarbitrary

A third kind of sign brings a concept to mind by means of a direct physical connection between itselfand its object For example if someone is walking down a street and suddenly encounters the smell offreshly baking bread he or she might find the concept of a bakery coming to mind This kind of sign iscalled an index

The Smell of Baking Bread As An Index

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 529

529

To summarize -- there are three basic types of signs the icon the index and the symbol Each bringsto mind concepts that are related to the perceivers previous experience with objects in the worldEach operates in a different way

Icon -- a sign that resembles its object

If this brings to mind a kind of fruit it is acting as an icon 1006

Index -- a sign that is physically connected to its object

If the rain touching your face brings to mind the nearby storm it is acting as an index 1005

Symbol -- a sign whose relationship to its object is arbitrary

If this brings to mind an interstate highway in the United States it is acting as a symbol

icon symbol index 1007 1008 1009 1019

THE SEMIOTIC MODEL

The Semiotic Model provides a coordinated way of talking about how the thoughts in our minds can beexpressed in terms of the world outside of our minds The model contains three basic entities

bull the sign something which is perceived but which stands for something elsebull the concept the thoughts or images that are brought to mind by the perception of the sign

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 629

629

bull the object the something else in the world to which the sign refers

The model is most often represented as the semiotic triangle

This version of the semiotic model is adapted from the work of the American philosopher Charles SPierce Pierce is generally acknowledged as an important pioneer in the study of signs

Notice that

bull the sign and the concept are connected by the persons perceptionbull the concept and the object are connected by the persons experiencebull the sign and the object are connected by the conventions or the culture of the social group

within which the person lives

These connections are important to the study of how meaning arises during the daily encounters withthe many signs that fill the human environment The remaining sections of this tutorial investigatesome of the ways that meaning arises as people make use of signs during the process ofcommunication

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 729

729

Art and the Semiotics of Images Three Questions About Visual Meaning

(Please note this page has many inline graphics and takes some time for a full load It is notbroken It also uses a good bit of JavaScript and Java)

George L Dillon

University of WashingtonJuly 1999

In the last five years the Internet has vastly enhanced our ability to display images to each other andwe can now think of ourselves not just as viewers and consumers of images but as makers and usersof them ourselves Indeed if on the Internet we do not use images we appear stuck in print cultureand oblivious to the possibilities of the new medium We can of course avoid giving these impressionsby including some wallpaper and a few bits of eye candy without thereby getting very far at all intographics as a mode of conveying meaning Schools and colleges certainly offer very little guidanceoutside of the area of technical communication At present we have more questions than answersamong which three seem quite fundamental

1 how language-like are images2 how do images and words work when they are both present3 how do scenes of people gazing and posing convey visual meaning

I will expand briefly on each of these questions and then take them in order So Kress and vanLeeuwen declare Reading Images The Grammar of Visual Design Routledge 1996 p 17 SuzanneK Langer is also often quoted

Some say that images work via a second communicative system one fully as expressive as naturallanguage but separate and structured independently of it Others find visual and verbal meaningsmore dissimilar than similar with the visual lacking a kind of determinacy for which verbal languageseems better suitedSo Paul Messaris Visual Literacy Image Mind and Reality Westview 1994 and Visual Persuasion

The Role of Images in Advertising amp Sage Publications 1997 so also Michael Titzmann cited inphoto text text photo ed Andreas Hapkemeyer and Peter Weiermair Edition Stemmle1996 p 10 This question of the nature and indeterminacy of visual meaning will be the first point we will take up

The second question is obviously related namely how do the two signalling systems work when theyare placed together In principle visual meanings may be entirely separate from verbal ones but as apractical matter we rarely find pure images with no text attaching to them Some 35 years ago RolandBarthes wrote of our very common practices of surrounding images with words which help to specifyand stabilize the interpretations of particular imagesRoland Barthes The Rhetoric of the Image in Image Music Text trans Stephen Heath Hill andWang 1977 pp 38-39 The original date of publication was 1964

all images are polysemous they imply underlying their signifiers a floating chain of signifieds thereader able to choose some and ignore others Polysemy poses a question of meaning and thisquestion always comes through as a dysfunctionHence in every society various techniques aredeveloped intended to fix the floating chain of signifieds in such a way as to counter the terror ofuncertain signs the linguistic message is one of those techniquesAmong these linguistic messages are captions labels placards guidebooks brochures and fliers--all bits of institutional apparatuses which select and present texts and images for the publicBut see Shane Coopers random captioner and the random2 phase of Jody Zellens All the NewsThats Fit to Print They are the tools of curators teachers and editors They in turn are parts of aneven larger body of institutions and practices which stabilize how images are to be interpreted andused That is when an image is used in a textbook or a treatise we assume it is there to illustrate andsupport the meanings and information provided by the text When an image occurs in anadvertisement we assume that it is there to help sell a product as by depicting an instance of

someone enjoying possession and consumption of the product Thus we have in these standarddeployments of text and image the harmonious relations of explication (by text) and illustration (byimage)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 2: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 229

229

Such a statement as The word cat stand for a certain small mammal is not either true or false Itstruth depends upon agreement between the speakers that it be true In terms of such agreement theyunderstand each other or where disagreement occurs they will meet with misunderstanding --Gregory Bateson

A sign is a pattern of data which when perceived brings to mind something other than itself

Although this definition appears simple on the surface it has complex implications Please pause tolook at Figure 1 for a moment or two

Figure 1

Now briefly and to yourself describe the thoughts that Figure 1 brought to your mind It may help ifyou write these down

waiting

please look at the picture and form your

thoughts before you continue

This situation illustrates the three fundamental building blocks which together with the rules thatdescribe how they relate to one another will be used to construct the Semiotic Model ofCommunication

bull The first of these building blocks is the data or the perceived pattern of dark-on-light that toan observer is Figure 1 This will be called the sign

bull The second building block is the real-world animal that Figure 1 resembles This will be calledthe object In the terminology of the semiotic model the sign is said to refer to its object --similarly the object is sometimes called the referent of the sign

bull The third building block is the thought that forms in the mind of a reader as he or she gazes at

Figure 1 This will be called the concept

These three elements relate to one another as a semiotic system

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 329

329

At the beginning of the exercise did this sign bring to mind a large African or Indiananimal Or memories of a trip to the zoo Or images recalled from a favorite book read as a child or a

television show or a movie Perhaps it brings to mind an American political party or perhaps thenotion of memory (as in a large animal with a trunk and big ears that never forgets ) Notice thatwhatever the sign brings to mind the concept is related to the readers past experience with theobject

This is always the case with signs and one of the advantages of the semiotic model lies in its ability tohighlight relationships among the sign the concepts the sign brings to mind and the experience of thereader The next picture illustrates this relationship

ICON INDEX AND SYMBOL

Why does a particular sign bring to mind a particular concept For example why does bring

to mind an animal while does not In this case the connection lies in theresemblance of the sign to the object 1004

It might be that one day during a trip to the zoo the reader saw a large animal -- and so later when heor she sees a printed image that resembles the animal that earlier experience is brought to mind

Connection-by-resemblance is one of the three fundamental ways that signs concepts andexperiences relate This particular kind of sign is called an icon If a sign is a perception that refers to

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 429

429

or brings to mind something other than itself an icon is a type of sign that resembles the thing that it

refers to Thus is an icon because it resembles the animal that it brings to mind 1020

You may have noticed that in our discussion of we have carefully refrained from using theword elephant The reason for this is that the word elephant is itself a sign though a different kind of

sign than

Note what do the s mean in the last paragraph

This second type of sign is called a symbol Symbols and the objects that they bring to mind arerelated in an arbitrary manner This means that there is no known reason why the symbol and theobject are related For example there is no reason why the large animal under discussion might not

be tagged by a different word -- nordnet for example or frindlemat or perhaps barracudaElephant is used simply because over the years it has come to be used -- no one knows why

Elephant Used As A Symbolarbitrary

A third kind of sign brings a concept to mind by means of a direct physical connection between itselfand its object For example if someone is walking down a street and suddenly encounters the smell offreshly baking bread he or she might find the concept of a bakery coming to mind This kind of sign iscalled an index

The Smell of Baking Bread As An Index

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 529

529

To summarize -- there are three basic types of signs the icon the index and the symbol Each bringsto mind concepts that are related to the perceivers previous experience with objects in the worldEach operates in a different way

Icon -- a sign that resembles its object

If this brings to mind a kind of fruit it is acting as an icon 1006

Index -- a sign that is physically connected to its object

If the rain touching your face brings to mind the nearby storm it is acting as an index 1005

Symbol -- a sign whose relationship to its object is arbitrary

If this brings to mind an interstate highway in the United States it is acting as a symbol

icon symbol index 1007 1008 1009 1019

THE SEMIOTIC MODEL

The Semiotic Model provides a coordinated way of talking about how the thoughts in our minds can beexpressed in terms of the world outside of our minds The model contains three basic entities

bull the sign something which is perceived but which stands for something elsebull the concept the thoughts or images that are brought to mind by the perception of the sign

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 629

629

bull the object the something else in the world to which the sign refers

The model is most often represented as the semiotic triangle

This version of the semiotic model is adapted from the work of the American philosopher Charles SPierce Pierce is generally acknowledged as an important pioneer in the study of signs

Notice that

bull the sign and the concept are connected by the persons perceptionbull the concept and the object are connected by the persons experiencebull the sign and the object are connected by the conventions or the culture of the social group

within which the person lives

These connections are important to the study of how meaning arises during the daily encounters withthe many signs that fill the human environment The remaining sections of this tutorial investigatesome of the ways that meaning arises as people make use of signs during the process ofcommunication

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 729

729

Art and the Semiotics of Images Three Questions About Visual Meaning

(Please note this page has many inline graphics and takes some time for a full load It is notbroken It also uses a good bit of JavaScript and Java)

George L Dillon

University of WashingtonJuly 1999

In the last five years the Internet has vastly enhanced our ability to display images to each other andwe can now think of ourselves not just as viewers and consumers of images but as makers and usersof them ourselves Indeed if on the Internet we do not use images we appear stuck in print cultureand oblivious to the possibilities of the new medium We can of course avoid giving these impressionsby including some wallpaper and a few bits of eye candy without thereby getting very far at all intographics as a mode of conveying meaning Schools and colleges certainly offer very little guidanceoutside of the area of technical communication At present we have more questions than answersamong which three seem quite fundamental

1 how language-like are images2 how do images and words work when they are both present3 how do scenes of people gazing and posing convey visual meaning

I will expand briefly on each of these questions and then take them in order So Kress and vanLeeuwen declare Reading Images The Grammar of Visual Design Routledge 1996 p 17 SuzanneK Langer is also often quoted

Some say that images work via a second communicative system one fully as expressive as naturallanguage but separate and structured independently of it Others find visual and verbal meaningsmore dissimilar than similar with the visual lacking a kind of determinacy for which verbal languageseems better suitedSo Paul Messaris Visual Literacy Image Mind and Reality Westview 1994 and Visual Persuasion

The Role of Images in Advertising amp Sage Publications 1997 so also Michael Titzmann cited inphoto text text photo ed Andreas Hapkemeyer and Peter Weiermair Edition Stemmle1996 p 10 This question of the nature and indeterminacy of visual meaning will be the first point we will take up

The second question is obviously related namely how do the two signalling systems work when theyare placed together In principle visual meanings may be entirely separate from verbal ones but as apractical matter we rarely find pure images with no text attaching to them Some 35 years ago RolandBarthes wrote of our very common practices of surrounding images with words which help to specifyand stabilize the interpretations of particular imagesRoland Barthes The Rhetoric of the Image in Image Music Text trans Stephen Heath Hill andWang 1977 pp 38-39 The original date of publication was 1964

all images are polysemous they imply underlying their signifiers a floating chain of signifieds thereader able to choose some and ignore others Polysemy poses a question of meaning and thisquestion always comes through as a dysfunctionHence in every society various techniques aredeveloped intended to fix the floating chain of signifieds in such a way as to counter the terror ofuncertain signs the linguistic message is one of those techniquesAmong these linguistic messages are captions labels placards guidebooks brochures and fliers--all bits of institutional apparatuses which select and present texts and images for the publicBut see Shane Coopers random captioner and the random2 phase of Jody Zellens All the NewsThats Fit to Print They are the tools of curators teachers and editors They in turn are parts of aneven larger body of institutions and practices which stabilize how images are to be interpreted andused That is when an image is used in a textbook or a treatise we assume it is there to illustrate andsupport the meanings and information provided by the text When an image occurs in anadvertisement we assume that it is there to help sell a product as by depicting an instance of

someone enjoying possession and consumption of the product Thus we have in these standarddeployments of text and image the harmonious relations of explication (by text) and illustration (byimage)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 3: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 329

329

At the beginning of the exercise did this sign bring to mind a large African or Indiananimal Or memories of a trip to the zoo Or images recalled from a favorite book read as a child or a

television show or a movie Perhaps it brings to mind an American political party or perhaps thenotion of memory (as in a large animal with a trunk and big ears that never forgets ) Notice thatwhatever the sign brings to mind the concept is related to the readers past experience with theobject

This is always the case with signs and one of the advantages of the semiotic model lies in its ability tohighlight relationships among the sign the concepts the sign brings to mind and the experience of thereader The next picture illustrates this relationship

ICON INDEX AND SYMBOL

Why does a particular sign bring to mind a particular concept For example why does bring

to mind an animal while does not In this case the connection lies in theresemblance of the sign to the object 1004

It might be that one day during a trip to the zoo the reader saw a large animal -- and so later when heor she sees a printed image that resembles the animal that earlier experience is brought to mind

Connection-by-resemblance is one of the three fundamental ways that signs concepts andexperiences relate This particular kind of sign is called an icon If a sign is a perception that refers to

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 429

429

or brings to mind something other than itself an icon is a type of sign that resembles the thing that it

refers to Thus is an icon because it resembles the animal that it brings to mind 1020

You may have noticed that in our discussion of we have carefully refrained from using theword elephant The reason for this is that the word elephant is itself a sign though a different kind of

sign than

Note what do the s mean in the last paragraph

This second type of sign is called a symbol Symbols and the objects that they bring to mind arerelated in an arbitrary manner This means that there is no known reason why the symbol and theobject are related For example there is no reason why the large animal under discussion might not

be tagged by a different word -- nordnet for example or frindlemat or perhaps barracudaElephant is used simply because over the years it has come to be used -- no one knows why

Elephant Used As A Symbolarbitrary

A third kind of sign brings a concept to mind by means of a direct physical connection between itselfand its object For example if someone is walking down a street and suddenly encounters the smell offreshly baking bread he or she might find the concept of a bakery coming to mind This kind of sign iscalled an index

The Smell of Baking Bread As An Index

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 529

529

To summarize -- there are three basic types of signs the icon the index and the symbol Each bringsto mind concepts that are related to the perceivers previous experience with objects in the worldEach operates in a different way

Icon -- a sign that resembles its object

If this brings to mind a kind of fruit it is acting as an icon 1006

Index -- a sign that is physically connected to its object

If the rain touching your face brings to mind the nearby storm it is acting as an index 1005

Symbol -- a sign whose relationship to its object is arbitrary

If this brings to mind an interstate highway in the United States it is acting as a symbol

icon symbol index 1007 1008 1009 1019

THE SEMIOTIC MODEL

The Semiotic Model provides a coordinated way of talking about how the thoughts in our minds can beexpressed in terms of the world outside of our minds The model contains three basic entities

bull the sign something which is perceived but which stands for something elsebull the concept the thoughts or images that are brought to mind by the perception of the sign

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 629

629

bull the object the something else in the world to which the sign refers

The model is most often represented as the semiotic triangle

This version of the semiotic model is adapted from the work of the American philosopher Charles SPierce Pierce is generally acknowledged as an important pioneer in the study of signs

Notice that

bull the sign and the concept are connected by the persons perceptionbull the concept and the object are connected by the persons experiencebull the sign and the object are connected by the conventions or the culture of the social group

within which the person lives

These connections are important to the study of how meaning arises during the daily encounters withthe many signs that fill the human environment The remaining sections of this tutorial investigatesome of the ways that meaning arises as people make use of signs during the process ofcommunication

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 729

729

Art and the Semiotics of Images Three Questions About Visual Meaning

(Please note this page has many inline graphics and takes some time for a full load It is notbroken It also uses a good bit of JavaScript and Java)

George L Dillon

University of WashingtonJuly 1999

In the last five years the Internet has vastly enhanced our ability to display images to each other andwe can now think of ourselves not just as viewers and consumers of images but as makers and usersof them ourselves Indeed if on the Internet we do not use images we appear stuck in print cultureand oblivious to the possibilities of the new medium We can of course avoid giving these impressionsby including some wallpaper and a few bits of eye candy without thereby getting very far at all intographics as a mode of conveying meaning Schools and colleges certainly offer very little guidanceoutside of the area of technical communication At present we have more questions than answersamong which three seem quite fundamental

1 how language-like are images2 how do images and words work when they are both present3 how do scenes of people gazing and posing convey visual meaning

I will expand briefly on each of these questions and then take them in order So Kress and vanLeeuwen declare Reading Images The Grammar of Visual Design Routledge 1996 p 17 SuzanneK Langer is also often quoted

Some say that images work via a second communicative system one fully as expressive as naturallanguage but separate and structured independently of it Others find visual and verbal meaningsmore dissimilar than similar with the visual lacking a kind of determinacy for which verbal languageseems better suitedSo Paul Messaris Visual Literacy Image Mind and Reality Westview 1994 and Visual Persuasion

The Role of Images in Advertising amp Sage Publications 1997 so also Michael Titzmann cited inphoto text text photo ed Andreas Hapkemeyer and Peter Weiermair Edition Stemmle1996 p 10 This question of the nature and indeterminacy of visual meaning will be the first point we will take up

The second question is obviously related namely how do the two signalling systems work when theyare placed together In principle visual meanings may be entirely separate from verbal ones but as apractical matter we rarely find pure images with no text attaching to them Some 35 years ago RolandBarthes wrote of our very common practices of surrounding images with words which help to specifyand stabilize the interpretations of particular imagesRoland Barthes The Rhetoric of the Image in Image Music Text trans Stephen Heath Hill andWang 1977 pp 38-39 The original date of publication was 1964

all images are polysemous they imply underlying their signifiers a floating chain of signifieds thereader able to choose some and ignore others Polysemy poses a question of meaning and thisquestion always comes through as a dysfunctionHence in every society various techniques aredeveloped intended to fix the floating chain of signifieds in such a way as to counter the terror ofuncertain signs the linguistic message is one of those techniquesAmong these linguistic messages are captions labels placards guidebooks brochures and fliers--all bits of institutional apparatuses which select and present texts and images for the publicBut see Shane Coopers random captioner and the random2 phase of Jody Zellens All the NewsThats Fit to Print They are the tools of curators teachers and editors They in turn are parts of aneven larger body of institutions and practices which stabilize how images are to be interpreted andused That is when an image is used in a textbook or a treatise we assume it is there to illustrate andsupport the meanings and information provided by the text When an image occurs in anadvertisement we assume that it is there to help sell a product as by depicting an instance of

someone enjoying possession and consumption of the product Thus we have in these standarddeployments of text and image the harmonious relations of explication (by text) and illustration (byimage)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 4: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 429

429

or brings to mind something other than itself an icon is a type of sign that resembles the thing that it

refers to Thus is an icon because it resembles the animal that it brings to mind 1020

You may have noticed that in our discussion of we have carefully refrained from using theword elephant The reason for this is that the word elephant is itself a sign though a different kind of

sign than

Note what do the s mean in the last paragraph

This second type of sign is called a symbol Symbols and the objects that they bring to mind arerelated in an arbitrary manner This means that there is no known reason why the symbol and theobject are related For example there is no reason why the large animal under discussion might not

be tagged by a different word -- nordnet for example or frindlemat or perhaps barracudaElephant is used simply because over the years it has come to be used -- no one knows why

Elephant Used As A Symbolarbitrary

A third kind of sign brings a concept to mind by means of a direct physical connection between itselfand its object For example if someone is walking down a street and suddenly encounters the smell offreshly baking bread he or she might find the concept of a bakery coming to mind This kind of sign iscalled an index

The Smell of Baking Bread As An Index

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 529

529

To summarize -- there are three basic types of signs the icon the index and the symbol Each bringsto mind concepts that are related to the perceivers previous experience with objects in the worldEach operates in a different way

Icon -- a sign that resembles its object

If this brings to mind a kind of fruit it is acting as an icon 1006

Index -- a sign that is physically connected to its object

If the rain touching your face brings to mind the nearby storm it is acting as an index 1005

Symbol -- a sign whose relationship to its object is arbitrary

If this brings to mind an interstate highway in the United States it is acting as a symbol

icon symbol index 1007 1008 1009 1019

THE SEMIOTIC MODEL

The Semiotic Model provides a coordinated way of talking about how the thoughts in our minds can beexpressed in terms of the world outside of our minds The model contains three basic entities

bull the sign something which is perceived but which stands for something elsebull the concept the thoughts or images that are brought to mind by the perception of the sign

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 629

629

bull the object the something else in the world to which the sign refers

The model is most often represented as the semiotic triangle

This version of the semiotic model is adapted from the work of the American philosopher Charles SPierce Pierce is generally acknowledged as an important pioneer in the study of signs

Notice that

bull the sign and the concept are connected by the persons perceptionbull the concept and the object are connected by the persons experiencebull the sign and the object are connected by the conventions or the culture of the social group

within which the person lives

These connections are important to the study of how meaning arises during the daily encounters withthe many signs that fill the human environment The remaining sections of this tutorial investigatesome of the ways that meaning arises as people make use of signs during the process ofcommunication

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 729

729

Art and the Semiotics of Images Three Questions About Visual Meaning

(Please note this page has many inline graphics and takes some time for a full load It is notbroken It also uses a good bit of JavaScript and Java)

George L Dillon

University of WashingtonJuly 1999

In the last five years the Internet has vastly enhanced our ability to display images to each other andwe can now think of ourselves not just as viewers and consumers of images but as makers and usersof them ourselves Indeed if on the Internet we do not use images we appear stuck in print cultureand oblivious to the possibilities of the new medium We can of course avoid giving these impressionsby including some wallpaper and a few bits of eye candy without thereby getting very far at all intographics as a mode of conveying meaning Schools and colleges certainly offer very little guidanceoutside of the area of technical communication At present we have more questions than answersamong which three seem quite fundamental

1 how language-like are images2 how do images and words work when they are both present3 how do scenes of people gazing and posing convey visual meaning

I will expand briefly on each of these questions and then take them in order So Kress and vanLeeuwen declare Reading Images The Grammar of Visual Design Routledge 1996 p 17 SuzanneK Langer is also often quoted

Some say that images work via a second communicative system one fully as expressive as naturallanguage but separate and structured independently of it Others find visual and verbal meaningsmore dissimilar than similar with the visual lacking a kind of determinacy for which verbal languageseems better suitedSo Paul Messaris Visual Literacy Image Mind and Reality Westview 1994 and Visual Persuasion

The Role of Images in Advertising amp Sage Publications 1997 so also Michael Titzmann cited inphoto text text photo ed Andreas Hapkemeyer and Peter Weiermair Edition Stemmle1996 p 10 This question of the nature and indeterminacy of visual meaning will be the first point we will take up

The second question is obviously related namely how do the two signalling systems work when theyare placed together In principle visual meanings may be entirely separate from verbal ones but as apractical matter we rarely find pure images with no text attaching to them Some 35 years ago RolandBarthes wrote of our very common practices of surrounding images with words which help to specifyand stabilize the interpretations of particular imagesRoland Barthes The Rhetoric of the Image in Image Music Text trans Stephen Heath Hill andWang 1977 pp 38-39 The original date of publication was 1964

all images are polysemous they imply underlying their signifiers a floating chain of signifieds thereader able to choose some and ignore others Polysemy poses a question of meaning and thisquestion always comes through as a dysfunctionHence in every society various techniques aredeveloped intended to fix the floating chain of signifieds in such a way as to counter the terror ofuncertain signs the linguistic message is one of those techniquesAmong these linguistic messages are captions labels placards guidebooks brochures and fliers--all bits of institutional apparatuses which select and present texts and images for the publicBut see Shane Coopers random captioner and the random2 phase of Jody Zellens All the NewsThats Fit to Print They are the tools of curators teachers and editors They in turn are parts of aneven larger body of institutions and practices which stabilize how images are to be interpreted andused That is when an image is used in a textbook or a treatise we assume it is there to illustrate andsupport the meanings and information provided by the text When an image occurs in anadvertisement we assume that it is there to help sell a product as by depicting an instance of

someone enjoying possession and consumption of the product Thus we have in these standarddeployments of text and image the harmonious relations of explication (by text) and illustration (byimage)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 5: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 529

529

To summarize -- there are three basic types of signs the icon the index and the symbol Each bringsto mind concepts that are related to the perceivers previous experience with objects in the worldEach operates in a different way

Icon -- a sign that resembles its object

If this brings to mind a kind of fruit it is acting as an icon 1006

Index -- a sign that is physically connected to its object

If the rain touching your face brings to mind the nearby storm it is acting as an index 1005

Symbol -- a sign whose relationship to its object is arbitrary

If this brings to mind an interstate highway in the United States it is acting as a symbol

icon symbol index 1007 1008 1009 1019

THE SEMIOTIC MODEL

The Semiotic Model provides a coordinated way of talking about how the thoughts in our minds can beexpressed in terms of the world outside of our minds The model contains three basic entities

bull the sign something which is perceived but which stands for something elsebull the concept the thoughts or images that are brought to mind by the perception of the sign

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 629

629

bull the object the something else in the world to which the sign refers

The model is most often represented as the semiotic triangle

This version of the semiotic model is adapted from the work of the American philosopher Charles SPierce Pierce is generally acknowledged as an important pioneer in the study of signs

Notice that

bull the sign and the concept are connected by the persons perceptionbull the concept and the object are connected by the persons experiencebull the sign and the object are connected by the conventions or the culture of the social group

within which the person lives

These connections are important to the study of how meaning arises during the daily encounters withthe many signs that fill the human environment The remaining sections of this tutorial investigatesome of the ways that meaning arises as people make use of signs during the process ofcommunication

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 729

729

Art and the Semiotics of Images Three Questions About Visual Meaning

(Please note this page has many inline graphics and takes some time for a full load It is notbroken It also uses a good bit of JavaScript and Java)

George L Dillon

University of WashingtonJuly 1999

In the last five years the Internet has vastly enhanced our ability to display images to each other andwe can now think of ourselves not just as viewers and consumers of images but as makers and usersof them ourselves Indeed if on the Internet we do not use images we appear stuck in print cultureand oblivious to the possibilities of the new medium We can of course avoid giving these impressionsby including some wallpaper and a few bits of eye candy without thereby getting very far at all intographics as a mode of conveying meaning Schools and colleges certainly offer very little guidanceoutside of the area of technical communication At present we have more questions than answersamong which three seem quite fundamental

1 how language-like are images2 how do images and words work when they are both present3 how do scenes of people gazing and posing convey visual meaning

I will expand briefly on each of these questions and then take them in order So Kress and vanLeeuwen declare Reading Images The Grammar of Visual Design Routledge 1996 p 17 SuzanneK Langer is also often quoted

Some say that images work via a second communicative system one fully as expressive as naturallanguage but separate and structured independently of it Others find visual and verbal meaningsmore dissimilar than similar with the visual lacking a kind of determinacy for which verbal languageseems better suitedSo Paul Messaris Visual Literacy Image Mind and Reality Westview 1994 and Visual Persuasion

The Role of Images in Advertising amp Sage Publications 1997 so also Michael Titzmann cited inphoto text text photo ed Andreas Hapkemeyer and Peter Weiermair Edition Stemmle1996 p 10 This question of the nature and indeterminacy of visual meaning will be the first point we will take up

The second question is obviously related namely how do the two signalling systems work when theyare placed together In principle visual meanings may be entirely separate from verbal ones but as apractical matter we rarely find pure images with no text attaching to them Some 35 years ago RolandBarthes wrote of our very common practices of surrounding images with words which help to specifyand stabilize the interpretations of particular imagesRoland Barthes The Rhetoric of the Image in Image Music Text trans Stephen Heath Hill andWang 1977 pp 38-39 The original date of publication was 1964

all images are polysemous they imply underlying their signifiers a floating chain of signifieds thereader able to choose some and ignore others Polysemy poses a question of meaning and thisquestion always comes through as a dysfunctionHence in every society various techniques aredeveloped intended to fix the floating chain of signifieds in such a way as to counter the terror ofuncertain signs the linguistic message is one of those techniquesAmong these linguistic messages are captions labels placards guidebooks brochures and fliers--all bits of institutional apparatuses which select and present texts and images for the publicBut see Shane Coopers random captioner and the random2 phase of Jody Zellens All the NewsThats Fit to Print They are the tools of curators teachers and editors They in turn are parts of aneven larger body of institutions and practices which stabilize how images are to be interpreted andused That is when an image is used in a textbook or a treatise we assume it is there to illustrate andsupport the meanings and information provided by the text When an image occurs in anadvertisement we assume that it is there to help sell a product as by depicting an instance of

someone enjoying possession and consumption of the product Thus we have in these standarddeployments of text and image the harmonious relations of explication (by text) and illustration (byimage)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 6: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 629

629

bull the object the something else in the world to which the sign refers

The model is most often represented as the semiotic triangle

This version of the semiotic model is adapted from the work of the American philosopher Charles SPierce Pierce is generally acknowledged as an important pioneer in the study of signs

Notice that

bull the sign and the concept are connected by the persons perceptionbull the concept and the object are connected by the persons experiencebull the sign and the object are connected by the conventions or the culture of the social group

within which the person lives

These connections are important to the study of how meaning arises during the daily encounters withthe many signs that fill the human environment The remaining sections of this tutorial investigatesome of the ways that meaning arises as people make use of signs during the process ofcommunication

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 729

729

Art and the Semiotics of Images Three Questions About Visual Meaning

(Please note this page has many inline graphics and takes some time for a full load It is notbroken It also uses a good bit of JavaScript and Java)

George L Dillon

University of WashingtonJuly 1999

In the last five years the Internet has vastly enhanced our ability to display images to each other andwe can now think of ourselves not just as viewers and consumers of images but as makers and usersof them ourselves Indeed if on the Internet we do not use images we appear stuck in print cultureand oblivious to the possibilities of the new medium We can of course avoid giving these impressionsby including some wallpaper and a few bits of eye candy without thereby getting very far at all intographics as a mode of conveying meaning Schools and colleges certainly offer very little guidanceoutside of the area of technical communication At present we have more questions than answersamong which three seem quite fundamental

1 how language-like are images2 how do images and words work when they are both present3 how do scenes of people gazing and posing convey visual meaning

I will expand briefly on each of these questions and then take them in order So Kress and vanLeeuwen declare Reading Images The Grammar of Visual Design Routledge 1996 p 17 SuzanneK Langer is also often quoted

Some say that images work via a second communicative system one fully as expressive as naturallanguage but separate and structured independently of it Others find visual and verbal meaningsmore dissimilar than similar with the visual lacking a kind of determinacy for which verbal languageseems better suitedSo Paul Messaris Visual Literacy Image Mind and Reality Westview 1994 and Visual Persuasion

The Role of Images in Advertising amp Sage Publications 1997 so also Michael Titzmann cited inphoto text text photo ed Andreas Hapkemeyer and Peter Weiermair Edition Stemmle1996 p 10 This question of the nature and indeterminacy of visual meaning will be the first point we will take up

The second question is obviously related namely how do the two signalling systems work when theyare placed together In principle visual meanings may be entirely separate from verbal ones but as apractical matter we rarely find pure images with no text attaching to them Some 35 years ago RolandBarthes wrote of our very common practices of surrounding images with words which help to specifyand stabilize the interpretations of particular imagesRoland Barthes The Rhetoric of the Image in Image Music Text trans Stephen Heath Hill andWang 1977 pp 38-39 The original date of publication was 1964

all images are polysemous they imply underlying their signifiers a floating chain of signifieds thereader able to choose some and ignore others Polysemy poses a question of meaning and thisquestion always comes through as a dysfunctionHence in every society various techniques aredeveloped intended to fix the floating chain of signifieds in such a way as to counter the terror ofuncertain signs the linguistic message is one of those techniquesAmong these linguistic messages are captions labels placards guidebooks brochures and fliers--all bits of institutional apparatuses which select and present texts and images for the publicBut see Shane Coopers random captioner and the random2 phase of Jody Zellens All the NewsThats Fit to Print They are the tools of curators teachers and editors They in turn are parts of aneven larger body of institutions and practices which stabilize how images are to be interpreted andused That is when an image is used in a textbook or a treatise we assume it is there to illustrate andsupport the meanings and information provided by the text When an image occurs in anadvertisement we assume that it is there to help sell a product as by depicting an instance of

someone enjoying possession and consumption of the product Thus we have in these standarddeployments of text and image the harmonious relations of explication (by text) and illustration (byimage)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 7: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 729

729

Art and the Semiotics of Images Three Questions About Visual Meaning

(Please note this page has many inline graphics and takes some time for a full load It is notbroken It also uses a good bit of JavaScript and Java)

George L Dillon

University of WashingtonJuly 1999

In the last five years the Internet has vastly enhanced our ability to display images to each other andwe can now think of ourselves not just as viewers and consumers of images but as makers and usersof them ourselves Indeed if on the Internet we do not use images we appear stuck in print cultureand oblivious to the possibilities of the new medium We can of course avoid giving these impressionsby including some wallpaper and a few bits of eye candy without thereby getting very far at all intographics as a mode of conveying meaning Schools and colleges certainly offer very little guidanceoutside of the area of technical communication At present we have more questions than answersamong which three seem quite fundamental

1 how language-like are images2 how do images and words work when they are both present3 how do scenes of people gazing and posing convey visual meaning

I will expand briefly on each of these questions and then take them in order So Kress and vanLeeuwen declare Reading Images The Grammar of Visual Design Routledge 1996 p 17 SuzanneK Langer is also often quoted

Some say that images work via a second communicative system one fully as expressive as naturallanguage but separate and structured independently of it Others find visual and verbal meaningsmore dissimilar than similar with the visual lacking a kind of determinacy for which verbal languageseems better suitedSo Paul Messaris Visual Literacy Image Mind and Reality Westview 1994 and Visual Persuasion

The Role of Images in Advertising amp Sage Publications 1997 so also Michael Titzmann cited inphoto text text photo ed Andreas Hapkemeyer and Peter Weiermair Edition Stemmle1996 p 10 This question of the nature and indeterminacy of visual meaning will be the first point we will take up

The second question is obviously related namely how do the two signalling systems work when theyare placed together In principle visual meanings may be entirely separate from verbal ones but as apractical matter we rarely find pure images with no text attaching to them Some 35 years ago RolandBarthes wrote of our very common practices of surrounding images with words which help to specifyand stabilize the interpretations of particular imagesRoland Barthes The Rhetoric of the Image in Image Music Text trans Stephen Heath Hill andWang 1977 pp 38-39 The original date of publication was 1964

all images are polysemous they imply underlying their signifiers a floating chain of signifieds thereader able to choose some and ignore others Polysemy poses a question of meaning and thisquestion always comes through as a dysfunctionHence in every society various techniques aredeveloped intended to fix the floating chain of signifieds in such a way as to counter the terror ofuncertain signs the linguistic message is one of those techniquesAmong these linguistic messages are captions labels placards guidebooks brochures and fliers--all bits of institutional apparatuses which select and present texts and images for the publicBut see Shane Coopers random captioner and the random2 phase of Jody Zellens All the NewsThats Fit to Print They are the tools of curators teachers and editors They in turn are parts of aneven larger body of institutions and practices which stabilize how images are to be interpreted andused That is when an image is used in a textbook or a treatise we assume it is there to illustrate andsupport the meanings and information provided by the text When an image occurs in anadvertisement we assume that it is there to help sell a product as by depicting an instance of

someone enjoying possession and consumption of the product Thus we have in these standarddeployments of text and image the harmonious relations of explication (by text) and illustration (byimage)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 8: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 829

829

For that reason many who have dealt with the semiotics of images have based their discussions onimages in textbooks and above all in advertising Barthes did in Rhetoric of the Image saying that theintention of the advertising image is anything but elusive or problematic Gunther Kress and Theo vanLeeuwen build their semiotics of the visual on such a stable corpus of adverts and texts and it is anentirely reasonable way to proceed --except that in studying the fenced-in image some of thesignifying potentials suppressed by the standard cases will go unrecognised Conceptualist artists in

recent decades have worked to foreground and overturn the standard canons and to explorepossibilities of tension and struggle between images and text

The combination is not only archtypal for Godfrey he eventually takes it as a norm for engagementwith the world and questions whether artists who did text and image and have more more recentlydone just image are retreating into a bygone formalism and estheticism It could be argued that the heart of Conceptual art in the late 1960s was not as is often stated thenotion of the artwork being essentially linguistic but rather the notion that it was simultaneouslylinguistic and visual It is certain true that the combination of text and photograph became increasinglyits archetypal form (Godfrey pp 301-2)

Even the process of labelling itself which was foregrounded rather lightheartedly by Rene Magrittehas been pushed in disturbingly directions as Willie Doherty (see Godfrey pp 367-72) Relationsbetween text and image--whether contentious or harmonious-- will be the second question we will takeup

the Gaze

The standard scriptings of instruction and advertising also allow the viewer to place herself outside thehuman scenes that may be depicted Kress and van Leeuwen describe a two-valued relation to peopledepicted either they look at the viewer and so make a demand for recognition acknowledgement

response or they are not looking at the viewer and in a sense offer themselves for viewing as thirdpersons ( Reading Images pp 121-130) But artists and critics of recent decades have questioned theinnocence of the beholder and for that matter of the subject and artist as well Once we begin to thinkin terms of gaze and pose demandoffer gets complicated in a hurry Looking then is the thirdquestion to be taken up

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 9: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 929

929

The Reconfigured Eye Visual Truth in the Post- Photographic Era (MIT Press 1992)Mitchell is well answered by LevManovich in The Paradoxes ofDigital Photography Photography

After Photography Hubertus vAmelunxen Stefan Iglhaut FlorianRoumltzer eds G+B Arts 1996 pp57-65 and also available online

This little survey of graphic signification will draw on paintingphotography and digital graphics there being no sharp linedistinguishing the latter two and all three appearing viareproductions on the Web To be sure some (the postphotography folks like William J Mitchell) have argued that thecase is quite altered with digitally manipulated images which

give up the claim or even appearance of representing some partof the material world and J David Bolter and Richard A Grusinargue for a line of development in Western graphic culturetoward pure presentation (unmediated by a representer artist)which culminates in contemporary Net graphics There is somepoint to this--digital artists take their images where they findthem httpmusejhuedujournalsconfigurationsv00443bolterhtml whether in a box of oldphotographs scans of objects sitting on top of the scannerstock photos their browsers caches--and we may imagine thegaze of digital takingmaking as directed not through aviewfinder or past an easel but at a monitor screen But just aswe imagine ourselves in scenes of seeing (though at one

remove from the photographers or painters seeing) we cancontinue to do so at two removes perhaps more

One set of very substantial differences remains in the sheer torrent of unvetted images that pour downthrough the Net neither selected by editors nor labelled and explained by curators nor shown andreviewed in galleries The mass media have already filled our lives with a vast eclectic profusion ofstyles and meanings and now even amateurs can display their images on monitors around the worldThe danger is not so much of an anarchy of signifying practices however as much as a vastlylowered expectation of signification in web graphics If we do not pause and look and reflect alongsome of the lines traced here all the great effort to build bandwidth to disseminate graphics andhardware to display them will have been for naught

1 The (relative) indeterminacy of image meanings

For Barthes and for our discussion language functions as a medium with relatively explicitdeterminate meanings to which the meanings of images may on the whole be contrasted Imagessay nothing--they are mute they make no propositions about the world--and for that reason havebeen valued by modernist poets as a mode of meaning or apprehension that does not use discursivereason Victor Burgin ed Talking Photography (1982) To articulate this difference I will develop apoint suggested by Barthes and noted as well by Victor Burgin namely that images like texts have arhetoric of arrangements which signify but there is no syntax that articulates their parts and bindsthem into a whole

Though pictures are quite different from texts of natural language they are not wholly different andmany have sought parallels between the two media Like texts most pictures are composed of parts

though the parts are bits of image (and perhaps words) arranged on a surface When the variousshapes in a picture wash and flow and blend into each other and the background they do not seemvery much like words but when they have crisp edges as for example in the Dada photomontageintroduced here they have attracted the term word and their arrangement likened to a syntax

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 10: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1029

1029

Hannah Houmlch Cut with a Kitchen Knife through the Beer Belly of the current Weimar Republic(1920) For example Dawn Ades in her overview Photomontage (Thames and Hudson revised andenlarged edition 1996) says of this famous piece by Hannah Houmlch disparate elements photographsand scraps of text are thickly scattered over the surface but still remain legible like words on a page(p 30)--but a page crucially with words arranged on it not placed in sentences Further suchmontage is as they say flat which means that there is no topography of concepts no arranging into aspace ordered by perspective but only a topology of relatedness conveyed by touching andseparation and spatial order (See John Willats Art and Representation Princeton University Press1997 p 13 and c3) It is hard to tell what relative size or overlapping indicates Nonetheless theseplacements signify--here by contrast oxymoron antithesis and incongruity (catechresis) principally--but not by virtue of their grammatical role in sentences That is there is arrangement and compositionof the parts and these arrangements signify after the fashion of the artful patterning of words (thefigures of words of classical rhetoric) rather than the constructions of grammar or the formulae of logicRhetorical signifying is also notoriously polysemous words arranged in a list for example can conveyplenitude even to the point of overflowing (epitrochasm ) or equivalence or precise detailed attentionor hierarchical ordering And so we may say can images But for language these rhetorical figures ofarrangement are a secondary signifying system for images theyre all weve got As long as themeanings we have to convey pertain to objects in space a graphic display is fully as adequateperhaps superior to a verbal description (we often draw diagrams to clarify such meanings) But asPaul Messaris argues (using syntax metaphorically)as soon as we go beyond spatiotemporal interpretations the meaning of visual syntax becomes fluidindeterminate and more subject to the viewers interpretational predispositions than is the case with acommunicational mode such as verbal language which possesses an elaborate set of explicitindicators of analogy causality and other kinds of connections between two or more concepts ( Visual Literacy (1994) p xiii)

El Lissitsky The Constructor (1924) When the edges of the parts are blurry or they are overlaid andmerge one into the other then figures of identity duality (amphibole) and metaphor come more tomind Graphics that do this sort of thing move away from representation of objects in a physical space(with defined light source) toward what Kress and van Leeuwen call lowered or less realisticmodality--they ask to be taken more abstractly as a schematic diagram of the way the world might beor ought to be ideally or is in a certain underlying aspect) Of this well-known self-portrait by the

Russian Constructivist El Lissitsky (1924) Edward Tufte says

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 11: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1129

1129

Overlapping images express a multiplicity of links and metaphors the minds eye the hand ofcreation the coordination of hand and eye the hand and tool the integration of person and work thewholeness of artistic creation--and possibly even a halo for its saintly constructor ( Visual Explanations (1997) p 140)One can only agree with this but Tufte plunges forward into syntactic metaphorBy showing steps between the ideas in the mind to the reality of the paper Lissitsky illustrates the

process of graphic thinking and creation Each visual bridge acts as a verb to link up the nouns (mindeye hand compass image type grid paper) of artistic work That work on paper then reflects back(via the pointing arrow) to eye and thought The grid of the graph paper orders both worlds (p 141)Note that the metaphor the minds eye has now sprouted mind as a separate object in the pictureIf the visual bridges are verbs what verbs are they ISA Flows forth Tuftes flight of syntacticmetaphor obscures the difference between images and words and suggests a precision of articulationthat the picture does not have (Note that it only suggests that articulation he doesnt spell thesentences out language we are reminded can be used to intimate as well as to declare and often isin art criticism) To be sure Tuftes words are as much enthusiastic celebration of the picture asshrewd analysis of it but they do illustrate one of societys techniques of fencing in the image namelyby critical commentary here specifically by turning the image into a quasi-statement And it is to thesetechniques and institutional arrangements that we now turn

2 Text andversus Image

Whether or not images are inherently more polysemous than words it is very common to find (andseek) words around exhibited or published images--titles labels placards guides the artists wordsand so on Classically however the words are peripheral to the work and confined to backgroundinformation and perhaps a few interpretive hints and pointers to notable features of the work Artistsare notoriously sparing of words preferring to let the image speak for itself In mass media howeveras Barthes noted words are everywhere from speech bubbles to voice over to writing overlaid on theimage (poster or slogan fashion) and when conceptualist artists started writing extensivecommentaries next to or on their images they simultaneously broke down the imagetext andHighMass culture dividers

To see how much energy and interest can be generated fromsplitting of placard and image consider the Statuary series byJacqueline Hayden on wwwzonezerocom the first one of 10 ishere in the margin These pictures are presented one by one ina highlighted oval (museum lighting) against a rich dark maroonfield each comes with a little placard button that when pressedopens a window as here with the placard (The picture also canbe enlarged) The placard text in each case seems utterlyunaware of the modification Hayden has made to the antiquetorso and thus enacts the obliviousness of the Western fine artstradition to the look of bodies past the age of fifty The imagesare rather small platinum prints done with great care and finefinish and the exhibition is not a joke or mockery of age by youth

or of museum culture by the realities of the aging body orpreposterous vanity of those past their physical prime Thesetensions are evoked but not resolved (since images dont sayanything) rather the gaze they call forth is a compassionate oneseeking and finding a certain kind of beauty

But that is getting ahead of the story which begins with thestandard arrangement whereby text may discreetly assist us ingetting the image to float in the right directions

To begin with the simple determining function of text comparethe following two images from an exhibit catalog from whichsuperimposed words have been removed so that you can

experience their float without words you can then add thewords by clicking the Add Text button This first is anabundant display of supermarket prepared food and one could

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 12: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1229

1229

Catalog piece 1

imagine several possible lines of intention (they are all Kraftfood products they run heavily to cheese and preserves theyare a riot of color shape and detail that severely challengescomputer resolution they are unbounded in all directions) but(youve clicked it already havent you) the words (enlarged forlegibility) anchor the display to a very conventional dismissal of

American processed food

Catalog piece 2

In this second graphic the wordsPost Human seem to point tosome kind of future world or tendency it echoes the otherposts --certainly poststructualism is post humanist--but whatpart of the post human world are we contemplating and withwhat attitude The image is also a bit hard to make out becauseof the angles the woman may be partially submerged (butupside down) and the light is no help either Is this some kind ofcryosleep in zero gravity There are a lot of things that might be

called post human

There are better clues available than the words on the imagethis graphic like the preceding one comes from an exhibitioncatalog for a show sponsored by the Deste Foundation forContemporary Art in Athens USA (Ohio) in 1990 CalledArtificial Nature the catalog pursues the phrase post human through many pictures of the artificial replacing altering andglossing over traditional human limits It even provides anotherview of the striped lady who apparently is lying in a few inchesof water at the bottom of a whirlpool bath Clearly the text doesnot close down interpretation here or even give it muchassistance

If text completely gives way toimage it becomes typographyvisual shape Lettrist textile designtexture (as in faded adverts on oldurban brick walls) or ascii-art Agood place to explore turningvisual is The End of Print the Graphic Design of David Carson ed Lewis Blackwell and DavidCarson Chronicle Books 1995

In these first rather simple cases one has the impression thatthe image came first and the words were added to interpretwhat was already there When we speak of illustrationhowever we are usually thinking of adding an image to analready existing text and this relation too would seem to anchorthe image At times however the image seems to interpret thetext quite broadly or even undermine it Consider for examplethe following work from Wired magazine

Each issue of Wired includes a 4 page (2 double) spread beforethe Contents page which cites a line or two from a featuredarticle later in the magazine and functions as a teaser (or highly

graphic abstract) for the article The sentence to be quotedand graphicked is usually long enough to support the two stagesetup (double page one followed by double page two) as forexample additive or contrastive pairings or cause and effect

Data 1

The Data set of pages is built on lines from an article about aSeattle company that recovers old email even deleted emailThe lines seem rewritten over themselves The line in Data 1Backups containing millions of email messages are the digitalequivalent of formaldehyde offers a simile which is the basis ofthe green liquid look with its bit of magnified mosquito or cranefly in it

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 13: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1329

1329

Data 2

Turning the page the color changes to fiery red and hotteryellow to a lake of fire or furnace with old disks a key somemore cranefly wing numbers and labels The text saysexplicates the simile a medium where nothing decays Thefire could be taken as what puts companies in the hot seat butit can also attract traditional connotations of Hell the place

where nothing is forgotten or forgiven For me seeing a sort ofdolls face or mask in the fire invites this human association withthe digital eternally unforgotten This I should add carries thesignificance of the graphic far into a spiritual dimension that haslittle to do with the content of the article which mostly turns onCYA for corporations If the reader turns to the indicated pageand begins to read the article she likely will be disappointed bythe absence of metaphysical grandeur Which is to say that theartist takes the lines out of context and composes a visualmeditation upon them the graphic however is still anillustration of what the words propose

Market 1

Usually Wireds graphic serves the bit of quoted text the nextexample is unusual in its relation to the quoted words GaryWolfs featured article in June 1999 Wired profiles Sir JohnTempleton and his investments in religion specifically inshowing that good religion is good business The two double-page spread is built on lines from one of Templetons operativesand is neither explicitly endorsed nor derided in the text of thepiece In context it both celebrates the triumph of worldcapitalism and outlines the next area for it to annex namely therealm of moral values

Market 2

On the first two pages the two spray cleanser containers on theright margin seem to express the result of the end of thestruggle for markets Photographed in hard focus and brightlight against dead black with nothing but the text to support

them they illustrate what Kress and van Leeuwen callhyperreal modality which in this case links to sensualpleasure focussing on the consumer object typical of food anddrink adverts (p 169) (see also John Berger Ways of Seeing pp 140-141) When we match these pages with their textdeclaring the settling of the fundamental battle over the freemarket it is hard to avoid the effect of severe understatement(or underrepresentation) amounting to ironic deflation (the lateconsumer capitalist market economy as epitomized by thechoice of cleansers now dominates the scene--with BruceSpringsteens 57 channels and nothin on in the background)In the second pair of pages the two packs of cigarettes (on salein Japan I hear) fill the position of the cleansers and would

appear to represent the not yet realized victory of the market inthe sphere of morals (And here they bear their own texts(Peace and Hope) which push even beyond Fantastic andFabulous as Orwellian perversions of the words) Thegraphics thus mock the words from Templetons agent byreducing the grand phrases to their practical consequences indaily life capitalist marketing of morality would offer us immoralcommodity choices packaged with positive words--howbackward can people be to withhold assent In this display fromWired graphics comes as close as it can to making a counterstatement

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 14: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1429

1429

The Butter is Gone

This degree of distance between graphic and text is typical ofpolitical cartoons and demonstration placards The graphic stylehowever is usually not realistic but exaggerated and parodicwhich is to say of lower (less realistic) modality (as if this is theway the world would be if these views were real--conditional ifnot irrealis one might say not indicative) John Heartfields The

Butter is Gone (1935) is a famous exemplar The text is aquotation from a speech of Hermann Goumlrings Bronze hasalways made a nation strong butter and fat at best make apeople plump And so the butter being gone the family isdining on metal Although the graphic is a montage of photosthe swastika wallpaper and general absurdity of the actionslower the modality

What does Possession Mean toYou

Victor Burgin developed exactly the opposite relation of text andgraphic in his political work of the 1970s here the image isappropriated from an advert and the text written on it is socialcritique or theory One quite well-known one (Possession)

was done at the time of an exhibit of contemporary artists inNewcastle The Arts Council asked for some publicity postersand Burgin responded with Possession 200 copies of whichwere pasted up on the streets of Newcastle Burgin intended forthe diametrical opposition of text and image to catch the gazeand trigger thought Follow-up research indicated that not manypassersby remembered what the posters said much less whatthey implied For a few more years Burgin continued to exhibitlarge photographs with substantial text (often over 100 words)at odds in various ways with the image The effect is sometimesa rather professorial and preachy enumeration of thecontradictions of late capitalist consumer society but at othertimes it is more suggestive enigmatic or tensely ironic as

when he quotes Foucaults description of the Panopticon in apicture of a Berlin peep show with circular stage

Victor Burgin Life Demands a LittleGive and Take (1974)

In Life Demands a Little Give and Take text and image are inthe opposite relation to Possession namely the text is fromthe commercial advert and the image is from the street I am notsure how readily the image would make sense with no contextbut in a collection of pictures that deal with the contradictionbetween manipulative obfuscating culture (ideology) and realmaterial conditions it is not hard to see this picture as anexposure of the racist overtones of pale=beautiful That is wehave ordinary people waiting for a bus on a nondescript streetcorner in modern Britain among whom the cameras gaze fallson a woman who is distinctly not pale and who does not qualifyas one of the targeted audience of the fashion magazine spiel

Keith Arnatt Trouser-Word Piece

(1972)

Burgin was certainly not alone making text+image displays inthe 1970s much conceptualist art would fall under this rubricKeith Arnatt for example exhibited a similar display this timewith a philosophic theme Tony Godfrey who cites this worksays It is uncertain whether the photograph acted as a critiqueof the philosophy or was merely the pretext for quoting it (Conceptual Art p 172) which is a way of saying he is not surewhether the image illustrates or undermines the text He finds

the text unmemorable and finally unnecessary saying that itserves ultimately only to underline what is implicit In a sense

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 15: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1529

1529

you can always say that even when you dont say what is implicit but for me it does highlight certain themes inherent inthe situation (the contradictoriness of all self-authenticatinggestures uncertainty (a la Austin) of what accusation is beingdenied the making of such a photograph declaring oneself tobe a real artist--is it real art is the art more real with the

accompanying text The photo might in itself cast the viewerinto its reflexive abyss but the text certainly does help This isart that makes you think

See Knorrs work in Hapkemeyerand Weiermair and also in Other then Itself Writing Photography edsJohn X Berger and Olivier RichonCornerhouse Publications 1989

Not all Conceptualist artists played the big discourses of politicsand philosophy off against images some like Karen Knorrplayed bits of self description or art cliches off against exquisitelyphotographed interiors to engage the viewer in reflectionBergers and Richons own contributions to the collection areeven more oblique in the relation of text to image as if thetextual material is slipping out of alignment with the visual Thetexts certainly do not dominate over these images and this maypartly have to do with the extreme degree of deliberation andhigh degree of technical finish the images exhibit Without thetexts however I am not sure we would have much of a clue asto what context to place the images in (they do appear in sets inthese publications)

(Victor Burgin Between Blackwell 1986)

3 The scene of looking

Appreciation

m) Bernardart critic

orks such asrs of) The StudyItalian Art

nd Essays inthe age

allery Rome

Natalie Bookchin and LevManovich Porno_Pictorialism(1995) from Digital Snapshots

Victor Burgin Graffitication (1977)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 16: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1629

1629

The first image depicts looking as the classic scene of art appreciation which authorizes among otherthings the refined and learned connoisseur Mr Berenson to gaze upon the statue of a largely uncladwoman We are safely at a second remove standing behind the statue watching Mr Berenson gaze(with yearning the Chim memorial website has it) at the figure that does not meet his gaze (this isAntonio Canovas Paola Borghese as Venus and she is staring off down the length of her couch) Thesecond image which has been digitally manipulated has us once again gazing upon a scene of

gazing though this time we infer the gazers view from her legs The oval framing the scene suggestseither a peephole or a classic oval frame We do not see her expression to tell us what she makes ofher collection of images of women The title suggests erotic reverie The third image we owe to VictorBurgin complete with a lecture upon the voyeur as fetishist It is unmistakably the scene of guiltyviewing unauthorized by anything Photographs even manipulated ones give us very strongly theimpression that we are seeing some part of the world and sharing the view of it with the photographerwho saw it in his viewfinder We can very easily be drawn in to imagined scenes of picture makingand a good bit of the meaning these pictures hold for us has to do with how we play out the roles theycast us in These enterings into the scene are by no means confined to photographs the art criticMichael Fried has developed extensive and detailed theories about it in relation to nineteenth centuryFrench painting (and hence in relation to modernism gnerally) but cameras automatic vanishing pointperspective offers us a familiar world in which our own viewing point is always readily apparent

Photography offers us two stories about the making of photographs One call it the frozen moment oflife is associated with photojournalism street photography candids and snapshots It capitalizes onmodern photographys ability to capture some part of the way the world looks in a given place andinstant (modern because you need decently fast emulsions and sometimes good flash) Thephotographer may take many exposures from numerous angles and lens settings but she will look forand try to seize the decisive moment in which the fullest significance of the scene is manifest Therecan thus be only limited planning graininess high contrastcropping which breaks objects and blurgive authenticating testimony to the unplanned catching of the unstaged life of the moment

Michael Fried also describes two temporalities in painting as well (see Manets Modernism Universityof Chicago Press 1996 pp 290ff

The alternate story of the scene of taking photos contrasts on most of these points bringing it closer tostudio-composed oil painting Here nothing is left to chance--nothing occurs by chance--and the viewermay ponder as long as he wishes why this or that detail is exactly as it is It is a tableau vivant Thereis still the difference from painting that all objects are seen in the cameras eye in one exposure nonein the artists imagination only so that the actual moment of time assumption is still maintained Thisis perhaps why photography is so effective as a medium of pornography the photographer must havebeen just a few feet away from the subjects who were doing exactly what you see to each other (or tothemselves) (It is sometimes suggested that as people begin to realize what digital manipulation ofphotos can do--that the participants may never have been together in one place exchanged looks orbodily fluids--they will lose their appeal as a focus for fantasizing)

On either version of the basic story then there was a moment when the photographer looked into theviewfinder and saw the scene that ultimately appeared in a print or transparency The photographer is

thus the first viewer of the scene and we as viewers imagine ourselves with our eyes at the place ofthe taking lens--where that is we infer the lens to be This positioning in the scene is not just physicalhowever but moral as well that is we can easily put on what we think to be the artistic (or salaciousor reportorial ) attitude of the photographer--his or her gaze This line of thought seems to beheading toward suggesting that there is something dubious at least in plenty of cases about lookingand freezing the appearance of someone or ones for public distribution Didnt your mother teach younot to stare Above all not to stare at cripples wounds beggars deformities private parts rottingfood tubes protruding from the body and people talking with no listener in sight--as if looking (so thepsychoanalytic story goes) for what is wrong what is missing or for reassurance that it isnt reallymissing (the fetish) Victor Burgin Between 1977 So Victor Burgin gives us one image of the sceneof seeing--the voyeurs peek into the lighted room of an adjacent dwelling--(along with a little lectureabout the paradox of the photographic image as fetish) This is the classical viewervoyeur scene ofunlicensed transgressive seeing--the subject is exposed to our gaze unbeknownst to them We

assume they would not want us to be looking at them this way and the exposure is all on one sideThere is something transgressive here A border is being crossed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 17: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1729

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 18: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1829

1829

Velasquezs The Toilet of Venus

Walkman-induced neoabsorption also caught the eye of JanSaudek who gave Velasquezs Rokeby Venus a similarmakeover The Velasquez original fits nicely into this theme ofabsorption and gaze In it Venus turns her back toward us and

appears to be entirely absorbed in her own image in the mirrorheld by Cupid But wait if we can see her image then shecannot she sees our image and so more indirectly anddiscreetly than Olympia she gazes back (Click on thethumbnails--the filiation between the pictures is not evidentotherwise)

Jan Saudeks Walkman

Clearly Saudeks take on the irruption of Walkmans into modernlife is similar to Walls once again a gaze that existed in theoriginal is absorbed by the black hole of the personal listeningdevice The idea of these images spoke so much to Saudek

that he did a second Walkman version with a classicalNarcissus image Note here the very close attention toreplicating the inner and outer fabrics and the position of thefeet which is just different enough to make it clear the wholeassembly was photographed anew

Jeff Walls The Storyteller

Another of Jeff Walls depictions of modern life (in particularmodern life in the Pacific Northwest) turns on a remaking ofManets famous Dejeuner sur la Herbe (which keeps peepingthrought the moving reader slit in the online version of thispaper) as the very large transparency The Storyteller (229 x437 cm) Here too we have gatherings in public park spacesthough the setting on the landscaped banks of a freeway

overpass is a far cry from the Paris herb and the temperatureis cooler judging by the clothes of the figures and their little fireClearly it is Vancouver (Wall taught Art History at Simon FraserUniversity for many years) The principle point of contact withManets Dejeuner is the group of three most particularly theposture of the man elbow on knee Manets grouping is directlylifted from Marcantonio Raimondis The Judgment of Paris (--seeFried eacute Manets Modernism p 56) But the relations arestrikingly different the three members of the group are engagedin the womans story and no one naked or otherwise has anyawareness of or interest in us I find this a salutary treatment forthose who might yearn to go to Paris and live in theImpressionist period

(We should perhaps note that Manets Dejeuner is a verystrange painting--very hard to make narrative sense of (why isshe sitting there nekid the men clothed and no one paying theslightest attention except us) At least one reader namely theBarbie parodist Dean Brown has visually shown another storypainted over in the picture as we have it today

32 including the shooter

The set of engagements (and non-engagements) is furtherenriched when the photographer includes himself or herself in

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 19: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 1929

1929

the scene I am not thinking primarily of Cindy Sherman whoincludes herself as the main subject but of photographers whodepict themselves depicting Such acts require mirrors andbreak the conventional twining of viewers and photographerseyes That is the viewer cannot be the implicit photographerwhen she sees the photographer represented behind the

camera (assuming it is the camera that took the pictureshooting into a mirror) If she sees the photographer viewingthrough the taking lens where is she viewing from Theclassical precedents for such pictures are the grand canvasesof Velasquez ( Los Meninas may it rest in peace) and Courbet (The Painters Studio ) but as paintings the viewer as painteris less compelling That is we know that the painter can painthimself into the scene any day he pleases but the sense ofshared instant of time is so much stronger that these reflexivepictures are disorienting The one resolution I think is to backthe viewer away from imagining himself as interacting in ascene of photographing and promoting a kind of detachedanalysis (and perhaps admiration) of the artifice--or amusement

at what can easily come off as self-deprecating JonathanMillers On Reflection includes a couple of pages (pp 184-5) ofphotographers self-portraits with taking camera one byAndreas Kertesz uses a distorting lens and model to suggestthe queerness of the situation But perhaps the most copiousand now well-discussed body of such self-portraits is by HelmutNewton

Helmet Newton Self Portrait withWife June and Models (1981)

An introductionary essay by Urs Stahel to Helmut Newton Selections from his Photographic Work (Participating withoutConsequences Rules and Patterns of Newtons Voyeurism pp 19-30) discusses a number of Newtons pictures of himself atwork photographing nudes Among these is one (Self Portraitwith Wife June and Models Paris 1981) upon which VictorBurgin has lavished much semiotic and psychoanalytic attention(see InDifferent Spaces University of California Press 1996cc 2 and 3) Although Burgin begins with a textbook applicationof Barthesian semiotic analysis (first denotation--the non-codifieddescription of the scene and then connotation--the cultural codesand associations of raincoats FM spiked heels pinup posturefollowed by rhetorical patterning of antithesis and repetition)he moves toward explication of the feminist psychoanalyticargument of Laura Mulveys work (and toward personal themesengaged by the picture) What both Burgin and Stahel ignore isNewtons opening up of the scene of the work and theconsequences of glamour photography This is a scene fordramatic imagining what can the model be thinking as Newtonswife sits watching like a casting director Is she turning towardhim to receive instructions What can Newton be thinking as hepositions people (and make no mistake they are all positioned)and dons a raincoat Why does he make himself so short Whatexactly might June be thinking Is this a proper use of the VogueParis studio Whos paying the model and when we havefinished all that what about the other model It seems to me thispicture works exactly against Stahels title it drops the screensand baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracketand conceal

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 20: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2029

2029

Eduoard Manet The Bar at theFolies Bergegrave

The last of these pictures thematizing the acts of viewingmaking and seeing is a near contemporary of Newtons SelfPortrait namely Jeff Walls equally wellknown Picture forWomen Like many other Walls it has a precursor in Manetnamely The Bar at the Folies Bergegravere This too appears tohave a mirror this time behind the subject in which her

reflection along with that of a patron appears The geometryas has been noted by a number of critics does not seem to bequite right if we are standing more or less directly in front of her(though not meeting her gaze) then it is hard to know wherethe other customer is located or else where we are (Onecritical cartoon of the times drew the scene up supplying whatM Manet had forgotten to put it namely the figure of the othercustomer standing to the right back to our view In a sketch forthe painting Manet posed the girl looking sharply to her leftacross the viewers gaze to the customer) It is above all thewomans posture that echoes Manet Here we note a bit ofillusionism even in classic realism--it is hard to imagine giventhe scene Manet wants to evoke where he would set his easel

or how it would look if he chose to paint it in

Jeff Wall Picture for Women 1979

Wall however drops the illusion of being anywhere but hisstudio and also opens up the full apparatus of enhancedwarehouse lighting and wiring all of which set up superb parallelline grids to assist the eye in perspective The light standspartition the composition into a triptych rather classicallyoccupied by the the three principle persons the subject thephotographer and the camera eyeI (but the light favors her)The woman once again reversing Manet is looking directly atthe viewer in as level a gaze as one could imagine--notchallenging or flirtatious or submissive supplicating the listgoes on Well of course she isnt looking at you shes looking atthe camera but Wall stands a good distance away from thecamera and farther forward (that is a very long cable release hehas there) He appears to be looking off the mirror at her Butthe effect of moving away from the camera is to vacate thespace of the viewing eye which is then free for the viewer to fillThe central protagonist is the camera and the camera is you

Jeff Wall eds Thierry de DuveArielle Pelenc and Boris GroysPhaidon Press Ltd 1996

In his Survey The Mainstream and the Crooked Path to Jeff Wall (ed Thierry de Duve Arielle Pelenc and Boris GroysPhaidon 1996) Thierry Duve celebrates this photograph as abreakthrough modernist photograph For him this meansbroadly self-critical and self-referential and narrowlyconscious of the medium which in this case is thetransparency of the pictures surface (p 29) But I do not think

we are made aware of the materiality of the photographs (ortransparencys) surface rather I think that our awareness thatwe are looking at a photograph collapses Our brain tells us thewoman is posed in an utterly contrived position with her handsresting on the edge of a plywood sheet not more than 4 feetwide facing directly into a large plate glass mirror Butperceptually one or two (incompatible) conclusions seemevident either she and her assistant Mr Wall are waiting foryou to come to the camera to take the shot or they are about totake your picture This completes the turning of the tables onthe viewer who becomes finally the viewee Surely the titlePicture for Women is some sort of pointer Then herremarkable gaze becomes The Gaze the regard classically

directed from the male observor toward the female object nowhere reversed

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 21: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2129

2129

Conclusions

Throughout we have been working with the modern notion of artas de-automatization--as making conscious and evident thegrounds of normal day to day viewing--through the violation ofconventions some of them conventions of practical graphics

and some of them conventions of classical art appreciation Onthe issues of rhetorical signfication tension between text andimage and the scene of viewing we have been able to teaseout interpretations according to regular and one hopestransportable principles using text and figures of rhetoricalform though with no hope of a syntax-semantics mappingstrings of images onto logical forms The general point seemsfairly evident that insofar as a certain image does de-automatize it obtrudes its own making and functioning in waysthat would interfere with its use in advertising or instructionHence these are not the images and ways of signifying that willbe found in your basic corpus of practical working images

Over and over Tony Godfrey says of conceptualist art that itspurpose is not to be beautiful but to make you think Such artshould resist adaptation to advertizing or instructional usesespecially the former since the purpose of advertising is tofocus your thoughts on the object for sale not to make you thinkbeneath the surface But of course the industry employs manyvery clever people and its appetite for a new look is insatiableEven locating the viewer as the maker of the image can bebrought off say in a camera ad

Here is one last image--an advert forAgfas digital camera from the August1999 edition of Wired In broad outline of

course this is conventional to and beyondthe hackneyed point selling the cameraas a sex-appeal-enhancing possessionBut there is a special twist--this happyencounter occurs as the camera is beingused not just displayed Assuming thepicture is what you see you look upseems to refer to the moment when youlook up through the cameras viewfinder to shoot the woman in the second storywindow (this is why the window casementis appears so tipped inward at the top)she sees you in the act of shooting

approves of your somewhat cyborgianmien (which of course is not depicted)and blows you a kiss The crucial clue forthis interpretation is the slight verticalpinching in the middle of the picture (iethe top and bottom edges are not straightbut curve inward then outward again)This gives viewfinder look So you wantus to think about the scene of shootingOK we can use that to sell cameras tooincredibly easy to use ePhoto digitalcameras

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 22: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2229

2229

SRB ArchivesThis article appeared in Volume 6 (2) of The Semiotic Review of Books

SRB Insights Can Pictures Lie

Winfred Noth

Pictures have for a long time served as scapegoats to the apocalyptists in the domain of mediastudies The apocalyptic scenario of the power which pictures exert in manipulating and deceiving themasses appears as early as 1895 when Gustave LeBon in his Psychology of the Masses describesthe picture as a medium for manipulating the minds of the primitive ones The masses he writescan only think in images and can only be influenced by means of pictures Only pictures can frightenor persuade them and become the causes of their actions To them the unreal is almost as important

as the real They have a striking tendency not to make any difference (Lebon 1895 S 32)

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 23: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2329

2329

In a less elitist vein some modern critics of the mass media continue to deplore the decline of the ageof verbal reasoning in the face of our present-day immersion in visual media from advertising to thecomputer screen According to their scenario the tyranny of the viewers pictorial immersion results inuncontrolled emotional involvement with - and the resultant lack of critical distance from - the pictorialmessage (Buddemeier 1993 20)

Whatever foundations such warnings against the manipulative power of pictures may have we canonly focus on one of its aspects namely the question whether the alleged manipulative power ofpictorial messages could also derive from an inherent semiotic potential to lie that is the creation ofuntrue pictorial statements with the intent to deceive

According to Umberto Ecos Theory of Semiotics the question of whether phenomena can be used toconvey a lie should be considered as crucial evidence of their sign nature On the contrary somethingthat cannot be used to lie should not be considered as an object of semiotic investigation Eco (19767) states these ideas in the following much quoted passage

Semiotics is concerned with everything that can be taken as a sign A sign is everything which can betaken as significantly substituting for something else This something else does not necessarily haveto exist or to actually be somewhere at the moment in which a sign stands in for itThus semiotics is inprinciple the discipline studying everything which can be used in order to lie If something cannot beused to tell a lie conversely it cannot be used to tell the truth it cannot in fact be used to tell at all Ithink that the definition of a theory of the lie should be taken as a pretty comprehensive program for ageneral semioticsThere is little doubt that pictures can refer to something that does not exist or has even never existedbut do such pictures therefore lieSurrealism has given ample evidence of paintings referring to mereimaginary objectsConsider for example Salvadore Dalis Burning Giraffe (1935) which shows astrange woman with open drawers protruding from her legs We are hardly inclined to call the painterof this work a liar but even the category of truth at least in the positivist sense does not seemapplicable

Although it is clear that pictures can refer to factual reality and to the unreal the question whether they

can convey a truth or a lie remains disputed

What is the semiotic potential of pictures Can they express ideas that correspond to verbal messagesat all as the proverbial saying which states that Pictures can tell a thousand words suggests or isthe semiotic potential of a picture inferior to the one of language insofar as a picture is necessarilyvague and in principle unable to depict any truth about the world as some logocentric semioticiansclaim If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie

The question of truth or lie in pictures has a semantic a syntactic and a pragmatic aspect From asemantic point of view a true picture must be one which corresponds to the facts it depicts From asyntactic point of view it must be one which represents an object and conveys a predication about thisobject and from the pragmatic point of view there must be an intention to deceive on the part of theaddresser of the pictorial message

Let us begin with the semantic dimension of our topic Photographs seem to be prototype of visualmessages which are true because they fulfil the semantic criterion of correspondence to the factsUnder certain circumstances photographs are even recognized by the courts as documentaryevidence which may replace evidence by ocular inspection or by verbal testimony (Robert 1974 17)

A pertinent example is the legal status of a passport photo as a document for establishing the realidentity of the person presenting the passport to the authorities From the legal point of view truth inthe sense of correspondence between a signifier and its referential object can thus be derived fromphotographic pictures

Semiotically the correspondence of the photographic signifier with the object it depicts is grounded in

what Peirce described as the indexical and the iconic nature of photography Photographs correspondto the depicted world by their iconic nature because as Peirce (CP 2281) puts it we know that they

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 24: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2429

2429

are in certain respects exactly like the objects they represent In addition to this correspondence bysimilarity photographs also correspond to reality by their contiguity with the depicted object at themoment of their production There is a physical connection between the signifier and its referentialobject since as Peirce (CP 2281) argues photographs have been produced under suchcircumstances that they were physically forced to correspond point by point to nature By this relationof productive causality the photographic picture is defined as an indexical sign

It is primarily because of this indexical signature that we tend to see in the photographic signifier anaffirmation of the existence of the depicted object A semiotician who emphasized various aspects ofthis indexical nature of photography is Roland Barthes In his words the photograph is an emanationof past reality (Barthes 1980 88) one could think that photography always carries its referent withitself (1980 5) and the noema of photographyis its message this is the way it has been (198077)Family photos which remind us of real situations lived in the past press photos which document ahistorical event such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943 or scientificphotos which show a real world object in all its details are typical examples of indexical photographicreference and iconic correspondence between the photographic signifier and its object which testify tothe truth potential of the photograph

Nevertheless everyone knows that photographic correspondence can be manipulated The referential

object may be transformed in the picture and its viewers arrive at the illusive or deceptive impressionof a nonexisting object This deceptive potential of the medium was recognized early in the history ofphotography and made use of in techniques such as retouch colour filtering solarization doubleexposure

By retouching the signifier referring to an existing object could be made to disappear By montage anonexisting object could make its appearance on the scene Thus photography became a mediumwhich lent itself to manipulation deception fakes and forgeries The more recent developments incomputer graphics with the new possibilities of shape blending distortion simulation and othermodes of digital image manipulation have greatly increased this deceptive potential of the medium

Manipulations of the photographic image provide a rationale for Umberto Ecos (1984 223) argument

that photographs can lie However instead of a lie these are mere visual metaphor hyperbols not tobe taken seriously The difference between a really deceptive fake a genuine visual lie and our topicis in the pragmatic dimension of the photographic message From the semantic point of view ourexamples do exemplify the pictorial potential of lying Just like fakes manipulated photos are visualmessages which depict but do not correspond to the reality depicted

But before further specifying the pragmatic differences between lying and other modes of visualcommunication we have to examine the syntactic dimension of truth and falsehood in pictorialmessages

In language only sentences and not individual words can be true or false The statement The cat is on the mat may be true or false but not the individual words cat and mat Truth values can only bederived from sentences or propositions in which a subject or argument is in a syntactic relation to apredicate Is it possible to discover similar syntactic conjunctions of visual signs in pictures

Since there are no words nor verbal propositions in pictures let us use the more general semioticterminology which Peirce introduced in the framework of his theory of signs rheme as the moregeneral semiotic equivalent of words and dicent as the general equivalent of propositions Thequestion is then can pictures function as autonomous dicentic signs or do they only consist ofrhematic signs Do pictures only represent objects or can they represent objects together withpredications about these objects For three very different reasons the answers which the theory ofpictorial representation has given to this question have been negative These three arguments may becalled contextual incompleteness non-segmentability and dicentic vagueness

The argument of contextual incompleteness was first exposed by Gombrich (1960 58-59) In his view

pictures alone can never function like true or false statements Only when a picture is accompanied bya caption or label can the resulting text-picture message convey a true or false proposition Captionsbelow press photos or a name below the picture of an object are his examples The logician Bennett

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 25: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2529

2529

(1974 263) interprets the picture in such text-picture combinations as predicates in schemes ofpredication According to this view the photo of a Siberian Husky above the caption Siberian Huskyfunctions as the pictorial predicate which combines with the written name as its verbal argument toform a true verbal-pictorial statement An example of a false message of this kind would be one ofRene Magrittes paintings of objects with deceiving labels for example his work La Table IOceacutean le Fruit (1927)where the label table is attached to the picture of a green leaf and the label fruit to the

picture of a jug In such verbal-visual messages it is not the picture alone which forms the propositionand therefore Bennett (1974 259) concludes Pictures are not themselves true or false but only partsof things that can be true or false

Muckenhaupt (198488) in his book Text and Picture basically agrees with Bennett with respect tothis general argument of contextual incompleteness of pictures but believes that the image in the text-picture context does not function like a predicate but rather like the argument of a propositionAccording to this interpretation the picture of a car in a police photo functions as the pictorialargument of a dicentic message whose predicates are verbally expressed in the numbers on thelicense plate and on the speedometer

Whether the picture functions like an argument or as a predicate what these interpretations have incommon is that they consider the picture as an incomplete rhematic message which can function only

as part of a larger dicentic whole when it appears in conjunction with a verbal message Against thislogocentric thesis of the dicentic incompleteness of pictures I would like to argue that the function ofpictures in text-picture combinations says nothing about the semiotic potential of pictures seen withoutlabels or captions The thesis that pictorial messages can only be completed by their verbal anchorageis rather an indicator of the logocentric bias to be found in the current theory of pictorial representationIn fact although pictures without verbal anchorage may have become rare in our age of multimediacommunication such pictorial messages are by no means uncommon In pictorial genres such apaintings family photos or touristic slides the lack of verbal anchorage is even the rule

Nevertheless we still have to decide whether we can expect to find anything like the dicentic duality ofverbal argument-predicate structures in such pictorial messages and this brings us to the secondargument against the assumption of the dicentic structure of pictures the argument of non-

segmentability This argument is nicely developed in a paper by Jerry A Fodor entitled ImagisticRepresentation Fodor (1981 64-66) considers the possibility of a language called for the sake ofargument Iconic English in which pictures might take the role that words play in a natural languageHe concludes that no such pictorial language could exist because the linearization of arguments andpredicates would prevent such pictorial words from being interpreted as a propositional whole Fodorsexample is Suppose that in Iconic English the word John is replaced by a picture of John and theword green is replaced by a green patch Then the sentence John is green comes out as (say) apicture of John followed by a green picture But that doesnt look like Johns being green it doesntmuch look like anything

In his search for a pictorial equivalent to verbal propositions Fodor commits the error of projecting thelinearity of verbal language onto the visual domain where simultaneity is the structural principlerelating the rhematic elements in question Against Fodors logocentric bias we have to raise the

question why the mere picture of green-skinned John should not suffice to derive the holistic pictorialpropositional message John is green Would not the photo of our green John testify to his unusualcolour in an even much more convincing way than the verbal statement John is green We claimthat the argument John and the predicate is green must thus be sought in pictorial simultaneity andnot in contiguity or if the linguistic analogy is preferred the visual predicate is suprasegmental to thesegmental visual argument

The thesis of such a propositional structure in pictorial messages has actually been suggested earliernamely in the semiotic theory of codes of the 1960s when the search for analogies between verbaland nonverbal messages was on the agenda of semiotic studies Eco (1968 236) eg followingPrieto (1966) argues that pictures always have a propositional structure since even the roughestsilhouette of a horse does not correspond to the verbal sign horse but to a series of possiblepropositions of the type standing horse in profile the horse has four legs this is a horse etc This

early idea of a propositional structure in pictures however was not pursued very systematically sincethe discussion at the time was focused too much on the search for visual equivalents to the structural

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 26: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2629

2629

dividing line between the levels of first and second articulation in language ie to words andphonemes

Nowadays in the era of cognitive approaches to pictorial perception since new evidence for theinterrelation between visual cognition and propositional coding in our mental representation of pictureshas been found (cf eg Jorna 1990) the topic deserves to be resumed and further explored

After concluding that pictures do have the potential of fulfilling the criterion of propositional structurelet us turn to the third syntactic argument against the possibility of assigning truth values to picturesthe argument of dicentic vagueness This argument claims that pictorial messages are so ambiguousvague and polysemous that they cannot serve to prove any truth or falseness Both Gombrich andFodor have defended this point of view

As far as ambiguity is concerned Wittgenstein(1953 140b) is quoted as a witness who onceremarked that a man walking up a hill forward corresponds equally and in the same way to a mansliding down the hill backward This may well be so but there is ambiguity in language too whichcannot testify against the truth potential of sentences either Even the classical example of anunambiguous sentence The cat is on the mat may have an ambiguity to it since being on the mat isa slang metaphor which can make the sentence mean The cat is in trouble Furthermore the samepicture of Wittgensteins man which may be ambiguous in one respect may well convey other truthsabout this man eg facts about his face figure clothing or age

Gombrich develops two arguments to prove that pictures cannot convey anything like a statement andhence no true or false messages The first argument is that pictures are vague while sentences arenot Gombrich (1972 82) explains

The sentence from the prime The cat sits on the mat is certainly not abstract but although theprimer may show a picture of a cat sitting on a mat a moments reflection will show that the picture isnot the equivalent of the statement We cannot express pictorially whether we mean the cat (anindividual) or a cat (a member of a class)This argument is clearly logocentric It does not ask whether pictures can convey statements but asks

whether it can convey the same statement as a given sentence The answer would be different if thepicture were the point of departure in the comparison with verbal statements A particular photographof a cat on a mat being an indexical sign is certainly in the first place about an individual cat and notabout a member of a class Furthermore the sentence The cat is on the mat is in many respects muchvaguer than a photo While the hearer of the sentence has to rely on many supplementary pieces ofknowledge in order to ascertain the truth value of the verbal statement - eg which cat or which mat- the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth ofthis pictorial statement The individuality of the cat and the mat an be easily identified in many details

The logocentric bias behind Gombrichs argument is even clearer when he continues to discusspictorial polysemy as a reason to contest the assertive potential of pictures In his view although thesentence may be one possible description of the picture there are an infinite number of other truedescriptive statements you could make such as There is a cat seen from behind or for that matterThere is no elephant on the mat (Gombrich 1972 82)

Fodor (1981 66-67) derives the same argument from a different example

Suppose that the picture that corresponds to John is fat is a picture of John with a bulging tummyBut then what picture are we going to assign to John is tall The same picture If so therepresentational system does not distinguish the thought that John is tall from the thought that John isfat () The trouble is precisely that icons are insufficiently abstract to be the vehicles of truthAgainst Gombrichs and Fodors view that pictorial polysemy prevents pictures from being vehicles oftruth I would like to argue that a message which conveys a plurality of facts about the world must nottherefore be less true than a message that conveys only a single true statement Neither polysemy norambiguity can thus be accepted as general arguments against the truth potential of pictures

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 27: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2729

2729

Incidentally the degree of pictorial polysemy and ambiguity tends to be greatly overestimated Justlike the polysemy of language the plurality of pictorial meanings is restricted by contextual cotextualand cultural knowledge It is therefore absurd to conclude as Fodor (1981 68) does that the pictureof fat John corresponds equally to Johns being pregnant since if that is the way that John does lookwhen he is fat it is also I suppose the way that he would look if he were pregnant

Turning to the pragmatic aspect of pictorial truth we come back to the question whether pictures canassert at all This question has to be considered because only those false messages count as a liewhich are expressed in the assertive modality (cf Kjerup 1974 1978 Eaton1980 Korsmeyer 1985)Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goalto make the addressee believe in the truth of his or her proposition Nothing can be judged as true offalse if it is only expressed in the modality of possibility fictionality imagination exemplification or as amere question This is why we could see no deceptive intention in the photo of our Man Without Qualities in the manipulated photo of the buses in the soccer stadium or in our distorted portrait Butcan pictures assert at all Is not their function restricted to the mere showing of the real or theimaginary

At this point we have to consider one of the most serious arguments against the assertive potential ofpictures which is the argument of their pragmatic indeterminacy It was Wittgenstein (1953 22) who

developed it with the following example

Imagine a picture representing a boxer in a particular stance Now this picture can be used to tellsomeone how he should stand should hold himself or how he should not hold himself or how aparticular man did stand in such-and-such a place and so on One might (using the language ofchemistry) call this picture a proposition-radicalAccording to Wittgenstein the pragmatic function of pictures is thus open and undetermined (Noticethat Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it apropositional-radical) This idea of pragmatic openness of pictures is one which Peirce ascribed inparticular to the pictorial genre of photography by defining photos as indexical signs Insofar as theyfunction as indices photos are characterized by the criteria which Peirce (CP 3361) specified forindexicality in general namely The index asserts nothing it only says There It takes hold of our

eyes as it were and forcibly directs them to a particular object and there it stops

And yet pictures are used for assertive purposes in situations which exclude other pragmaticfunctions In language the speech act of asserting is effected by means of a proposition whichrepresents an actual state of affairs Police photos and scientific illustrations are equally used torepresent and hence to assert an actual state of affairs Only because they assert and not for anyother pragmatic function can they serve as legal or scientific documents of truth The assertivepotential is even inherent in the genre of photography Only a photo and not a painting of a crime willbe accented as a document of truth in court

Whenever signs can be used for asserting the truth they can also be used to deceive If they assertthey will be used as lies A daily newspaper which publishes a photo in its news section asserts thereality of the scene in question A manipulated photo of an honest politician shown in a scene toasting

notorious gangsters whom he has never seen in reality (cf Worth 1975 100) is thus a photographiclie Because of the documentary nature of this pictorial genre we take the photo as an assertion of thefalse scene as long as we are unaware of the manipulation A painting of the same scene could onlyserve as a lie if accompanied by the statement of a witness testifying to its truth

A final question to be considered is whether the assertive function of pictures can be derived frompictures alone or whether nonpictorial signs are required as indicators of their truth claim The answeris that pictures and sentences in this respect are both alike and different They are alike becausesentences in isolation cannot be judged for their truth either The cat is on the mat is a sentencewhose function may be to assert but it may also be used with a poetic or a metalingual purposebecause it rhymes or because it exemplifies a particular way of using language Thus both verbal andpictorial messages have to be interpreted within their larger context

The difference between verbal and pictorial assertions is that the contextual indicators of an assertionin the medium of language can be expressed in the same medium while those of pictorial messages

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 28: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2829

2829

cannot While we can verbally reinforce the credibility of our claims by illocutionary verbs such as Iassert that I declare that or I swear that and similar metalingual devices pictures have no suchmetasemiotic means of asserting their truth (cf Kjerup 1978 65) unless the inherent assertive force ofphotographs is counted as such a metasemiotic device

Notice however that the explicit contextual use of an illocutionary label of assertion in language is

rather the exception than the rule (cf Wittgenstein 1953 22) and that there are also many contextualindicators of truth or lies which are nonverbal eg the nonverbal reaction of blushing and similarreactions of interest in a lie detector test Other nonlinguistic determinants of the assertive power ofverbal utterances include the credibility of the witness or the situational probability of the truth of theverbal claim

Questioning the truth value of pictures has had a long philosophical tradition A logocentric biasagainst the truth potential of pictures can be found as early as with Plato who wrote Painting is farfrom truth and therefore apparently painting has the effect of reaching only little of everything andthat only in a shadow image (Politeia X 598b) The galactic evolution of pictures from Platos shadowimages to documentary and computer manipulated photographs has made a reconsideration of thetopic necessary Semiotics although not immune against logocentrism has provided tools foranalyzing the topic of truth or lie in pictures without the logocentric bias Sebeok (1986) eg has

shown that lying is by no means restricted to verbal semiosis since it can be found with animals andthe semiotic of pictures has made advances in investigating pictures as a system of signs autonomouswith respect of verbal language The result of our study was that pictures can be used to assert or todeceive about facts from the semantic syntactic and with certain reserves also from the pragmaticdimension This does not mean that asserting and lying are very typical modes of pictorial informationMost of the manipulative strategies of pictorial information in the media are not direct falsifications ofreality expressed in the assertive mood but manipulations by means of a plurality of indirect modes ofconveying meanings

References

Barthes Roland Camera lucida Reflections on photography London Cape (1980) 1982

Bennett John G Depiction and convention In The Monist 58 pp255-268 1974

Buddemeier Heinz Leben in kunstlichen welten Cyberspace Videoclips und das tagliche FernsehenStuttgart Urachhaus 1993

Eaton Marcia Truth in pictures Journal of aesthetics and art criticism 3915-26 1980

Eco Umberto (La struttura assente trans) Einfuhrung in die Semiotik Muchen Fink (1968) 1972

--- A theory of semiotics Bloomington Indiana University Press 1976

--- Semiotics and the philosophy of language Bloomington Indiana University Press 1984

Fodor Jerry A Imagistic representation In Ned Block ed Imagery Cambridge MA MIT Press 63-86 1981

Gombrich Ernst H Art and illusion London Phaidon (1960) 1968

---Symbolic images Edinburgh Phaidon (1972) 1975

Jorna Rene J Knowledge representation and symbols in the mind Tubingen Stauffenburg 1990

Kjerup Seren Doing things with pictures In The Monist 2 216-235 1974

--- Pictorial speech acts Erkenntnis 12 55-71 1978

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)

Page 29: Semiotcs

842019 Semiotcs

httpslidepdfcomreaderfullsemiotcs 2929

2929

Korsmeyer Carolyn Pictorial assertion Journal of Aesthetics and Art Criticism 43 257-265 1985

LeBon Gustave Psychologie des foules dt Psychologie der Massen Stuttgart Kroner (1895) 1973

Muckenhaupt Manfred Text and Bild Tugingen Narr 1986

Peirce Charles S Collected papers Vols 1-6 ed Hartshorne Charles and Weiss Paul vols 7-8 edBurks Arthur W Cambridge Mass Harvard University Press 1931-58

Prieto Luis J Messages et signaux Paris Presses Universitaires 1966

Robert Oliver Der Augenschein im Strafprozelss Zurich Schulthess 1974

Searle John R Speech acts Cambridge University Press 1969

Sebeok Thomas A Can animals lie In TA Sebeok I think I am a verb New York Plenum 126-130 1986

Wahner Matthias Mann ohne Eigenschaften - Man without qualities Munchen Stadtmuseum 1994

Wittgenstein Ludwig Philosophische Untersuchungen - Philosophical Investigations OxfordBlackwell 1953

Worth Sol Pictures cant say aint Versus 1285-108 1975

Winfried Soth is Professor of English Linguistics at the University of Kassel Germany He is the authorof several books and numerous articles in linguistics and the semiotics of art literature and the mediaHis Handbook of Semiotics appeared in English in 1990 (Indiana University Press)