PDF_How do pictorial representations refer …

Reading right here:

How do pictorial representations refer to theoretical contents?

Abstract

Using a practice-based methodology this paper explores one of the semiotic fundamentals of many works of design: the semiotic nature of meaningful pictorial references to theoretical content. To do so it classifies meaning-related pictorial references as either pictorial literality or pictorial metonymy or pictorial metaphor and analyses combinations of these as examples of meaningful reference. Pictorial metaphors, especially those that can be traced back to common linguistic metaphors, play a special role in this process.

Keywords: referencing meaning through pictorial representation, interpretability, pictorial literality, pictorial metonymy, pictorial metaphor, combinations of references

1         Introduction

The main question of this paper is: in what cases do observers construct meaningful references between images and abstract linguistic contents. How do pictorial representations relate to and comment on theoretical linguistic statements? These are complex semiotic questions that are of fundamental importance to many design and art works that are based on the relationship between language and pictorial information. These questions are particularly relevant to works of design that try to construct pictorial metaphors. Exploring these questions helps to understand the semiotic basis on which many design and art projects attain their intelligibility and charm.

This paper is based on my PhD dissertation (completed at the University of Portsmouth in cooperation with the Merz Akademie). It explored the possible relations between theoretical text and pictorial representations. It employed a combination of theoretical approach and research by practice. This research by practice consisted in transposing short passages of original philosophical text (via different forms of analogies) into animated film and exploring the interaction of text and film.

2         Artificial presence and reference

In my work, I analyse relations between two fundamentally different formats: text and image. In this paper I will concentrate on images that are primarily representations of spatial situations and contain no common symbols like words or arrows etc.[1] In the following, therefore, the term “image” denotes a pictorial representation of a spatial situation that is as free of symbols as possible.

Following Lambert Wiesing (2005), we can describe the image as providing the illusion of an artificial presence of spatial objects and situations. Images form their content with the help of these artificial objects, without explicitly referring to anything beyond the frame of the image (unless they make use of symbols). Thus the content of images is always physical. Text in contrast is a system of references, it is symbolic and deictic throughout. Words are symbols that permit a deictic reference to something. Text is thus always reference and description without ever being able to present a presence of its contents.[2] While text normally refers to things and concepts outside itself, images do not necessarily do so. Language can deal with all levels of abstraction. If we take the idea of the artificial presence seriously we could say that the only possibility of an image (without symbols) to refer to linguistic contents is to embody or enact aspects of these contents as artificial presence or as artificial movement.

3         Literality, metonymy, and metaphor

In language the relation between a word and what it denotes and thus refers to is either literal or metonymical or metaphorical, (if we accept the definitions of these terms given below). Literality, metonymy, and metaphor arise when words as symbols are applied to something. These terms represent different bilateral relations between a symbol and its current meaning. An image must also produce a particular reference in order to relate to a text. After looking closely at many examples of relations between images and texts, I conclude that a meaningful relation between a part or aspect of an image and text content can also only considered meaningful, if it is constructed as literal, metonymic, or metaphorical reference.[3]

I developed my view on literality, metonymy and metaphor in relation to Eleanor Roschs idea of categories and prototypes (Rosch, 1978). Rosch describes the word as such (in a non applied state) as a category that is defined by one or more prototypes. In an actual utilisation of a word, this prototype is employed in a literal, metaphorical and metonymical way. Here we have a fundamental difference between linguistic and pictorial references, since the image (in the form I am focussing on here) offers no symbols. Thus while images do not provide prototypes, they do provide aspects of artificial objects or movement that can be related in a literal, metonymical, or metaphorical way.

3.1        Literal reference

Literality in language means that a word stands for a content that fulfils all characteristics of the prototype of this word. The literal meaning is defined by a bundle of attributes that together must be fulfilled in order for something to be judged viable as a literal denotation of this word. Nearly all words have several prototypes, and therefore several literal meanings.

What would it mean to say that some artificial object or aspect of an image can be understood as standing in a literal relation to a textual meaning? It means that this object or aspect fulfils all the characteristics of the meaning of the text in question. An image of blooming chestnut trees is a literal reference to a sentence like: «The chestnut trees were blooming».

Pictorial literality is a common form in illustrations and film adaptations. It gives the content a singular gestalt and thereby defines many aspects that might not been mentioned in the text: the actual form of the objects, its colours, its dimensions, what is in the background, etc. Pictorial literality inevitably produces a surplus of details.

In my context it is important to consider that pictorial literality can hardly be used to refer to theoretical texts. Since images can show only spatial situations, objects, and movements within a physical world, it can only refer to pictorially representable contents. Any abstract, theoretical content cannot be literally referred to by an image.

3.2        Metonymic reference

Linguistic theorists describe metonymy as concerning contiguities (e.g.: Jakobson, 2002). Combining this idea with the idea of the prototype we can say that in the case of a metonymical use of a word, some aspect of the ideal-typical contiguity of the prototype of this word is denoted. In metonymy, a shift takes place; the word means the proximate, the bordering, or the broadening of the prototype. This contiguity corresponds to the environment («the room applauded») or typical usage of the material («a glass of water»), pars-pro-toto constructions («headhunter») or totum-pro-parte constructions («light the Christmas tree») etc.

Metonymy allows for clear denotations without having the necessity for the mot juste. It offers precise references, without itself having to be precise. Since this function makes quick formulation and understanding possible, metonymy is very widespread.

Metonymical reference is often accompanied by a metaphorical reference. The expression «the room applauded» for example is not only metonymical but can also be understood as a metaphorical reference to the all-embracing nature of the applause.

Pictorial metonymy means that an image refers to a linguistic content by showing contiguities of this content. Of course images can do this. Often, however, observers do not comprehend the metonymical connections between image and text because the artificial presence of pictorial representations emphasises the difference between the intended content and the image that depicts something contiguous to it, in other words, it depicts something else.

There is often an additional problem when trying to refer to theoretical content metonymically: the contiguities of theoretical contents are usually also theoretical and are therefore not easier to represent visually then the literal meaning.

3.3        Metaphorical reference

Combining Rosch’s ideas about categories and Gorge Lakoff and Mark Johnson’s (2004) ideas about metaphor, we can construct the following concept of the metaphor: in the metaphorical usage of a word, certain conceptual parts of its prototype are used separately from other conceptual parts. A bundle of characteristics of the prototype are abstracted from others; these conceptual contents are then carried over onto other contents that are greatly distanced from the prototype. Important components of the prototype, however, have been subtracted, so that the word can then be used in a completely different context.

All forms of referencing imply some classification by naming something with a particular word. The function of metaphor, however, is explicitly to interpret and comment on the content that is denoted by the metaphorical expression.

In a pictorial metaphor some aspect of the image combines a bundle of characteristics that can be projected onto the linguistic content. Metaphorical pictorial reference is the only pictorial reference able to refer to, interpret, and comment on theoretical contents.

4         Combinations of references and interpretability

The three figures below are stills from three short film sequences that refer to sentences by philosophers. When shown simultaneously with an audio version of these texts, observers described all three as standing in a meaningful relation to the text.[4] These sequences, together with the text can therefore serve as examples of a meaningful reference between image and theoretical text.

Figure 1 What cannot be understood through something else, must be understood through itself. (Spinoza, 1999, p. 48, translation: V.R.) For the Figures please see the PDF.

The films demonstrate that an image never consists in only one reference to a text. Instead an image is normally a mixture of parts that refer to the text and parts that do not. It can also combine literal, metonymical, and metaphorical references. In Figure 1, for example, viewers understood the mode of contact between the objects to be a metaphorical reference to the idea of «cannot be understood» and «must be understood.» The form of the objects, however, were not seen as metaphorical reference, but were rather assumed to be freely invented.

Since all information represented in an image gets a singular gestalt with details and a visual environment, we could argue that a literal or metaphorical reference always has to include “metonymic” parts that consist precisely in this surplus of details. Looking at all my experiments, it seems that one of the central qualities of visual metaphors, is that they have to include metonymical enhancements and thus provide very interesting possibilities for expanding metaphorical references through metonymical extensions of the metaphor. Visually defined details and contiguous aspects of a metaphor provide new extensions of a metaphor that can be interpreted again in a metaphorical way, despite the fact that they originally derived from a metonymical process.

In Figure 1 the contact between the visual objects was interpreted as metaphorical reference to «cannot be understood» and «must be understood» along the linguistically familiar metaphor «to understand = to grasp, to touch, to contact». In this case the depiction of this contact as aggressive in the first instance and as gentle in the second serves as an interpretation and commentary on the idea of «understanding». The reference can be delineated as a two-step relation: first a linguistic metaphor is found and then this linguistic metaphor is embodied to in a literal way.

Figure 2 An animal does not feel essentially different than an angel. (Perniola, 1999, p. 63, translation: V.R.) For the Figures please see the PDF!

Figure 3 Immeasurability is one of the dynamic features of the calm reverie. (Bachelard, 1994, p. 187, translation: V.R.) For the Figures please see the PDF!

In the case of Figure 2, observers judged the darker object with the contours as reference to «animal» and the lighter and transparent one as reference to «angel». The connection consists initially in a simplification of the body; in this way it can be understood as a metonymical reference similar to a pars-pro-toto construction. The meaning of the sequence, however, arises from a metaphorical interpretation that sees the depicted characteristics as the most important ones and interprets their joint characteristics as a metaphorical reference to the joint characteristics of angel and animal and the difference in their characteristics as a metaphorical reference to the main differences between «angels» and «animals».

Figure 3 was also experienced as somehow meaningful by most of the observers, but all of them had great difficulty naming an actual meaning. The reason, in my opinion, is that there is no known linguistic metaphor that would assist the interpretation. Nevertheless, viewers suspected the connection in the image to be somehow metaphorically structured. We could hypothetically assume that this film provides a totally new metaphor that is not based on a familiar linguistic metaphor.

In Figures 1-3 the actual reference to the meaning of the text can be pinpointed in a metaphorical reference. This was also true for almost all experiments conducted in the course of my dissertation. This is not very surprising, since literal reference is not possible and metonymical reference normally serves as a way to speak vaguely without being imprecise. If an image acquires a meaning as interpretation or commentary on the text it does so through metaphorical references. In fact all pictorial examples in my dissertation for which people could come up with a clear interpretation, were not only based on a metaphorical relation but rather on a metaphorical relation that could be traced back to common linguistic metaphors.[5] Without such a given metaphor no clear interpretation could be made. This led me to the conclusion that we rely heavily on language in the construction of metaphorical meaning in images.

5         Conclusion

The ideas developed above, apply to all references between designs (graphic and spatial) and abstract contents such as for example reliability, complexity, dynamics or inflation etc[6]. All designs can at some level be understood as spatial situations or as representations of spatial situations. Abstract terms can only be named and understood through language.

My investigations show, that design can refer to abstract terms and values only through a literal, metonymical or metaphorical relation, or through combinations of these. For example a design of a vacuum cleaner might resemble to a little dog. This is a (more or less) literal reference to cuteness and a metaphorical reference to qualities usually associated with dogs like loyalty, and it is a metonymical reference to a happy home implying contents like safety and family ties. A design showing a simplified ascending line on the cover of an annual report, can be understood as a (common) metaphorical reference to a positive dynamic. Its wine-red colour is a reference to other designs using this colour, typically in the context of royal affairs, expensive hotels and fairy tale books. Therefore the colour is a metonymical reference to the contents of all the contexts.

References

Bachelard, G., 1994. Poetik des Raumes [The poetics of space]. Frankfurt am Main: Fischer.

Jakobson, R., 2002. The metaphoric and metonymic poles. In R. Dirven, & R. Pörings (Eds.), Metaphor and metonymy in comparison and contrast. Berlin: Mouton de Gruyter.

Johnson, M. 1987. The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: University of Chicago Press.

Lakoff, G., & Johnson, M., 2004. Leben in Metaphern: Konstruktion und Gebrach von Sprachbildern [Metaphors we live by]. Heidelberg: Carl-Auer.

Peirce, C., 1998. Phänomen und Logik der Zeichen [Logic of signs]. Frankfurt am Main: Suhrkamp.

Perniola, M, 1999. Der Sexappeal des Anorganischen [Sex appeal of the inorganic]. Wien: Turia und Kant.

Rosch, E., 1978. Principles of categorization. In E. Rosch, & B. Lloyd (Eds.), Cognition and Categorization. Hillsdale: Lawrence Erlbaum Associates, pp. 27–48.

Spinoza, B., 2007. Ethik [Ethics]. Wiesbaden: Marixverlag.

Wiesing, L., 2005. Artifizielle Präsenz: Studien zur Philosophie des Bildes [Artificial presence: Studies on a philosophy of the image]. Frankfurt Main: Suhrkamp.

Footnotes:

[1] The term symbol is used here in the way Charles Sanders Peirce (1998, pp. 64–67) defines it.

[2] There is of course a presence of the texts own implementation as a script or voice, but no presence of the contents to which it refers.

[3] It must be stated, however, that such references can only be identified within a formal closeness that strongly suggests there to be some connection between an image and a particular part of a text.

[4] I evaluated the films with a group of fifty people including a group of philosophers, and a group of artists. I conducted interviews and passed out questionnaires, but did not perform a quantitative statistical investigation of the results.

[5] This can also be observed in graphs and diagrams, which usually employ pictorial metaphors that are based on known linguistic metaphors such as «up is much» (cf. Lakoff and Johnson, 2004, and Johnson, 1987).

[6] It is interesting to note that many abstract contents can be understood at the same time as emotional values. This might seem surprising at the first instant, because we are used to understanding abstraction in opposition or at least at a great distance to emotions.