Images in (Con)Text. Intermedial and Intersemiotic Paradigms of Representation in the Old Media

Abstract

The rise and constant development of new media have made us more aware of the overwhelming presence of images. This is a striking characteristic of the era that W. J. T. Mitchell referred to as the Pictorial Turn and Gottfried Boehme as Ikonische Wende. Yet the questions of meaning creation and interpretation across media, modalities, and sign systems have not been fully studied and explained in the domain of the old media. The theorisation of mixed media has contributed to reviving scholarly debates on the image-word relation, demonstrating the ideologically loaded discourses that underlie conceptual oppositions such as nature/convention, space/time, iconic/symbolic, epistemic/rhetoric. In light of theoretical reflections on visuality proposed by scholars across different disciplines, this study sets out to frame intermediality and multimodality within the word-image paradigms. In doing so, it seeks to demonstrate that a ‘dialectic of the dialogic reason’, as intermedial and multimodal frameworks entail, is more useful for critical analysis than a dialectic of stark oppositions. Indeed, an either/or logic underlies the age-old word-image paragone itself: the very rhetoric, in other words, of the epistemic turns (such as the Pictorial Turn), and the ontological monism that governs Western traditions. In particular, this study examines the word-image relation from four different perspectives: 1) as a dialectics built on hegemonic discourses and specific regimes of representation. 2) as situated between media and semiotic modalities in the old media and thus as essentially intermedial and multimodal; 3) as a translation process; 4) as a semiotic relation that requires complementary notions – such as Bakhtin’s dialogue, the hyphos, the rhizome – to be appreciated in full; These perspectives intertwine and complement each other, illuminating the complexity of the ways in which words and images interact: an interaction that cannot be reduced to total correspondence (ut pictura poësis) or total disagreement (paragone). Methodologically, this argument is developed in three stages. First, I analyse intermediality and multimodality from the perspective of media studies, using the semiotic theories of Charles S. Peirce and others. Second, I demonstrate that intermedial and multimodal creation is in its essence a logically abductive process, as formulated by Peirce. This process illustrates how intermedial and multimodal creation are fundamentally rhizomatic, to revisit Deleuze and Guattari’s definition. Third, I examine the intersemiotic recoding from one sign system to another as an act of translation. Since the dialectic unfolding in any act of translation is fundamentally dialogic and open to alterity, it follows that intermediality and multimodality play a pivotal role in disrupting the parameters of ontological monism that scholars have traditionally used to evaluate the word-image relation. Finally, this study provides a deeper as well as a wider insight into word-image debates by proposing additional conceptual frameworks for and definitions of multimodality, intermediality and mediality of culture. Indeed, the interconnectedness of all semiospheres – as particularly emphasised by the notion of mediality of culture – means that the image-word relation must be examined both within and outside the traditional literary field of analysis. In view of this, I have integrated definitions from literary criticism, Visual Culture Studies, media theory, theological and postmodern semiotics, art history and aesthetic theory. The three exempla examined illustrate the problem of ontological monism in the word-image relation, advocating for a dialogic approach to the essentially rhizomatic processes of intermedial and multimodal translation. In Chapter 1, I introduce the paragone between the visual and the verbal as interpreted in postmodern literary works, which allow me to articulate the slow transition from the Linguistic Turn to the Pictorial Turn. Antonio Tabucchi’s Il gioco del rovescio and Daniele Del Giudice’s Nel museo di Reims are quintessential examples of the ambiguous translation of the visual into the verbal, which underlies the epistemology dismantling the ut pictura poësis model of representation. The irreducibility of the word-image relation to paradigms of correspondence and the distance between source and target text do not compromise the possibility of dialogue and translation: mimetic translation is still achievable. In Chapter 2, I focus on how the poetics of representation of Antonio Tabucchi and Fra Angelico interact and intersect in the creative meaning making space I refer to as ‘intermedial difference’. In Chapter 3, I develop the notion of intermedial difference further through the study of the artist as a double (or multi-) talent, and the peculiar convergence of genres in the form of the ex voto. The intermedial encounter between the eclectic and innovative Dino Buzzati and the avant-gardist Yves Klein illuminates two aspects. First, the role of abductive creation; second, the need for a dialogic approach that includes elements of Otherness. In turn, these analyses warrant the reassessment of interpretive categories dominated by the logic of ontological monism and the paradigm of identity in representation. In conclusion, this study of intermediality and multimodality in the old media contributes to shedding light on the word-image relation from an epistemological perspective. More specifically, it widens and deeps the ways in which the dialectic of alterity versus identity can be understood and interpreted within word-image paradigms. Moreover, this interdisciplinary research radically reframes the role of intermedial and multimodal creation as catalyst for the introduction of elements of otherness – which are not only indispensable to open up new ways of meaning creation but also, and above all, to continuously (and creatively) unsettle monist and purist epistemologies – and offers a new, integrated methodological framework to accommodate and celebrate the hybridity of all cultural formations.