Imagining the human brain – audiovisual metaphors in a science television program

Created by Jasper Stratil, last modified by Janine Leona Schleicher on 21.09.2021

Sarah Greifenstein & Regina Brückner

 


Introduction

The various ways we think about the human brain and cognition are not only a matter that concerns the neurosciences, psychology or medicine. Over the last years, neuroscience has entered popular culture in a vast spectrum of media formats, science television programs and documentaries. As experimental settings and scholarly thought are complex and discipline-specific, delivering information to a lay audiences draws heavily on the use of metaphors, both language-based and multimodal ones. In this case study, we will look at one example to show how the metaphors elaborated in these programs shape a popular knowledge about how the human mind works, through their audiovisual staging.


Fig. 1
Fig. 1


In the last years scholarly work on metaphors in audiovisual media has gained weight especially in the field of conceptual metaphor theory. But instead of looking at already existing metaphors and conceptual schemata, we want to contribute a perspective that focusses on how film images - by means of their temporal unfolding - shape the figurative thinking of their spectators ad hoc in the situated process of watching a film. For such a dynamic perspective on audiovisual metaphors we build on Hermann Kappelhoff's and Cornelia Müller's approach (Kappelhoff / Müller 2011). We will analyse how audiovisuals aesthetically address spectators, how audiovisual metaphors emerge and are dynamically activated within a situated and embodied experience (Kappelhoff / Greifenstein 2017). As Cornelia Müller argues regarding multimodal metaphors:


“the expression of ‘activation of metaphoricity’ implies that the product metaphor is always a result of the procedure for establishing metaphoricity.” (Müller 2008)



By building on this dynamic view we want to reconstruct the process of establishing metaphoricity, how metaphors are activated, foregrounded, highligthed and made salient in the process of meaning making. In the context of film and media perception this process has to do with the act of experiencing aesthetic forms. Or as Mark Johnson says: 


"In the visual arts, it is images, patterns, qualities, colors, and perceptual rhythms that are the principal bearers of meaning. The obvious fact that we usually cannot put into words, what we have experienced in our encouter with an artwork does not make the embodied, perceptual meaning any less a type of meaning." (Mark Johnson 2007: 234)


This perspective will guide our analysis of a popular science television program about the brain. Our case study is the first part of a 2-part German television program called: „Das automatische Gehirn“ - the automatic brain - with the subtitle: „Magie des Unbewussten“ - the magic of the unconscious. The title already calls up a familiar metaphor - the brain as a machine. We will look at how this metaphor unfolds in time and how it becomes embodied in a very specific manner.


Fig. 2: DAS AUTOMATISCHE GEHIRN, D 2011, Directors: F. D’Amicis, P. Höfer, F. Röckenhaus.
Fig. 2: DAS AUTOMATISCHE GEHIRN, D 2011, Directors: F. D’Amicis, P. Höfer, F. Röckenhaus.


The Brain as a Machine


It is important to remember at this point that this main metaphor is not thought of as occurring in one or more isolated instances, but structures our experience as spectators. It is a dynamic metaphor, a metaphor over time, that emerges from specific relationships between moving images, sound and language. The metaphor is something that is staged in a very elaborate and artistic way and is composed of auditive, visual and verbal elements. The main metaphor in this film conceptualizes brain function as an automatically working machine. During the whole program the terms „brain“ and „unconscious“ are used synonymously, and we will follow this usage.



the brain as a machine

If we look at the timeline of the whole documentary film, we can see that verbal utterances relating to the metaphor, represented by the blue stripes, are distributed over the entire 50 minutes. For example the voice-over commentator says: "Every morning, when we open our eyes, unconscious circuits create the world that surrounds us in our inner eye." Other examples are: „our brain automatically selects what we perceive“ or: „there is a fully automated error monitoring system in our heads.“ 


Fig. 3
Fig. 3

If we match the occurrence of verbal metaphors onto instances where the brain is visualised and animated, we can see that their frequency is highly increased in these instances. Interestingly, here the brain image is the more abstract domain whereas language provides the imagery that makes us understand what we see.


Fig. 4
Fig. 4
 

Visually and acoustically, the metaphoric realm of machine is not illustrated in an obvious manner, but more subtly cued. For example, sounds of electricity are staged in conjunction with nerve cells. The colours used to represent the brain and nerves are the same colors that are used in science fiction films and more generally with regard to computers.

Clip 1: DAS AUTOMATISCHE GEHIRN (F. D’Amicis/P. Höfer/F. Röckenhaus, D 2011), Min. 0:00-0:13.

Not only sound and colours but also camera work contributes to the metaphor, even in scenes where the brain is not shown. Here we see a fast, abrupt and unnatural motion, like movement patterns of a robot.

Clip 2: DAS AUTOMATISCHE GEHIRN (F. D’Amicis/P. Höfer/F. Röckenhaus, D 2011), Min. 29:06-29:09.

Detailed Analysis


After this short overview, we will now go into a detailed film analysis of one scene to show the dynamics of the metaphor.


Building on Kappelhoff and Müller, we presume that a temporally developing audiovisual metaphor acts on the actual experience of viewers, so what we bodily and perceptively go through can become the source and target domain of the metaphor.  What unfolds over time is the mapping of certain aspects of the metaphoric field that become perceptible in the flow of the images. 


The scene we're going to analyse is concerned with unconscious processes that provoke us to actions essential to our lives. The climax of the scene is rescuing a boy from an approaching car.


The scene, which is 4 minutes long, consists of four parts:

  1. introduction of the general idea
  2. sequence of everyday situations
  3. an interview with a scientist
  4. a dramatic situation and the brain processes going along with it 


Fig. 5
Fig. 5

 
Clip 3: DAS AUTOMATISCHE GEHIRN (F. D’Amicis/P. Höfer/F. Röckenhaus, D 2011), Min 17:46-21:45.

The idea that the brain manages essential processes without us being conscious of them, is first introduced by the statement that “a lot of the things our brain attends to without us, are good for us”. This personification attributes agency to the brain and therefore highlights this aspect in the metaphor. Another personification metaphor appears in the next part: „Our unconscious tells us whether we like coffee or tea before we think about it for long.“


When Marta, the character we follow throughout the program, is going about her morning routine, having coffee, washing strawberries, eating and driving to work, a number of verbal metaphors belonging to the realm of automatic machines, and computers in particular, accompany these actions.


For example: „In split seconds it analyses enormous amounts of data far more complex than anything we think about consciously.“ Or: „no problem for unconscious circuits.“ And then: „our autopilot recognizes them far more reliably.“ Finally: „With new dangers, our brain switches to thinking.“


And although there are no explicit visualisations of these metaphors, there are a number of visual objects that - together with the spoken words - help activate the metaphor multimodally.


Fig. 6: Verbal Expressions
Fig. 6: Verbal Expressions

In the audiovisual staging, close-ups highlight different machines and their operation, like pressing the button on a coffee machine, using a smartphone, or operating a car. 


Fig. 7
Fig. 7

Taken by themselves, these would hardly qualify as elements of a metaphor but through their repetition, how long and prominently they are shown and especially the close temporal co-occurence with the verbal metaphors, they foreground the realm of machinery and automatism. So this part establishes the metaphor on a verbal level together with subtle visual cues.


Fig. 8
Fig. 8

In the third and and fourth part of the scene, the source domain can be experienced through different stagings of time.

An aspect of the „brain as a machine“ - metaphor is the proclaimed high speed by which it computes data.
The aspect of the brain‘s temporality now becomes central through a scientist explaining how our conscious mind lags behind our unconscious mind.

The described temporality becomes a real temporal experience for the viewer through the use of slow and fast motion.

We see slow motion shots of a young boy, running along with a football followed by shots of cars and a shot of Marta leisurely walking along the pavement, then, in a medium long shot, the ball jumps past Marta toward us.
The motion of the image now becomes extremely slow when the boy appears behind the door. Eery, equally slow and elongated sounds make the movement audible and create an extremely dramatic atmosphere.
A brief change of speed, from slow to normal to slow motion, highlights the different temporalities.
This orchestration lets us experience that this is in fact just a split second, a standstill of time, a number of pieces of information too quick to be processed consciously.
Clip 4: DAS AUTOMATISCHE GEHIRN (F. D’Amicis/P. Höfer/F. Röckenhaus, D 2011), Min 19:42-20:09.

Now we jump to an extreme close-up of an eye, accompanied by the voice-over: “everything we see enters our pupils as a bundle of rays of light”. The moment he says “enter”, the forward movement of the camera is experienced as entering the eye, as if we ourselves were those rays of light. A change of perspective now shows us a side view of an abstract image of an eye while the commentator explains how the light hits the retina.

Fig. 9
Fig. 9

Accompanied by a space-ship sound, we enter the eyeball in a very dynamic movement. Leaving the still legible image of the eye into animations of the brain, the commentator says: “coded in a million nerve impulses, the data rush through the optic nerve.”
The camera moves further forward into the image of the boy, behind the retina and quickly through a tunnel of strands in a greenish-blue colour, resembling cables.

The fast movement that we experience audiovisually corresponds to the verbal mentioning of data rushing. Through this mapping of the experienced speed and the verbally mentioned speed of the data, the multimodal metaphor is gaining more shape.

This extreme rush is contrasted with the slow motion that we have experienced before, and makes the machine-like velocity perceptible. In this moment, we experience the source, the machine-like velocity, and the target, the brain, simultaneously:

Fig. 10
Fig. 10

We see images of the brain and experience the rush of velocity, we hear a verbal metaphor and are at the same time put into the perspective of the rushing data.

So with this multimodal articulation, the aspect of speed is induced into the overarching metaphor.


Fig. 11
Fig. 11

Within the scene, we have now entered an open, dark space, with semi-transparent, disorderly strands of nerves that seem to spark from electricity. The commentator describes parts of the brain as elements of the machine, each having a specific function.


The thalamus as the gatekeeper who communicates with other parts such as the amygdala, the so called „panic switch in our heads”

We now see an animated transparant image of a head from outside, in which the relevant parts of the brain are illuminated.

The transmission of data is visualized by small light dots moving along paths. When the “panic switch” is mentioned, a bright red flickering light appears in one part of the brain while the camera moves closer.

The red flickering light corresponds to an alarm signal that we are familiar with in western culture. The idea, that the amygdala becomes activated and thus blinks, matches with the verbally mentioned „panic switch“.

In this part of the scene the machine‘s consisting of elements is staged and their activities and functions are visualised as light.


Clip 5: DAS AUTOMATISCHE GEHIRN (F. D’Amicis/P. Höfer/F. Röckenhaus, D 2011), Min 20:37-20:48.

In the final set of shots, three instances belonging to the field of data computing and electricity are verbalised and accompanied by an audiovisual staging that highlight aspects of electricity, light and signals.


Fig. 12
Fig. 12

What the analysis shows, is how a metaphor emerges that is constituted by verbal expressions as well as the audiovisual staging. Combining the elements described: agency, automatism, speed, consisting of elements, and electricity – we would suggest the following specification of the main metaphor:


The brain and the unconscious are an agent. This agent is a fast machine. This machine consists of circuits, data, databases, switches, modules, light dots and electronic activites. The elements of the machine analyse, control, light up, and signal.


This metaphor dynamically creates a vivid scenario, that we experience and understand as spectators. Only the complex interaction of verbal, visual and auditive elements shapes the process of metaphoric meaning making.


We went through the speed perceptively and bodily while at the same time comprehending the speed as being part of the represented and metaphoric. The speed I experience becomes the speed of what I understand to be the machine‘s speed.


Fig. 13
Fig. 13


What we hope to have shown is how complex the audiovisual orchestration of such a metaphor is. This complexity structures an imagination of the human brain that is not only cognitively present but also felt bodily and affectively.


This imagination is a specific construct with consequences for our understanding of ourselves, it is an imagination that is culturally and historically specific and perpetrated by different media like the science TV program we have just shown.


We will leave the question of how scientifically accurate such a media presentation and use of metaphors is to experts in that field. What we can say is that the spectacle of the visually compelling way by which images reminiscent of advertising and film genres engage us with all our senses, often seems to become more important than the actual scientific argumentation.




Bibliography


Cameron, Lynne: Metaphor and reconciliation. The discourse dynamics of empathy in post-conflict conversations. New York 2011.

Fahlenbrach, Kathrin: Audiovisuelle Metaphern. Zur Körper- und Affektästhetik in Film und Fernsehen. Marburg 2010.

Goschler, Juliana: Metaphern für das Gehirn. Eine kognitiv-linguistische Untersuchung. Berlin 2008.

Johnson, Mark: The meaning of the body. Aesthetics of human understanding. Chicago 2007.

Kappelhoff, Hermann / Greifenstein, Sarah: Metaphorische Interaktion und empathische Verkörperung: Thesen zum filmischen Erfahrungsmodus. In: Malte Hagener / Ingrid Vendrell Ferran (eds.): Empathie im Film. Perspektiven der Ästhetischen Theorie, Phänomenologie und Analytischen Philosophie. Bielefeld 2017.

Kappelhoff, Hermann / Müller, Cornelia: Embodied meaning construction. Multimodal metaphor and expressive movement in speech, gesture, and in feature film. In: Metaphor and the Social World 1/2 (2011), 121-153.

Müller, Cornelia: Metaphors dead and alive, sleeping and waking. A dynamic view. Chicago 2008.

Schmitt, Christina / Greifenstein, Sarah / Kappelhoff, Hermann: Expressive movement and metaphoric meaning making in audio-visual media. In: Cornelia Müller / Alan Cienki / Ellen Fricke / Silvia H. Ladewig / David McNeill / Jana Bressem (eds.): Body _ Language _ Communication: An International Handbook on Multimodality in Human Interaction. (Handbooks of Linguistics and Communication Science 38.2). Berlin / Boston 2014, 2080-2092.


Filmography

DAS AUTOMATISCHE GEHIRN. F. D’Amicis/ P. Höfer / F. Röckenhaus, D 2011.


DOI: http://dx.doi.org/10.17169/mae.2017.77

Attachments:

das automatische gehirn_vier teile.png (image/png)
das automatische gehirn_timeline_verbal_gehirn visual.png (image/png)
das automatische gehirn_timeline_verbal metaphors.png (image/png)
advene_das_automatische_gehirn_maschine.png (image/png)