Towards a Science of Embodiment
Accepted May 1, 2012
Keywords
embodiment, mirror neurons, gesture, artificial intelligence, cognition, action-homunculus
Abstract
We propose here a science of human body/embodiment. We discuss our motivations, provide scientific and artistic precedents/foreshadowings, and then offer an overall picture of this nascent science, as well as a sample, unsolved problem from the field of disability studies. We imagine a science whose core subject is the connection of the body — as it appears in an action-perception paradigm derived from mirror neurons, the embodied artificial intelligence (AI), and embodied theories of dance, music, and painting — to the cognition of emotions, language, mathematics and logic. We propose to construe the missing link between these two poles, action-body and cognitive stratum, by use of a theory of gestures developed from music theory. We conclude with some suggestions for the foundation of a science of human embodiment and an associated scientific journal.
How we were motivated
When Romina De Novellis, a PhD student in Anthropology, first proposed the foundation of a science of the human body to Guerino Mazzola, the response was one of skepticism. Mazzola was a professional mathematician and jazz pianist who had made seminal contributions to mathematical music theory1, published neurological research on Depth-EEG (e.g., Mazzola et al. 1989), programmed music software2, written handbook articles on music semiotics (Mazzola 1999), and published a computer-aided analysis (Hofmann et al. 1987) of Raffael’s Vatican fresco, The School of Athens. Mazzola nonetheless expressed a concern that despite the current intensity of emphasis on corporeal matters within the humanities, there might not be a corresponding urgency to follow suit in the sciences. We have, in recent decades, seen a number of sciences such as complexity theory, and cybernetics (the latter having experienced a series of rebirths) go through phases of fashionability. Of course, these scientific developments are not bare of substance, but it is unclear whether they deserve to be considered sciences in their own rights rather than subfields of existing sciences, defined by specific discourses within a broader context. Examples include catastrophe theory, a branch of mathematics, and complexity theory, a subfield within system science.
A further concern was that the terms body and embodiment have been used in many contexts, from scientific to artistic, and from hard to social sciences. We were skeptical about our ability to transcend the common understanding of these terms. They are as ephemeral as time which, according to Saint Augustine’s famous dictum, is clear if one does not ask but becomes mysterious when one is queried as to its meaning. At present a science of time seems to us unfeasible. There are not enough common methods or conceptual tools to produce a science of time comprising its physical, psychological and cognitive perspectives.
So why did we eventually agree to consider a science of human body/embodiment? After reviewing the relevant literature we arrived at two principal reasons. First was the abundance of valuable precedential work by philosophers (e.g., Shusterman 1999; Merleau-Ponty 1945), anthropologists, cognitive scientists, and AI scientists. Second, the combined interdisciplinary literature makes evident a convergence of insights and basic questions and problems that may be investigated through collaborations of methodology, common terminology, and concrete research themes.
What follows begins with a tour d’horizon of some of the relevant literature. We then give a synopsis of communalities and divergent perspectives. A missing link in the function of body and embodiment is the transformational mechanism of bodily dynamics to cognition such as language and mathematics. We then sketch an approach to a possible systematic incorporation of this missing link. This model, construed from mathematical gesture theory in music (e.g., Mazzola and Andreatta 2007), yields an elegant answer to the question of the body’s contribution to the construction of a subject, a topic that has been addressed in semiotic embodiment discourse (e.g., Violi 2006). Finally, we ask what happens next, and speak to the need for a society and journal devoted to the science of embodiment.
We next stress a point that seems at once evident but likely to be forgotten. By its very nature a science of the human body is highly multidisciplinary. Therefore neither our terminology nor our knowledge of the extant literature on this topic will be satisfactory to all. Our interdisciplinary experience has taught us that often scientific terminology is nothing more than a terminology of intimidation. We ask the reader’s forgiveness for any use of terminologies that are inconsistent with disciplinary rigor and for any mistakes in our attempt to bind the broken parts of reality, which we euphemistically call disciplines.
Some of what one knows
At present a historically ordered and philosophically complete account of important contributions to body and embodiment would be impossible. Instead, we discuss a thematically motivated list of insights which we consider vectors pointing to a future science of embodiment.
The first reference, perhaps the most radical and tragic, is Wolfgang Graeser (1906-1928), a German mathematician and music theorist, who studied mathematics, physics, music and oriental languages since the age of seventeen in Berlin and Zurich. He became famous for his symmetry-oriented analysis of Bach’s Art of Fugue (Graeser 1924). A dramatic change in his understanding of music took place when he saw how dancers were rehearsing Bach’s Goldberg Variations. It led to his understanding that the musical essence was expressed in the dancer’s bodily movement. His last writing was an essay on embodiment, in which he concluded: Now we comprehend the body uncaged and without veiling insinuations (Graeser 1927). This explosion in his understanding was more than an intellectual insight; it effectively opened to him a completely new view of the essence of art. But it was also a revolution in his understanding of human expression. At age twenty-two, he committed suicide, overwhelmed by his deep and lonely insights.
A similar revolution occurred to Guerino Mazzola when he presented his formal tools (e.g., Mazzola 2002) in shaping free improvisation at a conference (Mazzola 2002) at IRCAM in 2002. He learned that free improvisation is not about abstract algebraic tools, but about gestures that are embodied in the dancing hands of the acting pianist. This opened his approach to the vital shaping force of the body in the art of music, an approach supported by the insights of Cecil Taylor, the monstre sacré of improvised piano music, stating that: I try to imitate on the piano the leaps in space a dancer makes (Spellman 1994). Taylor, quite radically, opposes a representative type of Western music culture:
“David Tudor is supposed to be the great pianist of the modern Western music because he’s so detached. You’re damned right he’s detached. He’s so detached he ain’t even there. Like, he would never get emotionally involved in it; and dig, that’s the word, they don’t want to get involved with music. It’s a theory, it’s a mental exercise in which the body is there as an attribute to complement that exercise. The body is in no way supposed to get involved in it.”
Suddenly, the body became a core creator, not merely the carrier of detached spiritual entities. And the famous mime Marcel Marceau echoed Taylor in saying:
“Dans le vide de l’espace quelqu’un dessine, crée à travers son corps l’infini du temps. Les mains bavardent, le buste s’exalte, le regard s’illumine et la scène se remplit petit à petit”.
(In the emptiness of space somebody draws, creates with his body the infinity of time. The hands chatter, the bust heats up, the view is illuminated and the scene is filled bit by bit.3)
The body was recognized for not only inhabiting a given space-time, but for creating it. It was the visionary French poet and philosopher Paul Valéry who summarized these artistic perspectives on embodiment with his famous inscription about the fundamental role of the hand in human cognition and creativity4 on Paris’ Palais Chaillot:
“Dans ces murs voués aux merveilles
J’accueille et garde les ouvrages
De la main prodigieuse de l’artiste
égale et rivale de sa pensée
L’une n’est rien sans l’autre.”
(In these walls devoted to the marvels
I receive and keep the works
of the artist’s prodigious hand
equal and rival of his thought
one is nothing without the other.)
It is not surprising that Valéry wrote an essay on the philosophy of dance (Valéry 1957), in which he concludes not with a scholarly description of dance, but by suggesting to start dancing our thoughts instead of thinking about dance. This image of a thorough consciousness of the primacy of embodiment in the arts is completed by the fact that in dance and its theories, embodiment has been a strong and important approach, as borne out by Jaques-Dalcroze’s eurhythmics (Mead 1986) and by Rudolf Laban’s work that created a subtle dance notation. Furthermore, Laban’s geometric language was able to make evident the role of the human body in the definition of space5. At present, dance as a way of thinking in the embodied making of space-time has become the basic method of understanding embodied movement as thought (Puttke 2010). Nonetheless, it remains problematic to determine what precisely is the thought that dance is expressing or, more radically, what precisely is a thought.
A proclivity for embodied approaches to the arts was initiated by painters who understood what the action of painting means. Jackson Pollock stressed gestural action as opposed to illustration. And, in a remarkable series of interviews, Francis Bacon made clear that his diagrammatic gestures were more important to his creative work than mentally planned strategies:
“The marks are made, and you survey the thing like you would a sort of graph. And you see within this graph the possibilities of all types of fact being planted...In the way I work I don’t in fact know very often what the paint will do, and it does many things which are very much better than I could make it do. Is that an accident?... What has never yet been analyzed is why this particular way of painting is more poignant than illustration. I suppose because it has a life completely of its own...So the artist may be able to open up or rather, should I say, unlock the valves of feeling.” (Sylvester 1975).
These insights have inspired the French school of diagrammatic philosophers, led by Gilles Deleuze’s comments on Bacon (Deleuze 1981) and elaborated and deepened by gesture theorists and philosophers, such as Gilles Châtelet (Châtelet 1993) and Charles Alunni (Alunni 2004). It is remarkable that here we encounter thinkers who are approaching embodiment not directly in a body-centered discourse, but instead focus on gestures as dynamic structures that act upon and through the body in a physical sense, but more abstractly. We shall return to this important nuance of embodiment when discussing the problem of the subject that is generated upon the body’s basic existence. The French diagrammaticians typically focused on gestures as a dynamic layer of embodiment. This approach had been inaugurated by the French mathematician Jean Cavaillès6 (1994) who had claimed that comprendre, c’est attraper le geste et pouvoir continuer (Understanding is catching the gesture and being able to continue.) This characteristic French dancing thought (also shaped in Pierre Boulez’s reflection on gesture in music [Boulez 2002]) was in fact stated with respect to mathematical theories, and as such it was one of the very first principles of gestural embodiment in mathematics, an idea made fashionable through the work of and Núñez (2000), but also anticipated in Châtelet’s observation (1993) that the Fregean concept of a function f in mathematics is a dramatic (and questionable) abstraction that replaces the moving gesture from argument x to its functional value f(x) by a kind of disembodied teleportation, where the evidence of the functional relation is wrapped and hidden if not destroyed.
This important French approach to gestures7 reveals a delicate aspect of embodiment in that gestures are conceived as being presemiotic. Gestures — except when tamed by social codes — are not signs in a semiotic environment. They are not a realization of Ferdinand de Saussure’s famous signification process from the expressive signifiant to the content of signifié (Saussure 1922). Châtelet (1993) is very clear in this point:
“Le concept de geste nous semble crucial pour approcher le mouvement d’abstraction amplifiante des mathématiques. . .Un diagramme peut immobiliser un geste, le mettre au repos, bien avant qu’il ne se blottisse dans un signe, et c’est pourquoi les géomètres ou les cosmologistes contemporains aiment les diagrammes et leurs pouvoirs d’évocation préemptoire.”
(The concept of a gesture seems crucial to approach the amplifying movememt of abstraction in mathematics…A diagram can immobilize a gesture, put it to rest long before it is hidden in a sign, this is why geometers and contemporary cosmologists love diagrams and their power of preemptive evocation.)
A gesture can be immoblilized by a diagram (which in this French theory is a kind of disembodied gesture) before it becomes a sign. And Alunni confirms this creative presemiotic role of gestures:
“Ce n’est pas la règle qui gouverne l’action diagrammatique, mais l’action qui fait émerger la règle.”
(It is not the rule which governs diagrammatic action but it is action which causes the rule to emerge.)
This is a theory of gestures that diverges from the Anglo-Saxon school centered around Adam Kendon and David McNeill, where (McNeill 2005, p. 58) it is stated that the gesture is created by the speaker as a materialization of meaning.
The question relating to the semiotic state of a gesture not only separates this French school from the above-mentioned Anglo-Saxon8 school, it also reveals a deep problem concerning the semiotic understanding of embodiment. What does it mean to be embodied? What is that body, whose substance reifies gestures? Such questions have been dealt with by linguists, of course, since Maurice Merleau-Ponty (1945), the figurehead of embodied linguistics, exposed the role of words as gesturally active bodies, and not as clothes and carriers of our thoughts and emotions:
“La pensée n’est rien d’intérieur, elle n’existe pas hors du monde des mots.”
(Thinking has no interior, it does not exist outside of the world of words.)
In a very lucid exposition (Violi 2006), the Italian semiotician Patrizia Violi has discussed the semiotic embodiment problem. Her discourse rightly points out the polysemy of the word body, which is evident; the word means very different things in medicine, yoga, psychology, performance, and philosophy. But the underlying question is what Violi puts into evidence: What is the substance, the essence of this concept that persists in all these approaches?
“In order to develop a fully embodied theory of semiosis we certainly need a bringing together of body and subject, and to do this we must develop an approach to subjectivity which is quite different from the transcendental Ego that is implicit in the classical structuralist framework.”
It is remarkable how the Saussurean disembodied abstraction in his sign concept (signifiant/signifié) resembles the Fregean abstraction in his disembodied concept of a mathematical function mentioned above. Violi confronts the linguistic category of a subject with the ontological one of the self as a being (also referring to Emile Benveniste’s investigations on enunciation theory) and observes that the theory of enunciation removes the issue of embodiment altogether. And she concludes that we are confronted with the paradoxical fact that on the one hand there is a theory of embodiment without subject, on the other a theory of the subject without a body. We learn that Violi opens a question which by far transcends the conceptual/semiotic analysis of body or embodiment, but points to the problem of how substantial these concepts could be in the very making of thoughts. The linguistic category she alludes to is a gate to the problem of why embodiment is such a basic category of any type of articulation, verbal and beyond9. Or, to restate in the spirit of Merleau-Ponty: if the words are the thoughts’ body, what can we draw from such a body’s anatomy when we think?
A field where thinking and embodied making, the mindful gesture, have been a traditional topic of attempts at theorization is music in theory and performance. It is a classical saying that music must be thought in the making, in performance. These insights are shared by Theodor Wiesengrund Adorno (Adorno 2001), Roger Sessions (Sessions 1971), Alexandra Pierce (Pierce 2010), Renate Wieland (Uhde and Wieland 2002), or Manfred Clynes (Clynes 1985), and we cite one representative statement by Adorno (Mazzola 2010):
“Notation wants music to be forgotten, in order to fix it and to cast it into identical reproduction, namely the objectivation of the gesture, which for all music of barbarian cultures martyrs the eardrum of the listener. The eternization of music through notation contains a deadly moment: what it captures becomes irrevocable…Musical notation is an expression of the Christianization of music…It is about eternity: it kills music as a natural phenomenon in order to conserve it — once it is broken — as a spiritual entity: The survival of music in its persistence presupposes the killing of its here and now, and achieves within the notation the ban from its mimetic representation.”
Despite these harsh insights, Robert Hatten (2004) rightly asks: Given the importance of gesture to interpretation, why do we not have a comprehensive theory of gesture in music? This reflects the dichotomy between performance practice, where embodiment and gesturality are well-known perspectives, and theoretical understanding, which might also be difficult because music theory has been a very rigid, algebraically shaped formalism, which has had no chance to deal with topological considerations needed for gestural and embodied analysis.
This dichotomy appears condensed in David Lewin’s celebrated book on musical transformation theory (Lewin 2007), where he asks: If I am at ‘s’ and wish to get to ‘t’, what characteristic gesture should I perform in order to arrive there? He thinks about a dancer, about the embodied musical subject, and the metaphor is quite strong. However, Lewin’s transformational theory is all but gestural; it is a very conservative formalism of abstract mathematical functions in the spirit of Emmy Noether of modern algebra in the late 1920s, in which there is no topology, no continuity or even homotopy. Gesture research in music has however taken place in the field of computer-aided performance (Wanderley and Battier 2000) and in our own computerized implementation of musical performance gesture theory for the pianist’s hand (Müller and Mazzola 2003). This research makes clear that a precise mathematical conceptualization of gestures and their embodied aspects is feasible. We have published two papers (e.g., Mazzola and Andreatta 2007; Mazzola 2009) on this theory and shall come back to it in more detail below, when discussing the missing link between embodiment and cognition. The point in this conceptualization is to shed light on Violi’s question concerning subject and embodiment, as well as on the problem of how gestures and embodiment are related, a question that remained unanswered in the above French approach to gestural embodiment.
Violi’s question is also critical in the ongoing embodied AI research, as typically represented by the breathtaking investigations with robots and biological simulators conducted by Rolf Pfeifer and collaborators (Pfeifer and Bongard 2006). They quite radically state that,
“(E)mbodiment is an enabler for cognition or thinking: in other words, it is a prerequisite for any kind of intelligence. So, the body is not something troublesome that is simply there to carry the brain around, but it is necessary for cognition.”
The problem here is that the body can be simulated by electro-mechanical machines (sometimes even by what they call cheap design, simple robots that do things just because their mechanical configuration enables it, such as walking down the street), but there is no evidence or even experimental proof that the gestural dynamics of these machines can transcend the level of a banal vaudeville show. In Violi’s words: what must be added to transform the body into a subject? And vice versa: in what way is the subject acting upon its body? After the failure of the brain-oriented symbolic AI and the nerve-oriented connectionist AI, we have now arrived at the upside-down variant that starts at the feet and aims at reaching the brain’s performance through the body’s intelligence, so to speak. The critical question arises from the problem to understand the added value of action, of gesture, of what makes the dynamic body become a subject without any postulate of divine instantiation of a soul. In Pfeifer’s statement, the body was recognized as being necessary for intelligence. In a trivial sense, this is true, since disembodied clouds of intelligence have never been observed. But the underlying thesis is that it is also sufficient, that intelligence is an emergent property of complex dynamic body configurations.
This unsolved problem also underlies the open problems emerging from recent discoveries in neuroscience. There are a number of such discoveries and associated theories. Let us refer to the most prominent one associated with the discovery of mirror neurons by Giacomo Rizzolatti and collaborators in 1992. These neurons were discovered in the ventral premotor cortex of the monkey (e.g., Rizzolatti and Fabbri-Destro 2010).
“The fundamental characteristic of these neurons was that they discharged both when the monkey performed a certain motor act (e.g., grasping an object) and when it observed another individual (monkey or human) performing that or a similar motor act.”
Similar findings were made for humans, and also for acoustical, not only visual input. These results have been interpreted as evidence that human cognition is a closed circuit process. The passive perception is mirrored to the action layer, and there is growing evidence that vice versa action also induces sensorial neurons (visual cortex) to fire (Ebisch et al. 2008). It has induced a series of models of human cognition and intelligence, individually, socially and culturally. And it is astonishing that Cavaillès’ above statement, understanding means catching the gesture and continuing, is nothing else but the mirror neuron mechanism in neuroscientific terms. Perception activates the neurons for action, you continue the action when understanding.
This philosophy is akin to other neuroscientific approaches, such as Wolfgang Prinz’s theory (Prinz 1984) of common coding of perception, cognition, and action, stating that these three human performances share one and the same code, a theory that had been suggested by the American psychologist William James and neuroscientist Roger Sperry (1952). The theory of mirror neurons, which can be interpreted as an embodied version of ideas that were also virulent in the above French gesture theory, has entailed a number of models for different areas of science. Let us mention two of them. Neurologist Vilayanur S. Ramachandran argues (Ramachandran 2000) that mirror neurons are the agents for the human capacity for learning and imitation, and that they will play the same role for psychology that DNA did for biology.
“Mirror neurons will provide a unifying framework and help explain a host of mental abilities that have hitherto remained mysterious and inaccessible to experiments.”
He confirms the theory of Australian anthropologist Merlin Donald, who maintains (Merlin 1998) that mirror neurons offer a basic mechanism for a cultural mimesis thesis:
“Most importantly, the fundamentals of articulatory gesture, from which all languages are built, were put in place when mimetic capacity emerged.”
In the vein of Donald’s anthropological perspective, De Novellis considers the body’s nonverbal communication as an embryonic form of high culture. Therefore, to reach an integral vision of these phenomena and their origins, the investigation of infancy is essential. De Novellis’s inquiry refers to a process that is similar to Darwinian evolution in that it is structured in firmly linked stages of development. Her anthropological investigations refer to a 20th century perspective on the body, called Les mutations du regard by Alain Corbin, Jean-Jacques Courtine, and Georges Vigarello in the third volume of their comprehensive analytical work Histoire du corps (Corbin et al. 2006) These mutations affect not just the idea of the body as a means of communication but also as a projection of the mental and imaginative worlds, two elements important to Saussure in his semiotic analysis of human existence.
Curiously, the origins of the popular tradition of the physical body are recovered in the trance state, where the body becomes a communicative container of language. In the 1950s, Diego Carpitella and Ernesto De Martino (De Martino 1999), during a series of trips around southern Italy10, recovered many testimonies of women who represented the performative behaviors of the attarantate tradition in the area of Salento, where folk rituals and festivities give us a quite primordial representation of the body in trance, expressing a psychological journey of emotions. When dealing with trance and bodily manifestations of emotions, it becomes necessary to understand the body as a representation, as a performance of its proper psychological dynamics.
While the evolutionary emergence of the mimetic capacity is an anthropological argument, the problem we referred to in the review of Violi’s discussion remains unanswered. How can higher cognition emerge from the embodied action scheme in humans? How can we describe the evolution of language, logic, mathematics, music, and other arts, from the action-perception cycles and mirror neurons?
Can and should it all be put together?
In light of the many and diverse fields from the full range of discipline, in which embodiment plays an ever increasing role, we find numerous arguments in favor of the creation of a human body/embodiment science. We emphasize that we are not the first to have considered such a multidisiplinary endeavor; two such enterprises come from France and the United States.
The first is the initiative of Frenchman Bernard Andrieu11, entitled Le site du corps, and may be found online12. Andrieu describes his approach as follows:
“Pour étudier comment le corps a été étudié et interprété par la science et la philosophie, notre recherche est organisée autour de deux grands axes, développés en parallèle depuis 1986: la désincarnation scientifique du corps et la description du sujet incarné par les philosophies du corps. Notre postulat est de lier l’épistémologie et l’ontologie du corps en démontrant comment la modélisation du corps présuppose, en le laissant le plus souvent dans l’implicite, une conception ontologique du corps.”
(In order to study how the body has been investigated and integrated by science and philosophy, our research is organized around two great axes, developed simultaneously since 1986: the scientific disembodiment of the body and the description of the embodied subject by philosophies of the body. Our postulate is to connect epistemology and ontology of the body by showing how the body’s modeling presupposes, keeping it however mostly implicit, an ontological concept of the body.)
The initiative is very well organized, including a number of books13, research, international connections, and the French internet network. In particular, it lists all doctoral theses in France in the years 1971-2007 with the word corps in their titles. The site reveals a great awareness of the multidisciplinary nature of the subject and also offers a historical perspective of the body. Although the site is centered around French contributions and perspectives, we consider this initiative a vital step towards an institutionalization of a science of the human body. Andrieu’s initiative also comprises the journal Corps14.
The other enterprise is more philosophical, though not restricted to philosophical discourse. It was created and directed by Richard Shusterman15 who has institutionalized his theoretical and practical approaches in the Center for Body, Mind, and Culture at Florida Atlantic University16. Similar to Andrieu, Shusterman has created an incredibly rich resource for scholarly research, which also includes information on mind-body practices such as Feldenkrais, Alexander Technique, and Confucian and Zen approaches, which are beyond the purview of our discussion, but are relevant to understanding Shusterman’s comprehensive approach and his understanding of philosophy as a quest for a good life. His seminal paper, Somaethetics: A Disciplinary Proposal, published (Shusterman 1999) in 1999, presented a model of what could be regarded as a philosophy of the human body and embodiment. In his critical review of Alexander Baumgarten’s Aesthetica, Shusterman proposes a discipline of somaesthetics, defined as the critical, meliorative study of the experience and use of one’s body as a locus of sensory-aesthetic appreciation (aesthesis) and creative self-fashioning. In remarkable agreement with Andrieu’s approach, Schusterman states that,
“Beyond the essential epistemological, ethical, and sociopolitical issues, the body plays a crucial role in ontology.” He adds that “analytical philosophy examines the body as a criterion for personal identity and as the ontological ground (through its central nervous system) for explaining mental states.”
On this ground, Shusterman then defines three fundamental dimensions of somaesthetics:
(i) Analytical somaethetics, (describing) “the basic nature of bodily perceptions and practices and also of their function in our knowledge and construction of reality.”
(ii) Pragmatic somaethetics having a distinctly normative, prescriptive character — by proposing specific methods of somatic improvement and engaging in their comparative critique.”
(iii) Practical somaesthetics, “being concerned not with saying but with doing, this practical dimension is the most neglected by academic body philosophers, whose commitment to the discursive logos typically ends in textualizing the body.”
He views the disciplinary proposal in a double meaning: as a branch of learning or instruction and as a corporeal form of training and exercise. He strongly argues for a wider conception of philosophy, including the ancient idea of philosophy as an embodied practice, a way of life.
Shusterman’s broader view of philosophy is shared in De Novellis’s approach as a performing artist to the idea of a body science. Many artists and numerous theatrical and choreographic schools of practice put their body at the center of artistic research. The dancer-actor, for instance, transmits life experience through his or her body. Spectators then recognize their own experience in the mirror of the dancer-actor’s vital interpretation, an exchange where the body yields the only filter and link. In the performing arts, the importance lies in the body’s actions, gesture and space (such as the installation of the body in space).
Although the body is not necessarily the central object of the artistic work, it is addressed — by artist and viewer — via action and gesture in space. The reality of action lies in the possibility of gesture to express the very thoughts that stir the body in the first place, and to allow them to echo in space and eventually lead to further spatial repercussions. In this sense, De Novellis’s interest in the performing arts focuses on the body’s ability to tell stories that culminate in gestures. This is why her performance resides in gesture and in the presence and appearance of the body. It is the emotional journey that she wants to represent, the denunciation of what was, and the fact that the viewer must be patient to find out what will happen to the body. In De Novellis’s recent performances (Figure 1), a journey of moving images and gestures in urban space reveals the passage of what is being recounted as it emerges from the human body, becomes verbal, reaches those who observe the passage, and finally returns to the body.
Figure 1: Romina De Novellis at a public performance La Festa di Santa Barbara, Rome 2008 (photo: Hugues Roussell)
Given the above tour d’horizon, as well as Andrieu’s and Shusterman’s strong arguments for a science of the human body and embodiment, we can now attempt to describe the state of the art of the scientific discipline of embodiment. We focus on the analytical, rather than the practical dimension, and on the global picture of such a scientific proposal. We do not yet problematize the concepts of body or embodiment in the manner of Violi, but will postpone this topic until the next section. We embrace the artistic insights exposed in our tour d’horizon as a strong argument for further investigating the architecture of a science that could contribute to the unification of methodologies and findings regarding the role of the body in the construction of higher cognition, intelligence, knowledge, and culture. Recent insights from the hard sciences, neurobiology and embodied artificial intelligence, draw a picture of a new type of homunculus. Whereas the classical homunculus was a topographic image of the interface between the body’s limbs and their sensorio-motor innervations, we now display an integrated map of action and perception. This unified action-perception topography of the body is what we call the action-homunculus. It is the integral of all the dimensions that define our physiological existence as a space-time of action and perception. It is the global interface with extensional reality of the human existence. Although we are far from understanding its detailed mechanisms, we conjecture that this layer is the bodily basis for whatever we may experience physiologically as living beings.
This is one extremal set of data, from which we have to draw a line to the other extremal set of human existence: cognition, emotions, and higher human dimensions such as language, non-verbal semiotic systems (traffic, fashion, alimentation, social interaction, emotions, sexual and erotic codes, and so on), logic, mathematics, and the arts. Let us call this layer the cognitive stratum.
It is one of the essential, although implicit, axioms of all scientific and artistic approaches to embodiment that the layer of the action-homunculus (or any corresponding concept they would associate with the body’s basic reality) is the first level of a cognitive embryology leading to the cognitive stratum. We use the term cognitive embryology because it is a second axiom of this philosophy of embodiment that the cognitive stratum is the completion of a complex organic evolution and not something which is added at once and stemming from a metaphysical Platonic sky or from some divine fertilization. This is a strong assumption that contradicts the Leibniztian monadic principle: there are not two worlds, our thoughts are created from the action-homunculus and not imported from another reality.
This setup is not a fact, but a basic approach to an attempt to understand cognition and intelligence in a very precise evolutionary architecture. The cognitive layer is the grown-up shape of the action-homunculus. Although this approach is far from complete, we must ask whether such a strong fundamental assumption or principle can be the basis of a scientific (multi)discipline. The model of a scientific discipline par excellence is modern physics. It is an extremely successful science with a complex theory and an overwhelming arsenal of applications in other sciences and in practice. Physics may alter its shape as theory and experiments evolve, but it is built upon a very abstract and invariant principle, namely that nature is governed by a small set of laws. At present, the architecture of these laws is simple. Nature is completely described by three types of forces: electro-weak force, strong force and gravitation. And all efforts in the Big Science of elementary particle physics are united by the axiom that these forces should be expressions of one basic force, an endeavor coined theory of everything (ToE). Nothing confirms such a radical principle, but physicists agree that it is the big challenge.
In view of this exemplary model of a hard science, it is reasonable to make an analogous effort in the understanding of the human condition, namely that the cognitive stratum is the endpoint of a cognitive embryology based upon the action-homunculus.
What is the critical point in such a scientific enterprise? The central question is this: how is it possible to describe and understand the transition from the action-homunculus to the cognitive layer? In embodied AI, the problem is manifest. All efforts in robotics, and other cheap or expensive embodiment machines, are a desperate search for emergent properties. There is no general methodology that would suggest how higher cognitive performance can emerge from low-level actions. The same is true when dealing with neurophysiological approaches to action-perception. We know quite a lot about the mechanisms of action-perception cycles, but there is a critical missing link from the action-homunculus to the cognitive stratum. We suggest that a science of the body and of embodiment should declare as its principle that we must find this missing link in the same spirit as modern physics in search of the unified force in the ToE. Therefore, we propose a model of such a science that connects the action-homunculus to the cognitive stratum by a layer which may be called the action-to-cognition (A2C) layer. In the next section, we discuss a proposal for A2C, which is built upon the theory of gestures. It is not, however, a defining feature of this science but a proposal — motivated from gesture theory in music — of how such a layer may be conceived.
The missing link
An approach to deal with the missing link between the action-homunculus and the cognitive stratum is deduced from musical gesture theory. Its motivation stems from the well-known fact that, in music, thoughts are known to be embodied in the performer’s gestural actions. In this general shape, the idea is metaphorical and abstract. Can we hope to make this more precise? Let us first recapitulate what is given. We have to search for a connecting structure between action-homunculus and the cognitive stratum. Since the action-homunculus is already well-prepared to deal with gestures, actions are no longer a static body shape; actions can be taken as elementary structures to enable gestures. The action-homunculus is a body in action.
In music theory, a gesture has been introduced (Mazzola and Andreatta 2007) as being composed of three components (see also Figure 2): a skeleton D, a body X, and a map g from D to the set C(X) of curves on X. The skeleton is a directed graph, i.e., a set VD of vertices together with a set AD of arrows connecting the vertices. We allow multiple arrows between the same two vertices as well as loops, i.e., arrows whose heads and tails coincide. This means that a skeleton is an abstract scheme describing directed connections (arrows) between a set of points (vertices). The gesture’s body X is a topological space. The set C(X) of curves on X is the set of all continuous maps c: I → X from the real unit interval:
I = {x, 0 ≤ x ≤ 1}
The map g: D → C(X) maps every vertex v in VD to a point g(v) in X, and every arrow a in AD to a continuous curve g(a) in C(X) such that the configuration of these curves corresponds to the arrow configuration in the skeleton D. In other words, the body X contains a faithful image of the skeleton that consists of curves instead of abstract arrows. This definition is a modern instantiation of the classical definition of a gesture given by Hugues de Saint-Victor (Schmitt 1990): Gestus est motus et figuratio membrorum corporis, ad omnem agendi et habendi modum (Gesture is the movement and figuration of the body’s limbs with an aim, but also according to the measure and modality proper to the achievement of all action and attitude.) In mathematical music theory, this concept of a gesture has been applied with great success (Mazzola 2009) to the description and analysis of gestures.
Figure 2: The anatomy of a gesture: skeleton D, a directed graph, and body X: the realization of the skeleton in a space of continuous curves in a topological space X, here the space of fingertip positions in time for a pianist’s hand.
The concept is also empowered to create gestures of gestures, so-called hypergestures, which means that the theory also deals with higher-order gestures. Denote the space of all gestures g: D → C(X) by D@X. Then this is provided with a canonical topology (Mazzola and Andreatta 2007). So we may go on and consider hypergestures h: G → C(D@X), hyperhyper-gestures, and so on.
Such gestures are an ideal tool to mediate between action and cognition. Let us explain how the most important mathematical structure: the group of integers Z (…, -3, -2, -1, 0, 1, 2, 3,…) can be modeled by use of gestures. If we take the unit circle X = S1 and consider all gestures g with body S1, skeleton the loop L (one vertex and one arrow on that vertex), and send the vertex to a fixed point s on S1, then one can concatenate two such gestures and obtain a new gesture of the same type. If we further identify two such gestures, if they can be deformed into each by a so-called homotopy, we obtain the so-called fundamental group of the circle. It is known that this group is exactly the group Z of integers. In other words, the integers are generated from gestures. It can be shown that every mathematical group is the fundamental group of an adequate body space. This means that all those fundamental algebraic group structures can be described by gestures!
This is a precise example of how the transition of action to higher cognition can work when we use gestures. Although this example is operational, we are still confronted with Violi’s problem of how the subject and the body may be related to each other.
Understanding the musical theory of gestures from a more philosophical point of view may give us a hint of how this problem can be attacked. In the concept of a gesture, we have two material ingredients: the skeleton and the body. The skeletal ingredients act upon the body; the vertexes and arrows are embodied by points and continuous curves on the body. The latter may be thought of as being given from the action-homunculus as a space, where action takes place. When accessed from the layer of gestures, this body is not directly grasped, instead we only experience it through the configurations of continuous curves defined by the map g on the skeleton. Since the ontology of the body is not directly accessible, we are forced to act via gestures upon this body X. The body X only exists as an embodiment of the possible skeleta. This means that X is an abstract entity and that its ontology is only visible when we let our gestures act upon this body X. We would like to call the totality of gestural actions (all the maps g with variable skeleta D) the body’s embodiment. We cannot know X, but only how it feels to act on X. So is X forever hidden to our insight? Not really, and this is one of the most beautiful results from modern mathematics, from category theory, to be precise. The result is known as Yoneda’s lemma. In common language it is known as the Yoneda philosophy (e.g., Mazzola 2002), stating that a mathematical object is completely known if we know how it looks when observed from all objects of the same type. For example, a topological space is completely known if we know all the continuous maps from any other topological space. In our case the set of all gestures targeting one and the same body is precisely a situation as described by Yoneda’s philosophy. This means that once we know all gestures on X, the system C(X) is completely known, and then the points of X, being the constant curves on X, are also known. In other words, yes, the system of all gestures on X gives a lot of information about X.
Our interpretation of these facts is that the embodiment of the body by the system of all gestures creates a machinery which we may control and which allows us to instantiate X when experienced through our gestures. The ontology of X is not given directly, but is mediated through our gestural dynamics. And, if we use the recursive machinery of hypergestures, the passage to the cognitive stratum will occur in a series of hypergestural intermediate layers, unfolding from simple gestures on X to highly hypergestural ones in hypergesture spaces of type Dn@ Dn-1@ Dn-2@... D1@X.
It follows that we have good reasons to conjecture that the missing A2C link between the action-homunculus and the cognitive stratum might be realized by the category of gestures acting upon the action-homunculus’ body. Violi’s subject may then be interpreted as being the identification of the body when the system of gestures operates in the creation of the ontology as mediated by Yoneda’s lemma.
Some undisciplined thoughts
For the time being, we propose a collaboration in the creation of a science of the human body and its embodiment. We are not sure whether science of the human body is a good name since this gives a static impression, whereas science of embodiment would stress the action over the underlying matter. But this is a minor problem, and perhaps an artificial wording, similar to Shusterman’s somaesthetics, would solve this problem.
A first measure in the joint effort for such a science could be made in the founding of an International Society for Embodiment Science, together with an international journal. The latter would be multidisciplinary, open to hard sciences, the humanities, and the arts (for the latter whenever they embark in a reflection of the matter beyond l’art pour l’art).
Epilogue: A Sample Problem for a Nascent Science
One field that might benefit greatly from this new Big Science is disability studies (DS). DS, already a broad interdiscipline, includes contributions from the humanities, social sciences, arts, and professions such as education and social work. To date, the hard and applied sciences have been minor contributors, with the recent inclusion of computer science and information technology in the field’s publications and conferences a welcome development. Science-phobia is largely a function of a history of ill treatment of and negativity towards disabled people by medical practitioners and researchers. Because DS has functioned throughout its history in large part as a critique of medicine and related fields, a symbiosis grounded in critical research, as called for by Alex Lubet (Lubet 2011), is possible and would be welcome.
One problem which DS and the new body science might collaborate concerns whether the body speaks or expresses itself. Nowhere is this of greater interest than in music in general and jazz improvisation in particular, especially regarding prominent virtuosi with significant hand impairments. In classical music, where the impaired hand is simply not used, as in left-hand-only works like the Ravel G Major Concerto, the question cannot be raised. But jazz has produced a number of prominent players who use their impaired hands, most notably guitarist Django Reinhardt and pianist Horace Parlan. Lubet has demonstrated how these artists’ improvisatory styles are profoundly informed by their unique hands (Lubet 2011). Undoubtedly this is always the case; the disability only makes it more obvious and easily studied. But the question that remains for the new science of the body is the manner in which the hand — most obviously the impaired hand — in concert with the brain and neural network, creates the improvised musical idea.
The question is, as with Augustine’s conception of time, not so simple once one begins to think about it. One may presume that the brain and neural network simply adjust to the impaired hand, which improvises accordingly. Hamilton, Pascual-Leone, and Schlaug have found such a compensatory adaptation in peripherally (that is, not born) blind musicians, for whom regions of the brain normally associated with sight are reallocated to hearing (Hamilton et al. 2004). But Parlan was infected with polio as a child, long before he began to play, while Reinhardt suffered his hand injury at age 18, an age considerably beyond the brain’s critical period for second language acquisition (Lennenberg 1967). When and how the brain adjusts to injury, particularly in regard to such task-specific behavior as musical improvisation, is not fully understood. Another factor, as all performers and especially improvisers know, is the body’s failure to execute physically the musical idea.
The data that complicate this question are not complete without consideration of the case of pianist Paul Wittgenstein. Unlike contemporary left-hand-only pianists Gary Graffman and Leon Fleisher17, who have impaired right hands, Wittgenstein lost his right arm in combat in WWII. However, according to Wittgenstein pupil Erna Otten, his arm was gone, but not forgotten:
“I had many occasions to see how involved his right stump was whenever we went over the fingering of a new composition. He told me many times that I should trust his choice of fingering because he felt every finger of his right hand. At times I had to sit very quietly while he would close his eyes and his stump would move constantly in an agitated manner. This was many years after the loss of his arm.” Otten later added: “His finger choice was always the best!” (Sacks 2008).
Although Wittgenstein’s piano lesson is not a precise analog to Parlan’s or Reinhardt’s improvisation, since no musical composition took place, right-hand fingerings — his musical idea — were revealed, and perhaps invented, apparently at the speed of real time performance, in the absence of a physical right hand.
The question of the neuroanatomy of improvisation, raised here by the interdisciplinary field of disability studies, requires an answer (which will serve DS’s mission of providing greater opportunities for disabled people) that can be found only if it partners with the science of body/embodiment proposed herein. This human, humanistic, and humane concern queries, in quite different form, the master narrative of the proposed field, its missing link between the action-body and the cognitive stratum proposed earlier. The range of disciplines already involved should produce a big science with room for many and diverse collaborations.
Endnotes
- He is the president of the Society for Mathematics and Computation in Music (http://www.smcm-net.info) and has published a number of books on the subject, most prominently: The Topos of Music. Springer, Basel 2002.
- See the Internet site www.rubato.org to the music software Rubato.
- All translations have been performed by the authors.
- The magic of the hand has recently been described in a marvelous book by architecture theorist Juhani Pallasmaa: The Thinking Hand. Wiley 2009.
- See also the treatise: Fabriques de la danse. Presses Univesitaires de France, Paris 2007, by Simon Hecqwuet and Sabine Prokhoris, dealing with Laban’s subtle contribution to the notation of dance, where the movement is never formally described, but only pointed at and left to the dancer’s existential embodiment.
- Cavaillès was executed by the Nazis in 1944 for having participated in the résistence.
- There are other French approaches to gesture theory, such as Geneviève Calbris, for example.
- There are other Anglo-Saxon approaches to gesture, for example, by Jürgen Streeck, Eve Sweester, or Katharine Young.
- We should also refer to a beautiful book on semiotic aspects of ambodiment and gesturality: Popelard, M.-D. and A. Wall: Es faits et gestes. Editions Bréal, Rosny-sous-Bois Cedex 2003.
- There are numerous sound recordings and published writings of that trip.
- Professor at the Faculty of Sport/ UHP of Nancy University, France.
- See http://www.staps.uhp-nancy.fr/bernard
- Among others, one on a philosophy of the body, the skin, suntan, etc.
- http://www.staps.uhp-nancy.fr/bernard/revues_corps.htm
- Dorothy F. Schmidt Eminent Scholar in the Humanities, Professor of Philosophy and English, and Director of the Center for Body, Mind, and Culture at Florida Atlantic University. But see http://www.fau.edu/humanitieschair
- http://www.fau.edu/bodymindculture
- Fleisher resumed limited two-handed performance in 1995, after successful treatments with Rolfing and Botox. Because of his limitations, he has the status of being the only professional classical piano soloist who plays with an impaired hand. Alex Lubet discusses this in (Paralyzed on One) Sideman: Disability Studies Meets Jazz, through the Hands of Horace Parlan, keynote address, Guelph Jazz Festival Colloquium, September 9, 2010.
References
Adorno TW (2001) Zu einer theorie der musikalischen reproduktion. Suhrkamp, Frankfurt, Germany
Alunni Ch (2004) Diagrammes & catégories comme prolégomènes à la question: qu’est-ce que s’orienter diagrammatiquement dans la pensée? In: Batt N (ed) Penser par le diagramme. Presses Universitaires de Vincennes 22-2004, Vincennes, France
Boulez P (2002) L’Ecriture du geste. Christian Bourgois, Paris, France
Cavaillès J (1994) “Méthode axiomatique et formalisme” (1938) In: Oeuvres complètes de philosophie des sciences. Hermann, Paris, France, p. 178
Châtelet G (1993) Les enjeux du mobile. Seuil, Paris, France
Clynes M (1985) Secrets of life in music. In: Analytica, studies in the description and analysis of music in honour of Ingmar Bengtsson. Royal Swedish Academy of Music 47, Stockholm, Sweden, pp 3-15
Corbin A, Courtine JJ, Vigarello G (2006) Histoire du corps, T3, Les mutations du regard. Le XX siècle. Seuil, Paris, France
De Martino E (1999) La terra del rimorso. Bellaterra edizioni
Deleuze G (1981) Francis Bacon: logique de la sensation. Editions de la Différence, Paris, France
Ebisch SJH, Perrucci MG, Ferretti A, Del Gratta C, Luca Romani G, Gallese V (2008) The sense of touch: embodied simulation in a visuotactile mirroring mechanism for observed animate or inanimate touch. J Cogn Neurosci 20:1-13
Graeser W (1924) Kunst der fuge. In: Bach-Jahrbuch
Graeser W (1927) Der körpersinn. Beck, München, Germany
Hamilton RH, Pascual-Leone A, Schlaug G (2004) Absolute pitch in blind musicians. Neuroreport 15:803-806
Hatten R (2004) Interpreting musical gestures, topics, and tropes. Indiana University Press, Bloomington, IN
Hofmann GR, Krömker R, Mazzola G (1987) Rasterbild - bildraster, CAD-gestützte analyse von Raffaels “Schule von Athen”. Springer, Berlin, Germany
Lakhoff G, R Núñez (2000) Where mathematics comes from: how the embodied mind brings mathematics into being. Basic Books, New York, NY
Lennenberg E (1967) Biological foundations of language. Wiley, New York, NY
Lewin D (2007) Generalized musical intervals and transformations. Oxford University Press, New York, NY
Lubet A (2011) Music, disability, and society. Temple University Press, Philadelphia, PA, pp 42-88, 168-170
Mazzola G (1999) Semiotic aspects of musicology: semiotics of music. In: Posner R et al. (eds) Semiotics handbook Vol. III. W. de Gruyter, Berlin, Germany
Mazzola G (2002) Les mathématiques de l’interprétation musicale. Séminaire MaMuX, IRCAM, Paris, May 18, 2002
Mazzola G (2002) The topos of music. Springer, Basel, Switzerland
Mazzola G (2009) Categorical gestures, the diamond conjecture, Lewin’s question, and the Hammerklavier Sonata. J Mathematics and Music 3
Mazzola G (2010) Musical performance. Springer, Heidelberg, Germany
Mazzola G, Andreatta M (2007) Formulas, diagrams, and gestures in music. J Mathematics and Music 1
Mazzola G, Wieser HG, Brunner F, Muzzulini D (1989) A symmetry-oriented mathematical model of counterpoint and related neurophysiological investigations by depth-EEG. In: Hargittai I (ed) Symmetry: unifying human understanding, II. CAMWA/Pergamon Press, New York, NY
McNeill D (2005) Gesture and thought. University of Chicago Press, Chicago, IL
Mead VH (1986) More than mere movement: Dalcroze Eurhythmics. Music Educators J 72:42-46
Merleau-Ponty M (1945) Phénoménologie de la perception. Gallimard, Paris, France
Merlin D (1998) Mimesis and the Executive Suite: missing links in language evolution. In: Hurford JR, Studdert-Kennedy M, Knight C (eds) Approaches to the evolution of language, Cambridge University Press, Cambridge, UK
Müller S, Mazzola G (2003) Constraint-based shaping of gestural performance. Proceedings of the ICMC 03, ICMA, Ann Arbor, MI
Pfeifer R, Bongard J (2006) How the body shapes the way we think. MIT Press, Cambridge, MA
Pierce A (2010) Deepening musical performance through movement: the theory and practice of embodied interpretation (musical meaning and interpretation). Indiana University Press, Bloomington, IN
Prinz W (1984) Modes of linkage between perception and action. In: Prinz W, Sanders AF (eds) Cognition and motor processes. Springer, Berlin, Germany, pp 185-193
Puttke M (2010) Learning to dance means learning to think! In: Bläsing B, Puttke M, Schack T (eds) The neurocognition of dance. Psychology Press, New York, NY
Ramachandran VS (2000) Mirror Neurons and imitation learning as the driving force behind the “great leap foward” in human evolution. Retrieved January 15, 2012 from http://www.edge.org/documents/archive/edge69.html
Rizzolatti G, Fabbri-Destro M (2010) Mirror neurons: from discovery to autism. Exp Brain Res 200:223-237
Sacks O (2008) Musicophilia: tales of music and the brain. Vintage Books, New York, NY
Saussure F de (1922) Cours de linguistique générale. Payot, Paris, France
Schmitt JC (1990) La raison des gestes dans l’occident médiéval. Gallimard, Paris, France
Sessions R (1971) Questions about music. W. W. Norton and Company, New York, NY
Shusterman R (1999) Somaesthetics: a disiplinary proposal. J Aesthetics and Art Criticism 57:299-313
Spellman AB (1994) Four lives in the bebop business. Limelight Editions, New York, NY
Sperry RW (1952) Neurology and the mind-body problem. American Scientist 40:291-312
Sylvester D (1975) Interviews with Francis Bacon. Pantheon Books, New York, NY
Uhde J, Wieland R (2002) Forschendes üben. Bärenreiter, Kassel, Germany
Valéry P (1957) Philosophie de la danse (1938). In: Œuvres I, Variété, Théorie poétique et esthétique, pp 1390-1403. Gallimard, Paris, France