Part IV: Can you interact with nonexistent physical objects and spaces?

Using embodied cognition as a framework, the preceding post revealed the probable role that the gestures and body movement played in creation of the objects. In this post, we focus on the causes of the accuracy of designer’s interaction with the objects and spaces. It will address the unanswered question,

How could designers corporally interact with a virtual, physically non-existent object that they created in their mind’s eye? And that too, with surprising accuracy?

About this post

It is planned to rely on multiple sources to answer these questions. First, we will depend on research literature, particularly on embodied cognition. This does not deal specifically with gestures in imagery and sketching. So, some elements of speculation cannot be denied, but at least, it offers a broad framework to support the ideas. Second source is the data that is available as series of experiments. We will also take support from anecdotal example, more to understand ideas than to support the conclusions.

So, the discussion that follows has following aims. 1] To offer a plausible explanation to the participants relying on specific kinds of gestures and movements during creation of these objects and spaces. It argues on why and when gestures and body movements vigorously mimic and do not mimic the physical characteristics of the shapes. 2] To explain the source of accuracy of designer’s interactions with the physically non-existent virtual objects while creating the shapes, and 3] To explain why, during summing up their solutions towards the end of the experimental sessions, the participants are able to repeat the identical gestures and body movements.

The creators selectively chose gestures and/or body movements to suit the design task at hand. The post will argue that their decisions to deploy or not to deploy them were prompted by the useful contributions that gestures and movements made to the evolving thoughts. Let us try to develop these arguments further.

During discussion in the preceding blog on embodied cognition, there were references to the close relationship between visual system and the motor system. This suggests that cognitive processes also encode the sensory-motor aspects as series of movements.1 The argument is built on this central idea of the partnership between the visual and the motor system. So, we first address the partnership related question,

Can the creator’s actions be viewed as close-knit partnership between visual decisions, movements and the thought process?

If the answer is yes, it will give us a framework to answer the questions that we started with.

The casual support to the idea of this partnership comes from several examples in our every day routine tasks that demand relying on mix of motor and visual system. They seem to work together in partnership. We perform such tasks efficiently because these two systems seem to work in sync. But are all partnerships same? For the purpose of this paper, we plan to categorize the partnership into two broad groups. We are specifically focus on the second, which is of interest to us in the context of this paper. So we only briefly touch the first and move on.

1] Creative partnership where gestures and movements are based on convention or tradition

In this category, the planned visual effects are achieved through ‘pre-designed’ body postures, movements and visual expressions. The visual and the motor systems work in sync. They balance the cerebral and the corporal.

The major difference is that the gestures and movements are not free to evolve, but were defined by codes. They are motivated by convention or tradition, and are often formally learnt, practiced and produced. These events give ample evidence of how visual and motor systems work in sync.

Typical day-to-day example would include writing and production of your own signature and learning to write new script.2 Formal example to understand the category would be live performances of classical dance, street play, fashion shows and even performances of acrobats in circuses.3 These gestures and movements are either controlled by tradition (as in classical dance forms) or by convention (as in writing, plays and fashion shows).

These categories of gestures and movements are unlike those produced in our experiments. Explanations of learning, producing and use of these gestures are available as footnotes cited above. This ensures that we maintain our focus on the second category i.e. the spontaneous and intuitively produced gestures and movements.

2] Supporting the partnership with intuitive and spontaneously produced gestures and movements

A typical example to explain the idea would be the gestures accompanying natural speech. These gestures are spontaneously produced to support the evolving thoughts. They reflect the thinking in some ways. They show how the cerebral and corporal work in partnership.

Better and more convincing examples in this category are the gestures produced when giving route directions to a remotely located third party. They are truly embodied creations used everyday. The gestures clearly offer a corporal support to the cerebral processes.

The synchronization between the visual and motor systems continues like in earlier cases. It is not dictated by practice, but by the emerging thoughts and ideas. Our focus is on corporal actions (gestures and body movements) as supports to the cerebral actions (thinking process). Spotlight on this is justified because we feel that this is how the designers and architects have used gestures in our experiments.

A formal and compelling example is a live music concert. The singers move their hands and torso. Accompanying artists too move their bodies in sync. These gestures and movements support the primary task of creation and production of live music. (Also see ‘Can we prevent designer from sketching’ in this blog series)

Such gestures are very similar to the way they occur in our experiments. They are in sync with the evolving thoughts and ideas and so, reflect the thought process. They are very much a part of the process of creation. Such spontaneous use of gestures would be of interest to us in the current discussions.

Like in the live concert example, in our experimental sessions, the gestures were spontaneously produced. They appear to respond to the creator’s evolving thoughts. Most of them seem to incrementally shape the ideas of objects and spaces. They often mimic the shapes and/or indicate the locations of features in the space, as if these exist in front of them.

Let us focus on these experiments with architects and designers that we reported in this series of posts. The participants in the experiments were not briefed and had the option of using or not using the gestures. They were free to deploy gestures and movements, if and when they chose.

Identifying partnership to interact with the physically non-existent shapes and spaces

Transcripts show these gestures and movements do not appear as consciously ‘designed’, but look spontaneous and well integrated with the act of creation. They had clear roots in the thought process that resulted into in creation of objects/spaces. Obviously, individual differences do exist in the way movements were produced. Yet they display internal consistency.

Such occurrences are also true of most speaking gestures. The creators seem to develop the rules of their own and sometimes improvise on the spot. Such individualized gestural behavior is also common in live music during concerts. Yet, there is a difference between gestures in music and in design.4

By revisiting events in the experiments reported in the earlier posts, we plan to understand these spontaneous gestures and body movements of the designers and architects during the act of creation. Because there are differences in the nature of the assignments handled as well as in the way the designers and architects used gestures and movements, it is better to discuss them separately.

Revisiting SP’s actions: Casserole case

Let us look at SP’s performance under experimental condition one, where he was blindfolded. (See ‘Part II: Reflecting on gestures as design behavior’ in this blog series) Video transcripts show large segments, where SP moves his hands to shape the casserole, responding to his evolving ideas and thoughts.5 The transcripts suggest that he was describing the shape to himself and experiencing it. But there is much more to these simple actions.

The actions dealing with the shape issues show close correlations between the decisions dealing with evolving image and the accompanying gestural kinesthetic sequences. (See video 1 and 2) SP’s palms and fingers are actually seen mimicking the shape in 3D space as truthfully as he could. (See also video 4 in the next section)

Video 1 : The segment defines the overall shape. Watch how the palms and fingers define the curved surface. The relative angles define the angular relationships of surfaces with each other.

Video 2 : In the later part, the segment discusses complex detailing, where the legs are conceptualised to be folded-up during packing.

On the face of it, SP seems to be mentally shaping the evolving visualization of 3D geometry as an image in his short-term memory. This mimicking of the shape with hand gestures perhaps ensures a close partnership between, 1] sets of sequential (gestural) kinesthetic actions in 3D space, leaving traces of motor system in action, 2] the image in the mind’s eye, which he continues to modify, visualize and consolidate through visual system. Both processes seem to happen simultaneously.

Accurate recall of form of the object

SP could repeat identical movements while revisiting the feature for modifying at different points of time during designing, as well as during final description of the shape created. So accurate was the description of his final image and reproduction of hand movements and gestures that shaped that image, that even other designers could decode them accurately and reconstruct the casserole and bowl shapes from the video. The accuracy in performance is surprising.

Holding a stable image in the mind’s eye as well as reproducing gestures that shaped the image with surprising accuracy is not a trivial fit, particularly considering that the gestures were spontaneously produced on the spot, they were shape dependent and were not practiced, nor bound by conventions.

The visual processes and motion processes are working in sync and together they are responding to the emerging ideas and thoughts. The close synchronization and partnership between the two systems seems to have led to stronger and stable creations that you could recall and modify at will.

Revisiting architects’ actions

Architects, when blindfolded, moved in the real world space and used gestures differently. The large sizes of the enclosing wall envelopes and other architectural elements were beyond the grasp of their hands. So, they figured out the way to compensate. Two of them visually built spaces by walking around, as if they were on a real site. They developed the spaces around themselves and used body movements to create the layout and feel the scale of the spaces.

It appears that they chose to rely more on the visual system to generate the walled enclosures in the mind’s eye. Simultaneously, they depended on body movements and the motor system for the geometry of the layout and to scale the spaces. To indicate relative positions of features and walls, they selectively used gestures and virtual locations of their body, on the virtual site that they visualized. In such cases, gestures played a secondary role, more as a complement to the body movements.

In two out of four architects, their physical movements in the hall could be mapped to the layout of spaces they were creating in their mind’s eye. These physical movements in the hall could have left interlinked memory traces. Transcripts show that their movements match the walkthroughs in space when they were creating in their mind’s eye. That is how they could effortlessly move in and out of these built spaces, ‘feel’ them, and to get the ‘view’ from the position they visualized they were standing at. This explains why, when asked what their built-form looked like, they raised their head and even stepped back to ‘see’, though the image was only in their mind’s eye! (See video 3)

Video 3 : The architect was asked to describe what his building looked like. While answering, he took a step back, looked up as if he was scanning the face of the building.

Accurate recall of architectural design

As a part of protocols of the experiment, the architects were asked to describe their final solutions after they completed their design. Some preferred to walk the talk while describing; and these walking movements were identical to what they had performed when they conceived these solutions. This also explains why during descriptions of the final solutions, two architects could walk accurately through the spaces that they had created. Another architect when asked, could walk back to the virtual entrance without error.

None of these performances are trivial. Probably, their actions may have led to strong visual and motor memory traces that they could fall back on.

When gestures? When body movement?

Architects and designers appear to spontaneously produce gestures and movements when they are a convenient support in 1] Development of the image in the mind’s eye and/or operating on it, 2] Conceptualization of enclosure spaces around them, 3] Exploring spatial relationships of elements and scaling the shapes and spaces.

Preference to depend on visual system, and use or not to use motor system (gestures and/or movements or mix the two) depended on individual capabilities, design context as well as the complexity of the design assignment or its details. It also depended on the context of the experiments. (i.e. Sitting or allowed to move) SP’s actions are restricted by the fact that he was sitting with a table in front. Besides, his design assignment was also a tabletop object and so, within his grasp. He used hand gestures to shape all the features.

It is interesting to note that the choices between the gestures and body movements were largely interchangeable. There is also sufficient evidence of effortless switching between the two. For example, architects, who generally walked and used body movements, could adopt quickly to the use gestures, when they found this efficient. For example, when the idea of entrance demanded a sculpted shape and flow of surfaces, she used the hands and palms almost the way SP used them. (See video 4) Similarly, some occasionally went back to their traditional methods and scaled down the creations to see the built spaces as a small table-top model, being inspected from top and even shaped it using gestures to match their evolving thoughts. (See video 5)

Video 4 : The segment where the architect shapes the entrance with her palms, as if she is sculpting it by hands. The limitations of the grasp are clearly visible.

Video 5 : The architect switched to treating the creation as a scale model that she visualized and partly shaped it using gestures.

Exception to the rule

Though most of the designers and architects chose to use gestures, to construe that they are essential part of the thinking process may be too hasty. As we saw earlier in the context of mental rotation, individuals either use visual system, favour the motor system or a mix of the two, to perform such tasks. They choose what they consider appropriate for the task or what they are comfortable with.

Amongst several experiments conducted, there is one unique example in this series where the architect sat on the chair throughout, with hands on the hand rests of the chair and developed and spoke of his ideas without any gestures or body movements. He produced quality work and was as detailed as others, who used gestures. He walked through the building virtually, describing people populating the spaces, the ambiance of light and shades and so on. He seems to have depended on visual system completely. When the experiment was over, he was asked about it. He replied that he could have used gestures, but was able to visualize it clearly in imagery equally well. He did exercise his choice to depend on visual system.

In all my initial research writings on these experiments, I explained gestures and body movements by referring to them as ‘thinking with body’. Later, I hinted to its roots in hinted of spatial intelligence.6 I had no clear idea of their relationships with evolving thoughts and ideas. It was during the writing this blog, that I realized the full force of spatial intelligence perspective. Later, I realized that embodied cognition and embodied design offer a even more effective framework to explain why the creator’s gestured and moved their body. If we look at these events and actions through this new lens, lot of events and actions are easily explained.

These are of course post-facto explanations and there are some speculations involved. But, I thought they are intuitively appealing. However, it is best leave it to the cognitive scientists to explain how and what prompted these actions.

Summing up

Have we answered these questions that we started with?

How could designers corporally interact with a virtual, physically non-existent object that they created in their mind’s eye? And that too, with surprising accuracy?

We now have reasonably clear explanations to these questions. In doing that we have also identified several unique, non-trivial and surprising performances of the designers and architects that need a special mention. We have listed them below.

Non-trivial discoveries in design behaviour

1] Our results of the experiments reported so far seem to support the idea that the visualization of objects/spaces by the visual system was assisted by motor system that contributed to creating, altering and maintaining the continuously transforming shape and spaces. These two linked systems seem to lend stability to the image and allow the designer to revisit them at will, and interact with them with accuracy.

2] Video transcripts give a feeling that the creators were treating the creations as if they were real events unfolding in front of them. There is sufficient evidence of the creators being present in the situation and taking decisions, accounting for the location and orientations of the features. Because of the visual and motor systems working as partners in the process of evolving and modifying objects emerging in their mind’s eye, designers seem to accurately and almost effortlessly, interact with these nonexistent entities using their gestures and body movements

3] Creators are able to recall and reconstruct the current states as an image with a fair accuracy. They could also describe their images in details with certainty. This suggests that the images that they recall in their mind’s eye are relatively stable at any point of time. Even the physical actions that shaped these creations could be recalled and repeated without error.

It appears that the entire history of the gradual evolution of these shapes and layout spaces is available to the creators.

4] The actions like gestures and movements of the body accompanying the speech strings were so accurate that could be decoded by third party to understand and reach the final design in the creator’s mind.

None of these are easy to explain, but an attempt is worthwhile. The reasons not only include the cognitive aspects, but go beyond. So, in the next post we will attempt this.

Preview of the next post

To effortlessly perform these non-trivial actions at different points of time in the design process is a surprising fit. We have tried to attribute this to stability to the otherwise fragile mind’s eye images.

Can you attribute the performance in these non-trivial tasks directly to the linking of visual and motor systems at a cognitive level? Or are there also other factors in these actions?

To perform these actions at different points of time in the design process is a surprising. We have tried to attribute it to the duel encoding that may have given stability to the otherwise fragile mind’s eye images. In the next post, let us explore other answers to this. Though we can never be sure, based on the evidence before us, we can at least come up with some conjectures.

Notes and references

1     Pande, P., & Chandrasekharan, S., (2017) Representational competence: towards a distributed and embodied cognition account. Studies in science education, 1-13

2     In many routine tasks, along with visual system, the motor system (movement memories) is also primarily used during learning as well as in performance. Most common example is the reproduction of our own signature. While signing, the flow is monitored by visual system as well as by the motor system. They have to work in sync and as partners.

Another example. In traditional method of learning to write, the children are given learning templates that indicate starting point/s and the direction of pencil movement. They are also encouraged to pronounce the sounds. The directions on the template control the final appearance of the marks on the paper as well as the correct movements of the hand. The practice also ensures creation of movement memories.

Video 6: Child practicing script. The appearance of the letter is controlled visually and the movements by the instructions given on the template. (video courtsey Santosh Khirsagar)

That explains why, little children can play the game of letter recognition by writing on each other’s back with finger. These are clear case of cerebral and corporal working together.

3     Creative partnership, where gestures and movements are based on convention or tradition

In classical dance, the gestures and body movements are directly responsible for the primary visual effect. So, the visual and the motor systems work in sync. The planned visual effects are achieved through ‘pre-designed’ body postures, movements and visual expressions. The freedom to evolve and change is within the framework of tradition and convention. The partnership has to be consciously and painstakingly learnt through practice till they are well synchronised.

Modern dance has the freedom to innovate new gestures and movements. Choreographers of modern dance and street play can explore new creative directions. Because they are driven by creator’s thoughts during conceptualiaation, they are closer to our experiments in design. They balance the cerebral and the corporal.

Closer to our example is when the modern dance performance is being choreographed, particularly modern dance. We could have included such cases to balance the findings. However, we are forced to exclude them from the current discussions because we did not have the opportunities to study the practices and actions involved during choreography.

4     In live music, the singer produces gestures spontaneously. The listener is aware of the gestures and uses them to understand and appreciates the performance. He may even synchronize his bodily responses to these gestures. In design, the final creation of objects and spaces stand independently and do not even offer clues to the gestures that were used to create the object. They play no visible role in the appreciation of the final output. In the act of design, gestures act more like backstage tools.

5     That the transcripts also show different kinds of gestures and all the gestures do not necessarily mimic the shape. For classification of these gestures refer to reference 6 below.

6      Athavankar U. A., (1999) Gestures, Imagery and Spatial Reasoning. In: Garo JS, Tversky B (eds) Visual and Spatial Reasoning. Preprints of the International Conference on visual and spatial Reasoning, (VR 99) MIT, Cambridge, June 15–17, 1999, pp 103–128.

Advertisements

Part III : Is design a corporal act

In the last post, we viewed the act of design as a cerebral act. We also touched the role of intelligence, particularly spatial intelligence in design problem solving. We tried to explore the role the gestures and body movements play in design problem solving. Developing this idea further, we cited several facets of spatial intelligence from the protocols of the experiments. However, visibly rigorous gestures and body movements suggested that one must also go beyond spatial intelligence to explain designer’s actions. Let us reword the earlier questions,

Why did the designer move hands in the air to mimic the shape he was creating? Why did the architects and designers physically move through the mental spaces that they created in their mind’s eye?

Are there more effective explanations to gestures and body movements in the previously reported designer actions?

Part III searches for alternative theoretical framework that directly confronts the issue. It treats design as a corporal act that supports and sometimes dominates the cerebral act, i.e. the body and its movements take part in understanding, reasoning, judgment and in conceptualizing of idea in problem solving. It is called as embodied cognition, a framework that I realized design community will more easily identify with. This theory draws from many fields like sports and music, to present more convincing answers to why designers used gestures and body movements.

Both the theories show advantages of getting the body involved while thinking, particularly when innovations are spatial in nature. Let us take this perspective to argue our case. Keeping embodied cognition framework as a backdrop the post will move on to explain some of the design(erly) behaviour in the experiments reported in earlier posts.

Design as an embodied cognitive act

The discussion in this post extensively relies on the new framework proposed by embodied cognition. To understand the nuances of this framework, it is best to contrast it with its predecessor, the classical information processing model. With a short introduction to the relevant features of the classical model, we can return to embodied cognition.

Classical information processing model

In our last post, we treated designing as a cerebral act. This is consistent with the classical model, which focuses entirely on cerebral processing. Designer is viewed as an information processor, who manipulates symbols during problem solving. The model suggests that all cognitive processing is (or is best) done by neural processes inside the brain. The influences of computer model are visible in its focus on working memory load, long-term memory, information storage and finally the resultant action produced.

In the classical model, the external representations (like sketches or diagrammes) are seen as help to offload information. External representation has a limited passive role of acting as a vehicle or a transmission media that carries information for processing.1  It is assumed that the cognitive system passively receives information from the external representations.

On the other hand, embodied cognition takes an opposite view. So, after a brief introduction to the theoretical framework on which these new concepts are based, we will explore if it offers better insights into the observed design(erly) behavior in our experimental data.2

Understanding embodied cognition

Embodied cognition theory considers that the cognitive processes are distributed across internal and external representations. Interactions with external representations (like pictures and sketches) are the central process driving meaning and understanding.3 (Note 3) Embodied cognition concepts propose that many features of cognition are shaped by the characteristics and aspects of the physical body and these influences have significant causal role in cognitive processes beyond the brain. Embodiment assumes that what happens in the mind is depending on properties of the body, such as kinesthetics.

It is also argued that that the brain and the cognitive processes are developed for action, particularly motor action. So, the body and the motor systems are closely involved in most cognitive operations. It also recognizes that the cognitive processing is distributed across internal and external representations.3 (Note 3)

In this framework, the local environment is seen as an actual extension of the body’s cognitive process that is explored through active perceptual and motor actions. Ideal way to understand the concept is to imagine that if we had different physical attributes and capabilities, we would have understood and responded to the world differently. There is a conceptual shift from the classical model in suggesting that cognitive process also encode sensory-motor aspects during interaction. As we will see later in this post, both are key to understanding the role of gestures and body movements of the designers. Even behavior is seen as a result of embodied cognitive responses. Examples will help to understand the ideas and concepts.

Human response to musical renderings is an excellent example. In this approach, spontaneous bodily movement (moving hands, head and torso) contributes to musical meaning formation. Indeed most listeners do respond to music with body movements. On the other hand, in disembodied approach (i.e. cerebral) the focus is on perception and analysis of musical structure.

Take another example. In spatial task like rotating of object in space, most individuals use visual processes by imagining rotational movements in the mind’s eye. However, some complement this by using hand /palm gestures to mimic the movement of shape in space.  They favour motor processes to perform the same tasks, often with higher overall performance. It points to the influences of the motor system on high-level reasoning and cognition.

A more convincing example is the use of mental abacus by experts. Some expert abacus users rely heavily on visuo-motor operations by implementing the same sensory motor actions internally in their mind. Others use the same action covertly leading to gestures as if they were using a physical abacus. In mentally playing an imagined musical instrument, similar covert physical actions are sometimes visible. These examples from divergent areas convincingly explain the concepts of embodied cognition and active bodily involvement.

Embodied creation

Creative endeavors in arts recognize the role of bodily involvement and responses. The embodied cognitive responses have important role to play in creations. For example, production of live music integrates the corporal and the cerebral act. Most singers and musician spontaneously produce gestures and body movements when they sing or play a musical instrument.4 (Note 4) It is difficult (if not impossible) to see production of music as a disembodied act.

The spontaneous partnership of body in learning as well as creations is not unusual.5 (note 5) We also extensively use spontaneous gestures when talking on telephone, knowing fully well that the person on the other side is not able to see them.

How does embodied cognition explain design(erly) behavior? How do we then explain the use of sketching? And benefit of gestures and body movements accompanying mental imagery? All these are in someway used to represent the objects and spaces in our experiments. We will deal with these issues later in this post. Based on this exposure to embodied cognition framework, let us revisit the  discussion on the results of these series of experiments.

Corporal acts and creations

Like embodied music creations, the tasks of shaping and locating elements in space are embodied design actions. Such events are referred as embodied imagination and are derived from the idea of embodied cognition. They are based on the premise that the actions of the body can (and do) participate in the development of thought and ideas. Embodied design explorations rely on be gestures or movements of the body and visualization in the mind’s eye.

Embodied imagination explains how, in blindfolded conditions, SP generated and manipulated the objects in condition 1, and architects conceived spaces around them in condition 3 (mentioned in the preceding post). Designer’s moves and reflections in these experiments show embodied imagination in action. These are ideal examples of embodied spatial creations.

Gestures and movements as corporal acts

Watch SP’s hands during creation of the casserole shape in condition 1. They move to create, refine and perfect the shape of the object. Several processes seem to be at work here.

First, SP’s gestures seem to complement the visual system deployed in the development of his visualization i.e. the visual system acting on the internal mind’s eye display. He could then creatively react to the display. Second, the fact that SP developed the virtual shape in front of him at a fixed distance shows that his body played a role in locating and scaling the virtual shape in front of him.

Second, the embodied creation demands being there, virtually or physically, to create objects and building ‘things’ and locate decision with respect to the body. Though SP’s visualization was virtual, the protocols and the design decisions seem to suggest that the designer was present physically in these visualizations. SP was sitting on a chair and. sculpted the object in his visualization as a tabletop creation in ‘front of him’ and scaled it using ‘his hand movements’. The actions were within a spatial range that ‘his grasp’ permitted. See video 1 below.

 

Video 1: Everything that SP created was small enough to be within is grasp, so he could shape it with hand gestures

In the next series of experiments (condition 3), the architects were let into a large hall and blindfolded. The responses were dramatically different. Corporal involvement became more intense. Architects chose to physically move in the hall and imagined as if they were on a ‘real’ site and the environment. Their physical movements as well as navigation (embodied cognition) seem to have contributed not only to their understanding of the spaces but also to creatively altering of the spatial arrangements. Architects immersed themselves in the space they created and almost ‘walked and lived’ in the visualized enclosures. (See video 2) In virtually constructing the built spaces, they appear to have relied on the visual as well as movement memory!

Video 2: The architects were so immersed that appears as if they ‘walked and lived’ in the spaces that they created.

Third, in describing design actions, they referred to design, as well as site features with respect to their bodies. So it is not surprising that the transcripts are full of references like ‘in the front, on my left/right, behind me’ and so on. (See transcripts in figure 2) It is impossible to make sense of the spaces that the architect’s created without accounting for the their body, its location and orientation.6

Video 3: In the immersive steps, the architects were able to tell where they were when asked.

Figure 1

Figure 2: Terms like my left, right, in front, occur extensively in the transcripts.

Is design a cerebral act or a corporal act?

There is much that we could learn from both the standpoints. As a designer, I feel that the act of design switches between the cerebral and corporal along the different stages in the design process. Early analytical studies are cerebral and when the action moves towards synthesis, the designer seems to frequently switch between the two. Frankly, it is best left to the cognitive scientist to solve the puzzle.

Summing up

Preceding post (Part II: Reflecting on gestures as design behavior) treated design as a cerebral act and focused on spatial intelligence and its role in design problem solving. This post viewed designing as a cerebral act and looked at it through the classical information-processing model. It tried to explain designer’s gestures and body movements as a cerebral act.

The current post pursues the same questions that we started with. ‘Why did the designer move hands in the air to mimic the shape he was creating? How could the architects and designers physically move through the mental spaces that they created in their mind’s eye?’ This post explores new theory that offers a better explanation to designer’s actions.

The theory that is relied on is embodied cognition and in many ways it takes an opposite standpoint. It claims that what happens in the mind is dependent on properties of the body. The local environment is seen as an extension of body and its cognitive processing.

The theory also proposes that the brain and cognitive processes are developed for motor actions and they extract information through active perceptual and motor actions. The physical actions like gestures are seen as complimenting the visual processes. Through several convincing examples, it shows how the body actually participates in thinking as well as development of ideas. This post, (as well as the subsequent posts) focus on this connection between cognitive processing and motor actions.

Representations, both Internal (mind’s eye) and external (sketching), play key role in this theory. The interaction with representations is considered central to deriving meaning and understanding. The current post views design behavior as a result of embodied cognitive actions. The post also touches the extension of this idea to embodied imagination and creations as a corporal act. It suggests that such corporal acts observed in the experimental tasks may have influenced thinking and dictated design decisions.

Preview of the next post

The next post will focus on the reasons for accuracy of designer’s interaction. It will address the question,

How could designers corporally interact with a virtual, physically non-existent object that they created in their mind’s eye? And that too, with surprising accuracy?

The answers will be somewhat speculative. It is possible that the effect of mimicking the physical characteristics of the shapes and the spaces had something to do with accuracy with which they interacted with the objects and built spaces during the creation as well as the recall in final description. What ensured the vigorous and accurate interactions with physically non-existent objects?

We will deal with these important questions in the next post. It will also argue that the choice to use gestures is with the designer and his decision is influenced by useful contributions that movements make to the evolving thoughts.

Notes and reference

  1. Interestingly Robert McKim has also talked of sketches as vehicle of thoughts. See McKim, R., (1972) Experiences in Visual Thinking. Brook/Cole, California
  2. For more on embodied cognition refer to Wilson, M., (2002). ‘Six views of embodied cognition’. Psychonomic Bulletin & Review. 9 (4): 625-635
  3. Pande, P., & Chandrasekharan, S., (2017) Representational competence: towards a distributed and embodied cognition account. Studies in science education, 1-13
  4. Even in the radio era when the musicians could not be seen, the gestures were not absent, but were less conspicuous.
  5. There is also conscious production of practiced bodily responses, as part of learning a spatial task. Most sportsmen practice their playing actions in the air, to reinforce their embodied responses, but their relationship with thoughts is slightly different.
  6. Location of objects or elements with respect to the body is common and is considered almost innate. George Lakoff points out that human movement revolves around standing erect and moving the body in an up-down motion. Humans innately have these concepts of up and down. Lakoff and Johnson contend this is similar with other spatial orientations such as front and back too.

 

Designers imagine, touch and walk the talk

This post is third in the series three posts that explores potentials of mind’s eye by blindfolding designers and architects and asking them to develop their design proposals, completely relying on internal resources and visualizing in the mind’s eye. They extensively used gestures while solving the design problem. On the other hand, in my earlier posts on teaching sketching for designers, I had focused extensively in getting the whole body involved in the act. I was interested in the relationship between the body movement and the thoughts.

Leaving the architects standing

To broaden the scope, I treated hand gestures as only a part of the larger aspect like conscious moving of body. I decided to refocus on the role that the body can play in solving design problem. In all my previous experiments, architects sat on a chair and then they where blindfolded. This had actually constrained their movement of the body. In these experiment, they were then blindfolded and left standing in a middle of a large empty hall, wearing a wireless collar mike in. I had hypothesized that they will move around and use their movements in some constructive ways.

Experiment design

This time we invited four architect volunteers with two/three years experience after graduation to participate. They were given a site plan with no indication of what functions to accommodate. Site given was rectangular, with one curved corner along the access road. It had gentle contours. The plan indicated trees (some to be preserved) and location of existing storm water drainage. The architects were asked to remember these features and verbally recalled them before they were given project requirements.

The design problem given was a hangout space with a cafeteria for student community on the campus. It also had a space that could be used by the students to display outcomes of their hobbies. The detailed account of the experiment and the results were published.1,2

The questions addressed were,

Will the freedom of movement impact visualization strategies?

Will the architects move their body to ‘feel’ the space that they develop in their mind’s eye?

If yes, can these movements be mapped to the spaces that they develop?

The design of the experiment ensured that the architects had to completely depend on their mental imagery and internal resources. We were exploring how the architects would use the freedom to move constructively and take advantage of the space in the hall. We were also looking for changes that may occur because the built spaces were much larger than their body and they have to be designed from inside as well as from outside.

Indeed there were visible changes in the strategies that architects used in visualization and in solving the design problem. They constructed a site in their visualization intervened in it creating architectural spaces around them, even walked around in the spaces created and when necessary altered them. Looking back, it seems like a peculiar interaction. The use of gestures was differed from the way Industrial designer used them. The difference is that the built spaces were too large to be physically sculpted! Their gestures and movements had different functions now. There were differences in the way four architect approached the idea of using the space. The visualization strategies also differed. We plan to discuss some key observations like 1] their presence on the virtual site that they visualized; 2] the strategies that they evolved to visualize the spaces and details; and 3] the way they interacted with their creations. The interested readers could refer to a detailed paper on this experiment.3

‘Presence’ on the site: Virtual or physical?

The video suggests that all the four architects were also seeing themselves virtually standing on the site, visualizing built spaces around them. How can a person be on a virtual site? This might sound strange, but it is true. Yet it needed to be proved beyond doubt by relying on concrete visible evidence of the designer was bodily present on the virtual site. The proof came from several sources in the video data and the transcripts.

We analyzed the transcripts to locate words and phrases that contained frequent references to self, such as ‘I am’, ‘on both sides of me’, ‘front of me, left side of me’, ‘on my right side’. (See figure 1) Surprisingly, most architects dropped references to north and referred everything with respect to the body, its location and orientation. In all four transcripts, there is only a single isolated reference to the North direction. (See video 1) In practice, North direction is critical in architectural practice and comes up often in conversations. Similarly, heat gain, ventilation and rain directions are worked out with North in mind.

 

Figure 1: Shows how concurrent speech that reveals the architect being on the virtual site

 

Video 1: There is only one reference to North direction in the four case studies is unusual. Their virtual presence on the visualized site was so dominating that most of them did not refer to North.

Occasionally the architect was asked a question ‘Where are you now?’ When declaring their location, they relied on the virtual space. They declared their locations with references to the features of the site or of new design that they were developing in their mind’s eye. (See video 2,3) Besides, when asked to go back to a specific feature or a location in the proposed design (like entrance), they made finer adjustments in their movements to reach almost the exact spot in the physical world! (See video 4) Such events further supported the idea of presence on the site. The boundaries between the physical world and virtual world seem to have blurred.

 

Video 2, 3: The architects were asked during the blindfolded session “Where are you now?” Observe the answers. All the architects were on the virtual site that they visualized and they were clear where they were located on it. (Architect in video 1 also ‘knows’ where he is.)

Video 4: Watch the architect making finer adjustments in his locations, before finally declaring where he is.

This presented sufficient evidence of the architects being on the virtual sites that they visualized in his mind’s eye. But we still need answers to the question,

Why were the architects on the virtual site? And how did it contribute to design decisions?

Beyond presence

It was not a mere static presence that could be observed. They walked up and down; exploring the site and conceptualizing built spaces around them. They seem to be imagining themselves constructing and using the spaces that they constructed. Most started with developing a built space from inside first but when required, they came out of the building to see what they had created. (See video 5)

Video 5: When working on the outside of the built space, architect steps out spontaneously.

How does one get valid clues to what they were seeing and experiencing? We mapped the combinations of gestures, body part movements and speech strings from the transcripts, second by second, and plotted the contents with respect to the site plan. (See earlier figure 1) The combined rich descriptions typically indicated site landmarks and new built features. On the other hand, simultaneous gestures, hand, neck and body movements showed the locations of these features. (See figure 2) This helped us reconstruct and map what they were seeing at that a given point of time and how it transformed. (See video 6)

Figure 2: Shows how concurrent speech with gestures and movements recorded and later mapped to reconstruct experiences.

 

Video 6: Watch how the site moves as the architect turns. With the site, all the built features also turn.

Virtually being on the site allowed him to operate in the ‘virtual’ space that he could build on, alter and experience from inside and outside. This was lot quicker than if had chosen the normal path of sketching these out and altering them. Watch the videos carefully. Their visualizations were far quicker to generate, lot more pliable and could match the speed of the evolving thoughts, ideas and fantasies.

Visualization strategies

The architects had to adapt to the unfamiliar situation spontaneously. So, to imagine consistency in the visualization strategies used by the four architects looked unrealistic. Nobody had any training in handling mental imagery. (Unlike sketching, the pedagogic implications of use of mental imagery have not been explored and so it had never been a part of any design curricula) Most often, it accompanies thoughts spontaneously and remains a very personal experience. The architects had to adapt to the new of ways on the spot. So, instead of searching for consistency in visualization strategy, we decided to look for differences between architects.

Architects moved on the virtual site as well as in the physical space, but the speed and the vigour of the movements and lengths covered were different. At times even the purposes of the movements seem to be different. The strategies could be grouped into three classes. Each architect treated one of the strategies as primary, but switched occasionally to other strategies when they found it convenient.

Strategy 1: Visualizing, working and moving in-situ

Two of the architects built the virtual building spaces and layouts around them and comfortably moved within it. (See video 7) They created the spaces and walked into them and altered them if necessary. The distances and spaces around were very ‘real’ to them. When asked to sum up their design, they were able to walk into the virtual built space that they created almost without error. The results clearly show that the two architects had developed a bodily feel for the virtual spaces that they built and oriented their body within these spaces. (See video 8)

Video 7: Architect created a virtual site and built the spaces around him, while pacing up and down. He is clearly developing his ideas by being inside the built spaces.

Video 8: Architects were uncomfortable if they miscalculated the body orientation and corrected it. Video shows one such example.

The visualization, particularly when in-situ, was very accurate. In summing up the design, when the architects were asked to sum up is design ideas, he could walk the same spaces with surprising accuracy. So accurate was this movement, that in spite of the eye mask, when asked to go back to the entrance, the architect carefully moved back and adjusted his steps to reach the correct location in the physical space! (See video 9)

Video 9: Watch architect going back to the exact location that she had planned as entrance. The last bit of adjustment showed surprising accuracy in the bodily feel.

When asked to describe elevation of the built space, one of them stepped out of the virtual space, to ‘see’ the building elevation and the entrance and reworked on it. (See video 10) Incidentally, these are the only gestures that sculpted the shape of the entrance interactively.

Video 10: Architect steps out and sculpts the entrance to the cafeteria. It appears as if she was developing the ideas in-situ.

Strategy 2: Carrying site on the shoulder

Two of the four architects moved into an in-situ space. Other two moved, but not with the same purpose and vigour. Their moving was a free wheeling movement. It had no direct relationship with the visualized space around. It appears that when they moved, they always had site with them. It appeared as if they were carrying the site with them as if it was attached to their body. So, the site moved and rotated, when they moved and turned. How does one prove such a strange conclusion?

Using the combination of concurrent speech strings, with gestures, neck and body movements, we could map their image that they were seeing in their mind’s eye. These dynamic map shows that when the architect moved or turned, the site also moved and turned, as if they were carrying it with them. (See video 6 shown earlier) It is not clear what was the advantage of this strategy to the designer, but they do look comfortable. Perhaps they were not aware of their own strategy of site moving with them.

Strategy 3: Shift over to a scale model

Some of the architects occasionally shifted to another strategy. From the body movements, gestures and speech, it appeared as if they were working on a scale model. They were outside the site now and perhaps viewed it from the top and created built spaces. Their hand gestures either showed location of the spaces or described shapes (often site contours, structure, roof) Gestures suggest that they are perhaps working on a small-scale model floating in the air, at a short distance from the body. (See video 11) This is closest to the earlier discussion on industrial designer (SP) working on a virtual model in front of him.

Video 11: The gestures suggest that the architect is working on a model close to her body.

General Observations

By allowing the architects to move around, we made the canvas larger. It revealed many new things. We derived three major visualization strategies from the data. For most, though one of the visualization strategies remained as a primary strategy, they were able to effortlessly switch to the other strategies when required. In spite of differences in strategies, all of them were able to solve the problem effectively.

With the broader canvas, the body movements and gestures were used to perform newer functions. They pointed out locations and directions of features that designers was conceiving or referring to. They indicated shapes and in rare case, they even sculpted the shape and the contours.

We also realized that the concurrent gesture + speech combination is so rich in information that it was possible to map what they were seeing in their mind’s eye using their body as a reference. This material acquired richness and revealed lot more of what was going on in the designer’s mind and what he was visualizing. The speech + gesture combination clearly revealed that the architect built spaces, while imagining a virtual site in his mind’s eye. He also used his body as a reference to locate things around.

In my earlier writings, I attributed this to ‘thinking with body’. Reflecting back now I found new explanations that closely match how and why architects and designers performed the way they did. It would be worth going to these areas by considering all the findings of the three articles together. We plan to do this in the next post.

I have come to believe that one of the important characteristics of designerly thinking is the inner urge to solve problems through the design expertise. The attitude can be captured as ‘whether there is a client or not, I want to, and need to solve the problem’. Designerly thinking is based on ambition as well as the excitement solving the problem, a point that writings on design methods miss completely. So, the constraints, like not allowing usual processes like sketching, blindfolding or left standing in the middle of the hall, don’t seem to bother them. None of them complained about the constraints imposed. They took the challenges in their stride, and in fact adapt to them, by spring back with spontaneous alternative strategies and approaches.

Sum up

This article is third in a series of posts that presented findings of the experiments on designers and architects, when they were blindfolded and asked to design. Sufficient evidence was presented in the previous two posts to conclude the most of them were able to handle design problems and come up with solutions and that too with amazing dexterity. This post focuses on the next objective, the role that body and its deliberate actions can potentially play in supporting visualization.

This time four architects were given a design problem and were let into a large hall. They were blindfolded, wore a cordless collar mike and were asked to work on an architectural project. The experiments were video taped and all the speech strings were transcribed for detailed analysis. The expectation was that they might use the freedom to move around while thinking of the solutions and this might impact their visualization. Indeed, they spontaneously responded to the new situation with different visualization strategies.

There was sufficient evidence in their speech as well as in the body movement and gestures to show that they were present on the site visualized in their mind’s eye. When asked, they would declare their location in the built form that they were developing. So, it is not surprising that they located everything around them with respect to their current location and orientation on the virtual site. The way they operated, even this virtual presence looked almost real to them!

Some of them used a strategy where they built the site in their mind’s eye, developed their ideas as virtual built forms, manipulated them in their visualization, but actually moved and interacted with their creations through actual physical movements in the real world. So accurate was their mapping between the virtual and real world that, when asked, they could physically walk back to the exact location in the built form and it would also tally with the physical location in the hall!

Another popular strategy was when they moved they carried the site with them. The site turned when they turned. Though the physical movement did not impact visualization, it was probably required to keep them active in the 3D space on the virtual site. Last, and perhaps the least used, was a conventional strategy of working on a small-scale model like situation. The fact that it was rarely used is surprising; as most of them are used to working with the scaled versions of their creation all the time, and that is how they are taught to develop ideas.

This data showed that body movements were far more pronounced and gestures played a supporting role. There were rare occasions when the gestures were used as tools to model the idea. Perhaps, the bigger size of the built form and the fact they had worked primarily from inside the spaces may have made it difficult to use gestures as shaping tools, the way industrial designers could.

The post concludes by listing the architects’ actions, visualizations and particularly the movements in the physical world. In the next post, we will review the findings of all the three articles in this series through the theoretical framework of spatial intelligence and embodied design.

Preview of the next post

The next post will take a bird’s eye view of experiments on imagery. We will address following questions.

While visualizing, how do designers benefit by use of body movements and gestures?

Why do they feel it necessary to move the body?

Does it support spatial decisions and design thinking?

The theoretical support for the mental events in this series come from work on forms of human intelligence and from findings in cognitive psychology. We will touch areas like 1] Spatial intelligence as well as; 2] Embodied cognition (We touched this in earlier post “out-of-box ideas to teach sketching”) as well as its spin-offs like embodied design and imagination.

Notes and references.

1 Athavankar. U. (2008) Exploring the boundaries of spatial intelligence, Conference on Research and Training in Spatial Intelligence, Evanston.
2 Athavankar. U., Prasad B., Guruprasad.K., Patsute R. and Sharma S. (2008) Reaching out in the mind’s space, In Design Computing and Cognition ’08, Eds. Goel A., Gero J, 321-340. Springer.
3 Athavankar, U., (1999) Gestures, mental imagery and spatial reasoning, In Visual and Spatial Reasoning, MIT, Eds Garo and Barbara Tversky. 103-128. MIT