Product semantics

‘Product semantics’ is the next series that is planned to be uploaded. Several papers and publications on this topic are already available on http://www.academia.edu. I hope to reorganise these into a series of posts in a non-research language.

I hope to make ‘Product semantics’ active as soon as my series of posts on design thinking and visualisation are over.

This could be around October or November this year.

Uday Athavankar

July 1, 2018

Advertisements

The idea of musing. No. 1

I hope to add some of my stay thoughts on design and other issues in this category. The writings are likely to be on various aspects of design and design related areas. I thought this was the most appropriate title for this category.

It will be a few more days before I put up my first post.

Uday Athavankar

July 1, 2018

Part V: Explaining designer’s nontrivial actions

To effortlessly perform these non-trivial actions at different points of time in the design process is a surprising fit. This was discussed in detail in the last post. These experiments were performed on several designers and architects, with varying levels of design experience and with different design problems. To perform these actions and movements spontaneously and effortlessly is not easy.

In the preceding post we attributed this performance to potential linking of the two information channels, the visual and the motor. We referred to this as linked-memory traces. We have tried to attribute stability of the otherwise fragile mind’s eye images to the linking and partnership of visual and motor information channel.

What does linking involve?

This is often referred to as common encoding in literature on cognition. We will briefly touch these ideas in this post. This could perhaps explain why architects and designers were able to perform nontrivial tasks effortlessly.

Linked memory traces?

The fact that the designers repeat identical gestures when returning to earlier decisions of form or layouts suggests that these sequences of gestures were processed as relatively independent motor information channel, similar to the visual information channel, that dealt with the status of the image. It is plausible that these were linked and were readily available for independent as well as simultaneous access again. That is why, on several occasions in the transcripts when discussing any element, SP is seen to be moving his hands exactly in the same contoured paths, coinciding with the path taken earlier. Similarly, some of the architects were able to go back to the indicated locations effortlessly. The conceptualizing and maintaining of the shape/layout in the mind, perhaps used visual as well as motion systems to encode this as separate and yet interlinked information.

We have so far referred to this as linked actions, where two channels, in this case visual and motor information channels, work in sync. Using gestures and movements that mimic the shapes and layouts are examples of visual and motor systems working in close partnership and possibly creating a stable entity in the mind. Though speculative, plausible explanation could be the idea of common encoding creating stable representation that they could recall anytime later.

Common encoding theory

Common encoding, because it is grounded in perception, is close to the idea of embodied cognition. The theory is based on connecting what we see and hear with our motor action. The shared common code links perceptions and actions in a cycle, which is considered as fundamental logic of the nervous system. What is relevant to us is how this close partnership facilitates the events that we encountered in our experiments. It shows that seeing an event activates the action associated with that event. The inverse is also true. Performing an action activates the associated perceptual event.1 We represent observed, executed and imagined actions in a manner that allows us to make predictions. So, for a given action, it supports prediction and anticipation of action outcomes, giving us control on the actions planned. When you conceive the action, you learn what the movements will lead to.

Could we attribute the accuracy in the generation and regeneration of the shapes and layouts during different stages of the design process to common encoding?

The intricate ways the motor actions are linked to visual system appears to suggest that whatever SP and the architects created had the advantage of common encoding. This seems to have also aided later recall of shapes and spaces, when finally describing their creations. We suspect that such an encoding allowed SP and the architects to interact accurately with the objects and spaces during conceptualization.

With the data that we have, this does offer a plausible, though somewhat speculative explanation to creating stable regeneration of ideas and explains the accuracy of designers’ interaction with physically non-existent virtual objects and spaces. Perhaps we should wait for a firmer answer from the cognitive scientists.

The questions that we plan to address in this post are,

Can you attribute the performance in these non-trivial tasks exclusively to common encoding? Or are there also other factors in these actions at play? And finally, what else accounts for the consistency in the performance?

About this post

In this post, we plan to explore other plausible explanations of how these nontrivial tasks could have been achieved and why such a performance was possible? The use of word explore is deliberate. Based on the evidence before us, we can at best come up with some conjectures. With limited current experimental data available to us, we can never be sure.

It is planned to end this argument discussing the question, ‘Was the performance natural and spontaneous?’ or was it because of the influence of the way the experiment was designed? The post will conclude with the pedagogic influence of the findings.

Let us return to the core issue of explaining the nontrivial performance. The explanations are divided into two groups.

  1. Digging deeper in cognition

This group of explanations explores roots in literature 0n cognition. None of these were apparent when the experiments were conceived. However data post facto indicates these possibilities. The group include, think aloud effect? and recency effects in memory.

  1. Other related issues

The group consists of collection of other reasons that have roots more in the way experiment was conceived, designed and unfolded in implementation. It includes, availability of decision logic and redundancy due to internal consistency.

Let us start with the first group.

Think aloud effect?

All of the designer’s actions are accompanied by think aloud protocols as part of experiment design. The effects of think aloud on the results are difficult to see in isolation. It is possible that articulation of designer’s logic as speech strings may be acting as an additional channel assisting image recall and accurate reconstruction. There is some evidence that point to this possibility.

Speech strings (Words), visual information (image), and body movement information (including gestures) are processed individually along different channels and are represented separately.2 But in recall, any one of the channels can be activated independently or both can be activated simultaneously. The concept is referred as duel encoding in cognition. (It differs from common encoding touch earlier)

The chances of recalling a stored item is higher because of the ability to code input in two different Channels. Several examples support the superiority of the duel encoding used in mental representation. Most commonly used example is use of multimedia presentations, which require both spatial and verbal working memory, but aid in superior recall later. Duel encoding has also found support in studies based on PET and fMRI. It seems to have been validated in range of circumstances, if not all.3

Recency effect

As defined in the experimental protocols, all the participants, in a single uninterrupted session, always completed the design. In most cases, sessions were completed in less than an hour. It is likely that the recency effects of the memory would have played a role in aiding the accuracy of recall and somehow contributed to interaction with the nonexistent objects easier.4

We will now turn our attention to the second group,

Availability of decision logic

The evolving thought process motivated all the decisions as well as the interactions. Uninterrupted sessions ensured that the decision logic that generated the physical configurations was always accessible to them to fall back on. They did not have to rely on memory extensively for recall. Known and articulated logic may have assisted in regenerating the shapes/spaces if needed.5 So, it is likely that the fragile images could be regenerated on demand.

Redundancy from internal consistency

Though gestures were produced spontaneously (not overtly planned), they could be understood and decoded by a third party. This would not have been possible without substantial internal consistency in their deployment. These rules were formed impromptu during the session by each individual participant. So, their individual style would have reflected in their actions. However, in each individual performance, there is extensive internal consistency in the way they mesh the visual and motor actions together.

Look at it from information theory perspective. Any rule formation creates redundancy and aids understanding and communication.6 In our experiments, actions in the visual system were supported by mimicking of physical characteristics of the shapes and movement, by hands gestures in casserole design, and by the body movements in architectural layouts. We suspect that the internal consistency used in individual rule-making for production of gestures and movements, may have created redundancy.

Rules create structures, and in this case correlational structure, accompanying the synchronicity and interlinking of visual and motor actions, thus leading to redundancy. Such structures are known to compensate for incompleteness or absence of one of the channel of information, without affecting the performance.7

Some of these explanations could be the sources of accuracy of designer’s interactions with the physically non-existent virtual objects while creating the shapes and the spaces. As mentioned earlier, these are conjectures based on what the data points out. They appear logical, but need a different experimental design and further testing rigor.

Why gesture?

Why did the architects and designers used gestures? There are several possibilities that cannot be ruled out. Could the accompanying corporal actions be because of the creators’ efforts to communicate with the experimenter directly or through the recording? Has think-aloud prompted corporal responses? Is the emphasis on gestures more than usual? None of these questions have easy answers. These are legitimate questions. Let us attempt to address them.

None of the participants were told to use gestures. When they began, they were only asked to design and simultaneously think aloud. After they declared that their respective designs were complete, they were asked and describe and/or sketch their ideas at the end. After they completed the session, the experimenter asked most of the participants ‘why they used gestures? Answers are interesting.

Few said, ‘we just started designing and were not aware of the bodily actions.’ This is not unusual with spontaneous speaking gestures. Some reflected that ‘we could have done without it too’, without assigning specific reasons. In fact, in earlier experiments when architects were seated while designing, one architect sat with no movements of his hands. (The details were reported in the preceding post.) Lastly, only one of them said, ‘I decided to use gestures so that the experimenter will know what is going on in my mind.’

Immersive Experience

Were they conscious of the fact that they were performing? Of course they were aware that the sessions are being video taped, at least for the first few minutes.

Because of the immersive nature of design in blindfolded condition, almost all of them were in the private world of their own. There is enough evidence of their presence in the environment that they created. The designers were fully ‘into’ this environment. Naturalness in their behavior suggests that they had forgotten that they are being recorded. This rules out the deliberateness in their action.

Are gestures critical for 2D conceptualization?

Some of the readers had commented that these findings seem valid mainly when dealing with 3D creations. Do gestures and body movements have advantage in 2D conceptualization?

Frankly, I do not know yet. I did not have the opportunity to work with visual communication professionals. So, it is not easy to give a clear answer to the question. Closest I came to them was when I worked with filmmakers who were asked to develop advertising clips for social messages. These are not exactly 2D creations, at least when conceptualizing the ideas. Besides film-makers, I did some work with ‘designers’ of temporal creations, like musicians, dancers and choreographers.

It will be nice if someone actually works with graphic designers by blindfolding them. I am sure something new and useful would be discovered. It will add to the knowledge base, if it challenges some of these finding.

Pedagogic implications

The ideas indicate potential influence on the pedagogy. Because design involves taking spatial decisions, gestures and movements have tremendous potentials to offer instantaneous support to the evolving creative thoughts during the act of design. Another positive outcome is that these movements help link visual and motor information channels and make them work synergistically.

Duel encoding and common encoding could compensate for the incompleteness/loss of information in the visual system or motor system in some ways. It also tells us, that if some one has a greater control over the visual system and has the ability to focus fully on the visual events, they can exclusively rely on it. For others, depending on both would be a better option.

There are several cultures that discourage use of gestures and body movements. Are they loosing on the spontaneity of gestures available as an ideal opportunity? Perhaps, yes. Else, they would have to learn to focus more on the visual system.

Summing up

The preceding post discussed how architects and designers effortlessly perform non-trivial actions at different points of time in the design process. These included repeatedly interacting with a physically nonexistent objects/spaces, with accuracy through gestures and body movements. To explain such a performance, the idea of partnership and linking of visual and motor information was introduced in the preceding post. The stability to the otherwise fragile mind’s eye images during these interactions was also attributed to this partnership.

This post takes the next step. It answers to ’Why such a performance was possible?’ It explores explanations of how these nontrivial tasks could have been achieved. The post tries to explore roots of such a performance in literature on cognition.

The idea of linking of the visual and motor system and their synchronous operations were explained though common encoding. Not restricting explanations to common encoding, the post then moves on to find other plausible issues that would have bearing on the performance. At the moment it best to treat these explanations as informed speculations. The experiments were never designed with these intentions and so; the data is not adequate to come up with firm conclusions. At best these can be treated as conjectures. These explanations are divided into two groups.

The first group of explanations digs deeper in literature in cognition. In spite of known limitations of short-term memory, how were they able to achieve non-trivial tasks? The post lists some possibilities.

1] Think aloud during designing was part of the experiment design. This in some ways articulated the thinking that drives design, and in turn the gestures and body movements.

Seen from a cognitive perspective, think aloud speech strings (language information), images (visual information) and body movements (motor information) are processed individually as different channels and are represented separately. This is referred as duel encoding and it may have contributed to the nontrivial results obtained or it least in making the task easier. Could they have assisted each other? It is known that the chances of recalling a stored item is higher because of our ability to encode input in two different Channels.

2] By design the experimental session were necessarily uninterrupted, the result could be possibly explained through recency effects in memory. The items are known to be more easily accessed because of there recent origin.

These two factors contributing to the results, at least partially, cannot be denied. None of these were apparent when the experiments were conceived. However post facto, the data indicates these possibilities.

The second group consists of reasons that have roots more in the way experiment was conceived, designed and conducted in implementation. It includes the following.

3] The decision logic was always available to the creator. This would have allowed the creators to regenerate shapes at will, without depending too much on visual memory.

4] The recall tasks also would have been facilitated by the redundancy. The internal consistency in the spontaneous gestures and movements of each creator suggests that they have developed rules for production of gestures and movements. Even though they may be unique to individuals, such consistency based on rules is known to create redundancy making such tasks manageable.

The post concludes these arguments by addressing the question, ‘Was it possible that the gestures and movements were a deliberate act by the designers?’ The answer relies on the short and direct post experiment discussion on respondents’ views on this. Their views substantially rule out this possibility. Besides, of the immersive nature of the sessions rules it out further. There is enough evidence of their presence in the environment that they created. The respondents were fully ‘into’ this environment. This rules out the idea of deliberate action even more.

The post ends with a short discussion on the pedagogic implications of these findings.

Preview of the next post

The next post will take an overall view the design act. By keeping the focus on visio-spatial decisions in design, it traces its roots to the pioneering work on visual thinking by Robert McKim in 1970s.

The post treats designer as a information processor, and tries to model the design act as information flow and actions. In this ‘in the head’ framework, designer can only ‘work with’ and ‘work on’ internal representation, manipulating it to get new solutions. The focus is on the cognitive actions and operations on the internal representations.

The question this post addresses is

‘As an information processor, what is the competence required for the designer to be effective? How does a designer learn these competencies?’

The later posts will address questions like,

‘Are there alternatives routes to consciously develop these competencies? Can this learning be fun?’

Notes and references

1 In the three stage classical approach, perception is linked to action through cognition. Common coding approach is built on directly linking perception and action. In common coding theory perceptual representation like seeing, hearing are directly linked to motor actions by a common code. The theory proposes that there is a shared common code for perception and action. That is why seeing an event activates the associated actions and performing actions activate associated perceptual event.

2 The word chair, as well as the visual image/s of a chair/s associated with it, are stored as separate information channels. Activation of one channel may be sufficient to recall the information in the other channel. Hearing the word chair can activate a picture of that object and vice versa.

3 Author does not have a formal background in cognitive psychology. So, to hazard a guess on the differences between common encoding and duel encoding is difficult. In most examples cited in common encoding, one of the channels deals with verbal information and the other with visual information. This may not be always correct. Note that the post is based on limited understanding of these issues.

4 There is a simple way of understanding recency effect in the long-term memory. More recent events are more easily and quickly accessed than the event from the past.

5 Athavankar, U., (1997) Mental Imagery as a Design Tool, Cybernetics and Systems, Vol 28, No 1, Jan-Feb, pp 25-42

6 Garner, W.R., Uncertainty and Structure as Psychological Concepts, John Wiley and Sons, Inc., New York, p.145.

7 Most common examples of redundancy by correlational structure are traffic signals. In stop sign, the colour red, round shape and location on top correlate. I go sign, green colour, arrow and locations correlate. That is how even colour-blind can navigate effortlessly.

 

Part IV: Can you interact with nonexistent physical objects and spaces?

Using embodied cognition as a framework, the preceding post revealed the probable role that the gestures and body movement played in creation of the objects. In this post, we focus on the causes of the accuracy of designer’s interaction with the objects and spaces. It will address the unanswered question,

How could designers corporally interact with a virtual, physically non-existent object that they created in their mind’s eye? And that too, with surprising accuracy?

About this post

It is planned to rely on multiple sources to answer these questions. First, we will depend on research literature, particularly on embodied cognition. This does not deal specifically with gestures in imagery and sketching. So, some elements of speculation cannot be denied, but at least, it offers a broad framework to support the ideas. Second source is the data that is available as series of experiments. We will also take support from anecdotal example, more to understand ideas than to support the conclusions.

So, the discussion that follows has following aims. 1] To offer a plausible explanation to the participants relying on specific kinds of gestures and movements during creation of these objects and spaces. It argues on why and when gestures and body movements vigorously mimic and do not mimic the physical characteristics of the shapes. 2] To explain the source of accuracy of designer’s interactions with the physically non-existent virtual objects while creating the shapes, and 3] To explain why, during summing up their solutions towards the end of the experimental sessions, the participants are able to repeat the identical gestures and body movements.

The creators selectively chose gestures and/or body movements to suit the design task at hand. The post will argue that their decisions to deploy or not to deploy them were prompted by the useful contributions that gestures and movements made to the evolving thoughts. Let us try to develop these arguments further.

During discussion in the preceding blog on embodied cognition, there were references to the close relationship between visual system and the motor system. This suggests that cognitive processes also encode the sensory-motor aspects as series of movements.1 The argument is built on this central idea of the partnership between the visual and the motor system. So, we first address the partnership related question,

Can the creator’s actions be viewed as close-knit partnership between visual decisions, movements and the thought process?

If the answer is yes, it will give us a framework to answer the questions that we started with.

The casual support to the idea of this partnership comes from several examples in our every day routine tasks that demand relying on mix of motor and visual system. They seem to work together in partnership. We perform such tasks efficiently because these two systems seem to work in sync. But are all partnerships same? For the purpose of this paper, we plan to categorize the partnership into two broad groups. We are specifically focus on the second, which is of interest to us in the context of this paper. So we only briefly touch the first and move on.

1] Creative partnership where gestures and movements are based on convention or tradition

In this category, the planned visual effects are achieved through ‘pre-designed’ body postures, movements and visual expressions. The visual and the motor systems work in sync. They balance the cerebral and the corporal.

The major difference is that the gestures and movements are not free to evolve, but were defined by codes. They are motivated by convention or tradition, and are often formally learnt, practiced and produced. These events give ample evidence of how visual and motor systems work in sync.

Typical day-to-day example would include writing and production of your own signature and learning to write new script.2 Formal example to understand the category would be live performances of classical dance, street play, fashion shows and even performances of acrobats in circuses.3 These gestures and movements are either controlled by tradition (as in classical dance forms) or by convention (as in writing, plays and fashion shows).

These categories of gestures and movements are unlike those produced in our experiments. Explanations of learning, producing and use of these gestures are available as footnotes cited above. This ensures that we maintain our focus on the second category i.e. the spontaneous and intuitively produced gestures and movements.

2] Supporting the partnership with intuitive and spontaneously produced gestures and movements

A typical example to explain the idea would be the gestures accompanying natural speech. These gestures are spontaneously produced to support the evolving thoughts. They reflect the thinking in some ways. They show how the cerebral and corporal work in partnership.

Better and more convincing examples in this category are the gestures produced when giving route directions to a remotely located third party. They are truly embodied creations used everyday. The gestures clearly offer a corporal support to the cerebral processes.

The synchronization between the visual and motor systems continues like in earlier cases. It is not dictated by practice, but by the emerging thoughts and ideas. Our focus is on corporal actions (gestures and body movements) as supports to the cerebral actions (thinking process). Spotlight on this is justified because we feel that this is how the designers and architects have used gestures in our experiments.

A formal and compelling example is a live music concert. The singers move their hands and torso. Accompanying artists too move their bodies in sync. These gestures and movements support the primary task of creation and production of live music. (Also see ‘Can we prevent designer from sketching’ in this blog series)

Such gestures are very similar to the way they occur in our experiments. They are in sync with the evolving thoughts and ideas and so, reflect the thought process. They are very much a part of the process of creation. Such spontaneous use of gestures would be of interest to us in the current discussions.

Like in the live concert example, in our experimental sessions, the gestures were spontaneously produced. They appear to respond to the creator’s evolving thoughts. Most of them seem to incrementally shape the ideas of objects and spaces. They often mimic the shapes and/or indicate the locations of features in the space, as if these exist in front of them.

Let us focus on these experiments with architects and designers that we reported in this series of posts. The participants in the experiments were not briefed and had the option of using or not using the gestures. They were free to deploy gestures and movements, if and when they chose.

Identifying partnership to interact with the physically non-existent shapes and spaces

Transcripts show these gestures and movements do not appear as consciously ‘designed’, but look spontaneous and well integrated with the act of creation. They had clear roots in the thought process that resulted into in creation of objects/spaces. Obviously, individual differences do exist in the way movements were produced. Yet they display internal consistency.

Such occurrences are also true of most speaking gestures. The creators seem to develop the rules of their own and sometimes improvise on the spot. Such individualized gestural behavior is also common in live music during concerts. Yet, there is a difference between gestures in music and in design.4

By revisiting events in the experiments reported in the earlier posts, we plan to understand these spontaneous gestures and body movements of the designers and architects during the act of creation. Because there are differences in the nature of the assignments handled as well as in the way the designers and architects used gestures and movements, it is better to discuss them separately.

Revisiting SP’s actions: Casserole case

Let us look at SP’s performance under experimental condition one, where he was blindfolded. (See ‘Part II: Reflecting on gestures as design behavior’ in this blog series) Video transcripts show large segments, where SP moves his hands to shape the casserole, responding to his evolving ideas and thoughts.5 The transcripts suggest that he was describing the shape to himself and experiencing it. But there is much more to these simple actions.

The actions dealing with the shape issues show close correlations between the decisions dealing with evolving image and the accompanying gestural kinesthetic sequences. (See video 1 and 2) SP’s palms and fingers are actually seen mimicking the shape in 3D space as truthfully as he could. (See also video 4 in the next section)

Video 1 : The segment defines the overall shape. Watch how the palms and fingers define the curved surface. The relative angles define the angular relationships of surfaces with each other.

Video 2 : In the later part, the segment discusses complex detailing, where the legs are conceptualised to be folded-up during packing.

On the face of it, SP seems to be mentally shaping the evolving visualization of 3D geometry as an image in his short-term memory. This mimicking of the shape with hand gestures perhaps ensures a close partnership between, 1] sets of sequential (gestural) kinesthetic actions in 3D space, leaving traces of motor system in action, 2] the image in the mind’s eye, which he continues to modify, visualize and consolidate through visual system. Both processes seem to happen simultaneously.

Accurate recall of form of the object

SP could repeat identical movements while revisiting the feature for modifying at different points of time during designing, as well as during final description of the shape created. So accurate was the description of his final image and reproduction of hand movements and gestures that shaped that image, that even other designers could decode them accurately and reconstruct the casserole and bowl shapes from the video. The accuracy in performance is surprising.

Holding a stable image in the mind’s eye as well as reproducing gestures that shaped the image with surprising accuracy is not a trivial fit, particularly considering that the gestures were spontaneously produced on the spot, they were shape dependent and were not practiced, nor bound by conventions.

The visual processes and motion processes are working in sync and together they are responding to the emerging ideas and thoughts. The close synchronization and partnership between the two systems seems to have led to stronger and stable creations that you could recall and modify at will.

Revisiting architects’ actions

Architects, when blindfolded, moved in the real world space and used gestures differently. The large sizes of the enclosing wall envelopes and other architectural elements were beyond the grasp of their hands. So, they figured out the way to compensate. Two of them visually built spaces by walking around, as if they were on a real site. They developed the spaces around themselves and used body movements to create the layout and feel the scale of the spaces.

It appears that they chose to rely more on the visual system to generate the walled enclosures in the mind’s eye. Simultaneously, they depended on body movements and the motor system for the geometry of the layout and to scale the spaces. To indicate relative positions of features and walls, they selectively used gestures and virtual locations of their body, on the virtual site that they visualized. In such cases, gestures played a secondary role, more as a complement to the body movements.

In two out of four architects, their physical movements in the hall could be mapped to the layout of spaces they were creating in their mind’s eye. These physical movements in the hall could have left interlinked memory traces. Transcripts show that their movements match the walkthroughs in space when they were creating in their mind’s eye. That is how they could effortlessly move in and out of these built spaces, ‘feel’ them, and to get the ‘view’ from the position they visualized they were standing at. This explains why, when asked what their built-form looked like, they raised their head and even stepped back to ‘see’, though the image was only in their mind’s eye! (See video 3)

Video 3 : The architect was asked to describe what his building looked like. While answering, he took a step back, looked up as if he was scanning the face of the building.

Accurate recall of architectural design

As a part of protocols of the experiment, the architects were asked to describe their final solutions after they completed their design. Some preferred to walk the talk while describing; and these walking movements were identical to what they had performed when they conceived these solutions. This also explains why during descriptions of the final solutions, two architects could walk accurately through the spaces that they had created. Another architect when asked, could walk back to the virtual entrance without error.

None of these performances are trivial. Probably, their actions may have led to strong visual and motor memory traces that they could fall back on.

When gestures? When body movement?

Architects and designers appear to spontaneously produce gestures and movements when they are a convenient support in 1] Development of the image in the mind’s eye and/or operating on it, 2] Conceptualization of enclosure spaces around them, 3] Exploring spatial relationships of elements and scaling the shapes and spaces.

Preference to depend on visual system, and use or not to use motor system (gestures and/or movements or mix the two) depended on individual capabilities, design context as well as the complexity of the design assignment or its details. It also depended on the context of the experiments. (i.e. Sitting or allowed to move) SP’s actions are restricted by the fact that he was sitting with a table in front. Besides, his design assignment was also a tabletop object and so, within his grasp. He used hand gestures to shape all the features.

It is interesting to note that the choices between the gestures and body movements were largely interchangeable. There is also sufficient evidence of effortless switching between the two. For example, architects, who generally walked and used body movements, could adopt quickly to the use gestures, when they found this efficient. For example, when the idea of entrance demanded a sculpted shape and flow of surfaces, she used the hands and palms almost the way SP used them. (See video 4) Similarly, some occasionally went back to their traditional methods and scaled down the creations to see the built spaces as a small table-top model, being inspected from top and even shaped it using gestures to match their evolving thoughts. (See video 5)

Video 4 : The segment where the architect shapes the entrance with her palms, as if she is sculpting it by hands. The limitations of the grasp are clearly visible.

Video 5 : The architect switched to treating the creation as a scale model that she visualized and partly shaped it using gestures.

Exception to the rule

Though most of the designers and architects chose to use gestures, to construe that they are essential part of the thinking process may be too hasty. As we saw earlier in the context of mental rotation, individuals either use visual system, favour the motor system or a mix of the two, to perform such tasks. They choose what they consider appropriate for the task or what they are comfortable with.

Amongst several experiments conducted, there is one unique example in this series where the architect sat on the chair throughout, with hands on the hand rests of the chair and developed and spoke of his ideas without any gestures or body movements. He produced quality work and was as detailed as others, who used gestures. He walked through the building virtually, describing people populating the spaces, the ambiance of light and shades and so on. He seems to have depended on visual system completely. When the experiment was over, he was asked about it. He replied that he could have used gestures, but was able to visualize it clearly in imagery equally well. He did exercise his choice to depend on visual system.

In all my initial research writings on these experiments, I explained gestures and body movements by referring to them as ‘thinking with body’. Later, I hinted to its roots in hinted of spatial intelligence.6 I had no clear idea of their relationships with evolving thoughts and ideas. It was during the writing this blog, that I realized the full force of spatial intelligence perspective. Later, I realized that embodied cognition and embodied design offer a even more effective framework to explain why the creator’s gestured and moved their body. If we look at these events and actions through this new lens, lot of events and actions are easily explained.

These are of course post-facto explanations and there are some speculations involved. But, I thought they are intuitively appealing. However, it is best leave it to the cognitive scientists to explain how and what prompted these actions.

Summing up

Have we answered these questions that we started with?

How could designers corporally interact with a virtual, physically non-existent object that they created in their mind’s eye? And that too, with surprising accuracy?

We now have reasonably clear explanations to these questions. In doing that we have also identified several unique, non-trivial and surprising performances of the designers and architects that need a special mention. We have listed them below.

Non-trivial discoveries in design behaviour

1] Our results of the experiments reported so far seem to support the idea that the visualization of objects/spaces by the visual system was assisted by motor system that contributed to creating, altering and maintaining the continuously transforming shape and spaces. These two linked systems seem to lend stability to the image and allow the designer to revisit them at will, and interact with them with accuracy.

2] Video transcripts give a feeling that the creators were treating the creations as if they were real events unfolding in front of them. There is sufficient evidence of the creators being present in the situation and taking decisions, accounting for the location and orientations of the features. Because of the visual and motor systems working as partners in the process of evolving and modifying objects emerging in their mind’s eye, designers seem to accurately and almost effortlessly, interact with these nonexistent entities using their gestures and body movements

3] Creators are able to recall and reconstruct the current states as an image with a fair accuracy. They could also describe their images in details with certainty. This suggests that the images that they recall in their mind’s eye are relatively stable at any point of time. Even the physical actions that shaped these creations could be recalled and repeated without error.

It appears that the entire history of the gradual evolution of these shapes and layout spaces is available to the creators.

4] The actions like gestures and movements of the body accompanying the speech strings were so accurate that could be decoded by third party to understand and reach the final design in the creator’s mind.

None of these are easy to explain, but an attempt is worthwhile. The reasons not only include the cognitive aspects, but go beyond. So, in the next post we will attempt this.

Preview of the next post

To effortlessly perform these non-trivial actions at different points of time in the design process is a surprising fit. We have tried to attribute this to stability to the otherwise fragile mind’s eye images.

Can you attribute the performance in these non-trivial tasks directly to the linking of visual and motor systems at a cognitive level? Or are there also other factors in these actions?

To perform these actions at different points of time in the design process is a surprising. We have tried to attribute it to the duel encoding that may have given stability to the otherwise fragile mind’s eye images. In the next post, let us explore other answers to this. Though we can never be sure, based on the evidence before us, we can at least come up with some conjectures.

Notes and references

1     Pande, P., & Chandrasekharan, S., (2017) Representational competence: towards a distributed and embodied cognition account. Studies in science education, 1-13

2     In many routine tasks, along with visual system, the motor system (movement memories) is also primarily used during learning as well as in performance. Most common example is the reproduction of our own signature. While signing, the flow is monitored by visual system as well as by the motor system. They have to work in sync and as partners.

Another example. In traditional method of learning to write, the children are given learning templates that indicate starting point/s and the direction of pencil movement. They are also encouraged to pronounce the sounds. The directions on the template control the final appearance of the marks on the paper as well as the correct movements of the hand. The practice also ensures creation of movement memories.

Video 6: Child practicing script. The appearance of the letter is controlled visually and the movements by the instructions given on the template. (video courtsey Santosh Khirsagar)

That explains why, little children can play the game of letter recognition by writing on each other’s back with finger. These are clear case of cerebral and corporal working together.

3     Creative partnership, where gestures and movements are based on convention or tradition

In classical dance, the gestures and body movements are directly responsible for the primary visual effect. So, the visual and the motor systems work in sync. The planned visual effects are achieved through ‘pre-designed’ body postures, movements and visual expressions. The freedom to evolve and change is within the framework of tradition and convention. The partnership has to be consciously and painstakingly learnt through practice till they are well synchronised.

Modern dance has the freedom to innovate new gestures and movements. Choreographers of modern dance and street play can explore new creative directions. Because they are driven by creator’s thoughts during conceptualiaation, they are closer to our experiments in design. They balance the cerebral and the corporal.

Closer to our example is when the modern dance performance is being choreographed, particularly modern dance. We could have included such cases to balance the findings. However, we are forced to exclude them from the current discussions because we did not have the opportunities to study the practices and actions involved during choreography.

4     In live music, the singer produces gestures spontaneously. The listener is aware of the gestures and uses them to understand and appreciates the performance. He may even synchronize his bodily responses to these gestures. In design, the final creation of objects and spaces stand independently and do not even offer clues to the gestures that were used to create the object. They play no visible role in the appreciation of the final output. In the act of design, gestures act more like backstage tools.

5     That the transcripts also show different kinds of gestures and all the gestures do not necessarily mimic the shape. For classification of these gestures refer to reference 6 below.

6      Athavankar U. A., (1999) Gestures, Imagery and Spatial Reasoning. In: Garo JS, Tversky B (eds) Visual and Spatial Reasoning. Preprints of the International Conference on visual and spatial Reasoning, (VR 99) MIT, Cambridge, June 15–17, 1999, pp 103–128.

Part III : Is design a corporal act

In the last post, we viewed the act of design as a cerebral act. We also touched the role of intelligence, particularly spatial intelligence in design problem solving. We tried to explore the role the gestures and body movements play in design problem solving. Developing this idea further, we cited several facets of spatial intelligence from the protocols of the experiments. However, visibly rigorous gestures and body movements suggested that one must also go beyond spatial intelligence to explain designer’s actions. Let us reword the earlier questions,

Why did the designer move hands in the air to mimic the shape he was creating? Why did the architects and designers physically move through the mental spaces that they created in their mind’s eye?

Are there more effective explanations to gestures and body movements in the previously reported designer actions?

Part III searches for alternative theoretical framework that directly confronts the issue. It treats design as a corporal act that supports and sometimes dominates the cerebral act, i.e. the body and its movements take part in understanding, reasoning, judgment and in conceptualizing of idea in problem solving. It is called as embodied cognition, a framework that I realized design community will more easily identify with. This theory draws from many fields like sports and music, to present more convincing answers to why designers used gestures and body movements.

Both the theories show advantages of getting the body involved while thinking, particularly when innovations are spatial in nature. Let us take this perspective to argue our case. Keeping embodied cognition framework as a backdrop the post will move on to explain some of the design(erly) behaviour in the experiments reported in earlier posts.

Design as an embodied cognitive act

The discussion in this post extensively relies on the new framework proposed by embodied cognition. To understand the nuances of this framework, it is best to contrast it with its predecessor, the classical information processing model. With a short introduction to the relevant features of the classical model, we can return to embodied cognition.

Classical information processing model

In our last post, we treated designing as a cerebral act. This is consistent with the classical model, which focuses entirely on cerebral processing. Designer is viewed as an information processor, who manipulates symbols during problem solving. The model suggests that all cognitive processing is (or is best) done by neural processes inside the brain. The influences of computer model are visible in its focus on working memory load, long-term memory, information storage and finally the resultant action produced.

In the classical model, the external representations (like sketches or diagrammes) are seen as help to offload information. External representation has a limited passive role of acting as a vehicle or a transmission media that carries information for processing.1  It is assumed that the cognitive system passively receives information from the external representations.

On the other hand, embodied cognition takes an opposite view. So, after a brief introduction to the theoretical framework on which these new concepts are based, we will explore if it offers better insights into the observed design(erly) behavior in our experimental data.2

Understanding embodied cognition

Embodied cognition theory considers that the cognitive processes are distributed across internal and external representations. Interactions with external representations (like pictures and sketches) are the central process driving meaning and understanding.3 (Note 3) Embodied cognition concepts propose that many features of cognition are shaped by the characteristics and aspects of the physical body and these influences have significant causal role in cognitive processes beyond the brain. Embodiment assumes that what happens in the mind is depending on properties of the body, such as kinesthetics.

It is also argued that that the brain and the cognitive processes are developed for action, particularly motor action. So, the body and the motor systems are closely involved in most cognitive operations. It also recognizes that the cognitive processing is distributed across internal and external representations.3 (Note 3)

In this framework, the local environment is seen as an actual extension of the body’s cognitive process that is explored through active perceptual and motor actions. Ideal way to understand the concept is to imagine that if we had different physical attributes and capabilities, we would have understood and responded to the world differently. There is a conceptual shift from the classical model in suggesting that cognitive process also encode sensory-motor aspects during interaction. As we will see later in this post, both are key to understanding the role of gestures and body movements of the designers. Even behavior is seen as a result of embodied cognitive responses. Examples will help to understand the ideas and concepts.

Human response to musical renderings is an excellent example. In this approach, spontaneous bodily movement (moving hands, head and torso) contributes to musical meaning formation. Indeed most listeners do respond to music with body movements. On the other hand, in disembodied approach (i.e. cerebral) the focus is on perception and analysis of musical structure.

Take another example. In spatial task like rotating of object in space, most individuals use visual processes by imagining rotational movements in the mind’s eye. However, some complement this by using hand /palm gestures to mimic the movement of shape in space.  They favour motor processes to perform the same tasks, often with higher overall performance. It points to the influences of the motor system on high-level reasoning and cognition.

A more convincing example is the use of mental abacus by experts. Some expert abacus users rely heavily on visuo-motor operations by implementing the same sensory motor actions internally in their mind. Others use the same action covertly leading to gestures as if they were using a physical abacus. In mentally playing an imagined musical instrument, similar covert physical actions are sometimes visible. These examples from divergent areas convincingly explain the concepts of embodied cognition and active bodily involvement.

Embodied creation

Creative endeavors in arts recognize the role of bodily involvement and responses. The embodied cognitive responses have important role to play in creations. For example, production of live music integrates the corporal and the cerebral act. Most singers and musician spontaneously produce gestures and body movements when they sing or play a musical instrument.4 (Note 4) It is difficult (if not impossible) to see production of music as a disembodied act.

The spontaneous partnership of body in learning as well as creations is not unusual.5 (note 5) We also extensively use spontaneous gestures when talking on telephone, knowing fully well that the person on the other side is not able to see them.

How does embodied cognition explain design(erly) behavior? How do we then explain the use of sketching? And benefit of gestures and body movements accompanying mental imagery? All these are in someway used to represent the objects and spaces in our experiments. We will deal with these issues later in this post. Based on this exposure to embodied cognition framework, let us revisit the  discussion on the results of these series of experiments.

Corporal acts and creations

Like embodied music creations, the tasks of shaping and locating elements in space are embodied design actions. Such events are referred as embodied imagination and are derived from the idea of embodied cognition. They are based on the premise that the actions of the body can (and do) participate in the development of thought and ideas. Embodied design explorations rely on be gestures or movements of the body and visualization in the mind’s eye.

Embodied imagination explains how, in blindfolded conditions, SP generated and manipulated the objects in condition 1, and architects conceived spaces around them in condition 3 (mentioned in the preceding post). Designer’s moves and reflections in these experiments show embodied imagination in action. These are ideal examples of embodied spatial creations.

Gestures and movements as corporal acts

Watch SP’s hands during creation of the casserole shape in condition 1. They move to create, refine and perfect the shape of the object. Several processes seem to be at work here.

First, SP’s gestures seem to complement the visual system deployed in the development of his visualization i.e. the visual system acting on the internal mind’s eye display. He could then creatively react to the display. Second, the fact that SP developed the virtual shape in front of him at a fixed distance shows that his body played a role in locating and scaling the virtual shape in front of him.

Second, the embodied creation demands being there, virtually or physically, to create objects and building ‘things’ and locate decision with respect to the body. Though SP’s visualization was virtual, the protocols and the design decisions seem to suggest that the designer was present physically in these visualizations. SP was sitting on a chair and. sculpted the object in his visualization as a tabletop creation in ‘front of him’ and scaled it using ‘his hand movements’. The actions were within a spatial range that ‘his grasp’ permitted. See video 1 below.

 

Video 1: Everything that SP created was small enough to be within is grasp, so he could shape it with hand gestures

In the next series of experiments (condition 3), the architects were let into a large hall and blindfolded. The responses were dramatically different. Corporal involvement became more intense. Architects chose to physically move in the hall and imagined as if they were on a ‘real’ site and the environment. Their physical movements as well as navigation (embodied cognition) seem to have contributed not only to their understanding of the spaces but also to creatively altering of the spatial arrangements. Architects immersed themselves in the space they created and almost ‘walked and lived’ in the visualized enclosures. (See video 2) In virtually constructing the built spaces, they appear to have relied on the visual as well as movement memory!

Video 2: The architects were so immersed that appears as if they ‘walked and lived’ in the spaces that they created.

Third, in describing design actions, they referred to design, as well as site features with respect to their bodies. So it is not surprising that the transcripts are full of references like ‘in the front, on my left/right, behind me’ and so on. (See transcripts in figure 2) It is impossible to make sense of the spaces that the architect’s created without accounting for the their body, its location and orientation.6

Video 3: In the immersive steps, the architects were able to tell where they were when asked.

Figure 1

Figure 2: Terms like my left, right, in front, occur extensively in the transcripts.

Is design a cerebral act or a corporal act?

There is much that we could learn from both the standpoints. As a designer, I feel that the act of design switches between the cerebral and corporal along the different stages in the design process. Early analytical studies are cerebral and when the action moves towards synthesis, the designer seems to frequently switch between the two. Frankly, it is best left to the cognitive scientist to solve the puzzle.

Summing up

Preceding post (Part II: Reflecting on gestures as design behavior) treated design as a cerebral act and focused on spatial intelligence and its role in design problem solving. This post viewed designing as a cerebral act and looked at it through the classical information-processing model. It tried to explain designer’s gestures and body movements as a cerebral act.

The current post pursues the same questions that we started with. ‘Why did the designer move hands in the air to mimic the shape he was creating? How could the architects and designers physically move through the mental spaces that they created in their mind’s eye?’ This post explores new theory that offers a better explanation to designer’s actions.

The theory that is relied on is embodied cognition and in many ways it takes an opposite standpoint. It claims that what happens in the mind is dependent on properties of the body. The local environment is seen as an extension of body and its cognitive processing.

The theory also proposes that the brain and cognitive processes are developed for motor actions and they extract information through active perceptual and motor actions. The physical actions like gestures are seen as complimenting the visual processes. Through several convincing examples, it shows how the body actually participates in thinking as well as development of ideas. This post, (as well as the subsequent posts) focus on this connection between cognitive processing and motor actions.

Representations, both Internal (mind’s eye) and external (sketching), play key role in this theory. The interaction with representations is considered central to deriving meaning and understanding. The current post views design behavior as a result of embodied cognitive actions. The post also touches the extension of this idea to embodied imagination and creations as a corporal act. It suggests that such corporal acts observed in the experimental tasks may have influenced thinking and dictated design decisions.

Preview of the next post

The next post will focus on the reasons for accuracy of designer’s interaction. It will address the question,

How could designers corporally interact with a virtual, physically non-existent object that they created in their mind’s eye? And that too, with surprising accuracy?

The answers will be somewhat speculative. It is possible that the effect of mimicking the physical characteristics of the shapes and the spaces had something to do with accuracy with which they interacted with the objects and built spaces during the creation as well as the recall in final description. What ensured the vigorous and accurate interactions with physically non-existent objects?

We will deal with these important questions in the next post. It will also argue that the choice to use gestures is with the designer and his decision is influenced by useful contributions that movements make to the evolving thoughts.

Notes and reference

  1. Interestingly Robert McKim has also talked of sketches as vehicle of thoughts. See McKim, R., (1972) Experiences in Visual Thinking. Brook/Cole, California
  2. For more on embodied cognition refer to Wilson, M., (2002). ‘Six views of embodied cognition’. Psychonomic Bulletin & Review. 9 (4): 625-635
  3. Pande, P., & Chandrasekharan, S., (2017) Representational competence: towards a distributed and embodied cognition account. Studies in science education, 1-13
  4. Even in the radio era when the musicians could not be seen, the gestures were not absent, but were less conspicuous.
  5. There is also conscious production of practiced bodily responses, as part of learning a spatial task. Most sportsmen practice their playing actions in the air, to reinforce their embodied responses, but their relationship with thoughts is slightly different.
  6. Location of objects or elements with respect to the body is common and is considered almost innate. George Lakoff points out that human movement revolves around standing erect and moving the body in an up-down motion. Humans innately have these concepts of up and down. Lakoff and Johnson contend this is similar with other spatial orientations such as front and back too.

 

Part II: Reflecting on gestures as design behavior

Problem solving is often treated as a cognitive act and problem solver as an information processor. This is also extended to design problem solving. Yet the act of design, standing on the fence between art and science, has different nuances. So, there is much to learn from how designers solve problems. Like all cognitive acts, designer’s internal information processing remains invisible. However, designers in the act of design display unique design(erly) behaviour. The clues to what goes on his head are through observing designer the visible results or overt actions in the act of problem solving,

As the designer progresses towards solutions, the spatial issues dominate his thinking. At that stage in the design process, design can be limitedly defined as act of resolving spatial issues to achieve predefined goals. Designer’s decisions involve designing elements and their specifications as well as arranging, composing and iteratively manipulating them in space to achieve stated goals. At this stage need to support the evolving thoughts through sketching or 3D representations becomes critical. Such design(erly) behavior includes, think aloud transcripts, mental imagery, gestures and body movements when working in blindfolded mode (in normal circumstances sketching and models produced).

These are the traits we have been decoding all along. In all our experiments we have tried to capture these and treated them as a focus of our analysis. Though we have treated them as separate external manifestations of what goes on the in the mind, the experiments suggest that they are closely interwoven with the thought processes. So, we have used visible clues as a ladder to understand how, why and when they assisted the thoughts that drove the decisions.

What does visible design(erly) behaviour tell us about the act of design? This is a continuation of the discussion started in the last post “Sketch or not to sketch? That is the question”. It ended with following questions,

Why do designers use gestures and movements of the body when they solve problems? Do these support spatial decisions, visualization and design(erly) thinking? And if so, how? When should they prefer gestures and body movements?

We have yet to find explanation to why designers and architects used gestures and physically move their body through mental spaces that they created and interacted with them.

So far we have referred to gestures and body movements as one entity, there are qualitative differences within and between them. A more detailed analysis of the classification of gestures and movements and the roles they play is available in my earlier papers.1,2 In answering the above question, we will only touch some of the key gestures and body movements that seem to have a role in understanding, thinking, conceptualizing and representing the creations. We will also focus on how and why they may have contributed to the evolving thoughts.

Plan of the discussion

We plan to look at external traits of design(erly) behavior in different experimental conditions and discuss their effectiveness in design problem solving. We will also compare the two modes, when using sketching and when blindfolded. Note that it deals with limited part of the design process, when designer is grappling with spatial issues.

It is conceived as a four part series. This post (Part II) primarily explains the role of gestures and body movements and tries to explore spatial intelligence as a theoretical framework to understand and explain the design(erly) behavior. In the next post we will focus on embodied cognition as a framework and the third will include the other special features.

Overview of the theoretical frameworks

We hope to find support in two key theories that come close to explaining this design(erly) behaviour. In this Part II, we will start with Howard Gardner’s theory of multiple intelligences.3 This theory relies on the computational capacity, activated by external or internal information and is consistent with our treatment of designing as a cerebral act. We would be particularly focusing on the two relevant areas in intelligence; 1] spatial intelligence and 2] bodily-kinesthetic intelligence.

In the subsequent post (Part III), we will seek support in the emerging theory that deals with embodied cognition. The concepts propose that many features of cognition are shaped by the characteristics and aspects of the physical body and these influences have significant causal role in cognitive process beyond the brain. It accounts for the active use of motor system, perceptual system and bodily interaction with the environment. It treats design as a corporal act.

Cognitive embodiment and spatial intelligence theories are built on different theoretical bases. The first relies on computational approach and thus treats problem solving as a cerebral act. The second argues for the role of bodily actions. It believes that thinking is also corporal in nature, at least in specific disciplines. In spite of their opposite theoretical stands, there is lot that can be learnt from these theories in understanding design(erly) behavior during problem solving.

A quick review of the experiments

In trying to find logical explanations we plan to discuss results keeping in mind all the experiments in this series. Participants in these experiments were industrial designers and architects solving problems typical to their disciplines. A quick overview of the series of experiments and the differences between them will offer the context for the discussions that follows.

Industrial design project

SP worked on a casserole project, a small tabletop domestic product. The findings of this experiment were discussed earlier in posts starting with ‘Can we prevent designers from sketching?” Short videos clips showed some of SP’s design actions. The readers could always revisit this material to refresh their memories. In this post, I plan to only briefly review the findings of these series taking into account the entire video records of the sessions .4

Let us start with a quick recap of the events in this experiment. The subject ‘SP’ was given a brief to design a casserole for domestic use. The project brief also demanded submitting designs of matching bowl set. Table 1.1 captures the set up. SP incrementally built the idea of the box, starting with fixing the volume and proportions of the rectilinear shape. It appeared as if he was shaping the volume with his hands moving in the air in front of him. He could manage this effectively as the actual size of the product was within his grasp. Most of these gestures were purposeful and were contributing to develop the casserole and bowl shapes. As he progressed in design, he started sculpting the shape and fine-tuned it. (refer Video 1) He assigned colour, transparency, textures and even finalized the product graphics. Some of his gestures simulated actions of the use of these products. He even simulated assembling of the components in manufacture. (Refer Video 2.) (Closest visualization of the scene would be to imagine musician acting as if he is playing an instrument in creating music without the instrument being there.)

Condition no. Design project Experimental condition and description

1

Industrial design project.

Table-top domestic products as projects.

Casserole, Salt-n-paper container etc.

Designer blindfolded

Designer sitting in an office chair and matching environment.

Small size, table-top product, Free flowing form.

Table 1.1. Designer was blindfolded and sat in a chair and solved the design problem

Video 1: The industrial designer shaping a virtual object in front of him as if it was real.

Video 2: The designer simulating assembly of parts

We deliberately changed conditions, where the architects solved a typical architectural problem. In condition 2, the architects were ushered in an office like environment, asked to sit and were given the site plan to see and recall. Then site plan was withdrawn and they were blindfolded. So, they had to develop the 3D representation image/s of the virtual site in their mind and worked with it. In condition 3 the procedure remained same, except that they were left standing in the middle of an empty hall, obviously without access to any sketching or modeling tools. Table 1.2 captures this.

Condition no. Design project Experimental condition and description

2

Architectural project Mid-size built space.

Student cafeteria with constrained site.

Designer blindfolded,

Designing while sitting in office like environment,

Mid-size site, Body enveloping spaces,

Constraints of construction.

3

Architectural project Mid-size built space,

Student cafeteria with constrained site.

Designer blindfolded,

Let in to large hall, can move if the designer chooses.

Mid-size site, Body enveloping built spaces.

Constraints of construction.

Table 1.2: Architects solving a problem. The differences in the conditions 2 and 3 are mainly in the working posture.

It is interesting to watch the full videos of the architects performing and ask ‘Why did the used gestures and movements?’ In this part we will rely on concepts like multiple intelligence and its subset, spatial intelligence to explain gestures. In part II, we will use embodied cognition as a theoretical framework. Both seem to offer new and credible explanations.

Treating design as a cerebral act

Designer is learning all the time by studying the problem and from the world that he lives in. He reacts to what he sees, understands and learns. The information that he absorbs and manipulates contributes to the thoughts and ideas. It influences the way he thinks and takes design decisions. The design actions are not just trial and error acts. They obviously involve intelligence. Thoughts during designing are driven by information and intentions. So, in this framework the designer is seen as information processor and design primarily as a cerebral act.

Traditional approach to cognition focuses on higher-level strategies like development of concepts, categories, reasoning and judgment. It is based on information processing and symbol manipulation resulting in production of output. Brain is an information processor and its actions are explained by computational approach.

Over the years the information processing act of design has evolved into a somewhat structured process. The design process followed ensures consistency in performance over time and projects and yet allows sufficient freedom to explore uncharted paths. It is reasonable to assume that the different stages of the process will demand different classes of information. It follows that these steps will demand specific forms of intelligence as well as unique skills. In the discussion that follows, we hope to identify some of the dimensions of spatial intelligence visible in these experiments, particularly in different segments of the design process. We hope to explain design(erly) behavior too.

Spatial intelligence is spatial reasoning and judgment

Driven by designer’s reactive thoughts, the new ideas are continuously generated, often as images. They are conceived and converted into sketches or seen in the minds eye. They are compared and judged. These images continue to evolve through designer’s reactive actions and interventions. The study of the entire process and the accompanying transcripts reveal the creative moves and reflections that SP generated throughout. (Refer Video 3)

Video 3: Industrial designer making move and reflecting on it. This continued throughout the initial part of the session.

When designers handle tasks like making effective design changes, developing alternative spatial layouts is obviously not a process-based on trial and error. To create and judge different alternatives demands reasoning, intelligence and considered judgment. It is a cerebral act. Design problem solving demands spatial judgment and spatial intelligence. All the solutions that SP developed, accepted and rejected during the session support the idea that design is a cerebral act. (condition 1)

This is also visible in conditions 2 and 3, where the architects thought, created and built the idea virtually, grasp and react to it and then alter it again. The architects too were visibly engage in modifying their ideas and trying to grasp them and figure out the implications. This involves anticipating the consequences of changes. Comparing and judging the ideas mentally needs intelligence and these cases spatial intelligence.

In blindfolded conditions 2 and 3, these steps were lot more difficult, because the dynamic events occur in the images in the mind’s eye, which are known to be extremely fragile. They had to hold these images as dynamic displays that they could react to, which demanded budgeting additional mental energy, over and above the energy spent on thinking of new ideas that keep the images in the state of flux.

Spatial intelligence in visualization to drawing

During the later stages of the design process, thoughts and ideas lead to some form of spatialization. Initially it often takes a form of mental image. Capabilities vary in simple tasks like depicting an image as ‘seen’ into these external forms. So, designers do need the ability to convert their mental images into series of sketching strokes, or in case of a model, actions on a clay block. Even more challenging is the tasks where the ideas are half-formed and designers expect the assistance from externalization efforts to concretize his ideas. These are not trivial tasks. The subsequent post (Part III) will touch this aspect in details.

How do we explain the architects exploiting the opportunities? They ‘walked’ the built spaces that designed. What explains the need for these movements?

Within this framework, attempt is made to explain gestures and movements as manifestations of bodily-kinesthetic intelligence. To be mobile in the space (navigating) to understand space and chasing dynamic changes are dimensions of spatial intelligence that emerged clearly in these studies.

Spatial intelligence and navigation

Architect’s navigating through the site to understand and create spaces is conspicuously visible in condition 3, where the architects were allowed to move after they were blindfolded. In condition 2 they seem to virtually move on the site though in real world they were sitting.

Normally the architects would move on the actual site exploring the features. It involves learning spatial arrangement by exposure to the available space by navigating back n forth. It needs spatial intelligence to learning about the spaces by walking or navigating i.e. by being mobile in that space and then connecting these pieces into a coherent mental model of the site in the mind. Don’t we develop notions of terrain maps by driving through it multiple times and then connecting the short segments together?

It is common to navigate in the space to understand it. In normal circumstances, to avoid mental energy required to maintain the evolving ‘things’ in the mind’s eye, architects and designers convert these perceptions in to a plan and add their notes.

In condition 2 they paced the virtually created site, going back n forth, turning and built spaces around their bodies. (Refer video 4 and 5) When allowed they physically moved in the hall, treating it as a virtual site. (refer Video 6) They were trying to absorb the features and spaces on the site and also constructing on it actively and interactively. (Refer to Video 6) Both the actions were almost simultaneous.

Video 4: Evidence of creation of virtual site in the mind’s eye

Video 5: Evidence of creation of the site in the mind’s eye

Video 6: Designer actively interacting with the shape created in the mind’s eye, treating it as if it is real.

Watching the video and listening to their think-aloud transcripts during the design action suggest that the architects moved on the virtual site from location to location. In condition 2, the navigation and movements were essentially on a virtual site in their mind. The navigation and movements were essentially virtual.

For architects, virtual and physical navigation on the site remained the most popular strategy to understand the current state as well as for conceptualizing the idea. The self-navigation ensured that the processes of sensing information remained lot more active and under designer’s control.

Spatial intelligence and chasing change

They virtually built spaces around them and changed them, reacted to their own decisions as a result of their evolving thought process. Spatial intelligence demands the ability to quickly grasp a rapidly changing environment. Chasing changes and handling rapidly changing situation is common also in architectural assignments.

The designer’s actions and behavior do find explanations in the theory of spatial intelligence. A more exhaustive study across different design disciplines and levels of expertise is sure to reveal more about the different dimensions of spatial intelligence.

Yet, it does not explain fully the natural urge to use gestures and body movements. For that it is necessary to turn to embodied cognition. Let us treat designing as a corporal act. More about it in the next post.

Summing up

These four posts will take an overview of the entire series of experiments that were conducted over a decade. Revisiting the experiments was refreshing in several ways. It allowed reflecting back on the objectives of the experiments. To find reasons for the consistencies in findings was a challenging task. Besides, the findings could be now explained with new theoretical frameworks. The discussion here is restricted only to part II.

These experiments differ in several ways. Industrial designers were given a typical design of small tabletop objects. Architects were given mid-size architectural projects like designing individual detached residential or commercial buildings on a defined site. In the later case, the conditions were further varied. In first case, the architects sat in an office like environment and in the second, they were left standing in a large hall. Because all were blindfolded, they could not have used sketching as handy representations. They spontaneously resorted to gestures and/or body movements to develop the ideas, thus revealing some new dimensions of design(erly) behavior.

The protocols of the sessions clearly show a close partnership between the designer’s evolving thoughts and the process of externalization. In this case, there is extensive dependence on gestures and body movements during conceptualization of new ideas. These posts are an attempt to explore answers the question,

Why do designers use gestures and movements of the body when they solve problems?

The design problem solving is viewed through two theoretical frameworks. Part II (this part) treats design as a cerebral act and considers designer as an information processor. It shows how intelligence, and this particular case visio-spatial intelligence, play role in problem solving and explain the need for use of gestures and movements.

Gardener’s multiple intelligence framework explains part of the design behavior and decisions through the idea of spatial intelligence. The protocols show how spatial intelligence, reasoning and judgment play a role in developing, comparing and selecting ideas and taking them forward.

In absence of external representations like sketching, the ideas were represented internally as fragile mind’s eye images. Yet, designer’s managed to overcome this limitation by supporting the ideas through sequential gestures. The sessions also bring forth another dimension of spatial intelligence involved in the process of creating drawings from visualization. As we will see later in part III, this is not a trivial act.

Attempt is made to explain gestures and movements through bodily-kinesthetic intelligence. To be mobile in the space to understand its nuances is a dimension of spatial intelligence has emerged clearly in these studies. The data showed how architects continued to navigate on the site virtually, to explore the features of the site as well as the spaces that they built by shifting their locations continuously.

What was more surprising was that they reacted to and altered what they created. They could quickly grasp the rapidly altered environment. Spatial intelligence is also chasing change in perception. The bodily actions and gestures were used to not only understand the configurations of object or spaces, but also for altering them.

The sessions clearly showed different ways in which designers deploy spatial intelligence. The post explains this through several events that occurred in the design sessions. Do see this as work in progress, till the next two parts are uploaded.

Preview of the next post

Are there more effective explanations to gestures and body movements?

Part III, the next part, will treat design as a corporal act using embodied cognition framework. This theory draws observations from many fields like sports and music, to present more convincing answers to why designers used gestures and body movements.

The post will also argue that the choice to use gestures is with the designer and his decision is influenced by contribution that movements make to the evolving thoughts. Both the theories show advantages of getting the body involved while thinking, particularly when innovations are spatial in nature.

Notes and reference

  1. Athavankar U. A., (1999) Gestures, Imagery and Spatial Reasoning. In: Garo JS, Tversky B (eds) Visual and Spatial Reasoning. Preprints of the International Conference on visual and spatial Reasoning, (VR 99) MIT, Cambridge, June 15–17, 1999, pp 103–128.
  2. Athavankar, U.A., Bokil, P., Guruprasad, K., Patsute, R., Sharma, S.: (2008) Reaching Out in the Mind’s Space. In: Design Computing and Cognition 2008, pp. 321–340
  3. The discussion here is based on Howard Gardner’s famous theory of ‘Multiple intelligences’
  4. The videos of the sessions, each lasting over one hour are not included for obvious reasons, though there are references to it. Also the videos included in this post have been shown in earlier posts. They are repeated here for readers who may not have seen them before.

 

Part I: To sketch or not to sketch? That is the question

As mentioned in the last post, this post was to deal with role that gestures and movement of the body play in spatialization of design ideas as sketches or images in the mind’s eye. We are deviating from that sequence. It was felt that an overview of the series would be more relevant before we move on to the complex role that gestures and movements play in design thinking.

Let us look back at several findings presented in the last seven uploads starting from ‘Why do designers sketch?’ We plan to take a bird’s eye view of the findings and reflect on them in this post. Now that we know the role sketching and mental imagery play in design problem solving, we can now reflect on the series.

Designing as resolving spatial issues

The designer’s decision making seem to be similar to most often-cited example; being in a room and working out alternative furniture arrangements to get the most effective layout at the end. Whether you use sketching or imagery most design problems are typically resolved through spatial decisions and spatial manipulations of elements (in this case furniture pieces). For this part of the discussion, let us treat designing as resolving spatial issues to achieve pre-determined functional goals through design actions. So, whether you are creating a space (layout problem) or a smaller object, what designer’s handle/create are the spatial elements and their locations in space. In part II, we will revisit these issues through the lens of design thinking.

There are two ways to handle such an assignment. In both approaches, actions are in this ‘real’ space, which becomes a context for situated cognitive activity. Whether you use sketching or decide to work it out in the mind’s eye, the process involves perception and monitoring of representations that are being continuously created and altered by reactive design actions. Most reactions typically result in yet another new representation.

Let us focus on each of the two ways to handle spatial issues.

Sketching as a medium for conceptualizing of objects

Though this series does not directly report research on sketching, the author had worked on the role of sketching prior to exploring mental imagery. Some of the statements rely on author’s earlier research in sketching.1, 2

The first approach is to use sketching pad and pen, if handy. The designer draws the room and thinks of new locations and/or orientations of furniture pieces, then quickly draws the new layout. Looking at the sketch he reviews the new idea for its effectiveness. He will often see and react to what he has sketched and come out with a new alternative to be executed as yet another new sketch. You can capture the process as iterative cycles that involves –

“Think -> Act (sketch) -> Review -> Transform -> Think again”

till he feels that he has met the demands made by the design problem. He may take a holistic approach or take one element or one function at a time and handle complexity incrementally.

If the designer has mastered sketching, this process is cognitively less taxing. There are several reasons. First, it also allows starting with a tentative idea and incrementally moving to a more complete and complex state. In the earlier post “Thinking through the messy sketches” we discussed this approach. Second, designer does not have to budget excessive mental energy to handle the actions of sketching. Third, when he creates a record in a media outside of his body (i.e. as sketches), he does not have to budget energy to memorize and recall his past action and decision. So, if he is quick and good at sketching, it is still a cognitively economical option.

Willful control on the act of sketching, particularly in representing objects in perspective, is a critical issue when implementing a sketch. Sketching is a learnt action and is often taught in a structured way in most design schools. There is enough literature on learning to sketch. My own work on control on sketching is available on D’Source. 2

If designer is bad at sketching, this approach can be counter-productive. He needs to budget more mental energy for the act of sketching, but this is at the cost of mental energy required to come up with new ideas. (Refer to earlier post ‘Why do designer’s sketch?’) Designer’s additional efforts to judge the correctness of the sketch can demand energy budget that could interfere with the ongoing thinking of the design problem. So, designer needs ability to sketch quickly and effortlessly, as if it is a routine and natural act.

That is why in the earlier post, “Out-of-the-box ideas for teaching sketching”, focus was on learning to draw with a ‘feel’ of the line drawn via kinesthetic feedback and by diverting attention away from the pencil tip. Reducing the need to depend too much on visual feedback while sketching, allows you to budget the mental energy saved to attend to the demands of design problem solving. Several ideas were presented to explain how the entire body could participate in the act of sketching. The kinesthetic feedback generated by sketching actions can be used to ‘feel’ the correctness of the sketch being executed. It is critical to learn to sketch effortlessly and with natural actions.

The external display that sketching creates reduces the effortful activity of holding the ideas in your memory. However, the mental imagery is not completely eliminated. You have to mentally decide the new location of a furniture piece first, try to review it and quickly sketch what you have imagined. Thus you avoid spending mental energy on holding ideas in his mind and comparing them later. In short, competent sketching ability helps him conserve mental energy so that he can concentrate on developing the next idea.

There is another way to solve the problem of room layout. Imagine now that designer does not have the paper and pen or chooses not to use it. After all, the

Eureka movements may not occur when you have paper and pencil in your hand. Indeed with Archimedes, it occurred in the bathtub!

Designer will have to then depend on his visualization abilities.

Mental imagery as a medium for conceptualizing of objects

For most people it is so natural to see images in their mind’s eye, that many times they are not even aware of it. So, it is difficult to talk of mental imagery. We see images in our dreams too. Like a dreamer, the blindfolded designer also believes that he is in a ‘real’ environment and is awake.

What is the difference in the images that designers’ see when they are designing blindfolded and what everyone sees in dreams?

Unlike in dreams, the blindfolded designer is indeed awake. More important to note that the designer mostly experiences ‘constructed as well as work in progress’ images that he conceives, modifies and builds in time and space. The site that the architects visualized was ‘real’ space that experienced by walking in and out. It is a highly immersive experience that designers choose to willfully enter in.

It is important to view intentionally constructed images as unique experiences and as ‘work in progress’. This continues till the designer decides to artificially stop. Ideas through images continue to evolve, sometime even when the designer is not consciously working on it. Remember Archimedes?

What is critical is the ability to willfully control image generation and transformation operations in response to the thoughts that drive these operations.

This is why the experiences of imagery during design sessions are different from what happens in dreams. The willful control is needed to handle the unique operations that designer performs. Interestingly, he learns to do this without any formal learning programme.

Designing and the mind’s eye

Let us return to the furniture layout problem, but this time designer does not have a sketchbook with him. Thoughts of the designer continue to drive new solutions and prompt new ideas. He can try out alternatives by physically shifting the furniture pieces around, but that will be not only time consuming but also strenuous. So, the designer would have to solve the room layout problem by relying extensively on his internal resource and working out solutions in their mind’s eye.

Designer could move the furniture pieces around, reorient them, think of creating alternatives mentally. He could view them and compare the alternative solutions. All along, he has to view the current state in the mind’s eye, if necessary bring back the earlier ideas from memory and react to this internal display creatively to generate even newer ideas. He has to hold all of them in memory and compare and contrast them. No doubt a mentally stressful task.

These operations are far more taxing than recalling of the images seen earlier. You can clearly see that the visualization actions obviously go far beyond just recalling the images and inspecting them. Obviously, designer needs more complex capabilities for such a creative use of mind’s eye.

We knew so little about how designers willfully control mental imagery event. By blindfolding the designers in our experiments we forced a situation, so that these capabilities could be studied. The series is biased towards a detailed treatment of mental imagery because, unlike sketching, we were ignorant of the issues involved. If this ability to willfully perform and control operations on the images in the mind’s eye is critical for new ideas,

Should we not identify the operations that designers perform and/or learn to perform?

Fortunately, some of the operations that can be performed on mental images have been identified in cognitive studies. Most well studied operation is mental rotation of objects along three axes.3 Further, Kosslyn identified three broad operations. They included, holding the image and inspecting it (Image inspection technically referred as scanning), generating the image (image generation) and lastly transforming the images (image transform). Last one is a powerful operation and includes several types of transformations.4

Most of these operations are clearly visible in our experiments. The videos clearly show that designers and architects iteratively switch between image inspection, image generation and image transformation. They cyclically inspect the results in their mind’s eye and reflect on these images. We will take these operations one by one.

Image Inspection (Scanning)

This operation usually deals with inspecting the recalled image from the memory (LTM and sometimes STM). Ideal examples of image recalls were when SP as well as the architects were asked to describe their final design after they declared that they have finished designing. The object seem to be virtually there in front of them (SP) or in case of architects, built space enveloping them. When they described, they appeared to be ‘seeing’ in their mind’s eye and ‘reading off’ the images. Similarly, architects occasionally recalled the site plan that they were given to memorize, but subsequently these drawings were often transformed into a 3D view with all its features to see in their mind’s eye.

These designers also recalled images that are part of their memories and use them as precedents. In our case, such images dealt with previously seen, and perhaps liked, examples of objects or built spaces or their features. For instance, in student pair experiments, there is a clear evidence of pergola roof, deck, Japanese garden and water fountain as precedents brought in as images from the memory. Needless to say that these precedents and features get transformed creatively to suit the new context, when they are used in design.

Obviously, the designers are able to hold their images in their mind’s eye, inspect (or scan) them and react to them as they work with the image. They would often inspect images to make judgments, spot inconsistencies, search for opportunities to alter and manipulate elements within the image. It serves the same function that sketches as display serve. It allows designer to review his ideas, creatively react to them and iterate. What exists as current, gradually move towards effective solution incrementally. This is similar to what occurs when sketching is used. Image inspection leads to spatial judgments and all design tasks are based on these judgments.

Yet, there are logical differences because of the nature of the media, in this case the sketchbook and the mind’s eye in which the display occurs. Mental images are fragile and fadeout if you are not attending to them actively. Cognitively, to hold and inspect an image is an active process that demands budgeting of mental energy, more so, when it is done purposefully.

Because designers and architects are involved in creating new objects and built forms, they tend to often inspect what they created and then manipulate it. So, most examples of inspection are part of the process of image generation and transformations.

Image generation

If you watch the video of SP in action in casserole experiment, it shows sufficient evidence of SP’s logic, ideas and thoughts directing his design actions and gestures. These often result in generating images of the object that he was developing. He develops the shape from scratch, builds it up step-by-step as an image. SP choses to ‘treat’ the overall image of the object as ‘real’ and even physically shapes it, making changes interactively as he goes ahead!

Most of SP’s actions in image generation were incremental and became more detailed as the time passed. Observe SP working with a virtual object in the front. See video 1 below.

Video 1: The designer created a virtual object in front of him and shaped it as if it was real for the entire period when he was designing.

As he advances with his design, he fondly sculpts and alters the shape. Similarly, when the architects were blindfolded and let into a big hall, they first create a gross layouts, detail them and as they go ahead, build the 3D spaces around them.

It is common to see the cyclic process that starts with generation of the image, then inspecting it, reacting and implementing changes in their mind’s eye. That explains the incremental nature of the development of the ideas. Note that there is little change in the design process as in both cases. There is only a change in the media used for displaying the current state of design.

Most of these videos have short pauses of few seconds when they are silent. Pauses seem to be useful to judge the current state of design idea. Pauses longer than three seconds were purposeful. They are used for inspections of the images in the mind’s eye and reflecting on it. You can clearly see this in video 1. They appear to be part of spatial judgment efforts and end in actions and decisions immediately after. Typically, they end up with flurry of activity, often resulting in a new idea or a modification that had existed as an earlier idea. SP pauses several times to ‘see’ the object in his mind’s eye and then reflects on it. He ‘looks’ around to inspect what he had created and pass judgment after a pause. After one such pause midway through the casserole session, SP sculpts the shape, pauses and comments like ‘That will look interesting’. See this in video 1 above.

Image transforms

Image transformation is a critical operation on the images. It is also the most often used operation to quickly alter the contents of the image seen. Most surprising finding was that in implementing the operation, designers physically interacted with their creations in the mind’s eye. In fact such interactions were at their best in image transformation. For example, while thinking of support to the casserole body, SP’s palms represented the folding legs of the casserole and movements of the palms simulate the leg movements.5

There are several complex transformations of objects or spaces that architects and designers seem to comfortably deploy while designing. For instance, SP’s casserole video has several examples of the shape or some element of it being reshaped, chopped, moved, flipped, rotated and so on. See this in video 2 and 3 below.

Video 2: Observe designer exploring the shaping (curving) of the bottom of the casserole.

Video 3: Observe designer shifting from shaping of the lid to folding legs.

These videos include actions like selectively moving the objects or elements, manipulating proportions and compositions, changing sizes, altering colours, exploring different backgrounds and even creating exploded structure. They appear to use more than one operation, often in continuous sequences, and that too with amazing ease. The gestures come handy in the process of working with it and on it. More about gestures in part II in the next post.

While modifying their ideas and exploring alternatives, architects too performed different transformations of the images. Architects however use gestures more often to indicate and locate than to shape the spaces, perhaps because of the scale of these spaces. Earlier posts do to indicate few examples of use of gestures in shaping some parts of the building. For instance, see video 4 below. However, these instances are fewer in numbers.

Video 4: See designer shaping the entrance with hands. 

The purpose of most of the transformations is to improve the effectiveness of the solution at hand. It is typically followed by image inspection, where the designer would try to assess the implications of the changes they had just made. The use is similar to the way the sketch is reviewed, except when working with mental imagery, it is lot more difficult.

These changing mental events are available in form of fragile images in their mind. They have to hold these static as well as sometimes-dynamic images in their mind’s eye, which demand budgeting mental energy. Additional energy is required for reacting to them, altering the image with new intentions and hold on to the new image.

What does the series reveal?

The discussion so far has also helped us separate two forms of representations that almost serve similar function. Most discourses on designing focus on the first form i.e. sketching, and do not acknowledge mental imagery as serious contender as a form of representation. It is understandable. Indeed, there are no capturable external representations to write about or comment on, when working with mental imagery. One of the contributions of this series is to discover methods that give access to the actions that occur in the mind’s eye.

The series argues that sketching as well as mental imagery serve similar role but deserve to be treated as separate acts.

We hope that this separation will remove the bias towards sketching and studies in understanding the role of mental imagery in design(erly) thinking will get the attention it deserves. It is hoped that the shift of spotlight will balance the studies of role of representations.

Why is attention to the mental imagery critical?

It is known that this ability is directly correlated with creative efforts. To willfully change, transform or manipulate the image or some of the elements of the image selectively to intentions, distinguish creative people from others. Artists, dancers, architects, designers and some scientists can willfully control what happens in the mind’s eye. Einstein was known to create events in his mind’s eye that he often referred as thought experiments. Mozart was known to compose his orchestra entirely in his mind’s ears, an audio equivalent of mind’s eye. He was also able to hear and follow each instrument in isolation. There is enough scientific evidence to suggest that the ability to handle events in the mind’s eye and creative work are closely related. Finke has spent several years researching the role of mental imagery in inventions using cognitive science framework.6

Understanding nuances of visualization

We now have a clearer understanding of the term visualization and what it means to design community. It is not just externalizing ideas in a sketchbook or on a computer screen. It should include perception and manipulation of representations in the mind’s eye.

Visualization is much used and abused word. At the simplest level, it can be defined as ‘seeing the object when it is withdrawn and is no more available to be perceived.’ We are then asked to recall what it looked like. Can we restrict visualization to just recalling and inspecting the image in the mind’s eye?

Ability to recall or generate an image is not unusual. Most people are able to recall and view images. Dreams also involve recall of images. Dreamers are convinced that they are in a real environment, are awake and are experiencing a real event in time and space. Situation is somewhat similar when the designers were blindfolded, except that they were actually awake. Besides recalling, lots of people generate and see new elements in their images in dreams (also under hallucination) that they have never seen before.

Learning to willfully control mental imagery

The major difference is in the ability to willfully control the events in the mind’s eye. Architects, designers and artists are able to perform several cognitive operations on the images in the pursuit of the creative work. However, unlike sketching which is systematically taught to them, they learn to handle mental imagery with no formal training.

Can we learn to willfully control events during visualization without design training?

To handle mental imagery, at least till this time, there is no structured training available. It is perhaps partly a natural talent and partly perfected on job through practice. We have little knowledge of how willful control on imagery can be taught through training.

The focus leaning towards sketching as a preferred representation could change in future as designers learn the advantages of mental imagery as a creative thinking tool and find strategies to develop educational material for everyone. I saw the absence of training as an opportunity to convert my understanding of controlling mental events by developing visual puzzles that can be solved by use of some of the cognitive operations that we discussed. I hope to present these games and visual puzzles in some later post.

Who else can benefit from such training? There are lots of people who are involved in design but are not competent in drawing. Their creative energies could be harnessed if they could be trained to be comfortable with mental imagery.

Learning to handle sketching

Sketching does come handy when the complexity of the design project is very high. However, it demands rigourous and systematic learning to model 3D ideas as 2D sketches and draw them skillfully. I personally believe that initial training in orthographic drawing trains your mind to switch between 2D and 3D effortlessly. It has long learning period. Obviously, only few professions need it and go through such training.

To think, model ideas in mind and simultaneously sketch them is a task that needs systematic learning and practice. The focus on drawing by feel and by kinesthetic feedback and not by sight has its roots in this idea.

Note that there are other forms of representations including verbal descriptions that are used in communication. Most people use language descriptions accompanied by gestures to explain design ideas. In fact, they are effectively used in brainstorming. I tried to study gestures with minimal use of language with some success.7 But discussion on these topics are outside the scope of this series.

Sum up

This post is an overview of what was presented in the several earlier posts on sketching and mental imagery. It argues that 1] sketching and mental imagery should be recognized as two separate forms of representations. 2] The representations are a critical constituent of design(erly) thinking, 3] Both act like mediums for conceptualizing design ideas, 4] Both of these forms serve similar functions. So, technically, they can substitute one another. The post also compares sketching and mental imagery from the point of expending of mental energy. It suggests necessity of removing the current bias towards studying sketching and treat studies in imagery on par.

Focusing on mental imagery, it identifies the three broad cognitive operations, Image inspection, Image generation and Image transformation, that can be performed on the imagery. These are explained through examples taken from the experiments cited before. It argues that the understanding of visualization in the context of design should necessarily include learning to willfully control cognitive operations in response to the designer’s intention.

At a broader level, the design process has commonalities irrespective of the forms of representations used. What differ are the cognitive operations when working with sketching or exclusively in mind’s eye.

To sketch or not to sketch?

We started our discussion by exploring the role that sketching and mental imagery play as conceptualization tool. Both the forms have their merits and demerits. The choice should depend on designer’s comfort level with these two forms of representations. To be able to competently and skillfully handle both forms of representation would be an ideal situation. The complexity would not trouble the designer. The design response would be quicker. As one of the participant said in the context of use of imagery, and I quote

“I carry my problem with me all the time now”

 

Preview of the next post

We have restricted the above discussion to the implications of production and modification of display through sketching or through use of mental imagery. However, our discussion does not explain why the architects moved, walked and used the spaces they created, nor does it explain the deeper role that gestures play. In part II, will address questions like,

Why do designers use gestures and movements of the body when they solve problems? Does it support spatial decisions, visualization and design(erly) thinking? And if so, how?

Next post will discuss how these two forms of representation affect design(erly) thinking and why in spite apparent similarities, they are conceptually different.

Notes and references

  1. Athavankar U., (1992) Rediscovery the Act of Sketching: Implication of its Support to the Creative Thought Process, Design Recherche, No. 2, pp 45-60
  2. Discussion on freehand drawing http://www.dsource.in/search/content/Freehand%20Sketching Dec 21,2017
  3. Shepard, R.N., Metzler, J., (1971) Mental rotation of three-dimensional objects. Science 171, 701–703
  4. Kosslyn S., (1983) Ghosts in the Mind’s Machine: Creating and Using Images in the Brain. Norton, New York
  5. Athavankar, U. A. (1999). Gestures, Imagery and Spatial Reasoning. In: Garo, J. S. & Tversky, B. (Eds) Visual and Spatial Reasoning. Preprints of the International Conference on visual and spatial Reasoning, (VR 99) MIT, Cambridge, pp 103-128.
  6. Finke R., (1990) Creative imagery: Discoveries and Inventions in Visualisation. Lawrence Erlbaum, New Jersey
  7. Varshney, S., 1998, Castles in air: A Strategy to model shapes in a computer, in proceedings of the conference ‘Third Asia Pacific Computer Human Interaction (APCHI ’98), Japan, July 1998, 350-355

Designers imagine, touch and walk the talk

This post is third in the series three posts that explores potentials of mind’s eye by blindfolding designers and architects and asking them to develop their design proposals, completely relying on internal resources and visualizing in the mind’s eye. They extensively used gestures while solving the design problem. On the other hand, in my earlier posts on teaching sketching for designers, I had focused extensively in getting the whole body involved in the act. I was interested in the relationship between the body movement and the thoughts.

Leaving the architects standing

To broaden the scope, I treated hand gestures as only a part of the larger aspect like conscious moving of body. I decided to refocus on the role that the body can play in solving design problem. In all my previous experiments, architects sat on a chair and then they where blindfolded. This had actually constrained their movement of the body. In these experiment, they were then blindfolded and left standing in a middle of a large empty hall, wearing a wireless collar mike in. I had hypothesized that they will move around and use their movements in some constructive ways.

Experiment design

This time we invited four architect volunteers with two/three years experience after graduation to participate. They were given a site plan with no indication of what functions to accommodate. Site given was rectangular, with one curved corner along the access road. It had gentle contours. The plan indicated trees (some to be preserved) and location of existing storm water drainage. The architects were asked to remember these features and verbally recalled them before they were given project requirements.

The design problem given was a hangout space with a cafeteria for student community on the campus. It also had a space that could be used by the students to display outcomes of their hobbies. The detailed account of the experiment and the results were published.1,2

The questions addressed were,

Will the freedom of movement impact visualization strategies?

Will the architects move their body to ‘feel’ the space that they develop in their mind’s eye?

If yes, can these movements be mapped to the spaces that they develop?

The design of the experiment ensured that the architects had to completely depend on their mental imagery and internal resources. We were exploring how the architects would use the freedom to move constructively and take advantage of the space in the hall. We were also looking for changes that may occur because the built spaces were much larger than their body and they have to be designed from inside as well as from outside.

Indeed there were visible changes in the strategies that architects used in visualization and in solving the design problem. They constructed a site in their visualization intervened in it creating architectural spaces around them, even walked around in the spaces created and when necessary altered them. Looking back, it seems like a peculiar interaction. The use of gestures was differed from the way Industrial designer used them. The difference is that the built spaces were too large to be physically sculpted! Their gestures and movements had different functions now. There were differences in the way four architect approached the idea of using the space. The visualization strategies also differed. We plan to discuss some key observations like 1] their presence on the virtual site that they visualized; 2] the strategies that they evolved to visualize the spaces and details; and 3] the way they interacted with their creations. The interested readers could refer to a detailed paper on this experiment.3

‘Presence’ on the site: Virtual or physical?

The video suggests that all the four architects were also seeing themselves virtually standing on the site, visualizing built spaces around them. How can a person be on a virtual site? This might sound strange, but it is true. Yet it needed to be proved beyond doubt by relying on concrete visible evidence of the designer was bodily present on the virtual site. The proof came from several sources in the video data and the transcripts.

We analyzed the transcripts to locate words and phrases that contained frequent references to self, such as ‘I am’, ‘on both sides of me’, ‘front of me, left side of me’, ‘on my right side’. (See figure 1) Surprisingly, most architects dropped references to north and referred everything with respect to the body, its location and orientation. In all four transcripts, there is only a single isolated reference to the North direction. (See video 1) In practice, North direction is critical in architectural practice and comes up often in conversations. Similarly, heat gain, ventilation and rain directions are worked out with North in mind.

 

Figure 1: Shows how concurrent speech that reveals the architect being on the virtual site

 

Video 1: There is only one reference to North direction in the four case studies is unusual. Their virtual presence on the visualized site was so dominating that most of them did not refer to North.

Occasionally the architect was asked a question ‘Where are you now?’ When declaring their location, they relied on the virtual space. They declared their locations with references to the features of the site or of new design that they were developing in their mind’s eye. (See video 2,3) Besides, when asked to go back to a specific feature or a location in the proposed design (like entrance), they made finer adjustments in their movements to reach almost the exact spot in the physical world! (See video 4) Such events further supported the idea of presence on the site. The boundaries between the physical world and virtual world seem to have blurred.

 

Video 2, 3: The architects were asked during the blindfolded session “Where are you now?” Observe the answers. All the architects were on the virtual site that they visualized and they were clear where they were located on it. (Architect in video 1 also ‘knows’ where he is.)

Video 4: Watch the architect making finer adjustments in his locations, before finally declaring where he is.

This presented sufficient evidence of the architects being on the virtual sites that they visualized in his mind’s eye. But we still need answers to the question,

Why were the architects on the virtual site? And how did it contribute to design decisions?

Beyond presence

It was not a mere static presence that could be observed. They walked up and down; exploring the site and conceptualizing built spaces around them. They seem to be imagining themselves constructing and using the spaces that they constructed. Most started with developing a built space from inside first but when required, they came out of the building to see what they had created. (See video 5)

Video 5: When working on the outside of the built space, architect steps out spontaneously.

How does one get valid clues to what they were seeing and experiencing? We mapped the combinations of gestures, body part movements and speech strings from the transcripts, second by second, and plotted the contents with respect to the site plan. (See earlier figure 1) The combined rich descriptions typically indicated site landmarks and new built features. On the other hand, simultaneous gestures, hand, neck and body movements showed the locations of these features. (See figure 2) This helped us reconstruct and map what they were seeing at that a given point of time and how it transformed. (See video 6)

Figure 2: Shows how concurrent speech with gestures and movements recorded and later mapped to reconstruct experiences.

 

Video 6: Watch how the site moves as the architect turns. With the site, all the built features also turn.

Virtually being on the site allowed him to operate in the ‘virtual’ space that he could build on, alter and experience from inside and outside. This was lot quicker than if had chosen the normal path of sketching these out and altering them. Watch the videos carefully. Their visualizations were far quicker to generate, lot more pliable and could match the speed of the evolving thoughts, ideas and fantasies.

Visualization strategies

The architects had to adapt to the unfamiliar situation spontaneously. So, to imagine consistency in the visualization strategies used by the four architects looked unrealistic. Nobody had any training in handling mental imagery. (Unlike sketching, the pedagogic implications of use of mental imagery have not been explored and so it had never been a part of any design curricula) Most often, it accompanies thoughts spontaneously and remains a very personal experience. The architects had to adapt to the new of ways on the spot. So, instead of searching for consistency in visualization strategy, we decided to look for differences between architects.

Architects moved on the virtual site as well as in the physical space, but the speed and the vigour of the movements and lengths covered were different. At times even the purposes of the movements seem to be different. The strategies could be grouped into three classes. Each architect treated one of the strategies as primary, but switched occasionally to other strategies when they found it convenient.

Strategy 1: Visualizing, working and moving in-situ

Two of the architects built the virtual building spaces and layouts around them and comfortably moved within it. (See video 7) They created the spaces and walked into them and altered them if necessary. The distances and spaces around were very ‘real’ to them. When asked to sum up their design, they were able to walk into the virtual built space that they created almost without error. The results clearly show that the two architects had developed a bodily feel for the virtual spaces that they built and oriented their body within these spaces. (See video 8)

Video 7: Architect created a virtual site and built the spaces around him, while pacing up and down. He is clearly developing his ideas by being inside the built spaces.

Video 8: Architects were uncomfortable if they miscalculated the body orientation and corrected it. Video shows one such example.

The visualization, particularly when in-situ, was very accurate. In summing up the design, when the architects were asked to sum up is design ideas, he could walk the same spaces with surprising accuracy. So accurate was this movement, that in spite of the eye mask, when asked to go back to the entrance, the architect carefully moved back and adjusted his steps to reach the correct location in the physical space! (See video 9)

Video 9: Watch architect going back to the exact location that she had planned as entrance. The last bit of adjustment showed surprising accuracy in the bodily feel.

When asked to describe elevation of the built space, one of them stepped out of the virtual space, to ‘see’ the building elevation and the entrance and reworked on it. (See video 10) Incidentally, these are the only gestures that sculpted the shape of the entrance interactively.

Video 10: Architect steps out and sculpts the entrance to the cafeteria. It appears as if she was developing the ideas in-situ.

Strategy 2: Carrying site on the shoulder

Two of the four architects moved into an in-situ space. Other two moved, but not with the same purpose and vigour. Their moving was a free wheeling movement. It had no direct relationship with the visualized space around. It appears that when they moved, they always had site with them. It appeared as if they were carrying the site with them as if it was attached to their body. So, the site moved and rotated, when they moved and turned. How does one prove such a strange conclusion?

Using the combination of concurrent speech strings, with gestures, neck and body movements, we could map their image that they were seeing in their mind’s eye. These dynamic map shows that when the architect moved or turned, the site also moved and turned, as if they were carrying it with them. (See video 6 shown earlier) It is not clear what was the advantage of this strategy to the designer, but they do look comfortable. Perhaps they were not aware of their own strategy of site moving with them.

Strategy 3: Shift over to a scale model

Some of the architects occasionally shifted to another strategy. From the body movements, gestures and speech, it appeared as if they were working on a scale model. They were outside the site now and perhaps viewed it from the top and created built spaces. Their hand gestures either showed location of the spaces or described shapes (often site contours, structure, roof) Gestures suggest that they are perhaps working on a small-scale model floating in the air, at a short distance from the body. (See video 11) This is closest to the earlier discussion on industrial designer (SP) working on a virtual model in front of him.

Video 11: The gestures suggest that the architect is working on a model close to her body.

General Observations

By allowing the architects to move around, we made the canvas larger. It revealed many new things. We derived three major visualization strategies from the data. For most, though one of the visualization strategies remained as a primary strategy, they were able to effortlessly switch to the other strategies when required. In spite of differences in strategies, all of them were able to solve the problem effectively.

With the broader canvas, the body movements and gestures were used to perform newer functions. They pointed out locations and directions of features that designers was conceiving or referring to. They indicated shapes and in rare case, they even sculpted the shape and the contours.

We also realized that the concurrent gesture + speech combination is so rich in information that it was possible to map what they were seeing in their mind’s eye using their body as a reference. This material acquired richness and revealed lot more of what was going on in the designer’s mind and what he was visualizing. The speech + gesture combination clearly revealed that the architect built spaces, while imagining a virtual site in his mind’s eye. He also used his body as a reference to locate things around.

In my earlier writings, I attributed this to ‘thinking with body’. Reflecting back now I found new explanations that closely match how and why architects and designers performed the way they did. It would be worth going to these areas by considering all the findings of the three articles together. We plan to do this in the next post.

I have come to believe that one of the important characteristics of designerly thinking is the inner urge to solve problems through the design expertise. The attitude can be captured as ‘whether there is a client or not, I want to, and need to solve the problem’. Designerly thinking is based on ambition as well as the excitement solving the problem, a point that writings on design methods miss completely. So, the constraints, like not allowing usual processes like sketching, blindfolding or left standing in the middle of the hall, don’t seem to bother them. None of them complained about the constraints imposed. They took the challenges in their stride, and in fact adapt to them, by spring back with spontaneous alternative strategies and approaches.

Sum up

This article is third in a series of posts that presented findings of the experiments on designers and architects, when they were blindfolded and asked to design. Sufficient evidence was presented in the previous two posts to conclude the most of them were able to handle design problems and come up with solutions and that too with amazing dexterity. This post focuses on the next objective, the role that body and its deliberate actions can potentially play in supporting visualization.

This time four architects were given a design problem and were let into a large hall. They were blindfolded, wore a cordless collar mike and were asked to work on an architectural project. The experiments were video taped and all the speech strings were transcribed for detailed analysis. The expectation was that they might use the freedom to move around while thinking of the solutions and this might impact their visualization. Indeed, they spontaneously responded to the new situation with different visualization strategies.

There was sufficient evidence in their speech as well as in the body movement and gestures to show that they were present on the site visualized in their mind’s eye. When asked, they would declare their location in the built form that they were developing. So, it is not surprising that they located everything around them with respect to their current location and orientation on the virtual site. The way they operated, even this virtual presence looked almost real to them!

Some of them used a strategy where they built the site in their mind’s eye, developed their ideas as virtual built forms, manipulated them in their visualization, but actually moved and interacted with their creations through actual physical movements in the real world. So accurate was their mapping between the virtual and real world that, when asked, they could physically walk back to the exact location in the built form and it would also tally with the physical location in the hall!

Another popular strategy was when they moved they carried the site with them. The site turned when they turned. Though the physical movement did not impact visualization, it was probably required to keep them active in the 3D space on the virtual site. Last, and perhaps the least used, was a conventional strategy of working on a small-scale model like situation. The fact that it was rarely used is surprising; as most of them are used to working with the scaled versions of their creation all the time, and that is how they are taught to develop ideas.

This data showed that body movements were far more pronounced and gestures played a supporting role. There were rare occasions when the gestures were used as tools to model the idea. Perhaps, the bigger size of the built form and the fact they had worked primarily from inside the spaces may have made it difficult to use gestures as shaping tools, the way industrial designers could.

The post concludes by listing the architects’ actions, visualizations and particularly the movements in the physical world. In the next post, we will review the findings of all the three articles in this series through the theoretical framework of spatial intelligence and embodied design.

Preview of the next post

The next post will take a bird’s eye view of experiments on imagery. We will address following questions.

While visualizing, how do designers benefit by use of body movements and gestures?

Why do they feel it necessary to move the body?

Does it support spatial decisions and design thinking?

The theoretical support for the mental events in this series come from work on forms of human intelligence and from findings in cognitive psychology. We will touch areas like 1] Spatial intelligence as well as; 2] Embodied cognition (We touched this in earlier post “out-of-box ideas to teach sketching”) as well as its spin-offs like embodied design and imagination.

Notes and references.

1 Athavankar. U. (2008) Exploring the boundaries of spatial intelligence, Conference on Research and Training in Spatial Intelligence, Evanston.
2 Athavankar. U., Prasad B., Guruprasad.K., Patsute R. and Sharma S. (2008) Reaching out in the mind’s space, In Design Computing and Cognition ’08, Eds. Goel A., Gero J, 321-340. Springer.
3 Athavankar, U., (1999) Gestures, mental imagery and spatial reasoning, In Visual and Spatial Reasoning, MIT, Eds Garo and Barbara Tversky. 103-128. MIT

 

 

Immersive mind’s eye experiences

In the last post we reported an experiment where a mid-career industrial designer was asked to develop his product idea when he was blindfolded. We saw how he successfully solved the design problem; and that too with amazing dexterity. The experiment objectively proved that he completely conceived the idea in his mind’s eye. It revealed the potentials of learning to handle mental imagery in design problem solving.

Mental imagery can potentially offer an effective alternative to sketching. However, such a conclusion would be termed a bit hasty considering that it is based on a single case, particularly because the results did look unbelievable.

This post reports efforts to dispel this doubt through a series of follow-up experiments with same or similar objectives. Later, going beyond, it also explores newer and more adventurous objectives. We will approach the finding in steps. We will start with the unfinished agenda first,

Could successfully conceiving and completing a design when blindfolded be considered freak results?

So, we decided to repeat the experiment. We invited SP again and gave him a different and a little more complex design project. SP was asked to design salt and pepper dispensers and common dinning table crockery; all stacked in a compact stand on the dinning table. This is a common product used by many middle class households in India. While the design task changed, rest of the experimental conditions were deliberately kept identical.

The results confirmed the findings of the first experiment. This time too SP sculpted his ideas using vigorous gestures, though the shape developed was much more complex. He also chose to focus on production using complex injection molding process and managed to completely avoid undercuts. All this, when he was blindfolded! The findings were triangulated as before, using transcripts and further supported by asking other designers to decode the design idea based on transcripts.1

The results of the second experiment unequivocally proves that these results were not freak instances and it is possible to generated design ideas and design details in the mind’s eye.

We invited other industrial designers and gave them same or similar design problems. Almost all of them were able to complete the design task when blindfolded. All of them were mid-career practicing designers in their 40s and above, with lot of product design projects behind them. Only one of them, in his fifties, said he would have preferred to sketch, but did solve the design problem effectively. It did give us sufficient evidence that,

Mind’s eye can serve as an effective substitute to a more popular alternative like sketching. But,

Can the success be explained because these products tend to be small in size and thus could be visualised and sculpted as virtual models in the front?

Could these results be attributed to their extensive experience as designers?

To eliminate these possibilities, we decided to offer similar experimental conditions to those who handle 3D objects, like architects. Again, I was not sure that it would work.

Architectural design problems are a different game

Architectural projects have different nuances. Unlike industrial design problems, the buildings tend to be client specific and are not mass-produced (in India). They tend to be large in size and have to be visualized both from inside as walkthroughs, as well as from outside. Of course new elements entered with architectural projects. A site for the building had to be specified, which they were to remember and recall before the project requirements were given to them. There were additional complexities like terrain conditions, climate and light that needed specifying region as well as north direction. As we will see later, these factors influenced visualization.

We gave two types of projects, like 1] give a site with specific size and site features and ask them to conceive the building and 2] give drawings of an already built space and ask them to develop interior layouts for a specific use along with furniture concepts. We had to make sure that the architects would be familiar with the functions they were asked to house. The building projects often included public spaces like information centers or large or small secluded bungalow on sea front or on a contoured site. The interior projects included crowded, but informal student hangout spaces and cafeteria.

Most of this work has been already published as research papers.2,3 So we plan to only include a short summary here, mainly contrasting it with industrial design project. None of the architects were perturbed by the strange experimental condition of blindfolding. They went on developing building ideas in their mind’s eye.

We realized how selective the mental imagery is. It often displays what is relevant to the context. Typically, the people imagined were actually stereotypes and had specific role to play. These stereotypes performed their assigned role in the spaces created, as if the creator was testing his layouts. Post session interviews confirmed that people were always appropriately dressed to match their defined role, but were as a rule faceless.4 Their dresses were important to establish their roles, but the faces were obviously not relevant to the role or the functions they performed.

Another major difference is the focus on controlling light and creating ambiance using natural as well as artificial lights appropriately. Architects not only work with spaces that are inhabited and used, but plan interesting lighting situations contributing to the ambiance. Creating such experiences is so much part of their routine, that its domination in mental imagery is not at all surprising. Indeed, their images were vivid experiences with detailed ambiance and were populated with people.

Designers, and particularly architects, depended on designs that they have seen and ‘noticed’ earlier and used them as precedents to develop new ideas. Some of these precedents come from their own previous successful works and from works that they have seen in design journals and as well as visited in real life. More popular were precedents that come from their favorite architecture gurus (masters). What they bring in through these precedents are interesting space organizing principles, lighting and ambiance or sometimes specific innovative architectural features of interest. Some of them tended to use analogies and metaphors in working out ideas that gave distinct edge to their solutions. I was taken aback by the ease with which they could handle the constraints of eye mask. Their verbal protocols (descriptions) as well as post-experiment interviews were full of emotions and drama.

Overall ‘seeing’ faceless people using designed spaces, effective use of precedents and creation of ambiance through controlling lighting dominated architect’s visualisations.

This may appear as a short anecdotal deviation. I could not resist blindfolding myself informally. So, I asked a student of mine to frame an architectural design problem and I blindfolded myself. The session lasted for over an hour. The experience was deeply immersive. The spaces I created were visualised in the evening light, which appeared to have been automatically selected. Besides functional layouts; the ambiance and time of the day became the focus. Interestingly, I was not aware of the actual time that I spent in the session, nor the time of the day when the experiment was actually conducted. I was of course careful not to include personal experiences in any of my research writings.

Working with architecture students

Note that the initial experimental work involved architects who were 35 plus, with varying experiences of design practice. (It was more of convenience sampling)

Is it then likely that they performed so effortlessly, because of their professional experience?

To eliminate this possibility, I shifted my focus to working with architectural students in their 3rd year. (age roughly 19 to 21 years), just when a design problem of relevant magnitude is introduced in the school design studios.

To replicate on larger audience of students, I had to change the experimental protocols. Video taping each session independently and analyzing transcripts   was beyond available means at my disposal. Besides, we had sufficient evidence from the earlier experiments that it is possible to design using mind’s eye. We did not have to prove that again.

So, I made student pairs where one of them was blindfolded and the other took notes, but only intervened for clarity when required. There were two conditions that we varied. First was to create pairs with boss and assistant relationship and the second was to establish partnership equality. Pair was separated after the design was declared complete and asked to independently draw the idea that they thought was final. We then compared these final sketches.

Overall, even these young students could effortlessly handle the project in blindfolded condition. (See video 1) There were surprising similarities in the sketches drawn by the pairs. The major deviations were in the scale of the building and the way it fitted on the site. (See figure 1,2) Most students were fluent and could explain their ideas to their partners verbally, often accompanied by gestures and sometime use precedents.

Video 1: Pair with one of the student (girl) blindfolded. The second acted like her equal partner. Watch her gestures and references to her body. Hear the description carefully to look at how the ambiance is emerging.

Figure 1,2: After the design assignment was completed, the pair was separated. The eye mask of the principal designer was removed and both were asked to sketch the design idea that they had mutually agreed on. Figure 1 shows the sketched plans and figure 2 shows the 3D view drawn by the blindfolded and sighted designers. The similarities are difficult to neglect.

Blindfolding the classroom

I became little more adventurous to explore what would happen if I blindfolded the entire architecture classroom (studio).5 There were 17 student pairs with each designer trying to explain his ideas verbally to his partner often with gestures. In most pairs, blindfolded student tried desperate tricks to explain his ideas using whatever means he could think of. (See video 2 and 3) In a closed classroom, with everyone speaking simultaneously, the noise it generated was very high. All of them were so much immersed in the process emotionally, that the commotion around did not disturb any of them. That is how immersive the imagery experiences can be!

Video 2,3: Pairs in immersive state figure out interesting ways to communicate their design ideas. Listen to the background noise. No pair was disturbed by it.

Leading to more adventures with sharing images

In the pair format of the experiment, we discovered new possibilities. These results indicated that the pair could share the mental images of creations, so far considered private. These pointed to exciting possibilities of shared imagery playing a role in teamwork and give the research a new direction. The question that we asked was,

Could pair in a team share a common image? If so, could this open up new collaborative possibilities for designers not too comfortable with sketching?

We started with co-design as our objective. So, in the first experiment, we gave an architectural problem to pair of professional architects, but separated them into adjoining rooms, connected through an audio or a limited video link. Both were not familiar with each other, nor did they have opportunity to meet before the experiment.6 The brief was to develop an information center for a historic monument across Mumbai west coast. They discussed the project over the audio link, discussed solutions and selected the best option. Unlike in the past experiment, there were scheduled pauses where the experimenter asked them the state of the design at those points. These breaks had some surprising points. In one such break, we asked one of them to guess ‘In the evolving space created, where is his design partner?’ He was prompt in his reply and said that ‘His partner is hovering around an indicated place on the site.’ We instantly checked this with the partner, and he confirmed this independently! When they agreed that the design task is complete, they remained separated and were asked to independently sketch out their shared final design idea. It was followed by post experiment interview. We also repeated the experiment pairing with two filmmakers with similar success.7

In this experiment, they could see each other’s sketches over the video link, but not see each other. The discussion was often based on 1] the partner’s reaction to the words used as well as 2] the video link access to each other’s sketches and diagrammes. Idiosyncrasies of the sketches did not hinder discussion. The results are significant for work in participatory design, as it proved that a pair of technologically linked designers could work together on a common project, share a common image and evolve a common solution. We then became bolder in our objectives and decided to investigate,

Is the access to each other’s sketches critical? In other words, was the video link critical?

Using substantially similar experimental protocols, we made a minor but significant change in the next experiment. We cut off the video link. They worked separately and in isolation, but could only discuss over audio link. In a way, they were required to figure out the evolving images in their partner’s mind and influence them with their new ideas till they agreed. Their final sketches showed that they were able to share a substantial part of new design proposal, though they had no opportunity to see what the other architect was sketching. There were of course some variations in the scale of the building.

The experiment did confirm our hunch that they had not only shared a common design idea but the image/s in their mind’s eye, so far considered as personal and private experience.

Anecdotal support

I am listing a few that I encountered in my experimental studies, hoping that others may want to take the idea further. Out of curiosity, we were simultaneously interviewing eminent film set designers and even eminent filmmakers. (We could not have expected them to sit through the elaborate experimental setup) These interviews contain interesting anecdotal information. Indeed, anecdotes do not make good science, but they do give push to newer experiments and ideas.

Most eminent artists seem to depend on mental imagery during creative phase. Not so surprisingly, filmmakers are only conscious of what the viewing frames will contain when the camera moves. They visualize details within a frame and had no idea of what was outside the frame of the camera, nor were they bothered about it. They use lot more precedents from their life in the film ideas that they develop, than what architects do. All of them seem to have library of images that they tend to fallback on for ideas. Interestingly, they do ‘see’ movements in the mind’s eye (shaped as a screen), visualize and hear background music scores and had hunches on who the music director could possibly be!

Another, eminent Indian classical dancer mentioned how, when she is visualizing a new steps for her own performance, she uses a mirror and her bodily actions to test her visualization. This is common. What turned out to be a surprise was when she choreographs for a group dance. She would then imagine a transparent box (roughly proportioned like a stage) in which she visualized her group movements. Surprisingly, she would view this box from a higher line of sight and not from the usual audience angle.

Similarly when asked, an accomplished Jazz musician could hear eminent musician playing a tune in her mind’s ears. Interestingly, when asked to imagine her playing piano while mentally hearing the sound, she said the tune was smoother when mentally playing it. She also felt frustrated that she is not able to reproduce this smoothness, when playing it physically. Interestingly, with no prompt, she imagined the keys of her piano moving up and down on to her tune! This does indicate somewhat autonomous nature of events in the mind’s eye.
Sum up

 In the last post, we reported experiment where a midcareer industrial designer was asked to develop his product idea, when he was blindfolded. We saw how he successfully solved the design problem and that too with amazing dexterity. The experiment objectively proved that he completely conceived the idea in his mind’s eye. It revealed the possibilities of learning to handle mental imagery in design problem solving.

Mental imagery can potentially offer an effective alternative to sketching. However, these conclusions could be termed a bit hasty considering that they would be based on a single case, particularly because the results did look unbelievable. This post reports efforts to dispel this doubt through a series of follow-up experiments with same or similar objectives.

First, to eliminate the possibility of the first results being criticized as freak, we invited the same designer (SP) to work with identical experimental procedures and protocols, but with a different design problem. SP was asked to design salt and pepper dispensers and common dinning table crockery; all stacked in a compact stand on the dinning table. Like in the first experiment, the sketched results were validated through several independent routes. Besides evolving an effective solution, SP concentrated on complex production problems, where he used injection molding in plastic while avoiding undercuts. All this when he was blindfolded! The results unambiguously confirmed the earlier findings of the first experiment.

The post then goes on to explore the next step with newer and more challenging objectives. This was achieved through series of new experiments with allied design professionals, like architects, filmmakers and so on. To begin with, we offered similar experimental conditions to architects tackling an architectural design problem. These problems are qualitatively different from what SP handled. First, the built forms tend to be very large. Second, they have an outside and an inside that is explored through mental walks. Third, architects face two kinds of design problems; creating a new built form and developing interiors in already built spaces. In this series, we tried both types. Lastly, built forms are not always sculptural (except in parts) and could not be shaped by gestures the way SP handled products. At best, only some elements of the building could be sculpted.

In spite of these differences, the architects conceived their built forms and interior spaces effortlessly. However, there were some striking differences. Besides solving spatial layout problems, they spent a lot of time visualizing and controlling lighting conditions and ambiance. Appropriately dressed stereotypical people populated most of the spaces they created, but they were always faceless. They often used precedents and some used metaphors as a design strategy.

Most of them, who participated were practicing architects/designers and were above 35 years of age. To eliminate the possibility of design experience influencing the results, in the next series, we decided to invite younger age group, mainly students in their third year of architecture. None of them had problems completing the design project. The results broadly confirmed our earlier findings.

Becoming little adventurous, we decided to explore blindfolding the entire architecture classroom. Videotaping each pair was beyond our means, nor was it a practical route. So, we altered the experimental protocols to pair two students with the principal designer being blindfolded and the other acted as a junior or an equal partner. They were separated when the idea was completely developed and were asked to independently sketch what the pair had jointly arrived at. These sketches were then compared. The similarity in the sketches presented by the pairs was apparent.

These experiments proved that experience was not an important factor and even at that young age, students could conceive their design ideas in their mind’s eye. Like their seniors, they also focused on creating ambience by controlling the light and landscaping the interiors. The spaces were conceived from outside as well as inside and they often walked through the spaces that they created.

The results confirmed that age and experience does not matter. But it proved something more significant, i.e. it is possible for a pair to share a common image. Realizing that this could have potential impact on work in co-design, we explored this direction further. In the first series, the pair was connected with video and audio link, but in the last one, we cut of the video link. In spite of this, the pair was able to share a common image with reasonable commonalities.

Finally, the post reinforces the findings that design ideas of reasonable complexity can be completely conceived in the mind’s eye. Besides, these efforts proved that the results reported in the earlier post were not freak occurrence.

Preview of the next post

 So far we have conclusively proved the abilities of the architects, designers, filmmakers and even design students in overcoming difficult situations like working with an eye mask and complete the entire or substantial part of design in their mind’s eye. We also looked at how they used gestures in different ways to help them think and reason out ideas. In a way it supported the idea of embodied cognition.

In the next post, we have pushed architects further to explore how they can use body and gestures in solving the design problems. The idea of cornering the architects with new challenges was not a bad one. As you will see in the next post, it did bring out interesting strategies and thinking styles.

Watch young architects using their bodies and movements in thinking of solutions with amazing dexterity!

 If you liked this series of posts and if you are looking forward to the next post, do follow the blog. That ensures that you get intimation as soon as the new post is uploaded.

Notes and references

  1. Athavankar U., (1997) Learning from the way Designers Model Shapes in their Mind, Cognitive Systems : from Intelligent Systems to Artificial life? J.R. Issac and V. Jindal, Tata McGraw-Hill, New Delhi, pp 221-232
  2. Singh A., (1999) The potential of mental imaging in architectural design process. In: Proceedings of International Conference on Design and Technology Educational Research and Curriculum Development, IDATER 99. University of Loughborough, England, pp 230–236
  3. Athavankar U., Garde A, Kuthiala S (2001) Interventions in the Mental Imagery: Design Process in a Different Perspective. Proceeding of the 5th asian Design Conference, International Syposium on Design Science, Seoul National University, Korea.
  4. It is not that industrial designers do not ‘see’ people using their products. In the first experiment, SP also reported such incidents. However, they largely depended on their own mental simulations to test the ideas.
  5. Athavankar U. & Mukherjee A., (2003). Blindfolded classroom getting design students to use mental imagery Human Behaviour in Design, U. Lindemann (Ed) Springer, 111-120
  6. Athavankar U., Gill N., Deshmukh H., (2000) Imagery as a Private Experience and Architectural Team Work. In: Scrivener S, Ball L J, Woodcock, Springer-Verlag (eds) Collaborative Design. London, pp 223–232
  7. Bhedasgaonkar, M., Jalote A. and Athavankar U., 2000, Co-design: sharing mental Imagery ?: Team Thinking in Filmmaking, Proceedings of CoDesign 20000, Coventry September 11-13, 2000, pp.87-92.

 

 

Can we prevent designer from sketching?

Blindfolded designer in action

So far, we discussed how sketching contributes to the design problem solving and partners with the evolving thought. It externalizes thought, but surprisingly also contributes to clarification of the emerging thoughts. Sketching works as partner in the creative process, support unobtrusively and ideally should demand little expending of mental energy. With their unique role, we characterized such sketches as a thinking tool, a class by themselves. We also divided the act into smaller actions and modeled them as a cognitive act. We looked at how budgeting of mental energy needs to be balanced between problem solving and creating instructions for sketching the idea.

The thinking sketches look like inevitable partner in design thinking and design problem solving. There are no reasons to doubt these statements. However, it automatically implies that you need quick and effortless sketching abilities, if you want to choose design or architecture as a career. There is more than adequate support to such a statement. There are lots of examples of great architects and designers being extremely good at sketching. (and even drafting) FLW is a good example of this. This justifies out effort listed in the last post that documents innovative ways of how sketching can be taught to students who may not necessarily start with good sketching. But

“Can such a statement be generalized. Should careers in design be restricted to those who are already good in sketching?”

Some books on great designers do list their sketching abilities. There is not doubt that sketching is a good skill to have. This has also been a subject research and documentation. (i.e. Leonardo’s sketches) However, why all books on great designers do not discuss their sketching abilities nor show their sketching samples? Should we interpret this silence, as ‘They do not have great sketching skills to talk about’ and yet they have succeeded?

To investigate this, it was possible to study sketches of all great names in design, but this was beyond my reach and means. Besides, there are several creative people connected with art and design, who do not use sketching. I intuitively felt that it was not fair to expect everyone to be competent in sketching. So, I decided to look at

“Is there an alternative route to sketching? What will happen if you stop an architect or a design from sketching?”

I decided to pursue this alternative route to discover answers through properly designed experiments in which designers participated. This led to series of studies and experiments that I conducted between 1995 and 2008 AD. The results inform us of the untapped capabilities of human mind and special abilities used by designers and architects in solving design problems. Reflecting back, I thing it was fortunate that I asked this question to myself. Else we would never have known the hidden capabilities would have remained unreported.

The primary research questions that we address were,

“Would the designer solve a design problem when blindfolded and thus prevented from sketching? If yes, how?”

Capturing designer in action

This demanded developing a strange scenario and a new experimental protocol, where the design process was captured live when the designer is solving a typical design problem.

Design problem

The design project was taken from local company manufacturing range of Thermowares as consumer durable and gift sets. Their range included insulated containers, food boxes and vacuum flasks used in households. The problem selected was to design a casserole that can retain food temperature. It should be used to store as well as serve hot/cold food item and can be offered as a gift item. One of the leading local industrial designer voluntarily participated in it. He is referred to as SP.

Challenge

We needed to develop experiment protocols to ensure beyond doubt that the actual visualization of the solutions and design actions are completed when the designer is not allowed to sketch.

SP was given a written brief to read and it was cross-checked that he remembered it thoroughly. He was then told that he will wear an eye mask and will develop design in blindfolded conditions. He was asked to concurrently speak-aloud whatever is passing through his mind.1 All the steps in the session were video recorded. When he was satisfied with his creation, he was told to verbally sum up the final design solution. Then, the eye mask was removed and he was asked to sketch his ideas as quickly as he can and not add new ideas during sketching.

Watching design action

Few general observations before we move on to surprises. SP was comfortable with the think-aloud process and finished designing in 56.5 minutes, after reading and recalling the project brief. Further, he took 7 to finish sketching his idea at the end. The session was fully recorded on video with a front and a top camera.

What happened was far beyond what we expected. SP developed alternatives for every design feature, evaluated them by simulating its use mentally and selected the most effective design approach to proceed. (See video 1) He played with different features, manipulated their locations in space to explore the most effective configuration. Halfway through, while simulating the use of design in his mind, he discovered a major functional mistake and reconfigured the new solution quickly, all this in his mind! A quick glimpse at the entire video record showed that throughout the session SP meticulously and mentally sculpted the shape and made sure that he responded to all functional and even production issues. (See video 2 & 3)

Video 1: SP developed features keeping function in mind, evaluated them by simulating its use mentally and selected the most effective design approach to proceed

Video 2: Watch SP as if he is sculpting the shape with his hands, as if he interacting with something real in front of him

SP 3, SP 5

 

Video 3, 5: Watch SP anticipating even production related issues. Later he assembled the product components with his hands.

He was comfortable taking decisions based on aesthetic judgment, decided on colour and product graphics before he declared that he has completed the design assignment. (See video 4) The detailed account of the session has been published in a paper earlier.2 The only visible difference was that he appeared to be developing the shape, features as well as manipulating and assembling the parts in his mind using hand gestures! The videos bear this out.

SP 4

 

Video 4: Watch SP take form decisions and refer to product graphics.

Let us return to some of the questions that we started with.

How do we make sure that SP completed the entire design in his mind when he was blindfolded?

How do we make sure that he did not add new ideas during the 7 minutes, when he sketched the final solution? After all, sketching does prompt ideas!

That he was not given enough time to add new ideas during sketching is not sufficient to prove the hypothesis scientifically. To ensure this, the video recording of the session was completely transcribed and later systematically coded. We then listed all the features in his final sketch and matched them with descriptions and references to each feature in the transcripts during the blindfolded conditions. To doubly make sure, we also checked references to features and descriptions in the final summing-up part of the transcripts. Results were surprising. Ninety-five percent of the features in the final sketch had unambiguous matching descriptions in the transcriptions. Obviously, SP had visualized all details in the mind’s eye.

Going beyond doubt

To make it triply sure (validity through triangulation), we asked two new designers to look at SP’s video after the final solutions were edited out. Based on his recorded think-aloud and the gestures, they were asked to reconstruct and sketch the final solution that SP had in his mind. Sketches that both the designers produced were very close to what SP had sketched.3

In the earlier posts, we had emphasized the role that sketching plays as an act, process and as a display in design problem solving. The obvious question that needed to be answered was,

What compensated for the sketching and the lack of visible display?

Is it likely that SP used his mental imagery capabilities to visualize and detail the ideas? For most designers this is not a question worth brooding on. When visualizing, they routinely develop and see their ideas in their mind. It is referred as seeing in the mind’s eye.

Mind’s eye in action

So common is the use of mind’s eye and so real are their experiences, that nobody in the design community ever discusses it, unless someone shows an exceptionally high standard. So, it is not surprising that design literature does list anecdotal evidence on use of mental imagery by the gifted designers. Frank Lloyd Write is known to have visualized the entire idea and details of his famous building ‘Falling water’ in his mind and was able to quickly draft it when Kaufman decided to visit his studio at a short notice. McKim mentions how inventors like Tesla and James watt developed their complete ideas in their mind.4 Mozart had the ability to hear his orchestra and every instrument in his mind’s ear and wrote his final score directly.

Anecdotes and experiences don’t make good science. Besides, there are no accounts of not so gifted designers and creators and their visualization abilities. For this, we need to take a short detour to understand how we use mental imagery and the way mind’s eye works.

On mental imagery and the mind’s eye

Experience of mental image is defined as ‘seeing in the absence of actual visual input in front of you’. To the person experiencing this, the image looks real. (Most convincing and yet difficult to prove example would be experience of dreams) Mental images were not studied because they could not be measured till Shepard and Metzler showed how this could be done.5 Subsequently, there are many studies of mental imagery capabilities. Kosslyn studied mental imagery extensively and listed its characteristics (Fragility, density …) as well as the operations that you can perform on it, like image scanning, image generation and transformations.6, 7 There is also literature that shows how creativity and mental imagery work in synergy.8 With these theoretical back ups the idea of mind’s eye has acquired greater acceptance.

Let us return to the experiment that we started with. Most of the videos above show how SP was continuously using hand gestures to shape an invisible object in front of him. He was obviously working in his mind’s eye. Its virtual-ness turned out to useful, because such a model was quick to manipulate and the change could be ‘observed’ instantly. He interacted with the model with his hand gestures, shaped it, felt the shape and the curves and used the shapes to test if they would work. He used his hand gestures as if he was sculpting a virtual product shape in front of him. (See earlier videos) All these gestural interactions with the virtual model were as real as it would have been with a physical model that he would have created under normal conditions. The gestures were used as much to think and manipulate the virtual object as for communicating the idea.

There is sufficient evidence in research literature to show that there is interrelationship between motor experiences and high-level spatial reasoning. For example, when presented with spatial problems such as mental rotation tasks those who use motor actions (like moving and tilting hands) perform better than those who exclusively depend on visual processes like handling the task in the minds eye. (Ref) That explains surprising accuracy of his gestures and hand movement was surprising. So, when this recording (Audio+video) was shown to two new industrial designers, they could reconstruct the final idea with a fair accuracy. We will focus on the role of gestures and body movements in the future posts.

The structure of the experiment also raised other related questions,

Was the designer’s thinking hampered when he was blindfolded? Was he forced to deviate from the normal design process?

It is difficult to come to a conclusive answer, without comparing this process with the normal process accompanied by sketching. But the transcriptions show that all the typical traits associated with creative problem solving were visible. For example, He systematically identified and tackled all the functional problems one by one. He continued to use ‘moves and reflections’. His moves displayed non-linear shifts, in that he shifted from feature to feature and returned to them again. He iterated extensively, revisiting his earlier decisions several times. His creative explorations remained non-linear.

For most of the ideas that he generated, he simulated its use in his mind’s eye and identified potential problems, and even modified his solutions.

What compensated lack of sketching?

Mind’s eye offered a display that could quickly generate and regenerate image display. It served as a pliable model that he could quickly manipulate in response to his evolving thoughts. It is fast to change, but is fragile and would normally demand budgeting of mental energy to retain and regenerate it. If this is so,

Why the energy budget was not an issue here?

There is no clear answer to this question. I can only venture an answer. Holding images in the mind is indeed difficult. It is true that they need to be regenerated frequently to remain visible in the mind’s eye and that requires budgeting of mental energy. However, most of such findings on energy budget and limitations of short-term memory are based on showing the participants completely new and unfamiliar pictures or words and ask them to recall. As against this, SP used a clear logic and reasoning to evolve the form, which clearly reflected in his speech. So, in case the image is lost due to its fragility, he could regenerate it easily using the logic.

The experiment clearly shows that mental imagery could be one of the viable substitutes to sketching. Perhaps we should correct our earlier statement. What design thinking needs is an ability to represent an object in some form that act as a stable display, that allows you to manipulate it quickly and effortlessly. Such a definition ideally fits sketching, but is inclusive enough to legitimately accommodate other forms of representations like mental imagery.

Could these results have been a freak case? Is the ability restricted to a gifted few? Or is based on years of in design that SP had?

This is a unique ability that designers seem to acquire during their education and practice. In fact, most professional designers who participated in the later experiments told me that, it gives them flexibility to work whenever and wherever they choose. SP himself commented, by using mental imagery “I carry the problem with me in my mind.”

Sum up

The article seeks answer to the question, ‘Is sketching as a representation tool an indispensible part of design problem solving?’ If yes, then this should be treated as an essential skill in design and architecture careers. The answer is explored through a carefully designed experiment, in which the designer is given a design problem to solve and he is blindfolded and thus prevented from sketching.

The fact that designer solved complete design problem when he was blindfolded was ensured by the way experiment was designed. The results show a clear and unambiguous answer that confirms that designers can do without sketching and they compensate this loss with their abilities to create images in their mind’s eye, manipulate them and work with them to develop solutions. In fact, in this case, he created a virtual model in front of him, interacted with it with his hands and altered it willfully. It also showed that he could effortlessly respond to this strange situation and that his design process was not altered.

Mental images are known to be fragile and not easy to work with. They also demand budgeting of more portion of mental energy to retain and process them. So, designer’s visible and effortless switchover to handling of imagery is not easy to explain. Perhaps because the images were generated and regenerated based on his reasoning, he does not seem to face the problem of diversion of excess mental energy. That also explains why designer’s design process does not visibly change.

The designer extensively used hand gestures while generating ideas and for interacting with the virtual model that he created in the front. He perhaps also used them to communicate his ideas. What is worth noting was that his interactions were amazingly accurate.

The results force us to correct our earlier statement. What design thinking needs is an ability to represent an object in some form that act as a relatively stable display, but allows you to manipulate it quickly and effortlessly. Such a definition no doubt fits sketching, but is inclusive enough to legitimately accommodate other forms of representations like mental imagery. Even if designer develops competence is handling one of them, he should be able to make a reasonable headway in design career.

Preview of the next post

When I conceived this experiment reported in this post, I had no confidence that I will discover new findings. Reflecting back, it could have been because of my love for sketching. I was more than surprised by these results and the findings. But it left a nagging feeling,

“Can this result be a freak case? Or is it because of years of experience of designing that SP had?” Or “Is this ability restricted to a gifted few?”

This subsequently led to series of experiments with designers and architects. More about it in the following post.

Notes and references

1 There is sufficient evidence to show that such think-aloud exercises reveals part of the contents of the short-term memory in action. Note that what is captured is what he naturally chose to speak aloud and may not represent everything that passed through his mind. These are referred as think-aloud sessions. Evidence shows that it approximates what he is thinking about. (In fact, most designers and architects are comfortable talking while designing)

2 Athavankar U., (1997) Mental imagery as a design tool. Cybernetics and Systems, 28 (1), 25-42.

3 Athavankar U., (1999) Gestures, imagery and spatial reasoning, In J. Gero & B. Tversky (Eds.), Visual and Spatial Reasoning (pp. 103-128). Preprints of the International Conference on Visual Reasoning (VR99), MIT

4 McKim R. H., (1972) Experiences in visual thinking. Brooks/Cole, California

5 Shepard R. and Metzler J., (1971) “Mental rotation of three dimensional objects.” Sci
ence. 171(972):701-3

6 Kosslyn S., (1983 ) Ghost’s in the mind’s machine, creating and using images in the brain. Norton, New york

7 There are also groups in cognitive psychology who dispute this, leading to what is now termed as mental imagery debate

8 Finke R., (1990) Creative imagery, discoveries and inventions in visualization. New Jersy, Lawrence Erlbaum