Part II: Reflecting on gestures as design behavior

Problem solving is often treated as a cognitive act and problem solver as an information processor. This is also extended to design problem solving. Yet the act of design, standing on the fence between art and science, has different nuances. So, there is much to learn from how designers solve problems. Like all cognitive acts, designer’s internal information processing remains invisible. However, designers in the act of design display unique design(erly) behaviour. The clues to what goes on his head are through observing designer the visible results or overt actions in the act of problem solving,

As the designer progresses towards solutions, the spatial issues dominate his thinking. At that stage in the design process, design can be limitedly defined as act of resolving spatial issues to achieve predefined goals. Designer’s decisions involve designing elements and their specifications as well as arranging, composing and iteratively manipulating them in space to achieve stated goals. At this stage need to support the evolving thoughts through sketching or 3D representations becomes critical. Such design(erly) behavior includes, think aloud transcripts, mental imagery, gestures and body movements when working in blindfolded mode (in normal circumstances sketching and models produced).

These are the traits we have been decoding all along. In all our experiments we have tried to capture these and treated them as a focus of our analysis. Though we have treated them as separate external manifestations of what goes on the in the mind, the experiments suggest that they are closely interwoven with the thought processes. So, we have used visible clues as a ladder to understand how, why and when they assisted the thoughts that drove the decisions.

What does visible design(erly) behaviour tell us about the act of design? This is a continuation of the discussion started in the last post “Sketch or not to sketch? That is the question”. It ended with following questions,

Why do designers use gestures and movements of the body when they solve problems? Do these support spatial decisions, visualization and design(erly) thinking? And if so, how? When should they prefer gestures and body movements?

We have yet to find explanation to why designers and architects used gestures and physically move their body through mental spaces that they created and interacted with them.

So far we have referred to gestures and body movements as one entity, there are qualitative differences within and between them. A more detailed analysis of the classification of gestures and movements and the roles they play is available in my earlier papers.1,2 In answering the above question, we will only touch some of the key gestures and body movements that seem to have a role in understanding, thinking, conceptualizing and representing the creations. We will also focus on how and why they may have contributed to the evolving thoughts.

Plan of the discussion

We plan to look at external traits of design(erly) behavior in different experimental conditions and discuss their effectiveness in design problem solving. We will also compare the two modes, when using sketching and when blindfolded. Note that it deals with limited part of the design process, when designer is grappling with spatial issues.

It is conceived as a four part series. This post (Part II) primarily explains the role of gestures and body movements and tries to explore spatial intelligence as a theoretical framework to understand and explain the design(erly) behavior. In the next post we will focus on embodied cognition as a framework and the third will include the other special features.

Overview of the theoretical frameworks

We hope to find support in two key theories that come close to explaining this design(erly) behaviour. In this Part II, we will start with Howard Gardner’s theory of multiple intelligences.3 This theory relies on the computational capacity, activated by external or internal information and is consistent with our treatment of designing as a cerebral act. We would be particularly focusing on the two relevant areas in intelligence; 1] spatial intelligence and 2] bodily-kinesthetic intelligence.

In the subsequent post (Part III), we will seek support in the emerging theory that deals with embodied cognition. The concepts propose that many features of cognition are shaped by the characteristics and aspects of the physical body and these influences have significant causal role in cognitive process beyond the brain. It accounts for the active use of motor system, perceptual system and bodily interaction with the environment. It treats design as a corporal act.

Cognitive embodiment and spatial intelligence theories are built on different theoretical bases. The first relies on computational approach and thus treats problem solving as a cerebral act. The second argues for the role of bodily actions. It believes that thinking is also corporal in nature, at least in specific disciplines. In spite of their opposite theoretical stands, there is lot that can be learnt from these theories in understanding design(erly) behavior during problem solving.

A quick review of the experiments

In trying to find logical explanations we plan to discuss results keeping in mind all the experiments in this series. Participants in these experiments were industrial designers and architects solving problems typical to their disciplines. A quick overview of the series of experiments and the differences between them will offer the context for the discussions that follows.

Industrial design project

SP worked on a casserole project, a small tabletop domestic product. The findings of this experiment were discussed earlier in posts starting with ‘Can we prevent designers from sketching?” Short videos clips showed some of SP’s design actions. The readers could always revisit this material to refresh their memories. In this post, I plan to only briefly review the findings of these series taking into account the entire video records of the sessions .4

Let us start with a quick recap of the events in this experiment. The subject ‘SP’ was given a brief to design a casserole for domestic use. The project brief also demanded submitting designs of matching bowl set. Table 1.1 captures the set up. SP incrementally built the idea of the box, starting with fixing the volume and proportions of the rectilinear shape. It appeared as if he was shaping the volume with his hands moving in the air in front of him. He could manage this effectively as the actual size of the product was within his grasp. Most of these gestures were purposeful and were contributing to develop the casserole and bowl shapes. As he progressed in design, he started sculpting the shape and fine-tuned it. (refer Video 1) He assigned colour, transparency, textures and even finalized the product graphics. Some of his gestures simulated actions of the use of these products. He even simulated assembling of the components in manufacture. (Refer Video 2.) (Closest visualization of the scene would be to imagine musician acting as if he is playing an instrument in creating music without the instrument being there.)

Condition no. Design project Experimental condition and description


Industrial design project.

Table-top domestic products as projects.

Casserole, Salt-n-paper container etc.

Designer blindfolded

Designer sitting in an office chair and matching environment.

Small size, table-top product, Free flowing form.

Table 1.1. Designer was blindfolded and sat in a chair and solved the design problem

Video 1: The industrial designer shaping a virtual object in front of him as if it was real.

Video 2: The designer simulating assembly of parts

We deliberately changed conditions, where the architects solved a typical architectural problem. In condition 2, the architects were ushered in an office like environment, asked to sit and were given the site plan to see and recall. Then site plan was withdrawn and they were blindfolded. So, they had to develop the 3D representation image/s of the virtual site in their mind and worked with it. In condition 3 the procedure remained same, except that they were left standing in the middle of an empty hall, obviously without access to any sketching or modeling tools. Table 1.2 captures this.

Condition no. Design project Experimental condition and description


Architectural project Mid-size built space.

Student cafeteria with constrained site.

Designer blindfolded,

Designing while sitting in office like environment,

Mid-size site, Body enveloping spaces,

Constraints of construction.


Architectural project Mid-size built space,

Student cafeteria with constrained site.

Designer blindfolded,

Let in to large hall, can move if the designer chooses.

Mid-size site, Body enveloping built spaces.

Constraints of construction.

Table 1.2: Architects solving a problem. The differences in the conditions 2 and 3 are mainly in the working posture.

It is interesting to watch the full videos of the architects performing and ask ‘Why did the used gestures and movements?’ In this part we will rely on concepts like multiple intelligence and its subset, spatial intelligence to explain gestures. In part II, we will use embodied cognition as a theoretical framework. Both seem to offer new and credible explanations.

Treating design as a cerebral act

Designer is learning all the time by studying the problem and from the world that he lives in. He reacts to what he sees, understands and learns. The information that he absorbs and manipulates contributes to the thoughts and ideas. It influences the way he thinks and takes design decisions. The design actions are not just trial and error acts. They obviously involve intelligence. Thoughts during designing are driven by information and intentions. So, in this framework the designer is seen as information processor and design primarily as a cerebral act.

Traditional approach to cognition focuses on higher-level strategies like development of concepts, categories, reasoning and judgment. It is based on information processing and symbol manipulation resulting in production of output. Brain is an information processor and its actions are explained by computational approach.

Over the years the information processing act of design has evolved into a somewhat structured process. The design process followed ensures consistency in performance over time and projects and yet allows sufficient freedom to explore uncharted paths. It is reasonable to assume that the different stages of the process will demand different classes of information. It follows that these steps will demand specific forms of intelligence as well as unique skills. In the discussion that follows, we hope to identify some of the dimensions of spatial intelligence visible in these experiments, particularly in different segments of the design process. We hope to explain design(erly) behavior too.

Spatial intelligence is spatial reasoning and judgment

Driven by designer’s reactive thoughts, the new ideas are continuously generated, often as images. They are conceived and converted into sketches or seen in the minds eye. They are compared and judged. These images continue to evolve through designer’s reactive actions and interventions. The study of the entire process and the accompanying transcripts reveal the creative moves and reflections that SP generated throughout. (Refer Video 3)

Video 3: Industrial designer making move and reflecting on it. This continued throughout the initial part of the session.

When designers handle tasks like making effective design changes, developing alternative spatial layouts is obviously not a process-based on trial and error. To create and judge different alternatives demands reasoning, intelligence and considered judgment. It is a cerebral act. Design problem solving demands spatial judgment and spatial intelligence. All the solutions that SP developed, accepted and rejected during the session support the idea that design is a cerebral act. (condition 1)

This is also visible in conditions 2 and 3, where the architects thought, created and built the idea virtually, grasp and react to it and then alter it again. The architects too were visibly engage in modifying their ideas and trying to grasp them and figure out the implications. This involves anticipating the consequences of changes. Comparing and judging the ideas mentally needs intelligence and these cases spatial intelligence.

In blindfolded conditions 2 and 3, these steps were lot more difficult, because the dynamic events occur in the images in the mind’s eye, which are known to be extremely fragile. They had to hold these images as dynamic displays that they could react to, which demanded budgeting additional mental energy, over and above the energy spent on thinking of new ideas that keep the images in the state of flux.

Spatial intelligence in visualization to drawing

During the later stages of the design process, thoughts and ideas lead to some form of spatialization. Initially it often takes a form of mental image. Capabilities vary in simple tasks like depicting an image as ‘seen’ into these external forms. So, designers do need the ability to convert their mental images into series of sketching strokes, or in case of a model, actions on a clay block. Even more challenging is the tasks where the ideas are half-formed and designers expect the assistance from externalization efforts to concretize his ideas. These are not trivial tasks. The subsequent post (Part III) will touch this aspect in details.

How do we explain the architects exploiting the opportunities? They ‘walked’ the built spaces that designed. What explains the need for these movements?

Within this framework, attempt is made to explain gestures and movements as manifestations of bodily-kinesthetic intelligence. To be mobile in the space (navigating) to understand space and chasing dynamic changes are dimensions of spatial intelligence that emerged clearly in these studies.

Spatial intelligence and navigation

Architect’s navigating through the site to understand and create spaces is conspicuously visible in condition 3, where the architects were allowed to move after they were blindfolded. In condition 2 they seem to virtually move on the site though in real world they were sitting.

Normally the architects would move on the actual site exploring the features. It involves learning spatial arrangement by exposure to the available space by navigating back n forth. It needs spatial intelligence to learning about the spaces by walking or navigating i.e. by being mobile in that space and then connecting these pieces into a coherent mental model of the site in the mind. Don’t we develop notions of terrain maps by driving through it multiple times and then connecting the short segments together?

It is common to navigate in the space to understand it. In normal circumstances, to avoid mental energy required to maintain the evolving ‘things’ in the mind’s eye, architects and designers convert these perceptions in to a plan and add their notes.

In condition 2 they paced the virtually created site, going back n forth, turning and built spaces around their bodies. (Refer video 4 and 5) When allowed they physically moved in the hall, treating it as a virtual site. (refer Video 6) They were trying to absorb the features and spaces on the site and also constructing on it actively and interactively. (Refer to Video 6) Both the actions were almost simultaneous.

Video 4: Evidence of creation of virtual site in the mind’s eye

Video 5: Evidence of creation of the site in the mind’s eye

Video 6: Designer actively interacting with the shape created in the mind’s eye, treating it as if it is real.

Watching the video and listening to their think-aloud transcripts during the design action suggest that the architects moved on the virtual site from location to location. In condition 2, the navigation and movements were essentially on a virtual site in their mind. The navigation and movements were essentially virtual.

For architects, virtual and physical navigation on the site remained the most popular strategy to understand the current state as well as for conceptualizing the idea. The self-navigation ensured that the processes of sensing information remained lot more active and under designer’s control.

Spatial intelligence and chasing change

They virtually built spaces around them and changed them, reacted to their own decisions as a result of their evolving thought process. Spatial intelligence demands the ability to quickly grasp a rapidly changing environment. Chasing changes and handling rapidly changing situation is common also in architectural assignments.

The designer’s actions and behavior do find explanations in the theory of spatial intelligence. A more exhaustive study across different design disciplines and levels of expertise is sure to reveal more about the different dimensions of spatial intelligence.

Yet, it does not explain fully the natural urge to use gestures and body movements. For that it is necessary to turn to embodied cognition. Let us treat designing as a corporal act. More about it in the next post.

Summing up

These four posts will take an overview of the entire series of experiments that were conducted over a decade. Revisiting the experiments was refreshing in several ways. It allowed reflecting back on the objectives of the experiments. To find reasons for the consistencies in findings was a challenging task. Besides, the findings could be now explained with new theoretical frameworks. The discussion here is restricted only to part II.

These experiments differ in several ways. Industrial designers were given a typical design of small tabletop objects. Architects were given mid-size architectural projects like designing individual detached residential or commercial buildings on a defined site. In the later case, the conditions were further varied. In first case, the architects sat in an office like environment and in the second, they were left standing in a large hall. Because all were blindfolded, they could not have used sketching as handy representations. They spontaneously resorted to gestures and/or body movements to develop the ideas, thus revealing some new dimensions of design(erly) behavior.

The protocols of the sessions clearly show a close partnership between the designer’s evolving thoughts and the process of externalization. In this case, there is extensive dependence on gestures and body movements during conceptualization of new ideas. These posts are an attempt to explore answers the question,

Why do designers use gestures and movements of the body when they solve problems?

The design problem solving is viewed through two theoretical frameworks. Part II (this part) treats design as a cerebral act and considers designer as an information processor. It shows how intelligence, and this particular case visio-spatial intelligence, play role in problem solving and explain the need for use of gestures and movements.

Gardener’s multiple intelligence framework explains part of the design behavior and decisions through the idea of spatial intelligence. The protocols show how spatial intelligence, reasoning and judgment play a role in developing, comparing and selecting ideas and taking them forward.

In absence of external representations like sketching, the ideas were represented internally as fragile mind’s eye images. Yet, designer’s managed to overcome this limitation by supporting the ideas through sequential gestures. The sessions also bring forth another dimension of spatial intelligence involved in the process of creating drawings from visualization. As we will see later in part III, this is not a trivial act.

Attempt is made to explain gestures and movements through bodily-kinesthetic intelligence. To be mobile in the space to understand its nuances is a dimension of spatial intelligence has emerged clearly in these studies. The data showed how architects continued to navigate on the site virtually, to explore the features of the site as well as the spaces that they built by shifting their locations continuously.

What was more surprising was that they reacted to and altered what they created. They could quickly grasp the rapidly altered environment. Spatial intelligence is also chasing change in perception. The bodily actions and gestures were used to not only understand the configurations of object or spaces, but also for altering them.

The sessions clearly showed different ways in which designers deploy spatial intelligence. The post explains this through several events that occurred in the design sessions. Do see this as work in progress, till the next two parts are uploaded.

Preview of the next post

Are there more effective explanations to gestures and body movements?

Part III, the next part, will treat design as a corporal act using embodied cognition framework. This theory draws observations from many fields like sports and music, to present more convincing answers to why designers used gestures and body movements.

The post will also argue that the choice to use gestures is with the designer and his decision is influenced by contribution that movements make to the evolving thoughts. Both the theories show advantages of getting the body involved while thinking, particularly when innovations are spatial in nature.

Notes and reference

  1. Athavankar U. A., (1999) Gestures, Imagery and Spatial Reasoning. In: Garo JS, Tversky B (eds) Visual and Spatial Reasoning. Preprints of the International Conference on visual and spatial Reasoning, (VR 99) MIT, Cambridge, June 15–17, 1999, pp 103–128.
  2. Athavankar, U.A., Bokil, P., Guruprasad, K., Patsute, R., Sharma, S.: (2008) Reaching Out in the Mind’s Space. In: Design Computing and Cognition 2008, pp. 321–340
  3. The discussion here is based on Howard Gardner’s famous theory of ‘Multiple intelligences’
  4. The videos of the sessions, each lasting over one hour are not included for obvious reasons, though there are references to it. Also the videos included in this post have been shown in earlier posts. They are repeated here for readers who may not have seen them before.



Part I: To sketch or not to sketch? That is the question

As mentioned in the last post, this post was to deal with role that gestures and movement of the body play in spatialization of design ideas as sketches or images in the mind’s eye. We are deviating from that sequence. It was felt that an overview of the series would be more relevant before we move on to the complex role that gestures and movements play in design thinking.

Let us look back at several findings presented in the last seven uploads starting from ‘Why do designers sketch?’ We plan to take a bird’s eye view of the findings and reflect on them in this post. Now that we know the role sketching and mental imagery play in design problem solving, we can now reflect on the series.

Designing as resolving spatial issues

The designer’s decision making seem to be similar to most often-cited example; being in a room and working out alternative furniture arrangements to get the most effective layout at the end. Whether you use sketching or imagery most design problems are typically resolved through spatial decisions and spatial manipulations of elements (in this case furniture pieces). For this part of the discussion, let us treat designing as resolving spatial issues to achieve pre-determined functional goals through design actions. So, whether you are creating a space (layout problem) or a smaller object, what designer’s handle/create are the spatial elements and their locations in space. In part II, we will revisit these issues through the lens of design thinking.

There are two ways to handle such an assignment. In both approaches, actions are in this ‘real’ space, which becomes a context for situated cognitive activity. Whether you use sketching or decide to work it out in the mind’s eye, the process involves perception and monitoring of representations that are being continuously created and altered by reactive design actions. Most reactions typically result in yet another new representation.

Let us focus on each of the two ways to handle spatial issues.

Sketching as a medium for conceptualizing of objects

Though this series does not directly report research on sketching, the author had worked on the role of sketching prior to exploring mental imagery. Some of the statements rely on author’s earlier research in sketching.1, 2

The first approach is to use sketching pad and pen, if handy. The designer draws the room and thinks of new locations and/or orientations of furniture pieces, then quickly draws the new layout. Looking at the sketch he reviews the new idea for its effectiveness. He will often see and react to what he has sketched and come out with a new alternative to be executed as yet another new sketch. You can capture the process as iterative cycles that involves –

“Think -> Act (sketch) -> Review -> Transform -> Think again”

till he feels that he has met the demands made by the design problem. He may take a holistic approach or take one element or one function at a time and handle complexity incrementally.

If the designer has mastered sketching, this process is cognitively less taxing. There are several reasons. First, it also allows starting with a tentative idea and incrementally moving to a more complete and complex state. In the earlier post “Thinking through the messy sketches” we discussed this approach. Second, designer does not have to budget excessive mental energy to handle the actions of sketching. Third, when he creates a record in a media outside of his body (i.e. as sketches), he does not have to budget energy to memorize and recall his past action and decision. So, if he is quick and good at sketching, it is still a cognitively economical option.

Willful control on the act of sketching, particularly in representing objects in perspective, is a critical issue when implementing a sketch. Sketching is a learnt action and is often taught in a structured way in most design schools. There is enough literature on learning to sketch. My own work on control on sketching is available on D’Source. 2

If designer is bad at sketching, this approach can be counter-productive. He needs to budget more mental energy for the act of sketching, but this is at the cost of mental energy required to come up with new ideas. (Refer to earlier post ‘Why do designer’s sketch?’) Designer’s additional efforts to judge the correctness of the sketch can demand energy budget that could interfere with the ongoing thinking of the design problem. So, designer needs ability to sketch quickly and effortlessly, as if it is a routine and natural act.

That is why in the earlier post, “Out-of-the-box ideas for teaching sketching”, focus was on learning to draw with a ‘feel’ of the line drawn via kinesthetic feedback and by diverting attention away from the pencil tip. Reducing the need to depend too much on visual feedback while sketching, allows you to budget the mental energy saved to attend to the demands of design problem solving. Several ideas were presented to explain how the entire body could participate in the act of sketching. The kinesthetic feedback generated by sketching actions can be used to ‘feel’ the correctness of the sketch being executed. It is critical to learn to sketch effortlessly and with natural actions.

The external display that sketching creates reduces the effortful activity of holding the ideas in your memory. However, the mental imagery is not completely eliminated. You have to mentally decide the new location of a furniture piece first, try to review it and quickly sketch what you have imagined. Thus you avoid spending mental energy on holding ideas in his mind and comparing them later. In short, competent sketching ability helps him conserve mental energy so that he can concentrate on developing the next idea.

There is another way to solve the problem of room layout. Imagine now that designer does not have the paper and pen or chooses not to use it. After all, the

Eureka movements may not occur when you have paper and pencil in your hand. Indeed with Archimedes, it occurred in the bathtub!

Designer will have to then depend on his visualization abilities.

Mental imagery as a medium for conceptualizing of objects

For most people it is so natural to see images in their mind’s eye, that many times they are not even aware of it. So, it is difficult to talk of mental imagery. We see images in our dreams too. Like a dreamer, the blindfolded designer also believes that he is in a ‘real’ environment and is awake.

What is the difference in the images that designers’ see when they are designing blindfolded and what everyone sees in dreams?

Unlike in dreams, the blindfolded designer is indeed awake. More important to note that the designer mostly experiences ‘constructed as well as work in progress’ images that he conceives, modifies and builds in time and space. The site that the architects visualized was ‘real’ space that experienced by walking in and out. It is a highly immersive experience that designers choose to willfully enter in.

It is important to view intentionally constructed images as unique experiences and as ‘work in progress’. This continues till the designer decides to artificially stop. Ideas through images continue to evolve, sometime even when the designer is not consciously working on it. Remember Archimedes?

What is critical is the ability to willfully control image generation and transformation operations in response to the thoughts that drive these operations.

This is why the experiences of imagery during design sessions are different from what happens in dreams. The willful control is needed to handle the unique operations that designer performs. Interestingly, he learns to do this without any formal learning programme.

Designing and the mind’s eye

Let us return to the furniture layout problem, but this time designer does not have a sketchbook with him. Thoughts of the designer continue to drive new solutions and prompt new ideas. He can try out alternatives by physically shifting the furniture pieces around, but that will be not only time consuming but also strenuous. So, the designer would have to solve the room layout problem by relying extensively on his internal resource and working out solutions in their mind’s eye.

Designer could move the furniture pieces around, reorient them, think of creating alternatives mentally. He could view them and compare the alternative solutions. All along, he has to view the current state in the mind’s eye, if necessary bring back the earlier ideas from memory and react to this internal display creatively to generate even newer ideas. He has to hold all of them in memory and compare and contrast them. No doubt a mentally stressful task.

These operations are far more taxing than recalling of the images seen earlier. You can clearly see that the visualization actions obviously go far beyond just recalling the images and inspecting them. Obviously, designer needs more complex capabilities for such a creative use of mind’s eye.

We knew so little about how designers willfully control mental imagery event. By blindfolding the designers in our experiments we forced a situation, so that these capabilities could be studied. The series is biased towards a detailed treatment of mental imagery because, unlike sketching, we were ignorant of the issues involved. If this ability to willfully perform and control operations on the images in the mind’s eye is critical for new ideas,

Should we not identify the operations that designers perform and/or learn to perform?

Fortunately, some of the operations that can be performed on mental images have been identified in cognitive studies. Most well studied operation is mental rotation of objects along three axes.3 Further, Kosslyn identified three broad operations. They included, holding the image and inspecting it (Image inspection technically referred as scanning), generating the image (image generation) and lastly transforming the images (image transform). Last one is a powerful operation and includes several types of transformations.4

Most of these operations are clearly visible in our experiments. The videos clearly show that designers and architects iteratively switch between image inspection, image generation and image transformation. They cyclically inspect the results in their mind’s eye and reflect on these images. We will take these operations one by one.

Image Inspection (Scanning)

This operation usually deals with inspecting the recalled image from the memory (LTM and sometimes STM). Ideal examples of image recalls were when SP as well as the architects were asked to describe their final design after they declared that they have finished designing. The object seem to be virtually there in front of them (SP) or in case of architects, built space enveloping them. When they described, they appeared to be ‘seeing’ in their mind’s eye and ‘reading off’ the images. Similarly, architects occasionally recalled the site plan that they were given to memorize, but subsequently these drawings were often transformed into a 3D view with all its features to see in their mind’s eye.

These designers also recalled images that are part of their memories and use them as precedents. In our case, such images dealt with previously seen, and perhaps liked, examples of objects or built spaces or their features. For instance, in student pair experiments, there is a clear evidence of pergola roof, deck, Japanese garden and water fountain as precedents brought in as images from the memory. Needless to say that these precedents and features get transformed creatively to suit the new context, when they are used in design.

Obviously, the designers are able to hold their images in their mind’s eye, inspect (or scan) them and react to them as they work with the image. They would often inspect images to make judgments, spot inconsistencies, search for opportunities to alter and manipulate elements within the image. It serves the same function that sketches as display serve. It allows designer to review his ideas, creatively react to them and iterate. What exists as current, gradually move towards effective solution incrementally. This is similar to what occurs when sketching is used. Image inspection leads to spatial judgments and all design tasks are based on these judgments.

Yet, there are logical differences because of the nature of the media, in this case the sketchbook and the mind’s eye in which the display occurs. Mental images are fragile and fadeout if you are not attending to them actively. Cognitively, to hold and inspect an image is an active process that demands budgeting of mental energy, more so, when it is done purposefully.

Because designers and architects are involved in creating new objects and built forms, they tend to often inspect what they created and then manipulate it. So, most examples of inspection are part of the process of image generation and transformations.

Image generation

If you watch the video of SP in action in casserole experiment, it shows sufficient evidence of SP’s logic, ideas and thoughts directing his design actions and gestures. These often result in generating images of the object that he was developing. He develops the shape from scratch, builds it up step-by-step as an image. SP choses to ‘treat’ the overall image of the object as ‘real’ and even physically shapes it, making changes interactively as he goes ahead!

Most of SP’s actions in image generation were incremental and became more detailed as the time passed. Observe SP working with a virtual object in the front. See video 1 below.

Video 1: The designer created a virtual object in front of him and shaped it as if it was real for the entire period when he was designing.

As he advances with his design, he fondly sculpts and alters the shape. Similarly, when the architects were blindfolded and let into a big hall, they first create a gross layouts, detail them and as they go ahead, build the 3D spaces around them.

It is common to see the cyclic process that starts with generation of the image, then inspecting it, reacting and implementing changes in their mind’s eye. That explains the incremental nature of the development of the ideas. Note that there is little change in the design process as in both cases. There is only a change in the media used for displaying the current state of design.

Most of these videos have short pauses of few seconds when they are silent. Pauses seem to be useful to judge the current state of design idea. Pauses longer than three seconds were purposeful. They are used for inspections of the images in the mind’s eye and reflecting on it. You can clearly see this in video 1. They appear to be part of spatial judgment efforts and end in actions and decisions immediately after. Typically, they end up with flurry of activity, often resulting in a new idea or a modification that had existed as an earlier idea. SP pauses several times to ‘see’ the object in his mind’s eye and then reflects on it. He ‘looks’ around to inspect what he had created and pass judgment after a pause. After one such pause midway through the casserole session, SP sculpts the shape, pauses and comments like ‘That will look interesting’. See this in video 1 above.

Image transforms

Image transformation is a critical operation on the images. It is also the most often used operation to quickly alter the contents of the image seen. Most surprising finding was that in implementing the operation, designers physically interacted with their creations in the mind’s eye. In fact such interactions were at their best in image transformation. For example, while thinking of support to the casserole body, SP’s palms represented the folding legs of the casserole and movements of the palms simulate the leg movements.5

There are several complex transformations of objects or spaces that architects and designers seem to comfortably deploy while designing. For instance, SP’s casserole video has several examples of the shape or some element of it being reshaped, chopped, moved, flipped, rotated and so on. See this in video 2 and 3 below.

Video 2: Observe designer exploring the shaping (curving) of the bottom of the casserole.

Video 3: Observe designer shifting from shaping of the lid to folding legs.

These videos include actions like selectively moving the objects or elements, manipulating proportions and compositions, changing sizes, altering colours, exploring different backgrounds and even creating exploded structure. They appear to use more than one operation, often in continuous sequences, and that too with amazing ease. The gestures come handy in the process of working with it and on it. More about gestures in part II in the next post.

While modifying their ideas and exploring alternatives, architects too performed different transformations of the images. Architects however use gestures more often to indicate and locate than to shape the spaces, perhaps because of the scale of these spaces. Earlier posts do to indicate few examples of use of gestures in shaping some parts of the building. For instance, see video 4 below. However, these instances are fewer in numbers.

Video 4: See designer shaping the entrance with hands. 

The purpose of most of the transformations is to improve the effectiveness of the solution at hand. It is typically followed by image inspection, where the designer would try to assess the implications of the changes they had just made. The use is similar to the way the sketch is reviewed, except when working with mental imagery, it is lot more difficult.

These changing mental events are available in form of fragile images in their mind. They have to hold these static as well as sometimes-dynamic images in their mind’s eye, which demand budgeting mental energy. Additional energy is required for reacting to them, altering the image with new intentions and hold on to the new image.

What does the series reveal?

The discussion so far has also helped us separate two forms of representations that almost serve similar function. Most discourses on designing focus on the first form i.e. sketching, and do not acknowledge mental imagery as serious contender as a form of representation. It is understandable. Indeed, there are no capturable external representations to write about or comment on, when working with mental imagery. One of the contributions of this series is to discover methods that give access to the actions that occur in the mind’s eye.

The series argues that sketching as well as mental imagery serve similar role but deserve to be treated as separate acts.

We hope that this separation will remove the bias towards sketching and studies in understanding the role of mental imagery in design(erly) thinking will get the attention it deserves. It is hoped that the shift of spotlight will balance the studies of role of representations.

Why is attention to the mental imagery critical?

It is known that this ability is directly correlated with creative efforts. To willfully change, transform or manipulate the image or some of the elements of the image selectively to intentions, distinguish creative people from others. Artists, dancers, architects, designers and some scientists can willfully control what happens in the mind’s eye. Einstein was known to create events in his mind’s eye that he often referred as thought experiments. Mozart was known to compose his orchestra entirely in his mind’s ears, an audio equivalent of mind’s eye. He was also able to hear and follow each instrument in isolation. There is enough scientific evidence to suggest that the ability to handle events in the mind’s eye and creative work are closely related. Finke has spent several years researching the role of mental imagery in inventions using cognitive science framework.6

Understanding nuances of visualization

We now have a clearer understanding of the term visualization and what it means to design community. It is not just externalizing ideas in a sketchbook or on a computer screen. It should include perception and manipulation of representations in the mind’s eye.

Visualization is much used and abused word. At the simplest level, it can be defined as ‘seeing the object when it is withdrawn and is no more available to be perceived.’ We are then asked to recall what it looked like. Can we restrict visualization to just recalling and inspecting the image in the mind’s eye?

Ability to recall or generate an image is not unusual. Most people are able to recall and view images. Dreams also involve recall of images. Dreamers are convinced that they are in a real environment, are awake and are experiencing a real event in time and space. Situation is somewhat similar when the designers were blindfolded, except that they were actually awake. Besides recalling, lots of people generate and see new elements in their images in dreams (also under hallucination) that they have never seen before.

Learning to willfully control mental imagery

The major difference is in the ability to willfully control the events in the mind’s eye. Architects, designers and artists are able to perform several cognitive operations on the images in the pursuit of the creative work. However, unlike sketching which is systematically taught to them, they learn to handle mental imagery with no formal training.

Can we learn to willfully control events during visualization without design training?

To handle mental imagery, at least till this time, there is no structured training available. It is perhaps partly a natural talent and partly perfected on job through practice. We have little knowledge of how willful control on imagery can be taught through training.

The focus leaning towards sketching as a preferred representation could change in future as designers learn the advantages of mental imagery as a creative thinking tool and find strategies to develop educational material for everyone. I saw the absence of training as an opportunity to convert my understanding of controlling mental events by developing visual puzzles that can be solved by use of some of the cognitive operations that we discussed. I hope to present these games and visual puzzles in some later post.

Who else can benefit from such training? There are lots of people who are involved in design but are not competent in drawing. Their creative energies could be harnessed if they could be trained to be comfortable with mental imagery.

Learning to handle sketching

Sketching does come handy when the complexity of the design project is very high. However, it demands rigourous and systematic learning to model 3D ideas as 2D sketches and draw them skillfully. I personally believe that initial training in orthographic drawing trains your mind to switch between 2D and 3D effortlessly. It has long learning period. Obviously, only few professions need it and go through such training.

To think, model ideas in mind and simultaneously sketch them is a task that needs systematic learning and practice. The focus on drawing by feel and by kinesthetic feedback and not by sight has its roots in this idea.

Note that there are other forms of representations including verbal descriptions that are used in communication. Most people use language descriptions accompanied by gestures to explain design ideas. In fact, they are effectively used in brainstorming. I tried to study gestures with minimal use of language with some success.7 But discussion on these topics are outside the scope of this series.

Sum up

This post is an overview of what was presented in the several earlier posts on sketching and mental imagery. It argues that 1] sketching and mental imagery should be recognized as two separate forms of representations. 2] The representations are a critical constituent of design(erly) thinking, 3] Both act like mediums for conceptualizing design ideas, 4] Both of these forms serve similar functions. So, technically, they can substitute one another. The post also compares sketching and mental imagery from the point of expending of mental energy. It suggests necessity of removing the current bias towards studying sketching and treat studies in imagery on par.

Focusing on mental imagery, it identifies the three broad cognitive operations, Image inspection, Image generation and Image transformation, that can be performed on the imagery. These are explained through examples taken from the experiments cited before. It argues that the understanding of visualization in the context of design should necessarily include learning to willfully control cognitive operations in response to the designer’s intention.

At a broader level, the design process has commonalities irrespective of the forms of representations used. What differ are the cognitive operations when working with sketching or exclusively in mind’s eye.

To sketch or not to sketch?

We started our discussion by exploring the role that sketching and mental imagery play as conceptualization tool. Both the forms have their merits and demerits. The choice should depend on designer’s comfort level with these two forms of representations. To be able to competently and skillfully handle both forms of representation would be an ideal situation. The complexity would not trouble the designer. The design response would be quicker. As one of the participant said in the context of use of imagery, and I quote

“I carry my problem with me all the time now”


Preview of the next post

We have restricted the above discussion to the implications of production and modification of display through sketching or through use of mental imagery. However, our discussion does not explain why the architects moved, walked and used the spaces they created, nor does it explain the deeper role that gestures play. In part II, will address questions like,

Why do designers use gestures and movements of the body when they solve problems? Does it support spatial decisions, visualization and design(erly) thinking? And if so, how?

Next post will discuss how these two forms of representation affect design(erly) thinking and why in spite apparent similarities, they are conceptually different.

Notes and references

  1. Athavankar U., (1992) Rediscovery the Act of Sketching: Implication of its Support to the Creative Thought Process, Design Recherche, No. 2, pp 45-60
  2. Discussion on freehand drawing Dec 21,2017
  3. Shepard, R.N., Metzler, J., (1971) Mental rotation of three-dimensional objects. Science 171, 701–703
  4. Kosslyn S., (1983) Ghosts in the Mind’s Machine: Creating and Using Images in the Brain. Norton, New York
  5. Athavankar, U. A. (1999). Gestures, Imagery and Spatial Reasoning. In: Garo, J. S. & Tversky, B. (Eds) Visual and Spatial Reasoning. Preprints of the International Conference on visual and spatial Reasoning, (VR 99) MIT, Cambridge, pp 103-128.
  6. Finke R., (1990) Creative imagery: Discoveries and Inventions in Visualisation. Lawrence Erlbaum, New Jersey
  7. Varshney, S., 1998, Castles in air: A Strategy to model shapes in a computer, in proceedings of the conference ‘Third Asia Pacific Computer Human Interaction (APCHI ’98), Japan, July 1998, 350-355

Designers imagine, touch and walk the talk

This post is third in the series three posts that explores potentials of mind’s eye by blindfolding designers and architects and asking them to develop their design proposals, completely relying on internal resources and visualizing in the mind’s eye. They extensively used gestures while solving the design problem. On the other hand, in my earlier posts on teaching sketching for designers, I had focused extensively in getting the whole body involved in the act. I was interested in the relationship between the body movement and the thoughts.

Leaving the architects standing

To broaden the scope, I treated hand gestures as only a part of the larger aspect like conscious moving of body. I decided to refocus on the role that the body can play in solving design problem. In all my previous experiments, architects sat on a chair and then they where blindfolded. This had actually constrained their movement of the body. In these experiment, they were then blindfolded and left standing in a middle of a large empty hall, wearing a wireless collar mike in. I had hypothesized that they will move around and use their movements in some constructive ways.

Experiment design

This time we invited four architect volunteers with two/three years experience after graduation to participate. They were given a site plan with no indication of what functions to accommodate. Site given was rectangular, with one curved corner along the access road. It had gentle contours. The plan indicated trees (some to be preserved) and location of existing storm water drainage. The architects were asked to remember these features and verbally recalled them before they were given project requirements.

The design problem given was a hangout space with a cafeteria for student community on the campus. It also had a space that could be used by the students to display outcomes of their hobbies. The detailed account of the experiment and the results were published.1,2

The questions addressed were,

Will the freedom of movement impact visualization strategies?

Will the architects move their body to ‘feel’ the space that they develop in their mind’s eye?

If yes, can these movements be mapped to the spaces that they develop?

The design of the experiment ensured that the architects had to completely depend on their mental imagery and internal resources. We were exploring how the architects would use the freedom to move constructively and take advantage of the space in the hall. We were also looking for changes that may occur because the built spaces were much larger than their body and they have to be designed from inside as well as from outside.

Indeed there were visible changes in the strategies that architects used in visualization and in solving the design problem. They constructed a site in their visualization intervened in it creating architectural spaces around them, even walked around in the spaces created and when necessary altered them. Looking back, it seems like a peculiar interaction. The use of gestures was differed from the way Industrial designer used them. The difference is that the built spaces were too large to be physically sculpted! Their gestures and movements had different functions now. There were differences in the way four architect approached the idea of using the space. The visualization strategies also differed. We plan to discuss some key observations like 1] their presence on the virtual site that they visualized; 2] the strategies that they evolved to visualize the spaces and details; and 3] the way they interacted with their creations. The interested readers could refer to a detailed paper on this experiment.3

‘Presence’ on the site: Virtual or physical?

The video suggests that all the four architects were also seeing themselves virtually standing on the site, visualizing built spaces around them. How can a person be on a virtual site? This might sound strange, but it is true. Yet it needed to be proved beyond doubt by relying on concrete visible evidence of the designer was bodily present on the virtual site. The proof came from several sources in the video data and the transcripts.

We analyzed the transcripts to locate words and phrases that contained frequent references to self, such as ‘I am’, ‘on both sides of me’, ‘front of me, left side of me’, ‘on my right side’. (See figure 1) Surprisingly, most architects dropped references to north and referred everything with respect to the body, its location and orientation. In all four transcripts, there is only a single isolated reference to the North direction. (See video 1) In practice, North direction is critical in architectural practice and comes up often in conversations. Similarly, heat gain, ventilation and rain directions are worked out with North in mind.


Figure 1: Shows how concurrent speech that reveals the architect being on the virtual site


Video 1: There is only one reference to North direction in the four case studies is unusual. Their virtual presence on the visualized site was so dominating that most of them did not refer to North.

Occasionally the architect was asked a question ‘Where are you now?’ When declaring their location, they relied on the virtual space. They declared their locations with references to the features of the site or of new design that they were developing in their mind’s eye. (See video 2,3) Besides, when asked to go back to a specific feature or a location in the proposed design (like entrance), they made finer adjustments in their movements to reach almost the exact spot in the physical world! (See video 4) Such events further supported the idea of presence on the site. The boundaries between the physical world and virtual world seem to have blurred.


Video 2, 3: The architects were asked during the blindfolded session “Where are you now?” Observe the answers. All the architects were on the virtual site that they visualized and they were clear where they were located on it. (Architect in video 1 also ‘knows’ where he is.)

Video 4: Watch the architect making finer adjustments in his locations, before finally declaring where he is.

This presented sufficient evidence of the architects being on the virtual sites that they visualized in his mind’s eye. But we still need answers to the question,

Why were the architects on the virtual site? And how did it contribute to design decisions?

Beyond presence

It was not a mere static presence that could be observed. They walked up and down; exploring the site and conceptualizing built spaces around them. They seem to be imagining themselves constructing and using the spaces that they constructed. Most started with developing a built space from inside first but when required, they came out of the building to see what they had created. (See video 5)

Video 5: When working on the outside of the built space, architect steps out spontaneously.

How does one get valid clues to what they were seeing and experiencing? We mapped the combinations of gestures, body part movements and speech strings from the transcripts, second by second, and plotted the contents with respect to the site plan. (See earlier figure 1) The combined rich descriptions typically indicated site landmarks and new built features. On the other hand, simultaneous gestures, hand, neck and body movements showed the locations of these features. (See figure 2) This helped us reconstruct and map what they were seeing at that a given point of time and how it transformed. (See video 6)

Figure 2: Shows how concurrent speech with gestures and movements recorded and later mapped to reconstruct experiences.


Video 6: Watch how the site moves as the architect turns. With the site, all the built features also turn.

Virtually being on the site allowed him to operate in the ‘virtual’ space that he could build on, alter and experience from inside and outside. This was lot quicker than if had chosen the normal path of sketching these out and altering them. Watch the videos carefully. Their visualizations were far quicker to generate, lot more pliable and could match the speed of the evolving thoughts, ideas and fantasies.

Visualization strategies

The architects had to adapt to the unfamiliar situation spontaneously. So, to imagine consistency in the visualization strategies used by the four architects looked unrealistic. Nobody had any training in handling mental imagery. (Unlike sketching, the pedagogic implications of use of mental imagery have not been explored and so it had never been a part of any design curricula) Most often, it accompanies thoughts spontaneously and remains a very personal experience. The architects had to adapt to the new of ways on the spot. So, instead of searching for consistency in visualization strategy, we decided to look for differences between architects.

Architects moved on the virtual site as well as in the physical space, but the speed and the vigour of the movements and lengths covered were different. At times even the purposes of the movements seem to be different. The strategies could be grouped into three classes. Each architect treated one of the strategies as primary, but switched occasionally to other strategies when they found it convenient.

Strategy 1: Visualizing, working and moving in-situ

Two of the architects built the virtual building spaces and layouts around them and comfortably moved within it. (See video 7) They created the spaces and walked into them and altered them if necessary. The distances and spaces around were very ‘real’ to them. When asked to sum up their design, they were able to walk into the virtual built space that they created almost without error. The results clearly show that the two architects had developed a bodily feel for the virtual spaces that they built and oriented their body within these spaces. (See video 8)

Video 7: Architect created a virtual site and built the spaces around him, while pacing up and down. He is clearly developing his ideas by being inside the built spaces.

Video 8: Architects were uncomfortable if they miscalculated the body orientation and corrected it. Video shows one such example.

The visualization, particularly when in-situ, was very accurate. In summing up the design, when the architects were asked to sum up is design ideas, he could walk the same spaces with surprising accuracy. So accurate was this movement, that in spite of the eye mask, when asked to go back to the entrance, the architect carefully moved back and adjusted his steps to reach the correct location in the physical space! (See video 9)

Video 9: Watch architect going back to the exact location that she had planned as entrance. The last bit of adjustment showed surprising accuracy in the bodily feel.

When asked to describe elevation of the built space, one of them stepped out of the virtual space, to ‘see’ the building elevation and the entrance and reworked on it. (See video 10) Incidentally, these are the only gestures that sculpted the shape of the entrance interactively.

Video 10: Architect steps out and sculpts the entrance to the cafeteria. It appears as if she was developing the ideas in-situ.

Strategy 2: Carrying site on the shoulder

Two of the four architects moved into an in-situ space. Other two moved, but not with the same purpose and vigour. Their moving was a free wheeling movement. It had no direct relationship with the visualized space around. It appears that when they moved, they always had site with them. It appeared as if they were carrying the site with them as if it was attached to their body. So, the site moved and rotated, when they moved and turned. How does one prove such a strange conclusion?

Using the combination of concurrent speech strings, with gestures, neck and body movements, we could map their image that they were seeing in their mind’s eye. These dynamic map shows that when the architect moved or turned, the site also moved and turned, as if they were carrying it with them. (See video 6 shown earlier) It is not clear what was the advantage of this strategy to the designer, but they do look comfortable. Perhaps they were not aware of their own strategy of site moving with them.

Strategy 3: Shift over to a scale model

Some of the architects occasionally shifted to another strategy. From the body movements, gestures and speech, it appeared as if they were working on a scale model. They were outside the site now and perhaps viewed it from the top and created built spaces. Their hand gestures either showed location of the spaces or described shapes (often site contours, structure, roof) Gestures suggest that they are perhaps working on a small-scale model floating in the air, at a short distance from the body. (See video 11) This is closest to the earlier discussion on industrial designer (SP) working on a virtual model in front of him.

Video 11: The gestures suggest that the architect is working on a model close to her body.

General Observations

By allowing the architects to move around, we made the canvas larger. It revealed many new things. We derived three major visualization strategies from the data. For most, though one of the visualization strategies remained as a primary strategy, they were able to effortlessly switch to the other strategies when required. In spite of differences in strategies, all of them were able to solve the problem effectively.

With the broader canvas, the body movements and gestures were used to perform newer functions. They pointed out locations and directions of features that designers was conceiving or referring to. They indicated shapes and in rare case, they even sculpted the shape and the contours.

We also realized that the concurrent gesture + speech combination is so rich in information that it was possible to map what they were seeing in their mind’s eye using their body as a reference. This material acquired richness and revealed lot more of what was going on in the designer’s mind and what he was visualizing. The speech + gesture combination clearly revealed that the architect built spaces, while imagining a virtual site in his mind’s eye. He also used his body as a reference to locate things around.

In my earlier writings, I attributed this to ‘thinking with body’. Reflecting back now I found new explanations that closely match how and why architects and designers performed the way they did. It would be worth going to these areas by considering all the findings of the three articles together. We plan to do this in the next post.

I have come to believe that one of the important characteristics of designerly thinking is the inner urge to solve problems through the design expertise. The attitude can be captured as ‘whether there is a client or not, I want to, and need to solve the problem’. Designerly thinking is based on ambition as well as the excitement solving the problem, a point that writings on design methods miss completely. So, the constraints, like not allowing usual processes like sketching, blindfolding or left standing in the middle of the hall, don’t seem to bother them. None of them complained about the constraints imposed. They took the challenges in their stride, and in fact adapt to them, by spring back with spontaneous alternative strategies and approaches.

Sum up

This article is third in a series of posts that presented findings of the experiments on designers and architects, when they were blindfolded and asked to design. Sufficient evidence was presented in the previous two posts to conclude the most of them were able to handle design problems and come up with solutions and that too with amazing dexterity. This post focuses on the next objective, the role that body and its deliberate actions can potentially play in supporting visualization.

This time four architects were given a design problem and were let into a large hall. They were blindfolded, wore a cordless collar mike and were asked to work on an architectural project. The experiments were video taped and all the speech strings were transcribed for detailed analysis. The expectation was that they might use the freedom to move around while thinking of the solutions and this might impact their visualization. Indeed, they spontaneously responded to the new situation with different visualization strategies.

There was sufficient evidence in their speech as well as in the body movement and gestures to show that they were present on the site visualized in their mind’s eye. When asked, they would declare their location in the built form that they were developing. So, it is not surprising that they located everything around them with respect to their current location and orientation on the virtual site. The way they operated, even this virtual presence looked almost real to them!

Some of them used a strategy where they built the site in their mind’s eye, developed their ideas as virtual built forms, manipulated them in their visualization, but actually moved and interacted with their creations through actual physical movements in the real world. So accurate was their mapping between the virtual and real world that, when asked, they could physically walk back to the exact location in the built form and it would also tally with the physical location in the hall!

Another popular strategy was when they moved they carried the site with them. The site turned when they turned. Though the physical movement did not impact visualization, it was probably required to keep them active in the 3D space on the virtual site. Last, and perhaps the least used, was a conventional strategy of working on a small-scale model like situation. The fact that it was rarely used is surprising; as most of them are used to working with the scaled versions of their creation all the time, and that is how they are taught to develop ideas.

This data showed that body movements were far more pronounced and gestures played a supporting role. There were rare occasions when the gestures were used as tools to model the idea. Perhaps, the bigger size of the built form and the fact they had worked primarily from inside the spaces may have made it difficult to use gestures as shaping tools, the way industrial designers could.

The post concludes by listing the architects’ actions, visualizations and particularly the movements in the physical world. In the next post, we will review the findings of all the three articles in this series through the theoretical framework of spatial intelligence and embodied design.

Preview of the next post

The next post will take a bird’s eye view of experiments on imagery. We will address following questions.

While visualizing, how do designers benefit by use of body movements and gestures?

Why do they feel it necessary to move the body?

Does it support spatial decisions and design thinking?

The theoretical support for the mental events in this series come from work on forms of human intelligence and from findings in cognitive psychology. We will touch areas like 1] Spatial intelligence as well as; 2] Embodied cognition (We touched this in earlier post “out-of-box ideas to teach sketching”) as well as its spin-offs like embodied design and imagination.

Notes and references.

1 Athavankar. U. (2008) Exploring the boundaries of spatial intelligence, Conference on Research and Training in Spatial Intelligence, Evanston.
2 Athavankar. U., Prasad B., Guruprasad.K., Patsute R. and Sharma S. (2008) Reaching out in the mind’s space, In Design Computing and Cognition ’08, Eds. Goel A., Gero J, 321-340. Springer.
3 Athavankar, U., (1999) Gestures, mental imagery and spatial reasoning, In Visual and Spatial Reasoning, MIT, Eds Garo and Barbara Tversky. 103-128. MIT



Immersive mind’s eye experiences

In the last post we reported an experiment where a mid-career industrial designer was asked to develop his product idea when he was blindfolded. We saw how he successfully solved the design problem; and that too with amazing dexterity. The experiment objectively proved that he completely conceived the idea in his mind’s eye. It revealed the potentials of learning to handle mental imagery in design problem solving.

Mental imagery can potentially offer an effective alternative to sketching. However, such a conclusion would be termed a bit hasty considering that it is based on a single case, particularly because the results did look unbelievable.

This post reports efforts to dispel this doubt through a series of follow-up experiments with same or similar objectives. Later, going beyond, it also explores newer and more adventurous objectives. We will approach the finding in steps. We will start with the unfinished agenda first,

Could successfully conceiving and completing a design when blindfolded be considered freak results?

So, we decided to repeat the experiment. We invited SP again and gave him a different and a little more complex design project. SP was asked to design salt and pepper dispensers and common dinning table crockery; all stacked in a compact stand on the dinning table. This is a common product used by many middle class households in India. While the design task changed, rest of the experimental conditions were deliberately kept identical.

The results confirmed the findings of the first experiment. This time too SP sculpted his ideas using vigorous gestures, though the shape developed was much more complex. He also chose to focus on production using complex injection molding process and managed to completely avoid undercuts. All this, when he was blindfolded! The findings were triangulated as before, using transcripts and further supported by asking other designers to decode the design idea based on transcripts.1

The results of the second experiment unequivocally proves that these results were not freak instances and it is possible to generated design ideas and design details in the mind’s eye.

We invited other industrial designers and gave them same or similar design problems. Almost all of them were able to complete the design task when blindfolded. All of them were mid-career practicing designers in their 40s and above, with lot of product design projects behind them. Only one of them, in his fifties, said he would have preferred to sketch, but did solve the design problem effectively. It did give us sufficient evidence that,

Mind’s eye can serve as an effective substitute to a more popular alternative like sketching. But,

Can the success be explained because these products tend to be small in size and thus could be visualised and sculpted as virtual models in the front?

Could these results be attributed to their extensive experience as designers?

To eliminate these possibilities, we decided to offer similar experimental conditions to those who handle 3D objects, like architects. Again, I was not sure that it would work.

Architectural design problems are a different game

Architectural projects have different nuances. Unlike industrial design problems, the buildings tend to be client specific and are not mass-produced (in India). They tend to be large in size and have to be visualized both from inside as walkthroughs, as well as from outside. Of course new elements entered with architectural projects. A site for the building had to be specified, which they were to remember and recall before the project requirements were given to them. There were additional complexities like terrain conditions, climate and light that needed specifying region as well as north direction. As we will see later, these factors influenced visualization.

We gave two types of projects, like 1] give a site with specific size and site features and ask them to conceive the building and 2] give drawings of an already built space and ask them to develop interior layouts for a specific use along with furniture concepts. We had to make sure that the architects would be familiar with the functions they were asked to house. The building projects often included public spaces like information centers or large or small secluded bungalow on sea front or on a contoured site. The interior projects included crowded, but informal student hangout spaces and cafeteria.

Most of this work has been already published as research papers.2,3 So we plan to only include a short summary here, mainly contrasting it with industrial design project. None of the architects were perturbed by the strange experimental condition of blindfolding. They went on developing building ideas in their mind’s eye.

We realized how selective the mental imagery is. It often displays what is relevant to the context. Typically, the people imagined were actually stereotypes and had specific role to play. These stereotypes performed their assigned role in the spaces created, as if the creator was testing his layouts. Post session interviews confirmed that people were always appropriately dressed to match their defined role, but were as a rule faceless.4 Their dresses were important to establish their roles, but the faces were obviously not relevant to the role or the functions they performed.

Another major difference is the focus on controlling light and creating ambiance using natural as well as artificial lights appropriately. Architects not only work with spaces that are inhabited and used, but plan interesting lighting situations contributing to the ambiance. Creating such experiences is so much part of their routine, that its domination in mental imagery is not at all surprising. Indeed, their images were vivid experiences with detailed ambiance and were populated with people.

Designers, and particularly architects, depended on designs that they have seen and ‘noticed’ earlier and used them as precedents to develop new ideas. Some of these precedents come from their own previous successful works and from works that they have seen in design journals and as well as visited in real life. More popular were precedents that come from their favorite architecture gurus (masters). What they bring in through these precedents are interesting space organizing principles, lighting and ambiance or sometimes specific innovative architectural features of interest. Some of them tended to use analogies and metaphors in working out ideas that gave distinct edge to their solutions. I was taken aback by the ease with which they could handle the constraints of eye mask. Their verbal protocols (descriptions) as well as post-experiment interviews were full of emotions and drama.

Overall ‘seeing’ faceless people using designed spaces, effective use of precedents and creation of ambiance through controlling lighting dominated architect’s visualisations.

This may appear as a short anecdotal deviation. I could not resist blindfolding myself informally. So, I asked a student of mine to frame an architectural design problem and I blindfolded myself. The session lasted for over an hour. The experience was deeply immersive. The spaces I created were visualised in the evening light, which appeared to have been automatically selected. Besides functional layouts; the ambiance and time of the day became the focus. Interestingly, I was not aware of the actual time that I spent in the session, nor the time of the day when the experiment was actually conducted. I was of course careful not to include personal experiences in any of my research writings.

Working with architecture students

Note that the initial experimental work involved architects who were 35 plus, with varying experiences of design practice. (It was more of convenience sampling)

Is it then likely that they performed so effortlessly, because of their professional experience?

To eliminate this possibility, I shifted my focus to working with architectural students in their 3rd year. (age roughly 19 to 21 years), just when a design problem of relevant magnitude is introduced in the school design studios.

To replicate on larger audience of students, I had to change the experimental protocols. Video taping each session independently and analyzing transcripts   was beyond available means at my disposal. Besides, we had sufficient evidence from the earlier experiments that it is possible to design using mind’s eye. We did not have to prove that again.

So, I made student pairs where one of them was blindfolded and the other took notes, but only intervened for clarity when required. There were two conditions that we varied. First was to create pairs with boss and assistant relationship and the second was to establish partnership equality. Pair was separated after the design was declared complete and asked to independently draw the idea that they thought was final. We then compared these final sketches.

Overall, even these young students could effortlessly handle the project in blindfolded condition. (See video 1) There were surprising similarities in the sketches drawn by the pairs. The major deviations were in the scale of the building and the way it fitted on the site. (See figure 1,2) Most students were fluent and could explain their ideas to their partners verbally, often accompanied by gestures and sometime use precedents.

Video 1: Pair with one of the student (girl) blindfolded. The second acted like her equal partner. Watch her gestures and references to her body. Hear the description carefully to look at how the ambiance is emerging.

Figure 1,2: After the design assignment was completed, the pair was separated. The eye mask of the principal designer was removed and both were asked to sketch the design idea that they had mutually agreed on. Figure 1 shows the sketched plans and figure 2 shows the 3D view drawn by the blindfolded and sighted designers. The similarities are difficult to neglect.

Blindfolding the classroom

I became little more adventurous to explore what would happen if I blindfolded the entire architecture classroom (studio).5 There were 17 student pairs with each designer trying to explain his ideas verbally to his partner often with gestures. In most pairs, blindfolded student tried desperate tricks to explain his ideas using whatever means he could think of. (See video 2 and 3) In a closed classroom, with everyone speaking simultaneously, the noise it generated was very high. All of them were so much immersed in the process emotionally, that the commotion around did not disturb any of them. That is how immersive the imagery experiences can be!

Video 2,3: Pairs in immersive state figure out interesting ways to communicate their design ideas. Listen to the background noise. No pair was disturbed by it.

Leading to more adventures with sharing images

In the pair format of the experiment, we discovered new possibilities. These results indicated that the pair could share the mental images of creations, so far considered private. These pointed to exciting possibilities of shared imagery playing a role in teamwork and give the research a new direction. The question that we asked was,

Could pair in a team share a common image? If so, could this open up new collaborative possibilities for designers not too comfortable with sketching?

We started with co-design as our objective. So, in the first experiment, we gave an architectural problem to pair of professional architects, but separated them into adjoining rooms, connected through an audio or a limited video link. Both were not familiar with each other, nor did they have opportunity to meet before the experiment.6 The brief was to develop an information center for a historic monument across Mumbai west coast. They discussed the project over the audio link, discussed solutions and selected the best option. Unlike in the past experiment, there were scheduled pauses where the experimenter asked them the state of the design at those points. These breaks had some surprising points. In one such break, we asked one of them to guess ‘In the evolving space created, where is his design partner?’ He was prompt in his reply and said that ‘His partner is hovering around an indicated place on the site.’ We instantly checked this with the partner, and he confirmed this independently! When they agreed that the design task is complete, they remained separated and were asked to independently sketch out their shared final design idea. It was followed by post experiment interview. We also repeated the experiment pairing with two filmmakers with similar success.7

In this experiment, they could see each other’s sketches over the video link, but not see each other. The discussion was often based on 1] the partner’s reaction to the words used as well as 2] the video link access to each other’s sketches and diagrammes. Idiosyncrasies of the sketches did not hinder discussion. The results are significant for work in participatory design, as it proved that a pair of technologically linked designers could work together on a common project, share a common image and evolve a common solution. We then became bolder in our objectives and decided to investigate,

Is the access to each other’s sketches critical? In other words, was the video link critical?

Using substantially similar experimental protocols, we made a minor but significant change in the next experiment. We cut off the video link. They worked separately and in isolation, but could only discuss over audio link. In a way, they were required to figure out the evolving images in their partner’s mind and influence them with their new ideas till they agreed. Their final sketches showed that they were able to share a substantial part of new design proposal, though they had no opportunity to see what the other architect was sketching. There were of course some variations in the scale of the building.

The experiment did confirm our hunch that they had not only shared a common design idea but the image/s in their mind’s eye, so far considered as personal and private experience.

Anecdotal support

I am listing a few that I encountered in my experimental studies, hoping that others may want to take the idea further. Out of curiosity, we were simultaneously interviewing eminent film set designers and even eminent filmmakers. (We could not have expected them to sit through the elaborate experimental setup) These interviews contain interesting anecdotal information. Indeed, anecdotes do not make good science, but they do give push to newer experiments and ideas.

Most eminent artists seem to depend on mental imagery during creative phase. Not so surprisingly, filmmakers are only conscious of what the viewing frames will contain when the camera moves. They visualize details within a frame and had no idea of what was outside the frame of the camera, nor were they bothered about it. They use lot more precedents from their life in the film ideas that they develop, than what architects do. All of them seem to have library of images that they tend to fallback on for ideas. Interestingly, they do ‘see’ movements in the mind’s eye (shaped as a screen), visualize and hear background music scores and had hunches on who the music director could possibly be!

Another, eminent Indian classical dancer mentioned how, when she is visualizing a new steps for her own performance, she uses a mirror and her bodily actions to test her visualization. This is common. What turned out to be a surprise was when she choreographs for a group dance. She would then imagine a transparent box (roughly proportioned like a stage) in which she visualized her group movements. Surprisingly, she would view this box from a higher line of sight and not from the usual audience angle.

Similarly when asked, an accomplished Jazz musician could hear eminent musician playing a tune in her mind’s ears. Interestingly, when asked to imagine her playing piano while mentally hearing the sound, she said the tune was smoother when mentally playing it. She also felt frustrated that she is not able to reproduce this smoothness, when playing it physically. Interestingly, with no prompt, she imagined the keys of her piano moving up and down on to her tune! This does indicate somewhat autonomous nature of events in the mind’s eye.
Sum up

 In the last post, we reported experiment where a midcareer industrial designer was asked to develop his product idea, when he was blindfolded. We saw how he successfully solved the design problem and that too with amazing dexterity. The experiment objectively proved that he completely conceived the idea in his mind’s eye. It revealed the possibilities of learning to handle mental imagery in design problem solving.

Mental imagery can potentially offer an effective alternative to sketching. However, these conclusions could be termed a bit hasty considering that they would be based on a single case, particularly because the results did look unbelievable. This post reports efforts to dispel this doubt through a series of follow-up experiments with same or similar objectives.

First, to eliminate the possibility of the first results being criticized as freak, we invited the same designer (SP) to work with identical experimental procedures and protocols, but with a different design problem. SP was asked to design salt and pepper dispensers and common dinning table crockery; all stacked in a compact stand on the dinning table. Like in the first experiment, the sketched results were validated through several independent routes. Besides evolving an effective solution, SP concentrated on complex production problems, where he used injection molding in plastic while avoiding undercuts. All this when he was blindfolded! The results unambiguously confirmed the earlier findings of the first experiment.

The post then goes on to explore the next step with newer and more challenging objectives. This was achieved through series of new experiments with allied design professionals, like architects, filmmakers and so on. To begin with, we offered similar experimental conditions to architects tackling an architectural design problem. These problems are qualitatively different from what SP handled. First, the built forms tend to be very large. Second, they have an outside and an inside that is explored through mental walks. Third, architects face two kinds of design problems; creating a new built form and developing interiors in already built spaces. In this series, we tried both types. Lastly, built forms are not always sculptural (except in parts) and could not be shaped by gestures the way SP handled products. At best, only some elements of the building could be sculpted.

In spite of these differences, the architects conceived their built forms and interior spaces effortlessly. However, there were some striking differences. Besides solving spatial layout problems, they spent a lot of time visualizing and controlling lighting conditions and ambiance. Appropriately dressed stereotypical people populated most of the spaces they created, but they were always faceless. They often used precedents and some used metaphors as a design strategy.

Most of them, who participated were practicing architects/designers and were above 35 years of age. To eliminate the possibility of design experience influencing the results, in the next series, we decided to invite younger age group, mainly students in their third year of architecture. None of them had problems completing the design project. The results broadly confirmed our earlier findings.

Becoming little adventurous, we decided to explore blindfolding the entire architecture classroom. Videotaping each pair was beyond our means, nor was it a practical route. So, we altered the experimental protocols to pair two students with the principal designer being blindfolded and the other acted as a junior or an equal partner. They were separated when the idea was completely developed and were asked to independently sketch what the pair had jointly arrived at. These sketches were then compared. The similarity in the sketches presented by the pairs was apparent.

These experiments proved that experience was not an important factor and even at that young age, students could conceive their design ideas in their mind’s eye. Like their seniors, they also focused on creating ambience by controlling the light and landscaping the interiors. The spaces were conceived from outside as well as inside and they often walked through the spaces that they created.

The results confirmed that age and experience does not matter. But it proved something more significant, i.e. it is possible for a pair to share a common image. Realizing that this could have potential impact on work in co-design, we explored this direction further. In the first series, the pair was connected with video and audio link, but in the last one, we cut of the video link. In spite of this, the pair was able to share a common image with reasonable commonalities.

Finally, the post reinforces the findings that design ideas of reasonable complexity can be completely conceived in the mind’s eye. Besides, these efforts proved that the results reported in the earlier post were not freak occurrence.

Preview of the next post

 So far we have conclusively proved the abilities of the architects, designers, filmmakers and even design students in overcoming difficult situations like working with an eye mask and complete the entire or substantial part of design in their mind’s eye. We also looked at how they used gestures in different ways to help them think and reason out ideas. In a way it supported the idea of embodied cognition.

In the next post, we have pushed architects further to explore how they can use body and gestures in solving the design problems. The idea of cornering the architects with new challenges was not a bad one. As you will see in the next post, it did bring out interesting strategies and thinking styles.

Watch young architects using their bodies and movements in thinking of solutions with amazing dexterity!

 If you liked this series of posts and if you are looking forward to the next post, do follow the blog. That ensures that you get intimation as soon as the new post is uploaded.

Notes and references

  1. Athavankar U., (1997) Learning from the way Designers Model Shapes in their Mind, Cognitive Systems : from Intelligent Systems to Artificial life? J.R. Issac and V. Jindal, Tata McGraw-Hill, New Delhi, pp 221-232
  2. Singh A., (1999) The potential of mental imaging in architectural design process. In: Proceedings of International Conference on Design and Technology Educational Research and Curriculum Development, IDATER 99. University of Loughborough, England, pp 230–236
  3. Athavankar U., Garde A, Kuthiala S (2001) Interventions in the Mental Imagery: Design Process in a Different Perspective. Proceeding of the 5th asian Design Conference, International Syposium on Design Science, Seoul National University, Korea.
  4. It is not that industrial designers do not ‘see’ people using their products. In the first experiment, SP also reported such incidents. However, they largely depended on their own mental simulations to test the ideas.
  5. Athavankar U. & Mukherjee A., (2003). Blindfolded classroom getting design students to use mental imagery Human Behaviour in Design, U. Lindemann (Ed) Springer, 111-120
  6. Athavankar U., Gill N., Deshmukh H., (2000) Imagery as a Private Experience and Architectural Team Work. In: Scrivener S, Ball L J, Woodcock, Springer-Verlag (eds) Collaborative Design. London, pp 223–232
  7. Bhedasgaonkar, M., Jalote A. and Athavankar U., 2000, Co-design: sharing mental Imagery ?: Team Thinking in Filmmaking, Proceedings of CoDesign 20000, Coventry September 11-13, 2000, pp.87-92.