Part I: To sketch or not to sketch? That is the question

As mentioned in the last post, this post was to deal with role that gestures and movement of the body play in spatialization of design ideas as sketches or images in the mind’s eye. We are deviating from that sequence. It was felt that an overview of the series would be more relevant before we move on to the complex role that gestures and movements play in design thinking.

Let us look back at several findings presented in the last seven uploads starting from ‘Why do designers sketch?’ We plan to take a bird’s eye view of the findings and reflect on them in this post. Now that we know the role sketching and mental imagery play in design problem solving, we can now reflect on the series.

Designing as resolving spatial issues

The designer’s decision making seem to be similar to most often-cited example; being in a room and working out alternative furniture arrangements to get the most effective layout at the end. Whether you use sketching or imagery most design problems are typically resolved through spatial decisions and spatial manipulations of elements (in this case furniture pieces). For this part of the discussion, let us treat designing as resolving spatial issues to achieve pre-determined functional goals through design actions. So, whether you are creating a space (layout problem) or a smaller object, what designer’s handle/create are the spatial elements and their locations in space. In part II, we will revisit these issues through the lens of design thinking.

There are two ways to handle such an assignment. In both approaches, actions are in this ‘real’ space, which becomes a context for situated cognitive activity. Whether you use sketching or decide to work it out in the mind’s eye, the process involves perception and monitoring of representations that are being continuously created and altered by reactive design actions. Most reactions typically result in yet another new representation.

Let us focus on each of the two ways to handle spatial issues.

Sketching as a medium for conceptualizing of objects

Though this series does not directly report research on sketching, the author had worked on the role of sketching prior to exploring mental imagery. Some of the statements rely on author’s earlier research in sketching.1, 2

The first approach is to use sketching pad and pen, if handy. The designer draws the room and thinks of new locations and/or orientations of furniture pieces, then quickly draws the new layout. Looking at the sketch he reviews the new idea for its effectiveness. He will often see and react to what he has sketched and come out with a new alternative to be executed as yet another new sketch. You can capture the process as iterative cycles that involves –

“Think -> Act (sketch) -> Review -> Transform -> Think again”

till he feels that he has met the demands made by the design problem. He may take a holistic approach or take one element or one function at a time and handle complexity incrementally.

If the designer has mastered sketching, this process is cognitively less taxing. There are several reasons. First, it also allows starting with a tentative idea and incrementally moving to a more complete and complex state. In the earlier post “Thinking through the messy sketches” we discussed this approach. Second, designer does not have to budget excessive mental energy to handle the actions of sketching. Third, when he creates a record in a media outside of his body (i.e. as sketches), he does not have to budget energy to memorize and recall his past action and decision. So, if he is quick and good at sketching, it is still a cognitively economical option.

Willful control on the act of sketching, particularly in representing objects in perspective, is a critical issue when implementing a sketch. Sketching is a learnt action and is often taught in a structured way in most design schools. There is enough literature on learning to sketch. My own work on control on sketching is available on D’Source. 2

If designer is bad at sketching, this approach can be counter-productive. He needs to budget more mental energy for the act of sketching, but this is at the cost of mental energy required to come up with new ideas. (Refer to earlier post ‘Why do designer’s sketch?’) Designer’s additional efforts to judge the correctness of the sketch can demand energy budget that could interfere with the ongoing thinking of the design problem. So, designer needs ability to sketch quickly and effortlessly, as if it is a routine and natural act.

That is why in the earlier post, “Out-of-the-box ideas for teaching sketching”, focus was on learning to draw with a ‘feel’ of the line drawn via kinesthetic feedback and by diverting attention away from the pencil tip. Reducing the need to depend too much on visual feedback while sketching, allows you to budget the mental energy saved to attend to the demands of design problem solving. Several ideas were presented to explain how the entire body could participate in the act of sketching. The kinesthetic feedback generated by sketching actions can be used to ‘feel’ the correctness of the sketch being executed. It is critical to learn to sketch effortlessly and with natural actions.

The external display that sketching creates reduces the effortful activity of holding the ideas in your memory. However, the mental imagery is not completely eliminated. You have to mentally decide the new location of a furniture piece first, try to review it and quickly sketch what you have imagined. Thus you avoid spending mental energy on holding ideas in his mind and comparing them later. In short, competent sketching ability helps him conserve mental energy so that he can concentrate on developing the next idea.

There is another way to solve the problem of room layout. Imagine now that designer does not have the paper and pen or chooses not to use it. After all, the

Eureka movements may not occur when you have paper and pencil in your hand. Indeed with Archimedes, it occurred in the bathtub!

Designer will have to then depend on his visualization abilities.

Mental imagery as a medium for conceptualizing of objects

For most people it is so natural to see images in their mind’s eye, that many times they are not even aware of it. So, it is difficult to talk of mental imagery. We see images in our dreams too. Like a dreamer, the blindfolded designer also believes that he is in a ‘real’ environment and is awake.

What is the difference in the images that designers’ see when they are designing blindfolded and what everyone sees in dreams?

Unlike in dreams, the blindfolded designer is indeed awake. More important to note that the designer mostly experiences ‘constructed as well as work in progress’ images that he conceives, modifies and builds in time and space. The site that the architects visualized was ‘real’ space that experienced by walking in and out. It is a highly immersive experience that designers choose to willfully enter in.

It is important to view intentionally constructed images as unique experiences and as ‘work in progress’. This continues till the designer decides to artificially stop. Ideas through images continue to evolve, sometime even when the designer is not consciously working on it. Remember Archimedes?

What is critical is the ability to willfully control image generation and transformation operations in response to the thoughts that drive these operations.

This is why the experiences of imagery during design sessions are different from what happens in dreams. The willful control is needed to handle the unique operations that designer performs. Interestingly, he learns to do this without any formal learning programme.

Designing and the mind’s eye

Let us return to the furniture layout problem, but this time designer does not have a sketchbook with him. Thoughts of the designer continue to drive new solutions and prompt new ideas. He can try out alternatives by physically shifting the furniture pieces around, but that will be not only time consuming but also strenuous. So, the designer would have to solve the room layout problem by relying extensively on his internal resource and working out solutions in their mind’s eye.

Designer could move the furniture pieces around, reorient them, think of creating alternatives mentally. He could view them and compare the alternative solutions. All along, he has to view the current state in the mind’s eye, if necessary bring back the earlier ideas from memory and react to this internal display creatively to generate even newer ideas. He has to hold all of them in memory and compare and contrast them. No doubt a mentally stressful task.

These operations are far more taxing than recalling of the images seen earlier. You can clearly see that the visualization actions obviously go far beyond just recalling the images and inspecting them. Obviously, designer needs more complex capabilities for such a creative use of mind’s eye.

We knew so little about how designers willfully control mental imagery event. By blindfolding the designers in our experiments we forced a situation, so that these capabilities could be studied. The series is biased towards a detailed treatment of mental imagery because, unlike sketching, we were ignorant of the issues involved. If this ability to willfully perform and control operations on the images in the mind’s eye is critical for new ideas,

Should we not identify the operations that designers perform and/or learn to perform?

Fortunately, some of the operations that can be performed on mental images have been identified in cognitive studies. Most well studied operation is mental rotation of objects along three axes.3 Further, Kosslyn identified three broad operations. They included, holding the image and inspecting it (Image inspection technically referred as scanning), generating the image (image generation) and lastly transforming the images (image transform). Last one is a powerful operation and includes several types of transformations.4

Most of these operations are clearly visible in our experiments. The videos clearly show that designers and architects iteratively switch between image inspection, image generation and image transformation. They cyclically inspect the results in their mind’s eye and reflect on these images. We will take these operations one by one.

Image Inspection (Scanning)

This operation usually deals with inspecting the recalled image from the memory (LTM and sometimes STM). Ideal examples of image recalls were when SP as well as the architects were asked to describe their final design after they declared that they have finished designing. The object seem to be virtually there in front of them (SP) or in case of architects, built space enveloping them. When they described, they appeared to be ‘seeing’ in their mind’s eye and ‘reading off’ the images. Similarly, architects occasionally recalled the site plan that they were given to memorize, but subsequently these drawings were often transformed into a 3D view with all its features to see in their mind’s eye.

These designers also recalled images that are part of their memories and use them as precedents. In our case, such images dealt with previously seen, and perhaps liked, examples of objects or built spaces or their features. For instance, in student pair experiments, there is a clear evidence of pergola roof, deck, Japanese garden and water fountain as precedents brought in as images from the memory. Needless to say that these precedents and features get transformed creatively to suit the new context, when they are used in design.

Obviously, the designers are able to hold their images in their mind’s eye, inspect (or scan) them and react to them as they work with the image. They would often inspect images to make judgments, spot inconsistencies, search for opportunities to alter and manipulate elements within the image. It serves the same function that sketches as display serve. It allows designer to review his ideas, creatively react to them and iterate. What exists as current, gradually move towards effective solution incrementally. This is similar to what occurs when sketching is used. Image inspection leads to spatial judgments and all design tasks are based on these judgments.

Yet, there are logical differences because of the nature of the media, in this case the sketchbook and the mind’s eye in which the display occurs. Mental images are fragile and fadeout if you are not attending to them actively. Cognitively, to hold and inspect an image is an active process that demands budgeting of mental energy, more so, when it is done purposefully.

Because designers and architects are involved in creating new objects and built forms, they tend to often inspect what they created and then manipulate it. So, most examples of inspection are part of the process of image generation and transformations.

Image generation

If you watch the video of SP in action in casserole experiment, it shows sufficient evidence of SP’s logic, ideas and thoughts directing his design actions and gestures. These often result in generating images of the object that he was developing. He develops the shape from scratch, builds it up step-by-step as an image. SP choses to ‘treat’ the overall image of the object as ‘real’ and even physically shapes it, making changes interactively as he goes ahead!

Most of SP’s actions in image generation were incremental and became more detailed as the time passed. Observe SP working with a virtual object in the front. See video 1 below.

Video 1: The designer created a virtual object in front of him and shaped it as if it was real for the entire period when he was designing.

As he advances with his design, he fondly sculpts and alters the shape. Similarly, when the architects were blindfolded and let into a big hall, they first create a gross layouts, detail them and as they go ahead, build the 3D spaces around them.

It is common to see the cyclic process that starts with generation of the image, then inspecting it, reacting and implementing changes in their mind’s eye. That explains the incremental nature of the development of the ideas. Note that there is little change in the design process as in both cases. There is only a change in the media used for displaying the current state of design.

Most of these videos have short pauses of few seconds when they are silent. Pauses seem to be useful to judge the current state of design idea. Pauses longer than three seconds were purposeful. They are used for inspections of the images in the mind’s eye and reflecting on it. You can clearly see this in video 1. They appear to be part of spatial judgment efforts and end in actions and decisions immediately after. Typically, they end up with flurry of activity, often resulting in a new idea or a modification that had existed as an earlier idea. SP pauses several times to ‘see’ the object in his mind’s eye and then reflects on it. He ‘looks’ around to inspect what he had created and pass judgment after a pause. After one such pause midway through the casserole session, SP sculpts the shape, pauses and comments like ‘That will look interesting’. See this in video 1 above.

Image transforms

Image transformation is a critical operation on the images. It is also the most often used operation to quickly alter the contents of the image seen. Most surprising finding was that in implementing the operation, designers physically interacted with their creations in the mind’s eye. In fact such interactions were at their best in image transformation. For example, while thinking of support to the casserole body, SP’s palms represented the folding legs of the casserole and movements of the palms simulate the leg movements.5

There are several complex transformations of objects or spaces that architects and designers seem to comfortably deploy while designing. For instance, SP’s casserole video has several examples of the shape or some element of it being reshaped, chopped, moved, flipped, rotated and so on. See this in video 2 and 3 below.

Video 2: Observe designer exploring the shaping (curving) of the bottom of the casserole.

Video 3: Observe designer shifting from shaping of the lid to folding legs.

These videos include actions like selectively moving the objects or elements, manipulating proportions and compositions, changing sizes, altering colours, exploring different backgrounds and even creating exploded structure. They appear to use more than one operation, often in continuous sequences, and that too with amazing ease. The gestures come handy in the process of working with it and on it. More about gestures in part II in the next post.

While modifying their ideas and exploring alternatives, architects too performed different transformations of the images. Architects however use gestures more often to indicate and locate than to shape the spaces, perhaps because of the scale of these spaces. Earlier posts do to indicate few examples of use of gestures in shaping some parts of the building. For instance, see video 4 below. However, these instances are fewer in numbers.

Video 4: See designer shaping the entrance with hands. 

The purpose of most of the transformations is to improve the effectiveness of the solution at hand. It is typically followed by image inspection, where the designer would try to assess the implications of the changes they had just made. The use is similar to the way the sketch is reviewed, except when working with mental imagery, it is lot more difficult.

These changing mental events are available in form of fragile images in their mind. They have to hold these static as well as sometimes-dynamic images in their mind’s eye, which demand budgeting mental energy. Additional energy is required for reacting to them, altering the image with new intentions and hold on to the new image.

What does the series reveal?

The discussion so far has also helped us separate two forms of representations that almost serve similar function. Most discourses on designing focus on the first form i.e. sketching, and do not acknowledge mental imagery as serious contender as a form of representation. It is understandable. Indeed, there are no capturable external representations to write about or comment on, when working with mental imagery. One of the contributions of this series is to discover methods that give access to the actions that occur in the mind’s eye.

The series argues that sketching as well as mental imagery serve similar role but deserve to be treated as separate acts.

We hope that this separation will remove the bias towards sketching and studies in understanding the role of mental imagery in design(erly) thinking will get the attention it deserves. It is hoped that the shift of spotlight will balance the studies of role of representations.

Why is attention to the mental imagery critical?

It is known that this ability is directly correlated with creative efforts. To willfully change, transform or manipulate the image or some of the elements of the image selectively to intentions, distinguish creative people from others. Artists, dancers, architects, designers and some scientists can willfully control what happens in the mind’s eye. Einstein was known to create events in his mind’s eye that he often referred as thought experiments. Mozart was known to compose his orchestra entirely in his mind’s ears, an audio equivalent of mind’s eye. He was also able to hear and follow each instrument in isolation. There is enough scientific evidence to suggest that the ability to handle events in the mind’s eye and creative work are closely related. Finke has spent several years researching the role of mental imagery in inventions using cognitive science framework.6

Understanding nuances of visualization

We now have a clearer understanding of the term visualization and what it means to design community. It is not just externalizing ideas in a sketchbook or on a computer screen. It should include perception and manipulation of representations in the mind’s eye.

Visualization is much used and abused word. At the simplest level, it can be defined as ‘seeing the object when it is withdrawn and is no more available to be perceived.’ We are then asked to recall what it looked like. Can we restrict visualization to just recalling and inspecting the image in the mind’s eye?

Ability to recall or generate an image is not unusual. Most people are able to recall and view images. Dreams also involve recall of images. Dreamers are convinced that they are in a real environment, are awake and are experiencing a real event in time and space. Situation is somewhat similar when the designers were blindfolded, except that they were actually awake. Besides recalling, lots of people generate and see new elements in their images in dreams (also under hallucination) that they have never seen before.

Learning to willfully control mental imagery

The major difference is in the ability to willfully control the events in the mind’s eye. Architects, designers and artists are able to perform several cognitive operations on the images in the pursuit of the creative work. However, unlike sketching which is systematically taught to them, they learn to handle mental imagery with no formal training.

Can we learn to willfully control events during visualization without design training?

To handle mental imagery, at least till this time, there is no structured training available. It is perhaps partly a natural talent and partly perfected on job through practice. We have little knowledge of how willful control on imagery can be taught through training.

The focus leaning towards sketching as a preferred representation could change in future as designers learn the advantages of mental imagery as a creative thinking tool and find strategies to develop educational material for everyone. I saw the absence of training as an opportunity to convert my understanding of controlling mental events by developing visual puzzles that can be solved by use of some of the cognitive operations that we discussed. I hope to present these games and visual puzzles in some later post.

Who else can benefit from such training? There are lots of people who are involved in design but are not competent in drawing. Their creative energies could be harnessed if they could be trained to be comfortable with mental imagery.

Learning to handle sketching

Sketching does come handy when the complexity of the design project is very high. However, it demands rigourous and systematic learning to model 3D ideas as 2D sketches and draw them skillfully. I personally believe that initial training in orthographic drawing trains your mind to switch between 2D and 3D effortlessly. It has long learning period. Obviously, only few professions need it and go through such training.

To think, model ideas in mind and simultaneously sketch them is a task that needs systematic learning and practice. The focus on drawing by feel and by kinesthetic feedback and not by sight has its roots in this idea.

Note that there are other forms of representations including verbal descriptions that are used in communication. Most people use language descriptions accompanied by gestures to explain design ideas. In fact, they are effectively used in brainstorming. I tried to study gestures with minimal use of language with some success.7 But discussion on these topics are outside the scope of this series.

Sum up

This post is an overview of what was presented in the several earlier posts on sketching and mental imagery. It argues that 1] sketching and mental imagery should be recognized as two separate forms of representations. 2] The representations are a critical constituent of design(erly) thinking, 3] Both act like mediums for conceptualizing design ideas, 4] Both of these forms serve similar functions. So, technically, they can substitute one another. The post also compares sketching and mental imagery from the point of expending of mental energy. It suggests necessity of removing the current bias towards studying sketching and treat studies in imagery on par.

Focusing on mental imagery, it identifies the three broad cognitive operations, Image inspection, Image generation and Image transformation, that can be performed on the imagery. These are explained through examples taken from the experiments cited before. It argues that the understanding of visualization in the context of design should necessarily include learning to willfully control cognitive operations in response to the designer’s intention.

At a broader level, the design process has commonalities irrespective of the forms of representations used. What differ are the cognitive operations when working with sketching or exclusively in mind’s eye.

To sketch or not to sketch?

We started our discussion by exploring the role that sketching and mental imagery play as conceptualization tool. Both the forms have their merits and demerits. The choice should depend on designer’s comfort level with these two forms of representations. To be able to competently and skillfully handle both forms of representation would be an ideal situation. The complexity would not trouble the designer. The design response would be quicker. As one of the participant said in the context of use of imagery, and I quote

“I carry my problem with me all the time now”

 

Preview of the next post

We have restricted the above discussion to the implications of production and modification of display through sketching or through use of mental imagery. However, our discussion does not explain why the architects moved, walked and used the spaces they created, nor does it explain the deeper role that gestures play. In part II, will address questions like,

Why do designers use gestures and movements of the body when they solve problems? Does it support spatial decisions, visualization and design(erly) thinking? And if so, how?

Next post will discuss how these two forms of representation affect design(erly) thinking and why in spite apparent similarities, they are conceptually different.

Notes and references

  1. Athavankar U., (1992) Rediscovery the Act of Sketching: Implication of its Support to the Creative Thought Process, Design Recherche, No. 2, pp 45-60
  2. Discussion on freehand drawing http://www.dsource.in/search/content/Freehand%20Sketching Dec 21,2017
  3. Shepard, R.N., Metzler, J., (1971) Mental rotation of three-dimensional objects. Science 171, 701–703
  4. Kosslyn S., (1983) Ghosts in the Mind’s Machine: Creating and Using Images in the Brain. Norton, New York
  5. Athavankar, U. A. (1999). Gestures, Imagery and Spatial Reasoning. In: Garo, J. S. & Tversky, B. (Eds) Visual and Spatial Reasoning. Preprints of the International Conference on visual and spatial Reasoning, (VR 99) MIT, Cambridge, pp 103-128.
  6. Finke R., (1990) Creative imagery: Discoveries and Inventions in Visualisation. Lawrence Erlbaum, New Jersey
  7. Varshney, S., 1998, Castles in air: A Strategy to model shapes in a computer, in proceedings of the conference ‘Third Asia Pacific Computer Human Interaction (APCHI ’98), Japan, July 1998, 350-355