Blog

Design as communication, Design as conversation : A semiotic investigation
This article was written as an introduction to semiotic engineering – a theory of HCI that uses concepts from semiotics and computer science to focus on the communication between designers and users during interaction time.The article attempts to capture some of the crucial elements of the theory reflected in Clarisse De souza`s seminal book “The Semiotic Engineering of Human-Computer Interaction”

Design as Communication
Semiotics is interested in the relationship between things, thoughts, and signs - a sign being a discrete unit of meaning. Signification is the process through which certain systems of signs are established by virtue of social and cultural conventions adopted by the users; who are the interpreters and producers of such signs. Semiotics can be defined as the study of signs, signification processes, and how signs and signification take part in communication.

Semiotic engineering proposes an account of Human Computer Interaction HCI that draws on theories and concepts from the field of semiotics and computer science. The synthesis of these fields of study is an attempt to bring together semiotics and HCI in a concise and consistent way; creating a new framework for investigating the relationship between a designer and a user of an artefact. This framework we can refer  to as “Design as Communication”.

The essence of semiotic engineering is in the communication that occurs between the designer and the user; it models design as a form of communication between these two actors. Clarisse de Souza describes, “…both designers and users are interlocutors in an overall communication process that takes place through an interface of words, graphics, and behaviour. Designers must tell users what they mean by the artefact they have created, and users must understand and respond to what they are being told …”[1]

In order for design to be effective this communication must contain a message(s) which will be mediated through the interface of the artefact. The message(s) must be clear and in some way targeted to the user’s requirements, enabling them to decode and understand it. This interaction forms a meta-communication i.e. an act of communication between two agents that also communicates something about the communication itself, or about the relationship between the two actors, or both.

To elaborate, the communication is initiated by the designer; he must first envision and construct a conceptual model of how the system functions and how it is likely to be used. If the designer is aware and perceptive he will consider the potential users of the system (archetypical users) and the context of usage (scenarios of how the system will be used by the users). This approach can lead him to identify the user requirements more accurately, helping him to interpret the users’ problems and to precisely define the “problem space”.

The designer must somehow be present in the interface, at the point of interaction, to tell the user how to use the signs that make up a system. The designer does this by mapping his conceptual model of the system onto a physical artefact. The designer’s conceptual model will be projected to the user at the point of interaction with the artefact. In a software system the designer might create a graphic user interface GUI which will allow him to map his conceptual model.

When the user enters the design communication, he is the receiver / interpreter and he performs his role with a preconceived, preconditioned model of how the artefact works. This “mental model” is an internal scale-model representation of an external reality. Norman describes, “In interacting with the environment, with others, and with the artefacts of technology, people form internal, mental models of themselves and of the things with which they are interacting. These models provide predictive and explanatory power for understanding the interaction” [3]

For the user to successfully use the artefact, characterized by positive utility and a good user experience, the user must have a similar mental model (the user's model) to that of the designer (the conceptual model). The ultimate challenge for the designer will be to narrow the gap between the conceptual model of the system and the mental model of the user; while trading-off against constraints.

There are three significant advantages to adopting the Design as Communication approach. The first is that it focuses the researcher’s attention on signs, in a sense once we begin to conceptualize design as a system of signs, in a meta-communication, between two actors and artefacts & interfaces become media - the entire design process becomes expanded but more intuitive.

The second is that Design as Communication provides a somewhat rational system for the analysis of communication and design problems. HCI naturally considers the contributions from the fields of computer science and psychology. Computer science focuses on the machine and does not adequately account for the human mind, while psychology focuses on the mind and does not adequately represent its study algorithmically. Design as Communication mediates this divide by building a bridge between mind and machine.

In effect, by using this framework a researcher can qualify and quantify design criteria as ‘good’ or ‘bad’ based on its ability to effectively and efficiently communicate the embedded messages of the design; which should ultimately convey positive utility and user experience for the user. Any difficulties and confusion experienced by the user is evaluated as a breakdown in the design communication. This is to say, the designer’s choice of signs fails to convey the intended message to the user effectively. Design and evaluation techniques in semiotic engineering thus involve identifying and preventing design communication breakdowns.

The Thrid advantage is that designers are reappointed to a role of significance in the design process, this is contrary to the prevailing logic of user-centred design which assigns user requirements a more prominent role. Within the semiotic engineering framework designers and users are assigned to the same role. According to De Souza semiotic engineering, “… brings designers onto the stage of HCI processes and assigns them an ontological position as important as that of the users”. [1]

Design as Conversation
Marcel Duchamp in his essay the “creative act” describes a metaphysical relationship between the artist and the spectator. He interprets that “All in all, the creative act is not performed by the artist alone; the spectator brings the work in contact with the external world by deciphering and interpreting its inner qualifications and thus adds his contribution to the creative act …” [2] Of course, art and design are two different things that operate differently with alternative objectives. Design is often viewed as a more rigorous form of art, or art with a clearly defined purpose. However, in both mediums the relationship between the actors are the same and they communicate with each other in a meta-communication.

From Duchamp’s description we can extract two important insights. First, we can identify that an interplay or dialogue exists between the spectator / user and the artist / designer and that both are actively involved in the creative act. This interplay is analogous to a conversation, in the sense that it requires input from both actors and the path is indeterminate. Ultimately, the process is negotiated by the needs and the intent of the actors involved, which makes it both a cooperative and a reciprocal process.

Secondly, we can deduce that the spectator / user enters the conversation not naively but conditioned by his environment and experience and bring this to bear on artefact. From a design viewpoint (where communication is deliberate and intentional in order to induce interpretation) the context of how a user enters this meta-communication - his precondition and his perception becomes a very important consideration.

The context is defined by all the circumstances that bear upon the situation at the point of interaction between the user and the artefact. The context has the power to change the meaning of a message - it is a snapshot of the environment framed by the user’s psychology. It is defined by the user’s first person perspective of reality which is conditioned by experience, knowledge, perception, culture, media etc. In terms of interface design it is defined by the user’s conception and understanding of the media and his engagement with technology in the digital environment. It is from this context and history that the user shapes the formation of his mental model about a particular artefact and related artefacts.

By introducing new technologies into the environment the designer changes the environment and re-condition the user context e.g. ipod, iphone, blackberry. According to Winograd and Flores, “The new design adapts to the context of the world that has previously existed and is now changed by the innovation”[4] “By creating a new artefact, the researcher sets the stage for new conversations, connections and communications” (winograd and flores, 1984). Conceiving design as conversation allows for a flexible approach to redefining the conventions and definitions of context; which will ultimately be negotiated through design.

Design as Conversation also reveals to designers the understanding that the value of their work depends not only on the intention and the vision they put into their representation but also the interpretation of that work from the people who experience it. Meaning is not simply encoded, transmitted and decoded; it is actively created and negotiated according to a complex interplay of sign systems of which designers are generally unaware. According to Roman Jacobson, “meaning resides in the total act of communication. It is not a stable, predetermined entity which passes, untrammeled, from sender to receiver”.

Once designers understand that meaning isn’t fix and something that must be negotiated; they can become aware that their work has an unstable meaning outside the complex set of factors that define it i.e. in the sense, “nothing is a sign unless it is interpreted as a sign” Pierce. According to De Souza, “… in a semiotic perspective, there is no room for a purely objective and stable account of meaning, whether the designers’ or the users’ …” [1]. This inexactness and instability in the design process is one of the inherent constraints of prevailing predictive methodologies that attempt to precisely identify, understand and predict information about what users want and need. The inherent subjectivity of “meaning determination” questions whether the user’s context, mental models and interactions with the artifact could be accurately captured by a researcher.

Predictive and Empirical approaches attempt to calculate a sort of “certainty” in an indeterminate process - whether this approach alone is efficient is questionable. These methods do add value to the design process, the main objective of usability testing is to improve the design of interfaces. However, when we approach design as conversation these models fall short and only focus on the user side of the design conversation. Because the designer’s voice is absent these models are at best complimentary and not absolute.

Too often prescriptive approaches minimize or neglect the creative process required in designing innovation. The designer must input into the conversation, his voice announcing innovation; in an open dialogue with the user. In this dialogue the user is empowered to discover the new features of the artefact, learn the system easily and derive positive utility and experience from usage. According to Erich Fromm, “The quest for certainty blocks the search for meaning. Uncertainty is the very condition to impel man to unfold his powers. Creativity requires the courage to let go of certainties”. [12]In effect, Designers should embrace an open situation that is not certain, that is conversational and that may lead to inspiration and serendipity.

References
1.    Clarisse Sieckenius de Souza, The Semiotic Engineering of Human-Computer Interaction, The MIT Press (March 1, 2005)
2.    Delivered at the convention of the American Federation of Arts, Houston, Tex, in April 1957, and published in Robert Lebel, Marcel Duchamp, New York, Grove Press, 1959.
3.    Gentner, Stevens, Mental Models (Cognitive Science Series), Lawrence Erlbaum Associates; 1 edition (May 1, 1983)
4.Winograd, T. and Flores, F. Understanding Computer sand Cognition: A New Foundation for Design,Addison-Wesley, Reading, MA. 1986
5.Read, S. J. & L. C.Miller (1995). Stories are fundamental to meaning and memory: For social lcreatures, could it be otherwise? In R. S. Wyer, (Ed.), Knowledge and Memory: the RealStory (pp. 139–152). Hillsdale, N J: Lawrence Erlbaum Associates.
6.    Stamper R (2001)Organisational Semiotics: Informatics without the computer? In: Liu K, Clarke R, Andersen P, Stamper R (eds) Informa-tion, Organisation and Technology: Studies in Organisational Semiotics. Kluwer Academic Publishers, Boston, MA, pp 115–171
7. Shannon, C., Weaver, W.: The Mathematical Theory of Communication. MIT Press, Cambridge.MA, 1949
8. Lakoff, G. (1993). The Contemporary Theory of Metaphor. In A. Ortony (ed.) Metaphors and Thought Cambridge University Press
9. Jodi Forlizzi, Understanding Experience in Interactive Systems,HCII and School of Design Carnegie Mellon University
10.Michael Mateas,Narrative Intelligence,Computer Science Department, Carnegie Mellon University
11. José Cordeiro and Joaquim Filipe, The Semiotic Pentagram Framework A perspective on the use of Semiotics within Organisational Semiotics
12.Erich Fromm - internationally renowned Jewish-German-American social psychologist, psychoanalyst, and humanistic philosopher.
13. Quote, Naomi Klein, author of,  No Logo, Flamingo; New Ed edition (15 Jan 2001)