Overview of multimodal literacy
A multimodal text conveys meaning through a combination of two or more modes, for example, a poster conveys meaning through a combination of written language, still image, and spatial design. Each mode has its own specific task and function (Kress, 2010, p. 28) in the meaning making process, and usually carries only a part of the message in a multimodal text. In a picture book, the print and the image both contribute to the overall telling of the story but do so in different ways.
Images may simply illustrate or expand on the written story, or can be used to tell different aspects of the story, even contradicting the written words (Guijarro and Sanz, 2009, p. 107).
Effective multimodal authors creatively integrate modes in various configurations to coherently convey the meaning required, ‘moving the emphasis backwards and forwards between the various modes' (Cope and Kalantzis, 2009. p. 423) throughout the text.
The complexity of the relationships between the various meaning or semiotic systems in a text increases proportionately with the number of modes involved. For example, a film text is a more complex multimodal text than a poster as it dynamically combines the semiotic systems of moving image, audio, spoken language, written language, space, and gesture (acting) to convey meaning.
