She discusses the benefits of using multimodal social semiotics as a framework and the methodologies that need to be developed to support the empirical work, including working with children as co-researchers to help capture and document their lives. We talk about the strengths and limitations of the approach, how to overcome its criticisms and whether to use it alongside other theoretical perspectives.
With all theories and methods, it's about knowing the reach and limitations of particular perspectives and potentially then combining theories and methods to compensate for those issues, but always being aware of the lenses that we adopt and that they're always going to give us partial perspectives.
Multimodal social semiotics examines meaning making in multiple modes, explains Dr Kate Cowan. It moves away from the idea of language in social semiotics to thinking about all the various modes that are involved in communication which might include also images, moving images, music, gestures, facial expression, body movement, and gaze. These are explored in addition to, or sometimes also in the absence of, linguistic modes like speech and writing. Hence, a multi-modal approach offers a means of trying to look at these multiple modes within one theoretical and conceptual frame and to be able to grasp not only individual modes but also how they work together within an ensemble. This is particularly relevant for studying communication in the digital age.
Children are not just receiving media, they're not just consuming or reading it, they are often actively producing this media. So in the clips that they're making for TikTok, Instagram or YouTube they are making decisions about what modes to use together and how: how to edit, whether to use a soundtrack, whether they overlay text, in what colour, how big and so on.
We end the conversation with a discussion of how to create theoretical novelty and how the wider social context might drive developments in theory and research enquiry.
Watch the full vlog with Kate Cowan
Video: CO:RE theories vlog series - an interview with Kate Cowan.
If you are experiencing issues with the video player, please watch the video here on our YouTube channel.
Dr Kate Cowan is a Lecturer in Education at the University of Sheffield and an Honorary Senior Research Fellow at IOE, UCL’s Faculty of Education and Society. She is an early years specialist researching children’s play and communication from a multimodal perspective. Her research covers areas including literacy, creativity and digital technologies. Kate’s work also involves developing multimodal methodologies, video-based methods and participatory approaches with children.
Some of her projects include 'A National Observatory of Children's Play Experiences During Covid-19’ (funded by the ESRC), 'Playing the Archive: Memory, Community and Mixed Reality Play’ (funded by the EPSRC), 'Valuing Young Children’s Signs of Learning: Observation and Digital Documentation of Play in Early Years Classrooms’ (funded by the Froebel Trust), 'Encounters on the Shop Floor: Embodiment and the Knowledge of the Maker' (V&A Research Institute - funded by the Andrew W. Mellon Foundation), and a doctorate linked to the MODE Project (funded by the ESRC).
Prior to that Kate worked as a nursery teacher and she remains committed to connecting research and practice. In addition to academic publication in books and papers, Kate has written for early years teachers, students and the general public.
Further reading by Kate Cowan
Cowan, K. (2020). Tracing the Ephemeral: Mapping young children’s running games. Designs for Learning Journal. 12(1), 81-93.
Cowan, K. (2019). Digital Languages: Multimodal meaning-making in Reggio-inspired early years education. Media Education Research Journal. 8 (2).
Cowan, K. (2014) Multimodal transcription of video: examining interaction in Early Years classrooms. Classroom Discourse, 5 (1) pp. 6-21.
Bezemer, J., & Cowan, K. (2020). Exploring reading in social semiotics: theory and methods. Education 3-13. doi:10.1080/03004279.2020.1824706
Potter, J., & Cowan, K. (2020). Playground as meaning-making space: Multimodal making and re-making of meaning in the (virtual) playground. Global Studies of Childhood. doi:10.1177/2043610620941527
Cowan, K. (2018). Multimodal technologies in LEGO House: A social semiotic perspective. Multimodal Technologies and Interaction. doi:10.3390/mti2040070
Further reading by other authors
Jewitt, C. (2010) (ed.) Routledge Handbook of Multimodal Analysis, London: Routledge.
Jewitt, C. (2008) Technology, Literacy and Learning: A Multimodal Approach, London: RoutledgeFalmer.
Jewitt, C., Bezemer, J., & O’Halloran, K. L. (2016). Introducing Multimodality. Routledge.
Kress, G. R. (2010) Multimodality. A social semiotic approach to communication London: RoutledgeFalmer.
Kress, G. and van Leeuwen, T. (2001) Multimodal Discourse: The Modes and Media of Contemporary Communication, London: Arnold.
Explore the multimodality approach