Skip to main content
School of Electronic Engineering and Computer Science

Metaphors we share: Leveraging cross-sensory associations in communication

This PhD project involves research in the areas of communication acoustics, cognitive linguistics, and crossmodal perception applied to modelling the dynamics of emergent communication systems. Academic disciplines spanning those areas have made strides in understanding how humans communicate and interact, yet scientific understanding of the function of cross-sensory associations in communication is still in its infancy. This project will leverage citizen science and gamification methods to study cross-sensory communication in the form of a novel experimental paradigm, going beyond existing research by assessing cross-sensory associations in a dyadic communication game, tasking participants with creating an emergent communication system using sound, shape, colour, and visual texture (https://www.seeingmusic.app/).

We will aim to explore several open questions with this paradigm, simultaneously hitting upon new ways of exploring cross-sensory associations, exploring the dynamics of emergent communication systems, and testing the role of fully continuous vs discrete meaning and form spaces in communication. The communication game will be developed from in-lab controlled experiments to a fully digital online tool that can be used for crowdsourcing large-scale behavioural data via public engagement. Prospective candidates should be curious, self-motivated, and have experience in one or more of the following: Communication Acoustics, Cognitive Science & Language, Multisensory Perception, Interactive Digital Media Engineering & Design.

The student will be based in the Centre for Digital Music research group.

This topic is co-supervised by Dr Charalampos Saitis, Dr Christine Cuskley (Newcastle University) and Prof Massimo Poesio