The design of interactions with sound and audio processes is a seminal activity in the creation of a performance, installation, a virtual sound environment, or interface for musical expression. The interaction design is often fixated by the interface without taking into account human factors and our diverse abilities to perceive the sound and interface affordances.
Human-Sound Interaction (HSI) look at human-centered interaction design aspects that determine the realisation and appreciation of musical works (installations, composition and performance), interfaces for sound design and musical expression, augmented instruments, sonic aspects of virtual environments, interactive audiovisual dance performances.
Balandino Di Donato, Christopher Dewey, and Tychonas Michailidis. 2020. Human-Sound Interaction: Towards a Human-Centred Sonic Interaction Design approach. In 7th International Conference on Movement and Comput-ing (MOCO '20), July 15–17, 2020, Jersey City/ Virtual, NJ, USA.ACM, NewYork, NY, USA, 4 pages. DOI: 10.1145/3401956.3404233 https://doi.org/10.1145/3401956.3404233
Background and Aims
Referring to principles of affordance of musical sound, and the diversity of abilities and human factors to perceive them, this workshop invites participants to take part in a collaborative activity for the exploration of Human-Sound Inter-action (HSI), defined as direct, engaging, natural and embodied interaction with sound :
direct as the impression or a feeling about an interface capable of being described in terms of concrete actions 
engaging as fostering the “feeling of directly manipulating the objects of interest”, where “the world of interest is explicitly represented and there is no intermediary between user and world”
embodied as an extension and incorporation of humans skills and abilities within the interaction design of a system 
Below is a video presentation of a recent work realised around the concept of Human-Sound Interaction :
We aim to create a wider interest around HSI across different communities to explore current knowledge and create new thinking. Through their different experiences practitioners at the intersection of music, sound art, performance and Human-Centred Interaction Design, we can explore how to make the sound experience more inclusive and diverse. Findings will contribute to a body of knowledge on Human-Sound Interaction shared with the community.
We invite the scientific and artistic community working on topics HSI relates, including:
Human-Centred Design approaches for sonic interaction design
Interactions designed by/with/for diverse musicians
Fostering diversity in music and sound art through interaction design
Sonic and cross-modal music affordances
Embodied interaction with sound
Interaction design for augmented instruments
Adaptive frameworks for interaction design
Interaction design for audio/musical interfaces
Interaction design for interactive sound art and installations
Impact of interaction design in artistic practice, culture, societal and gender aspects
Impact of human, artistic, cultural, societal and gender factors in designing interaction with sound
Inclusive interaction design strategies with sound
Sonic interactions in Virtual (VR), Augmented (AR) and Mixed (MR) realities
Interactive audio-visual dance performance
All you need
For these activities yout will be asked to use any tool that they are comfortable using for sketching and design ideas. For example: pen and paper, or a software such as
Microsoft Office, etc.
Cosider that you will have to share wit others using your screen or camera. Make sure you are all set for this.
In this half-day workshop, you are invited to take part in the following interaction design activities: warm-up,divergent, incubation break no.1, div-interaction, incuba-tion break no.2, and convergent.
Introduction (30 minutes)
Introduction to Human-Sound Interaction concept and the workshop schedule.
Warm-up (10 minutes)
You will generate five or more extremely bad HSI ideas. You can use any design approach, random metaphors, or sketching absurd musical or sound design contexts. “To invent, you need a good imagination and a pile of junk.” Thomas Edison .
Divergent (20 minutes)
In the first 10 minutes, you will be asked to select ideas from the warm-up session, or ideate between 2 and 4 new HSIs, without thinking at the potential interface that fosters such interactions.
Later, in the second 10 minutes, you will focus on the implementation of their HSI in the real world situation. Through sketching, you will have to think design an interface, installation, software mapping solution or any other artefact that can potentially satisfy the human-sound interaction you have previously designed. We will call this ``the prototype''.
Incubation break no.1 (20 minutes)
In this phase, you have the chance to let your ideas sink-in and have a rest.
You can do anything you wish, which does not require any particular cognitive load, but that allows them to keep their mind in a free flow state; for example, making coffee, washing up, rest, a nap, tidying up, not answering your emails or writing a paper.
Div-interaction (30 minutes)
At this point, you will enter a collaborative design phase. You will work in pairs with another participant.
In the first 15 minutes, you will share with another participant the prototype ideas only, thus, your companion will design an HSIs based on your prototypes; and, your companion will share with you the HSI design only; thus, you will have to design an interface prototype upon the HSI of your companion.
This process is then reversed, with you sharing with your companion the prototype, and your companion sharing with you the HSI.
This phase is crucial to the whole workshop. Here themes of Human-Interaction Design in a collaborative and iterative process will generate through a natural exchange of ideas. The order within which each participant will exchange the HSI, and the prototype will allow us to observe the adaptive, transferable and transformative aspects of the design process.
Incubation break no.2 (20 minutes)
As Incubation break no.1.
Convergent (40 minutes)
You will receive back your original HSI and prototype designs from the participant you paired up in the Div-interaction phase, and go through a convergent design process, in which you will finalise the HSI and the prototype. More to the point, in this phase, you will have to finalise their HSI designs and implementation.
Wrap-up (10 minutes)
Closing and remarks of the session.
After the workshop
It would be great to have your feedback. You will be asked to fill out a questionnaire about the format of the workshop, thoughts and comments.
Balandino Di Donato is a Digital Artists, Researcher and Lecturer in Creative Computing at the University of Leicester. During his PhD at Royal Birmingham Conservatoire (BCU), he explored the design of embodied interactions with audiovisual processes during music performance. He worked as Research Assistant at Goldsmiths, University of London on the realisation of EMG-interface and -driven AI as part of the ERC-funded project: BioMusic; and, as Research and Artistic Assistant at Centro Ricerche Musicali in Rome. He authored and contributed towards the development of software and interfaces for musical expression (Myo Mapper, Interga Live, TUI Metis). His research focuses on Digital Arts and Human-Centred Interaction Design.
B. Di Donato, C. Dewey, and T. Michailidis. 2020. Human-Sound Interaction: Towards a Human-Centred Sonic Interaction Design approach. In 7th International Conference on Movement and Computing (MOCO '20), July 15–17, 2020, Jersey City/ Virtual, NJ, USA.ACM, NewYork, NY, USA, 4 pages. DOI: https://doi.org/10.1145/3401956.3404233
P. Dourish. Where The Action Is: The Foundations of Embodied Interaction. MIT Press, Oct. 2001.
S. A. Grandhi, G. Joue, and I. Mittelberg. Understanding naturalness and intuitiveness in gesture production. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI, pages 821–824, Vancouver, BC, Canada, 2011.
D. A. Norman and S. W. Draper. User Centered System Design: New Perspectives on Human-computer Interaction. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1986.