NIME 2020 Human-Sound Interaction (HSI) Workshop

The design of interactions with sound and audio processes is a seminal activity in the creation of a performance, installation, a virtual sound environment, or interface for musical expression. The interaction design is often fixated by the interface without taking into account human factors and our diverse abilities to perceive the sound and interface affordances.
Human-Sound Interaction (HSI) look at human-centered interaction design aspects that determine the realisation and appreciation of musical works (installations, composition and performance), interfaces for sound design and musical expression, augmented instruments, sonic aspects of virtual environments, interactive audiovisual dance performances.

Related Publication
Balandino Di Donato, Christopher Dewey, and Tychonas Michailidis. 2020. Human-Sound Interaction: Towards a Human-Centred Sonic Interaction Design approach. In 7th International Conference on Movement and Comput-ing (MOCO '20), July 15–17, 2020, Jersey City/ Virtual, NJ, USA.ACM, NewYork, NY, USA, 4 pages. DOI: 10.1145/3401956.3404233

Background and Aims

Referring to principles of affordance of musical sound, and the diversity of abilities and human factors to perceive them, this workshop invites participants to take part in a collaborative activity for the exploration of Human-Sound Inter-action (HSI), defined as direct, engaging, natural and embodied interaction with sound [1]:

  • direct as the impression or a feeling about an interface capable of being described in terms of concrete actions [4]
  • engaging as fostering the “feeling of directly manipulating the objects of interest”, where “the world of interest is explicitly represented and there is no intermediary between user and world”[4]
  • natural “as being marked by spontaneity”[3]
  • embodied as an extension and incorporation of humans skills and abilities within the interaction design of a system [2]

Below is a video presentation of a recent work realised around the concept of Human-Sound Interaction [1]:

We aim to create a wider interest around HSI across different communities to explore current knowledge and create new thinking. Through their different experiences practitioners at the intersection of music, sound art, performance and Human-Centred Interaction Design, we can explore how to make the sound experience more inclusive and diverse. Findings will contribute to a body of knowledge on Human-Sound Interaction shared with the community.
We invite the scientific and artistic community working on topics HSI relates, including:

  • Human-Centred Design approaches for sonic interaction design
  • Interactions designed by/with/for diverse musicians
  • Fostering diversity in music and sound art through interaction design
  • Sonic and cross-modal music affordances
  • Embodied interaction with sound
  • Interaction design for augmented instruments
  • Adaptive frameworks for interaction design
  • Interaction design for audio/musical interfaces
  • Interaction design for interactive sound art and installations
  • Impact of interaction design in artistic practice, culture, societal and gender aspects
  • Impact of human, artistic, cultural, societal and gender factors in designing interaction with sound
  • Inclusive interaction design strategies with sound
  • Sonic interactions in Virtual (VR), Augmented (AR) and Mixed (MR) realities
  • Interactive audio-visual dance performance

All you need

For these activities yout will be asked to use any tool that they are comfortable using for sketching and design ideas. For example: pen and paper, or a software such as miro, Lucidchart, Microsoft Office, etc. Cosider that you will have to share wit others using your screen or camera. Make sure you are all set for this.

Workshop Programme

In this half-day workshop, you are invited to take part in the following interaction design activities: warm-up,divergent, incubation break no.1, div-interaction, incuba-tion break no.2, and convergent.

  • Introduction (30 minutes)
    Introduction to Human-Sound Interaction concept and the workshop schedule.
  • Warm-up (10 minutes)
    You will generate five or more extremely bad HSI ideas. You can use any design approach, random metaphors, or sketching absurd musical or sound design contexts. “To invent, you need a good imagination and a pile of junk.” Thomas Edison [1].
  • Divergent (20 minutes)
    In the first 10 minutes, you will be asked to select ideas from the warm-up session, or ideate between 2 and 4 new HSIs, without thinking at the potential interface that fosters such interactions.
    Later, in the second 10 minutes, you will focus on the implementation of their HSI in the real world situation. Through sketching, you will have to think design an interface, installation, software mapping solution or any other artefact that can potentially satisfy the human-sound interaction you have previously designed. We will call this ``the prototype''.
  • Incubation break no.1 (20 minutes)
    In this phase, you have the chance to let your ideas sink-in and have a rest. You can do anything you wish, which does not require any particular cognitive load, but that allows them to keep their mind in a free flow state; for example, making coffee, washing up, rest, a nap, tidying up, not answering your emails or writing a paper.
  • Div-interaction (30 minutes)
    At this point, you will enter a collaborative design phase. You will work in pairs with another participant. In the first 15 minutes, you will share with another participant the prototype ideas only, thus, your companion will design an HSIs based on your prototypes; and, your companion will share with you the HSI design only; thus, you will have to design an interface prototype upon the HSI of your companion. This process is then reversed, with you sharing with your companion the prototype, and your companion sharing with you the HSI.
    This phase is crucial to the whole workshop. Here themes of Human-Interaction Design in a collaborative and iterative process will generate through a natural exchange of ideas. The order within which each participant will exchange the HSI, and the prototype will allow us to observe the adaptive, transferable and transformative aspects of the design process.
  • Incubation break no.2 (20 minutes)
    As Incubation break no.1.
  • Convergent (40 minutes)
    You will receive back your original HSI and prototype designs from the participant you paired up in the Div-interaction phase, and go through a convergent design process, in which you will finalise the HSI and the prototype. More to the point, in this phase, you will have to finalise their HSI designs and implementation.
  • Wrap-up (10 minutes)
    Closing and remarks of the session.
  • After the workshop
    It would be great to have your feedback. You will be asked to fill out a questionnaire about the format of the workshop, thoughts and comments.


Balandino Di Donato, Informatics Department, University of Leicester

Balandino Di Donato is a Digital Artists, Researcher and Lecturer in Creative Computing at the University of Leicester. During his PhD at Royal Birmingham Conservatoire (BCU), he explored the design of embodied interactions with audiovisual processes during music performance. He worked as Research Assistant at Goldsmiths, University of London on the realisation of EMG-interface and -driven AI as part of the ERC-funded project: BioMusic; and, as Research and Artistic Assistant at Centro Ricerche Musicali in Rome. He authored and contributed towards the development of software and interfaces for musical expression (Myo Mapper, Interga Live, TUI Metis). His research focuses on Digital Arts and Human-Centred Interaction Design.

Ethical Standards

This workshop will be conducted complying with all University of Leicester's Research Ethics Policy. It provides the framework and guidelines for conducting research with integrity and promoting good practice in all aspects of research.


  1. B. Di Donato, C. Dewey, and T. Michailidis. 2020. Human-Sound Interaction: Towards a Human-Centred Sonic Interaction Design approach. In 7th International Conference on Movement and Computing (MOCO '20), July 15–17, 2020, Jersey City/ Virtual, NJ, USA.ACM, NewYork, NY, USA, 4 pages. DOI:
  2. P. Dourish. Where The Action Is: The Foundations of Embodied Interaction. MIT Press, Oct. 2001.
  3. S. A. Grandhi, G. Joue, and I. Mittelberg. Understanding naturalness and intuitiveness in gesture production. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI, pages 821–824, Vancouver, BC, Canada, 2011.
  4. D. A. Norman and S. W. Draper. User Centered System Design: New Perspectives on Human-computer Interaction. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1986.