AM'20 Human-Sound Interaction (HSI) Workshop

The design of interactions with sound and audio processes is a seminal activity in the creation of a performance, installation, a virtual sound environment, or interface for musical expression. The interaction design is often fixated by the interface without taking into account human factors and our diverse abilities to perceive the sound and interface affordances.
Human-Sound Interaction (HSI) look at human-centered interaction design aspects that determine the realisation and appreciation of musical works (installations, composition and performance), interfaces for sound design and musical expression, augmented instruments, sonic aspects of virtual environments, interactive audiovisual dance performances.
Participants are invited to a 3-hours interactive session (9:00 - 12:00 CEST), with hands-on design activities, and a 4-hours extended abstract presentation session (13:00 - 17:00 CEST). The workshop will take place 14 September 2020 remotely, via Zoom.
More details about the workshop submitted abstract, deadlines can be found below.

Background and Aims

Referring to principles of affordance of musical sound, and the diversity of abilities and human factors to perceive them, this workshop invites participants to take part in a collaborative activity for the exploration of Human-Sound Inter-action (HSI), defined as direct, engaging, natural and embodied interaction with sound [1]:

  • direct as the impression or a feeling about an interface capable of being described in terms of concrete actions [4]
  • engaging as fostering the “feeling of directly manipulating the objects of interest”, where “the world of interest is explicitly represented and there is no intermediary between user and world”[4]
  • natural “as being marked by spontaneity”[3]
  • embodied as an extension and incorporation of humans skills and abilities within the interaction design of a system [2]

Below is a video presentation of a recent work realised around the concept of Human-Sound Interaction [1]:

We aim to create a wider interest around HSI across different communities to explore current knowledge and create new thinking. Through their different experiences practitioners at the intersection of music, sound art, performance and Human-Centred Interaction Design, we can explore how to make the sound experience more inclusive and diverse. Findings will contribute to a body of knowledge on Human-Sound Interaction shared with the community.
We invite the scientific and artistic community working on topics HSI relates, including:

  • Human-Centred Design approaches for sonic interaction design
  • Interactions designed by/with/for diverse musicians
  • Fostering diversity in music and sound art through interaction design
  • Sonic and cross-modal music affordances
  • Embodied interaction with sound
  • Interaction design for augmented instruments
  • Adaptive frameworks for interaction design
  • Interaction design for audio/musical interfaces
  • Interaction design for interactive sound art and installations
  • Impact of interaction design in artistic practice, culture, societal and gender aspects
  • Impact of human, artistic, cultural, societal and gender factors in designing interaction with sound
  • Inclusive interaction design strategies with sound
  • Sonic interactions in Virtual (VR), Augmented (AR) and Mixed (MR) realities
  • Interactive audio-visual dance performance


This is a full-day workshop which includes a 3-hours interactive morning session, and a 4hours extended abstract presentation afternoon session. Separate zoom sessions will be made available here.

Interactive design session - Time: 9:00 - 12:00 CEST

During the morning session, you are invited to take part in the following interaction design activities: warm-up,divergent, incubation break no.1, div-interaction, incuba-tion break no.2, and convergent. For these activities you will be asked to use any tool that you are comfortable using for sketching and design ideas. For example: pen and paper, or a software such as miro, Lucidchart, Microsoft Office, etc. Cosider that you will have to share with others using your screen or camera. Make sure you are all set for this.

  • Introduction (30 minutes)
    Introduction to Human-Sound Interaction concept and the workshop schedule.
  • Warm-up (10 minutes)
    You will generate five or more extremely bad HSI ideas. You can use any design approach, random metaphors, or sketching absurd musical or sound design contexts.“To invent, you need a good imagination anda pile of junk.” Thomas Edison, quoted in [1].
  • Divergent (20 minutes)
    In the first 10 minutes, you will be asked to select ideas from the warm-up session, or ideate between 2 and 4 new HSIs, without thinking at the potential interface that fosters such interactions.
    Later, in the second 10 minutes, you will focus on the implementation of their HSI in the real world situation. Through sketching, you will have to think design an interface, installation, software mapping solution or any other artefact that can potentially satisfy the human-sound interaction you have previously designed. We will call this ``the prototype''.
  • Incubation break no.1 (20 minutes)
    In this phase, you have the chance to let your ideas sink-in and have a rest. You can do anything you wish, which does not require any particular cognitive load, but that allows them to keep their mind in a free flow state; for example, making coffee, washing up, rest, a nap, tidying up, not answering your emails or writing a paper
  • Div-interaction (30 minutes)
    At this point, you will enter a collaborative design phase. You will work in pairs with another participant. In the first 15 minutes, You will share with another participant the prototype ideas only; thus, your companion will design an HSIs based on your prototypes; and, your companion will share with you the HSI design only; thus, you will have to design an interface prototype upon the HSI of your companion. This process is then reversed, with you sharing with your companion the prototype, and your companion sharing with you the HSI.
    This phase is crucial to the whole workshop. Here themes of Human-Interaction Design in a collaborative and iterative process will generate through a natural exchange of ideas. The order within which each participant will exchange the HSI, and the prototype will allow us to observe the adaptive, transferable and transformative aspects of the design process.
  • Incubation break no.2 (20 minutes)
    As Incubation break no.1.
  • Convergent (40 minutes)
    You will receive back your original HSI and prototype designs from the participant you paired up in the Div-interaction phase, and go through a convergent design process, in which you will finalise the HSI and the prototype. More to the point, in this phase, you will have to finalise their HSI designs and implementation.
  • Wrap-up (10 minutes)
    Closing and remarks of the session.
  • After the workshop
    It would be great to have your feedback. You will be asked to fill out a questionnaire about the format of the workshop, thoughts and comments.

Extended abstracts presentations - Time: 13:00 - 17:00 CEST

In this session, accepted extended abstracts will be presented, together with their HSI prototype created in the morning. You will have ten minutes to present your work, and five minutes for questions. You are free to divide the time to present their work and what you have created during the interactive session as you wish.
Abstracts will be published under Creative Commons license via the HSI Zenodo repository.

Participation and Registration

Part 1 of the workshop is open to an unlimited number of participants. The whole Audio Mostly community is welcome to attend Part 2, here selected extended abstracts will be presented, and authors will be available to take questions from the audience.
Registration to the workshop and the Audio Mostly conference can be made here.

Important dates

  • 20 July 2020: open call
  • 26 August 2020: submission deadline
  • 31 August 2020: acceptance notification
  • 7 September 2020: camera ready and broadcast-ready presentation deadline
  • 14 September 2020: Workshop
    • 9:00 - 12:00 CEST - Part 1. Interactive Design
    • 13:00 - 17:00 CEST - Part 2. Extended abstracts
    • presentations

Extended abstracts submission instructions

A maximum of 14 extended abstract will be accepted for presentation to Part 2. Submission are anonymous in the form of an extended abstract (up to 2 pages including references) using the, using the Latex (prefered) or Word template. You will submit your abstract using the EasyChair conference management system. Submitted abstracts will be reviewed by the organisers. The submission deadline is 17 August 2020, and notification of acceptance will be sent on 28th August.


Balandino Di Donato, Informatics Department, University of Leicester

Balandino Di Donato is a Digital Artists, Researcher and Lecturer in Creative Computing at the University of Leicester. During his PhD at Royal Birmingham Conservatoire (BCU), he explored the design of embodied interactions with audiovisual processes during music performance. He worked as Research Assistant at Goldsmiths, University of London on the realisation of EMG-interface and -driven AI as part of the ERC-funded project: BioMusic; and, as Research and Artistic Assistant at Centro Ricerche Musicali in Rome. He authored and contributed towards the development of software and interfaces for musical expression (Myo Mapper, Interga Live, TUI Metis). His research focuses on Digital Arts and Human-Centred Interaction Design.

Tychonas Michailidis, DMT Lab, Birmingham City University

Tychonas Michailidis is a Lecturer in Digital Media Technology at the School of Computing and Digital Technology, Birmingham City university. He received his PhD degree in music technology and composition from Birmingham Conservatoire and a Master’s degree in Digital Arts in Performance. His research interests focus on human-computer interaction, gesture interaction, interactive systems, user experience, sensor technologies, electronic music performance and new media compositions. In particular, hi research examines how vibrotactile and haptic feedback experience can advance interactive systems as well as how it can be used as a communication tool in the arts and real-world applications.

Alessio Gabriele, CRM - Centro Ricerche Musicali of Rome

Alessio Gabriele is a Composer, Violinist, Researcher and Lecturer in Algorithmic Composition and Multimedia Systems and Technologies at Conservatoire of L’Aquila (Italy). His interest focuses on the relationships between music production, embodied music cognition, digital multimedia and ICT, aimed to closely integrate artistic language, human expression, scientific thought and technological resources. As a Composer and Senior Researcher at Centro Ricerche Musicali in Rome he authors acousmatic and instrumental works and contributes towards the design and development of Augmented Instruments, distributed Interactive and Adaptive Operas, HSI for music performance, Sound Art installations. As a violinist, Alessio performs baroque and contemporary music.

Ethical Standards

This workshop will be conducted complying with Audio Mostly's code of conduct and University of Leicester's Research Ethics Policy. These provides a framework and guidelines for conducting research with integrity and promoting good practice in all aspects of research.


  1. B. Di Donato, C. Dewey, and T. Michailidis. 2020. Human-Sound Interaction: Towards a Human-Centred Sonic Interaction Design approach. In 7th International Conference on Movement and Computing (MOCO '20), July 15–17, 2020, Jersey City/ Virtual, NJ, USA.ACM, NewYork, NY, USA, 4 pages. DOI:
  2. P. Dourish. Where The Action Is: The Foundations of Embodied Interaction. MIT Press, Oct. 2001.
  3. S. A. Grandhi, G. Joue, and I. Mittelberg. Understanding naturalness and intuitiveness in gesture production. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI, pages 821–824, Vancouver, BC, Canada, 2011.
  4. D. A. Norman and S. W. Draper. User Centered System Design: New Perspectives on Human-computer Interaction. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1986.