Scholarly article on topic 'Musical Experience and Sharing Musical Haptices'

Musical Experience and Sharing Musical Haptices Academic research paper on "Art (arts, history of arts, performing arts, music)"

Share paper
{"musical experience" / haptices / "social-haptic communication" / "dual-sensory impairment" / interaction}

Abstract of research paper on Art (arts, history of arts, performing arts, music), author of scientific article — Russ Palmer, Riitta Lahtinen, Stina Ojala

Abstract In this article we introduce a systematic, holistic way of experiencing music performances through touch. This method called social-haptic communication, which consists of various touch messages called haptices is frequently used in interpreting for sensory impaired people, but can also be applied to various other disability groups, especially in connection to therapeutic approaches. Haptices are mostly used in interpreting visual events, but here we expand the scope into acoustic events, more specifically into interpretation of music, hi this article we use a concert as an example of the versatility of haptices in sharing a music experience. The various techniques used in training to use haptices with music are also explained in further detail.

Academic research paper on topic "Musical Experience and Sharing Musical Haptices"

Available online at

SciVerse ScienceDirect

Procedia - Social and Behavioral Sciences 45 (2012) 351 - 358

The 5 th Intercultural Arts Education Conference: Design Learning

Musical experience and sharing musical haptices

Russ Palmera, Riitta Lahtinenb, Stina Ojalac*

a 11 Connaught House, 112 Connaught Avenue, Frinton-on-sea, CO13 9AA, Essex, UK

Finnish DeafblindAssociation, P.O.Box 40, 00030IIRIS, Finland

c Department of Information Technology, 20014 University of Turku, Finland


In this article we introduce a systematic, holistic way of experiencing music performances through touch. This method called social-haptic communication, which consists of various touch messages called haptices is frequently used in interpreting for sensory impaired people, but can also be applied to various other disability groups, especially in connection to therapeutic approaches. Haptices are mostly used in interpreting visual events, but here we expand the scope into acoustic events, more specifically into interpretation of music. In this article we use a concert as an example of the versatility of haptices in sharing a music experience. The various techniques used in training to use haptices with music are also explained in further detail.

© 2012 Published by Elsevier Ltd. Selection cnd/or peer review under responsibility ofProfessor Heikki Ruismaki and Adjunct Profedsor InkeriRuokonen

Keywords: musical experience; haptices; social-haptic communication; dual-sensory impairment; interaction

1. Introduction

Music consists of different textural elements such as rhythm, pitch, and melody (Roederer, 1973 among others). These elements can be illustrated through touch during an interactive process between two people for example if interpreters are interpreting music for sensory impaired people during a live performance or when listening to pre-recorded music. Traditionally, haptices have been used to interpret something that is visual in nature but here in this article we expand the use of haptices into interpreting

* Corresponding author. Tel.: +358-2-333-7495 E-mail address:

1877-0428 © 2012 Published by Elsevier Ltd. Selection and/or peer review under responsibility of Professor Heikki Ruismaki and Adjunct Professor Inkeri Ruokonen


something that is invisible and acoustic. In other words, we show a way of enhancing the perception of music through using haptices. Furthermore, haptices are still mostly used for interpreting visual events.

One aim of this research is to expand the social-haptic communication (basic haptices and haptemes) (Lahtinen, 2008) to include music. The other aim is to investigate how to interpret acoustic music onto the body using these methods and thus enhance the enjoyment of music through touch. This project is a pilot study in order to evaluate how compatible and flexible the haptices are in interpreting music. This being a pilot study we experimented several different approaches to the study question and filtered out those which were not compatible or applicable.

1.1. Getting to know haptices in relation to music — why and how

The use of haptices enables the hearing impaired person to be involved in a hearing culture. Without it, the person would be just sitting there without the possibility to be involved equally. For example if there is a music performance in the middle of speeches in an official event, haptices enable the person to know what is going on so that s/he is involved without interruptions because the lack of linguistic content, which is conveyed further by the interpreter. Haptices provide a method in describing something which does not have a verbal content. The music can be felt on the skin as vibrations and the haptices give "names" and connotations to the vibrations and the information about the sources of the vibrations. For example the drum sound creates vibration that can easily be felt on the skin by everyone, deafblind or hearing and sighted (Goldstein, 1989). It is usually that the hearing and sighted are not aware of the vibrations as much.

When listening to music from a CD there is not an orchestra that you point at, but at the beginning it might be useful to show where the music player is situated, so that the person listening to music can find the most suitable and convenient place for listening and feeling the vibrations. In this situation, it is usually possible for the interpreter to listen to the music beforehand. This enables preplanning for the listening event. When practicing listening to music you can start with works that have a clear texture (for example Sky (1979, 1992), Barry (1988) and Vangelis (1992) among others). In that way the practicing session is easier to construct. In later stages the music can be more versatile in instruments and may also include song.

The use of haptices varies from one person to another. This might be due to various reasons: the sensitivity of touch, the hearing status, the visual status, etc. The physiology of skin and its ability to feel movements in different parts of the body enables the use of touch in communication and information structure sharing. The touch messages can also be felt through layer of clothes and touch messages can be felt regardless of surrounding noise or diminished lighting, which might prevent the use of other communication methods, such as speech and signing. (Cahusac, 2002 on representation of sensitivity in brain regions; Hellstrom & Revonsuo, 1996 on information structure and receptor cells; Lahtinen, 2008 on information structure and communication).

Musical haptices can be used when sitting down (Figure 1), standing or lying down (Figure 2). The only decisive factor on perceiving and producing haptices is the ergonomic positioning of both the receiver and interpreter. There should also not be any concerns for losing balance. The most usual position for haptices is sitting side by side, since this is the position possible in a live concert situation. In this position the body parts used in receiving and producing haptices are hands, arms, and legs. The haptices are made using hand-to-hand and leg-to-leg contact. Usually the most natural place for the rhythm is the leg. The various uses of hands in relation to different parts of music are described further later in the article. The situation where the person is lying down could be used also in a children's concert or with a person with mobility impairment.

Fig. 1. Musical haptices when sitting side by side (Reprinted with permission from Lahtinen & Palmer, 2005)

Fig. 2. Musical haptices when lying on the floor listening to pre-recorded music (Reprinted with permission from Lahtinen & Palmer, 2005)

1.2. Going to a concert using haptices — an example of versatility

A music experience in a concert does not only consist of the performed songs or instrumental music but it is a holistic cultural experience. It consists of the "feeling" of the concert hall or the venue, the size of the orchestra or group as well as the applause from the audience (Figure 3), occasional standing ovations, and all the actions in the audience. The scheme of events within intermission is also an important part of the concert-going experience. In the following figure the hands depict the rhythm, duration and intensity of the audience and the head movement to the shoulder depict the manner and feeling of the hug received by the soloist at the end of the concert.

Part of the concert experience is the position of the seat in the concert hall in relation to the place of the orchestra (Campbell & Greated, 1987, 43). It affects the reverbation pattern from the orchestra (Beranek, 1979). When you are in your seat with the person using haptices there are things that are

always described before the concert programme starts. These are: the position of the seat with regards to the stage, the orchestra, places of different instruments, number of instruments, and what selection of musical haptices will be used in this particular concert. These methods will enable the user to have a more realistic view of where and what there is in a performance. The different instruments within the score can be defined using Deafblind Manual Alphabet or writing block letters on the palm. A body map of the orchestra can be drawn on the back, so that later in the concert the leading melodies can be pointed at the map on the back. It also helps the person to orientate for example towards the soloists.

Fig. 3. Using haptices to describe the way the soloist is being thanked on stage (Reprinted with permission from Lahtinen, 2008)

Musical haptices are used in sharing musical experiences when the auditive listening does not provide enough information. In this way haptices can support the functional hearing as well as give additional information or provide the only way of perceiving acoustic information apart from the feeling of vibrations. If and when the person using haptices is able to perceive lower sounds using the functional hearing, the supporting element based on haptices can be representation of the instrument in question or haptices can provide the "missing acoustic element", for example an instrument with higher sounds, such as a flute or a violin. If the person has a visual impairment in addition to the hearing impairment, there are no visual cues from the orchestra or the instruments. In this case haptices can be used to represent the places of the instruments playing in a particular moment. In using haptices the interpreter has to choose what to convey, since there are so many elements that it is not possible to convey all the information available.

If there is free seating order in the concert and the chairs can be moved around the music can be described on the back. Other alternative is standing face-to-face describing hand-to-hand. Music can also be described using dancing. This way usually the element being described is the rhythm by holistic body movements (cf. dancing waltz) and the aesthetics of the texture of music.

2. The application of different haptices

Haptices as defined by Lahtinen (2008) is a system of touch-based methods that allow two or more people to interact together to improve the quality of information during a spontaneous activity. During the interaction between these two people they are able to gain a holistic experience of the performance using

haptices in real time. This enables one to have a better experience of the music when it's being played or listened to in a spontaneous yet structured manner. The touch messages can be used in conveying multilayered information within the situation, in this case within a concert.

Different musical elements, such as rhythm, melody and dynamics, can be portrayed through touch elements, i.e. haptices. Haptices provide a method of encoding acoustic information, in this case music, through touch. They can be used to enhance the receiver's hearing. In other words they bring acoustic information accessible to people whose hearing is impaired.

From a user perspective this enables them to receive the information in higher standard and quality compared to that of language-based interpretation. It may also be applicable to a wider audience of disability groups, i.e. deafblind, visually-impaired, hard-of-hearing and profound learning disability groups. It can also include other subgroups within the therapy areas, such as the terminally ill patients. This can be used when the verbal communication becomes more difficult with family members or professionals.

2.1. Rhythm

Rhythmical elements can be illustrated by using a sequence of different taps in different tempos and/or pauses, which are produced onto a person's arm or hand This enables the receiver to follow the flow of the music performance i.e. group or concert performance. From a hearing or a sensory impaired point of view this is a very important element to follow, which is not always easy to hear with using different hearing aids and devices. The haptical elements (haptemes, see Lahtinen, 2008) provide an enhanced sensory perception of rhythm. If there is a drummer in the live performance group his actions would be portrayed using the taps onto the person's hand.

Some people can interpret music based solely on visual elements as this really depends on the person's position in relation to the live performance. So you might be able to see what the performers are doing but at the same time you might not be able to feel the rhythm from the music. If you are in a big stadium, you might get the visual clues for drummer's actions in advance of the physical sensations of feeling the beats. That's because of the distance involved between you and the performer. Also, some people may not be so sensitive for auditory information within music, i.e. non-musicians. They might then be able to pick up the visual clues to pass on to the receiver. But if you are close enough to be able to feel the rhythm of the performance you can nevertheless enhance it by using haptices.

2.2. Pitch

Pitch elements can be illustrated in several different ways using the arm as the ground for the interpretation so that the higher and lower notes occupy different areas of the arm (Lahtinen, 2008, 142; Lahtinen & Palmer, 2005). The basic concept of illustrating pitch elements i.e. low, middle and high tones, onto a person's arm through touch are defined by Lahtinen (2008), table 23. For example the receiver's arm can be used in three sections: hand and wrist for low tones (double bass, drums), lower arm for middle tones (guitar, piano) and upper arm for high tones (violin, flute). There are also other ways depending on the particular instruments used in the concert.

2.3. Melody

Melodic elements produced by different instruments either in a group or in an orchestra can be illustrated by a defined set of handshapes that denote different instruments. For example string instruments like violin or cello are illustrated by a handshape portraying the bowing action of the players.

The general overview of the performance is shown by the melody moving up or down, which also then includes the pitch elements and at the same time shows if an instrument is being plucked or bowed if it is a string instrument (on the acoustic differences between plucking and bowing a string instrument, please see Backus, 1977, 194-196). If a trumpet is playing this is portrayed with opening and closing of fist onto the arm. Both the melody and pitch are incorporated together (Backus, 1977, 55-62). The receiver's arm can also depict the instrument itself, that is, piano can be played on the receiver's arm, interpreting the movements on the keyboard. In this way the rhythm, intensity and melody of the piano score are incorporated in one movement.

2.4. Intensity

Differences in the intensity of music are illustrated by the changes of hand pressure onto the receiver's arm (Lahtinen 2008, 142). For example in a dr^atic piece whore there's a build-up of instrumentation this creates greater intensity in positioning of the hands. You can also illustrate greater intensity by using not one but two hands moving together, for example in opening and closing of hand movements. When creating a musical story as defined in Lahtinen and Palmer (2005), it is possible to portray pictorial elements as well, such as the sea lapping on the shore with opening and closing hand movement on to the receiver's arm. The rough sea could then be portrayed with either using two hands moving together or increasing the hand pressure of the single hand onto the receiver's arm.

2.5. Texture

As stated above, music comprises of different elements. These elements form the overall music texture. The texture is a holistic experience in acoustics, and it can be described onto the hand using different movements in parallel and in succession. When there are different instruments playing in the music, the introduction of a new element in the texture is done in changing the musical haptices on the hand. When interpreting music performance or pre-recorded music using haptices you have to pick the elements you are including in your interpretation because you cannot take it all in. In the holistic view of interpreting music the interpreter may be able to select the most meaningful, most prominent elements of the music that are meaningful for the person you are working with. For example for a profoundly deaf and blind person (no hearing, no sight), their perception of music might be more rhythm-based in comparison to someone who has partial hearing loss and partial sight loss (hearing aids or cochlear implants). In other words, the particular elements on the music texture for interpretation affect the haptices you are using. The selection of different haptices in the interpretation of a performance or a concert onto the body depends on what elements you illustrate.

3. Material and methods

When sitting or standing, rhythm can be expressed onto different body areas, such as hands, arms or shoulders. It is possible to express various elements of music through different areas of the body at the same time. This enhances a person's holistic experience during a live performance. Additional elements can be portrayed using musical haptices, these include selection of instruments, levels of tone, rhythm, melody, pauses and dynamics. Sometimes the receiver can represent a music instrument which is being played. If there is a hand-to-hand contact, as present in a tactile sign language discussion (Mesch, 1998 among others), the tapping rhythm can be produced directly onto the hand. In certain cases it is possible the receiver can copy individual instrument, such as a cymbal interacting with the interpreter. Different pitch elements between the frequencies of the sounds can be expressed for instance onto a person's arm,

which can be broken down to different areas: for example, the low tones onto the lower arm, middle tones onto the middle part and high tones onto the higher part respectively. The dynamics can be produced by varying degrees of hand pressure onto the arm or body. During a musical performance if there is a pause, break or even silence, this can be shown through withdrawal of contact or stopping the movement. (Lahtinen, 2008, 143).

The methods outlined in this article form the basis to a greater music appreciation and experience using touch based approaches which can be expanded further to a more sophisticated level depending upon the person's musical experience or knowledge. These approaches may be more applicable for those people who have a sensory impairment either with the hearing or sight. However, for those people who may be fortunate to have a cochlear implant with a combined dual-sensory loss may reach to a new level not experienced before.

For example Palmer explains below:

"As a person who was born as a severely hearing impaired person, who has never heard normal

acoustic sounds and using hearing aids from the age offour, the cochlear implant combined with musical haptices has enriched my musical appreciation to a higher level. In 2004 I was extremely sceptical if music would ever compete to my lifelong experience in using hearing aids. With the changing circumstances of my vision worsening over the years due to Retinitis Pigmentosa (RP) and Usher Syndrome, I rely on musical haptices to provide me with the auditive and visual clues I am missing through my sensory losses as described in Environmental Description book (Lahtinen, Palmer & Lahtinen 2010).

Furthermore, with the addition of receiving a second implant in June 2011, my musical experience has been expanded to a more stereophonic level, equal to that of a normal sighted and hearing person. Sometimes the experience has been extremely emotional, knowing that my musical quality and experience is being fulfilled compared to before using two cochlear implants." Russ Palmer (personal comment 18.12.2011).

4. Conclusions

In examining the evidence over the past three years there is a greater role for musical haptices which expands Lahtinen's (2008) philosophical context using social-haptic communication with interpreters and professionals working with deafblind people. These approaches can be adapted to other client groups particularly for visually impaired people, profound learning disabilities and it has therapeutic implications for the terminally ill and the elderly especially where other communication methods may not function so well.

Further research on musical haptices and systematic analysis for other stages of using musical haptices is needed to consolidate the artistic aspects of haptices in social-haptic communication. These results have to be recorded and collated amongst other client groups as well focusing on the different human interactive processes. This illustrates how flexible and versatile the whole social-haptic communication is as it can be applied to a wide range of different client groups in professional and therapeutic areas. In this way the invisible elements become more visible in this human context.

Russ PalmCT describes: "My recent experiences of attending some concerts at the Royal Albert Hall, Festival Hall in London and some concert halls in Finland have illustrated to me the value of combining acoustics, musical haptices and my perception of musical sounds through my cochlear implants to a new more qualitive experience which I was not expecting. For example when listening Holst "The Planets suite" I was able to both hear and feel the musical elements of the orchestra across a broad range of frequencies. Through my cochlear implants I picked up the higher frequencies and simultaneously the musical haptices enhanced the rhythmic and texture elements at the same time. One appreciates that this

is a single person's experience but through further research it may be possible to get a wider perspective of other users." Russ Palmer (personal comment 22.2.2012).


A. Literal references

Backus, J. (1977). The Acoustical Foundations of Music. W.W. Norton & Company.

Beranek, L.L. (1979). Music, acoustics & architecture. Krieger Publications.

Cahusac, P. (2002). The perception of touch. In D. Roberts (Ed.), Signals and Perception: The fundamentals of human sensation. Palgrave Macmillan.

Campbell, M. & Greated, C. (1987). The Musician's Guide to Acoustics. J.M. Dent & Sons Ltd.

Goldstein, E.B. (1989). Sensation and perception. Brooks/Cole Publishing Company, USA.

Lahtinen, R. (2008). Haptices and haptemes. A case study of developmental process in social-haptic communication of acquired deafblind people. Academic dissertation, University of Helsinki. Tampere: Cityoffset.

Lahtinen, R. & Palmer, R. (2005). Body Story. Creating Musical Images through Touch (CMIT). Tampere: Cityoffset.

Lahtinen, R., Palmer, R., & Lahtinen, M. (2010). Environmental Description for visually and dual sensory impaired people. Art-Print Oy.

Mesch, J. (1998). Teckensprak i taktil form — turtagning och fragor i dovblindas samtal pa teckensprak. Academic dissertation, University of Stockholm.

Palmer, Russ (2011). Personal comment on 18.12.2011.

Palmer, Russ (2012). Personal comment on 22.2.2012.

Roederer, J.G. (1973). Introduction to the Physics and Psychophysics of Music. Heidelberg Science Library, vol. 16. Springer-Verlag.

B. Music references

Barry, John (1988): Film Music of John Barry. Music CD. Sky (1979): Sky. Music CD. Sky (1992): Sky 2. Music CD.

Vangelis (1992): 1492 Conquest of Paradise. Music CD.