How Our Hands Help Us Speak

Having a conversation? Use your hands!

Source: Images by cottonbro/Pexels, fauxels/Pexels, Jessica Da Rosa/Unsplash

Successful communication involves much more than just talking. When having a face-to-face conversation, we use our body posture, facial expressions, and hands to get our message across. Still, some of us might think of these non-verbal signals as 'icing on the cake': the main communicative message is carried by the spoken words, the rest is 'decoration'. However, a recent study that I conducted with David Peeters shows that even the simplest flicks of the hands can change what speech sounds you hear.

Beat gestures on stressed syllables

One of the most frequently occurring hand movements we make while talking are beat gestures—simple up and down movements that do not carry much meaning in and of themselves, but only accompany the spoken message to highlight more prominent words. Still, research indicates that these beat gestures tend to fall on the stressed syllable of a highlighted word. So in a sentence like "I ob-JECT", you'd make a beat-like movement with your hand on the stressed syllable "-ject", but in a sentence like, "No, it's an OB-ject!", you'd make the same beat gesture on the stressed syllable "ob-". Now, interestingly, your audience also knows this and can actually use a beat gesture as an indication of a stressed syllable.

Hand gestures can make you hear a different word

In the study, participants saw videos of a talker producing a sentence ending in a word that could either have stress on the first syllable (e.g., "OBject") or the second syllable (e.g., "obJECT"). Sometimes the talker produced a beat gesture on the first syllable, and sometimes on the second. The participants' task was to indicate what they heard, "OBject" or "obJECT"? Results showed that participants were more likely to perceive stress on the syllable that was accompanied by a beat gesture. That is, if the talker produced a beat gesture on the first syllable, participants were more likely to hear "OBject" (not "obJECT"). Interestingly, this effect was even found when the audio mismatched the video: even when the talker quite clearly said "obJECT", participants were still more likely to respond hearing "OBject" if the beat gesture fell on the first syllable.

Try the effect yourself. This clip shows a Dutch talker producing the sentence "Nu zeg ik het woord [pla-to]" / "Now say I the word [pla-to]". The first part has the talker produce a beat gesture on the first syllable of [pla-to]. The second part has the exact same audio as the first part, but this time the beat gesture falls on the second syllable of [pla-to]. What do you hear, "PLAto" or "plaTO"?

Hand gestures can change what vowels you hear

But can beat gestures even change how you hear individual speech sounds? To answer this question, we used a vowel distinction in Dutch between short and long "a". In fact, they artificially created a vowel that lay kind of 'in between' the short and long versions of this vowel. As such, this artificial 'medium-length vowel' could either be perceived as a stressed longish version of short "a", or as an unstressed shortish version of long "aa". We presented a new set of participants with videos of the same talker as before, but this time he produced this 'medium-length vowel' in the first syllable of nonsense words (e.g., "b[a/aa]g-pif"). Participants were asked to indicate what they heard: "bagpif" with short "a", or "baagpif" with long "aa". Results showed that when the talker produced a beat gesture on the first syllable, participants perceived the medium-length vowel as a stressed longish version of short "a", reporting hearing "bagpif". But the same audio combined with a beat gesture on the second syllable led participants to perceive the medium-length vowel as an unstressed shortish version of long "aa", hearing "baagpif". Therefore, beat gestures can even influence what vowels you hear.

Try the effect yourself here. NOTE: this will work best with native speakers of Dutch. The clip shows a Dutch talker producing the sentence "Nu zeg ik het woord [b???g-pif]" / "Now say I the word [b???g-pif]". This last word contained a vowel midway between short "a" and long "aa" in the first syllable ("bagpif" vs. "baagpif"). The first part of the clip has the talker produce a beat gesture on the first syllable of [b???g-pif]. The second part has the exact same audio as the first part, but this time the beat gesture falls on the second syllable of [b???g-pif]. Native speakers of Dutch are more likely to hear "bagpif" with short "a in the first part, but "baagpif" with long "aa" in the second part. What do you hear, "bagpif" or "baagpif"?

Why is this important?

So we see that even the simplest flicks of the hands can influence what we hear. This is important for how we speak: Using simple beat gestures can help you get your message across, helping your audience even at the most basic level of decoding *what* you say. This may be especially important in situations where your spoken words are hard to hear, for instance in large crowds and noisy conditions. Also in the present day, with masks obstructing lip-reading, glass screens making it hard to hear what others say, or Zoom meetings with only (parts of) faces being visible, the use of simple but salient hand movements can considerably aid communication. So wash your hands, but use them too.