Language and cognition: Insights from the study of sign language iconicity

One of the central assumptions in language research has been that the relationship between a wordform and its referent is
arbitrary. However, languages do exhibit iconicity, non-arbitrary linkings between meaning and form, in which the phonetic
resources of a language are built up into an analogue of a mental image. For spoken languages, constraints of an acoustic
modality result in non-arbitrary linkings that are largely limited to onomatiopoeic words. Sign languages, on the other hand, are able
to exploit the potential for iconic expression for a wide range of basic conceptual structures because a visual/gestural modality is
better suited to such representations.
That sign languages take full advantage of iconicity, suggests that the largely arbitrary relation between form and meaning may not
be a characteristic intrinsic to language, but rather a consequence of using the acoustic modality. Thus, investigations on the
processing consequences of iconic relationships in sign languages can provide us with insight into how language can be influenced
by the specific medium (speech, sign) that is used. Moreover, investigating the processing consequences of iconic relationships in
sign language can provide us with insight into the more general issue of the relationship between language and imagery.
In principle iconicity, by virtue of creating a more direct link between linguistic and imagistic information, should facilitate
processing, and should be used whenever possible by sign language users. However, acquisition studies suggest that iconicity
does not affect language development for early native signers (although there is some evidence that iconicity plays a role in
learning a sign language as L2). Very little is known concerning language processing by adult signers, although studies of aphasic
signers clearly argue against equating iconic signs with pantomimes.
We have begun addressing the processing consequences of iconicity for adult native signers and L2 signers. In particular we have
investigated whether properties of a referent that are iconically represented in signs are more salient than other properties in
signers' mental representations of the corresponding entities. In a first series of studies we found that native BSL signers used
imagistic properties of signs in making off-line meaning similarity judgments on signs/words compared with English speakers. In
addition we showed that English speakers behaved more like the signers when asked to develop mental images for the words
before performing the same task. These findings indicate that language differences can bias users to attend more to those aspects
of the world encoded in their language than to those that are not; and that modality (spoken vs. signed) can modulate the cross-talk
between language and imagery. This finding leaves many questions open, however. Are these iconicity effects just the result of
metalinguistic strategies or are they genuine processing effects? Also, how pervasive are these effects of iconicity? In a second
study we used a picture-sign/word matching task (Is the sign/word the label for the picture?) with early ASL signers, L2 signers and
English speakers. We manipulated whether the picture rendered the iconic property of the referent salient (e.g., whiskers of a cat
well in view for the sign CAT) or not (e.g., profile of a cat sitting). We found iconicity effects in this on-line task showing that these
effects cannot be attributed solely to the use of metalinguistic strategies. In the last series of studies that I will present (experiments
using a speeded feature-verification task) we showed that effects of iconicity do not extend beyond language use (e.g., when tasks
use only pictures as materials). Thus, our studies to date show that indeed transparent links between linguistic form and meaning
have processing consequences but, importantly, these are limited specifically to language use.