交流手势认知理论
|
摘要
手势是交流互动中一种重要的非语言媒介,手势不仅可以辅助语言交流而且具有独立的交流性;作为和语言共同发生的非语言媒介,手势交流有助于降低交流认知负荷。文章重点归纳和述评了基于手势和语言表达关系的交流手势理论、交流手势激活理论、交流手势的认知节省理论。未来研究需要进一步考虑交流手势实验研究情境自然性和控制严格性间的平衡,交流手势和其他非语言因素间的关系,交流手势认知研究的现实意义。
Gesture is an important nonverbal medium of communication. Gesture can not only assist language communication but also has independent communication attribute. As a nonverbal medium co occurring with language media, gesture communication can help to reduce communication cognitive load.This paper mainly summarizes and reviews the theory of communicative gesture based on the expression relationship between gesture and language, the theory of communicative gesture activation, and the cognitive saving theory of communicative gesture. The future research should explore the balance between naturalness and control rigor of the experimental situation, the relationship between gesture and other nonverbal factors, and the practical significance of gesture cognition research.
Gesture is an important nonverbal medium of communication. Gesture can not only assist language communication but also has independent communication attribute. As a nonverbal medium co occurring with language media, gesture communication can help to reduce communication cognitive load.This paper mainly summarizes and reviews the theory of communicative gesture based on the expression relationship between gesture and language, the theory of communicative gesture activation, and the cognitive saving theory of communicative gesture. The future research should explore the balance between naturalness and control rigor of the experimental situation, the relationship between gesture and other nonverbal factors, and the practical significance of gesture cognition research.
引文
张恒超.(2013).参照性交流中的“听者设计”.心理发展与教育,29(5),552-560.
张恒超.(2017).共享因素对参照性交流双方学习的影响.心理学报,49(2),197-205.
张恒超.(2018).交流语言认知特征.心理科学进展,26(2),270-282.
Alibali,M.W.,Kita,S.,&Young,A.J.(2000).Gesture and the process of speech production:We think,therefore,we gesture.Language and Cognitive Processes,15(6),593-613.
Alibali,M.W.,Spencer,R.C.,Knox,L.,&Kita,S.(2011).Spontaneous gestures influence strategy choices in problem solving.Psychological Science,22(9),1138-1144.
Arbib,M.A.,Gasser,B.,&Barrès,V.(2014).Language is handy but is it embodied?.Neuropsychologia,55,57-70.
Arnold,J.E.,Kahn,J.M.,&Pancani,G.C.(2012).Audience design affects acoustic reduction via production facilitation.Psychonomic Bulletin&Review,19(3),505-512.
Beaudoin-Ryan,L.,&Goldin-Meadow,S.(2014).Teaching moral reasoning through gesture.Developmental Science,17(6),984-990.
Berezan,O.,Yoo,M.,&Christodoulidou,N.(2016).The impact of communication channels on communication style and information quality for hotel loyalty programs.Journal of Hospitality and Tourism Technology,7(1),100-116.
Bock,K.,&Cutting,J.C.(1992).Regulating mental energy:Performance units in language production.Journal of Memory and Language,31(1),99-127.
Brentari,D.,&Goldin-Meadow,S.(2017).Language Emergence.Annual Review of Linguistics,3(1),363-388.
Broaders,S.C.,&Goldin-Meadow,S.(2010).Truth is at hand:How gesture adds information during investigative interviews.Psychological Science,21(5),623-628.
Broaders,S.C.,Cook,S.W.,Mitchell,Z.,&Goldin-Meadow,S.(2007).Making children gesture brings out implicit knowledge and leads to learning.Journal of Experimental Psychology:General,136(4),539-550
Brooks,N.,&Goldin-Meadow,S.(2015).Moving to learn:How guiding the hands can set the stage for learning.Cognitive Science A Multidisciplinary Journal,40(7),1831-1849.
Cook,S.W.,Duffy,R.G.,&Fenn,K.M.(2013).Consolidation and transfer of learning after observing hand gesture.Child development,84(6),1863-1871.
de Marco,D.,de Stefani,E.,&Gentilucci,M.(2015).Gesture and word analysis:The same or different processes?.NeuroImage,117,375-385.
Drijvers,L.,&?zyürek,A.(2017).Visual context enhanced:The joint contribution of iconic gestures and visible speech to degraded speech comprehension.Journal of Speech,Language,and Hearing Research,60(1),212-222.
Duran,N.D.,&Dale,R.(2014).Perspective-taking in dialogue as self-organization under social constraints.New Ideas in Psychology,32,131-146.
Edelman,S.(2017).Language and other complex behaviors:Unifying characteristics,computational models,neural mechanisms.Language Sciences,62,91-123.
Glenberg,A.M.,&Gallese,V.(2012).Action-based language:A theory of language acquisition,comprehension,and production.Cortex,48(7),905-922.
Goldin-Meadow,S.(2015).From action to abstraction:Gesture as a mechanism of change.Developmental Review,38,167-184.
Goldin-Meadow,S.,Cook,S.W.,&Mitchell,Z.A.(2009).Gesturing gives children new ideas about math.Psychological Science,20(3),267-272.
Goldin-Meadow,S.,Nusbaum,H.,Kelly,S.D.,&Wagner,S.M.(2001).Explaining math:Gesturing lightens the load.Psychological Science,12(6),516-522.
Graziano,M.,&Gullberg,M.(2013).Gesture production and speech fluency in competent speakers and language learners.In Tilburg Gesture Research Meeting(TiGeR)2013.Tilburg University.
Grenoble,L.A.,Martinovic′,M.,&Baglini,R.(2014).Verbal gestures in Wolof.In R.Kramer,L.Zsiga,&O.Boyer(Eds.),Selected proceedings of the 44th annual conference on African linguistics.Somerville,MA:Cascadilla Press.
Hadar,U.,&Butterworth,B.(1997).Iconic gestures,imagery,and word retrieval in speech.Semiotica,115(1-2),147-172.
Hostetter,A.B.(2011).When do gestures communicate?Ameta-analysis.Psychological bulletin,137(2),297-315.
Hostetter,A.B.(2014).Action attenuates the effect of visibility on gesture rates.Cognitive Science,38(7),1468-1481.
Hostetter,A.B.,&Alibali,M.W.(2010).Language,gesture,action!A test of the Gesture as Simulated Action framework.Journal of Memory and Language,63(2),245-257.
Hostetter,A.B.,Alibali,M.W.,&Kita,S.(2007).I see it in my hands’eye:Representational gestures reflect conceptual demands.Language and Cognitive Processes,22(3),313-336.
Hu,F.T.,Ginns,P.,&Bobis,J.(2015).Getting the point:Tracing worked examples enhances learning.Learning&Instruction,35,85-93.
Iverson,J.M.,&Goldin-Meadow,S.(1998).Why people gesture when they speak.Nature,396(6708),228.
Kang,S.,Tversky,B.,&Black,J.B.(2015).Coordinating gesture,word,and diagram:Explanations for experts and novices.Spatial Cognition&Computation,15(1),1-26.
Kelly,S.D.,&Church,R.B.(1998).A comparison between children's and adults'ability to detect conceptual information conveyed through representational gestures.Child Development,69(1),85-93.
Kita,S.,&?zyürek,A.(2003).What does cross-linguistic variation in semantic coordination of speech and gesture reveal?Evidence for an interface representation of spatial thinking and speaking.Journal of Memory and language,48(1),16-32.
Koppensteiner,M.,Stephan,P.,&J?schke,J.P.M.(2016).Moving speeches:Dominance,trustworthiness and competence in body motion.Personality and Individual Differences,94,101-106.
Krauss,R.M.,Chen,Y.S.,&Gottesman,R.(2000).Lexical gestures and lexical access:A process model.In D.McNeill(Eds.)Language and gesture(pp.261-283).Cambridge,UK:Cambridge University Press.
Krauss,R.M.,&Weinheimer,S.(1964).Changes in reference phrases as a function of frequency of usage in social interaction:A preliminary study.Psychonomic Science,1,113-114.
Louwerse,M.M.(2011).Symbol interdependency in symbolic and embodied cognition.Topics in Cognitive Science,3(2),273-302.
Matovic,D.,Koch,A.S.,&Forgas,J.P.(2014).Can negative mood improve language understanding?Affective influences on the ability to detect ambiguous communication.Journal of Experimental Social Psychology,52,44-49.
McNeill,D.,&Duncan,S.(2000).Growth points in thinking-for-speaking.In D.McNeill.(Eds)Language and gesture(pp.141-161).Cambridge,UK:Cambridge University Press.
Morsella,E.,&Krauss,R.M.(2004).The role of gestures in spatial working memory and speech.The American Journal of Psychology,117(3),411-424.
Nicoladis,E.,Pika,S.,Yin,H.U.I.,&Marentette,P.(2007).Gesture use in story recall by Chinese-English bilinguals.Applied Psycholinguistics,28(4),721-735.
Novack,M.A.,Congdon,E.L.,Hemanilopez,N.,&Goldin-Meadow,S.(2014).From action to abstraction:Using the hands to learn math.Psychological Science,25(4),903-910.
Novack,M.A.,&Goldin-Meadow,S.(2016).Gesture as representational action:A paper about function.Psychonomic Psychonomic Bulletin&Review,24(3),652-665.
Novack,M.,&Goldin-Meadow,S.(2015).Learning from gesture:How our hands change our minds.Educational Psychology Review,27(3),405-412.
Novack,M.A.,Goldin-Meadow,S.,&Woodward,A.L.(2015).Learning from gesture:How early does it happen?.Cognition,142,138-147.
Perniss,P.,?zyürek,A.,&Morgan,G.(2015).The Influence of the visual modality on language structure and conventionalization:Insights from sign language and gesture.Topics in Cognitive Science,7(1),2-11.
Pine,K.J.,Bird,H.,&Kirk,E.(2007).The effects of prohibiting gestures on children's lexical retrieval ability.Developmental Science,10(6),747-754.
Ping,R.M.,&Goldin-Meadow,S.(2008).Hands in the air:Using ungrounded iconic gestures to teach children conservation of quantity.Developmental Psychology,44(5),1277-1287.
Ping,R.M.,Goldin-Meadow,S.,&Beilock,S.L.(2014).Understanding gesture:Is the listener’s motor system involved?.Journal of Experimental Psychology:General,143(1),195-204.
Post,L.S.,Gog,T.V.,Paas,F.,&Zwaan,R.A.(2013).Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning.Computers in Human Behavior,29(4),1450-1455.
Rauscher,F.H.,Krauss,R.M.,&Chen,Y.S.(1996).Gesture,speech,and lexical access:The role of lexical movements in speech production.Psychological Science,7(4),226-231.
Sacchi,S.,Riva,P.,&Aceto,A.(2016).Myopic about climate change:Cognitive style,psychological distance,and environmentalism.Journal of Experimental Social Psychology,65,68-73.
Sassenberg,U.,&van Der Meer,E.(2010).Do we really gesturemorewhen it is more difficult?Cognitive Science,34(4),643-664.
Sharma,V.,&Droch,B.(2015).Gesture-controlled user interfaces.Journal of Information Sciences and Computing Technologies,2(1),133-135.
Trofatter,C.,Kontra,C.,Beilock,S.,&Goldin-Meadow,S.(2015).Gesturing has a larger impact on problem-solving than action,even when action is accompanied by words.Language,Cognition and Neuroscience,30(3),251-260.
Weinberg,A.,Fukawa-Conolly,T.,&Wiesner,E.(2015).Characterizing instructor gestures in a lecture in a proof-based mathematics class.Educational Studies in Mathematics,90(3),233-258.
张恒超.(2017).共享因素对参照性交流双方学习的影响.心理学报,49(2),197-205.
张恒超.(2018).交流语言认知特征.心理科学进展,26(2),270-282.
Alibali,M.W.,Kita,S.,&Young,A.J.(2000).Gesture and the process of speech production:We think,therefore,we gesture.Language and Cognitive Processes,15(6),593-613.
Alibali,M.W.,Spencer,R.C.,Knox,L.,&Kita,S.(2011).Spontaneous gestures influence strategy choices in problem solving.Psychological Science,22(9),1138-1144.
Arbib,M.A.,Gasser,B.,&Barrès,V.(2014).Language is handy but is it embodied?.Neuropsychologia,55,57-70.
Arnold,J.E.,Kahn,J.M.,&Pancani,G.C.(2012).Audience design affects acoustic reduction via production facilitation.Psychonomic Bulletin&Review,19(3),505-512.
Beaudoin-Ryan,L.,&Goldin-Meadow,S.(2014).Teaching moral reasoning through gesture.Developmental Science,17(6),984-990.
Berezan,O.,Yoo,M.,&Christodoulidou,N.(2016).The impact of communication channels on communication style and information quality for hotel loyalty programs.Journal of Hospitality and Tourism Technology,7(1),100-116.
Bock,K.,&Cutting,J.C.(1992).Regulating mental energy:Performance units in language production.Journal of Memory and Language,31(1),99-127.
Brentari,D.,&Goldin-Meadow,S.(2017).Language Emergence.Annual Review of Linguistics,3(1),363-388.
Broaders,S.C.,&Goldin-Meadow,S.(2010).Truth is at hand:How gesture adds information during investigative interviews.Psychological Science,21(5),623-628.
Broaders,S.C.,Cook,S.W.,Mitchell,Z.,&Goldin-Meadow,S.(2007).Making children gesture brings out implicit knowledge and leads to learning.Journal of Experimental Psychology:General,136(4),539-550
Brooks,N.,&Goldin-Meadow,S.(2015).Moving to learn:How guiding the hands can set the stage for learning.Cognitive Science A Multidisciplinary Journal,40(7),1831-1849.
Cook,S.W.,Duffy,R.G.,&Fenn,K.M.(2013).Consolidation and transfer of learning after observing hand gesture.Child development,84(6),1863-1871.
de Marco,D.,de Stefani,E.,&Gentilucci,M.(2015).Gesture and word analysis:The same or different processes?.NeuroImage,117,375-385.
Drijvers,L.,&?zyürek,A.(2017).Visual context enhanced:The joint contribution of iconic gestures and visible speech to degraded speech comprehension.Journal of Speech,Language,and Hearing Research,60(1),212-222.
Duran,N.D.,&Dale,R.(2014).Perspective-taking in dialogue as self-organization under social constraints.New Ideas in Psychology,32,131-146.
Edelman,S.(2017).Language and other complex behaviors:Unifying characteristics,computational models,neural mechanisms.Language Sciences,62,91-123.
Glenberg,A.M.,&Gallese,V.(2012).Action-based language:A theory of language acquisition,comprehension,and production.Cortex,48(7),905-922.
Goldin-Meadow,S.(2015).From action to abstraction:Gesture as a mechanism of change.Developmental Review,38,167-184.
Goldin-Meadow,S.,Cook,S.W.,&Mitchell,Z.A.(2009).Gesturing gives children new ideas about math.Psychological Science,20(3),267-272.
Goldin-Meadow,S.,Nusbaum,H.,Kelly,S.D.,&Wagner,S.M.(2001).Explaining math:Gesturing lightens the load.Psychological Science,12(6),516-522.
Graziano,M.,&Gullberg,M.(2013).Gesture production and speech fluency in competent speakers and language learners.In Tilburg Gesture Research Meeting(TiGeR)2013.Tilburg University.
Grenoble,L.A.,Martinovic′,M.,&Baglini,R.(2014).Verbal gestures in Wolof.In R.Kramer,L.Zsiga,&O.Boyer(Eds.),Selected proceedings of the 44th annual conference on African linguistics.Somerville,MA:Cascadilla Press.
Hadar,U.,&Butterworth,B.(1997).Iconic gestures,imagery,and word retrieval in speech.Semiotica,115(1-2),147-172.
Hostetter,A.B.(2011).When do gestures communicate?Ameta-analysis.Psychological bulletin,137(2),297-315.
Hostetter,A.B.(2014).Action attenuates the effect of visibility on gesture rates.Cognitive Science,38(7),1468-1481.
Hostetter,A.B.,&Alibali,M.W.(2010).Language,gesture,action!A test of the Gesture as Simulated Action framework.Journal of Memory and Language,63(2),245-257.
Hostetter,A.B.,Alibali,M.W.,&Kita,S.(2007).I see it in my hands’eye:Representational gestures reflect conceptual demands.Language and Cognitive Processes,22(3),313-336.
Hu,F.T.,Ginns,P.,&Bobis,J.(2015).Getting the point:Tracing worked examples enhances learning.Learning&Instruction,35,85-93.
Iverson,J.M.,&Goldin-Meadow,S.(1998).Why people gesture when they speak.Nature,396(6708),228.
Kang,S.,Tversky,B.,&Black,J.B.(2015).Coordinating gesture,word,and diagram:Explanations for experts and novices.Spatial Cognition&Computation,15(1),1-26.
Kelly,S.D.,&Church,R.B.(1998).A comparison between children's and adults'ability to detect conceptual information conveyed through representational gestures.Child Development,69(1),85-93.
Kita,S.,&?zyürek,A.(2003).What does cross-linguistic variation in semantic coordination of speech and gesture reveal?Evidence for an interface representation of spatial thinking and speaking.Journal of Memory and language,48(1),16-32.
Koppensteiner,M.,Stephan,P.,&J?schke,J.P.M.(2016).Moving speeches:Dominance,trustworthiness and competence in body motion.Personality and Individual Differences,94,101-106.
Krauss,R.M.,Chen,Y.S.,&Gottesman,R.(2000).Lexical gestures and lexical access:A process model.In D.McNeill(Eds.)Language and gesture(pp.261-283).Cambridge,UK:Cambridge University Press.
Krauss,R.M.,&Weinheimer,S.(1964).Changes in reference phrases as a function of frequency of usage in social interaction:A preliminary study.Psychonomic Science,1,113-114.
Louwerse,M.M.(2011).Symbol interdependency in symbolic and embodied cognition.Topics in Cognitive Science,3(2),273-302.
Matovic,D.,Koch,A.S.,&Forgas,J.P.(2014).Can negative mood improve language understanding?Affective influences on the ability to detect ambiguous communication.Journal of Experimental Social Psychology,52,44-49.
McNeill,D.,&Duncan,S.(2000).Growth points in thinking-for-speaking.In D.McNeill.(Eds)Language and gesture(pp.141-161).Cambridge,UK:Cambridge University Press.
Morsella,E.,&Krauss,R.M.(2004).The role of gestures in spatial working memory and speech.The American Journal of Psychology,117(3),411-424.
Nicoladis,E.,Pika,S.,Yin,H.U.I.,&Marentette,P.(2007).Gesture use in story recall by Chinese-English bilinguals.Applied Psycholinguistics,28(4),721-735.
Novack,M.A.,Congdon,E.L.,Hemanilopez,N.,&Goldin-Meadow,S.(2014).From action to abstraction:Using the hands to learn math.Psychological Science,25(4),903-910.
Novack,M.A.,&Goldin-Meadow,S.(2016).Gesture as representational action:A paper about function.Psychonomic Psychonomic Bulletin&Review,24(3),652-665.
Novack,M.,&Goldin-Meadow,S.(2015).Learning from gesture:How our hands change our minds.Educational Psychology Review,27(3),405-412.
Novack,M.A.,Goldin-Meadow,S.,&Woodward,A.L.(2015).Learning from gesture:How early does it happen?.Cognition,142,138-147.
Perniss,P.,?zyürek,A.,&Morgan,G.(2015).The Influence of the visual modality on language structure and conventionalization:Insights from sign language and gesture.Topics in Cognitive Science,7(1),2-11.
Pine,K.J.,Bird,H.,&Kirk,E.(2007).The effects of prohibiting gestures on children's lexical retrieval ability.Developmental Science,10(6),747-754.
Ping,R.M.,&Goldin-Meadow,S.(2008).Hands in the air:Using ungrounded iconic gestures to teach children conservation of quantity.Developmental Psychology,44(5),1277-1287.
Ping,R.M.,Goldin-Meadow,S.,&Beilock,S.L.(2014).Understanding gesture:Is the listener’s motor system involved?.Journal of Experimental Psychology:General,143(1),195-204.
Post,L.S.,Gog,T.V.,Paas,F.,&Zwaan,R.A.(2013).Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning.Computers in Human Behavior,29(4),1450-1455.
Rauscher,F.H.,Krauss,R.M.,&Chen,Y.S.(1996).Gesture,speech,and lexical access:The role of lexical movements in speech production.Psychological Science,7(4),226-231.
Sacchi,S.,Riva,P.,&Aceto,A.(2016).Myopic about climate change:Cognitive style,psychological distance,and environmentalism.Journal of Experimental Social Psychology,65,68-73.
Sassenberg,U.,&van Der Meer,E.(2010).Do we really gesturemorewhen it is more difficult?Cognitive Science,34(4),643-664.
Sharma,V.,&Droch,B.(2015).Gesture-controlled user interfaces.Journal of Information Sciences and Computing Technologies,2(1),133-135.
Trofatter,C.,Kontra,C.,Beilock,S.,&Goldin-Meadow,S.(2015).Gesturing has a larger impact on problem-solving than action,even when action is accompanied by words.Language,Cognition and Neuroscience,30(3),251-260.
Weinberg,A.,Fukawa-Conolly,T.,&Wiesner,E.(2015).Characterizing instructor gestures in a lecture in a proof-based mathematics class.Educational Studies in Mathematics,90(3),233-258.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |