方差分析效果大小报告的新指标
|
摘要
心理学期刊论文中可重复性不高的现象,原因之一是研究结果的效果普遍较小。并且在报告效果大小的文章中,效果大小指标使用不当。在方差分析中最经常报告的是η~2和η■,但是在不同的研究设计中,这些效果大小是无法直接进行比较的。广义eta方(η■)是近年来新出现的一种新的效果大小指标,可克服的η~2和η■不足,灵活处理重复测量等多种研究设计下个体差异的计算问题,实现跨研究设计效果大小的可比性。论文结合实例介绍了η■的原理和计算方法,并对其优缺点、使用和报告等问题进行了讨论。研究人员在报告效果大小时要考虑到不同的研究设计和研究假设,并选择恰当的指标防止过高估计效果大小。
One of the reasons for the low repeatability in psychological research is that the effect size are generally small.And in the article reporting the effect size,the effect size indicator is not used properly.The most frequently reported indicators for ANOVA are eta squared and partial eta squared.But these two indicators are not comparable across different research designs.Generalized eta squared that has emerged in recent years can overcome traditional indicators' shortcomings,and achieve comparability by considering individual differences in many kinds of research designs.The present paper introduces what is generalized eta squared and how to calculate it with examples.Advantages,disadvantages and reporting of it are also discussed.Researchers should consider different research designs and research hypotheses and choose appropriate size measures to prevent overestimation.
One of the reasons for the low repeatability in psychological research is that the effect size are generally small.And in the article reporting the effect size,the effect size indicator is not used properly.The most frequently reported indicators for ANOVA are eta squared and partial eta squared.But these two indicators are not comparable across different research designs.Generalized eta squared that has emerged in recent years can overcome traditional indicators' shortcomings,and achieve comparability by considering individual differences in many kinds of research designs.The present paper introduces what is generalized eta squared and how to calculate it with examples.Advantages,disadvantages and reporting of it are also discussed.Researchers should consider different research designs and research hypotheses and choose appropriate size measures to prevent overestimation.
引文
胡竹菁,戴海琦.(2011).方差分析的统计检验力和效果大小的常用方法比较.心理学探新,31(3),254-259.
聂丹丹,王浩,罗蓉.(2016).可重复性:心理学研究不可忽视的实践.中国临床心理学杂志,24(4),618-622.
权朝鲁.(2003).效果量的意义及测定方法.心理学探新,23(2),39-44.
舒华.(1994).心理与教育研究中的多因素实验设计.北京:北京师范大学出版社.
郑昊敏,温忠麟,吴艳.(2011).心理学常用效应量的选用与分析.心理科学进展,19(12),1868-1878.
American Psychological Association.(1994).Publication manual of the American Psychological Association(4th ed.).Washington,DC:American Psychological Association.
American Psychological Association.(2001).Publication manual of the American Psychological Association(5th ed.).Washington,DC:American Psychological Association.
American Psychological Association.(2010).Publication manual of the American Psychological Association(6th ed.).Washington,DC:American Psychological Association.
Bakeman,R.(2005).Recommended effect size statistics for repeated measures designs.Behavior Research Methods,37(3),379-384.
Baker,M.(2016).1,500 scientists lift the lid on reproducibility.Nature,533(7604),452-454.
Cohen,J.(1988).Statistical power analysis for the behavioral sciences.Hillsdale,N.J:L.Erlbaum Associates.
Cumming,G.,& Fidler,F.(2009).Confidence intervals:Better answers to better questions.Zeitschrift Für Psychologie,217(217),15-26.
Cumming,G.(2014).The new statistics:Why and how.Psychological Science,25(1),7-29.
Francis,G.(2013).Replication,statistical consistency,and publication bias.Journal of Mathematical Psychology,57(57),153-169.
Franco,A.,Malhotra,N.,& Simonovits,G.(2015).Underreporting in psychology experiments:Evidence from a study registry.Social Psychological & Personality Science,7(1),1-5.
Glass,G.V.(1976).Primary,secondary,and meta-analysis of research.Educational Researcher,5(10),3-8.
Green,J.L.,Hedges,L.V.,& Levine,F.J.(2006).Standards for Reporting on Empirical Social Science Research in AERA Publications:American Educational Research Association.Educational Researcher,35(6),33-40.
Hedges,L.V.(1981).Distribution theory for Glass’s estimator of effect size and related estimators.Journal of Educational Statistics,6(2),107-128.
John,L.K.,Loewenstein,G.,& Prelec,D.(2012).Measuring the prevalence of questionable research practices with incentives for truth telling.Psychological Science,23(5),524-532.
Kendall,P.C.(1997).Editorial.Journal of Consulting and Clinical Psychology,65(1),3-5.
Keppel,G.(1991).Design and analysis:A researcher’s handbook.Upper Saddle River(NJ):Prentice-Hall.
Kirk,R.E.(1996).Practical significance:A concept whose time has come.Educational and Psychological Measurement,56(5),746-759.
Lawrence,M.A.(2011).ez:Easy analysis and visualization of factorial experiments.R package version 3.0-0.http://CRAN.R-project.org/package=ez
Levelt,P.,Noort,E.,& Drenth,P.(2012).Flawed science:The fraudulent research practices of social psychologist Diederik Stapel.Retrieved from http://www.tilburguniversity.edu/upload/3ff904d7-547b-40ae-85fe-bea38e05a34a_Final%20report%20Flawed%20Science.pdf
Miller,G.(2010).Cognition research.Investigation leaves field in the dark about a colleague’s work.Science,329(5994),890-891.
Murphy,& Kevin,R.(1997).Editorial.Journal of Applied Psychology,82(1),3-5.
National Center for Education Statistics.(2002).NCES statistical standards.Washington,DC:Department of Education.
Olejnik,S.,& Algina,J.(2003).Generalized eta and omega squared statistics:Measures of effect size for some common research designs.Psychological Methods,8(4),434-447.
Open Science Collaboration.(2015).Estimating the reproducibility of psychological science.Science,349(6251),aac4716.
Simmons,J.P.,Nelson,L.D.,& Simonsohn,U.(2011).False-Positive Psychology Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.Psychological Science,22(11),1359-1366.
Thompson.(1994).Guidelines for authors.Educational and Psychological Measurement,54,837-847.
聂丹丹,王浩,罗蓉.(2016).可重复性:心理学研究不可忽视的实践.中国临床心理学杂志,24(4),618-622.
权朝鲁.(2003).效果量的意义及测定方法.心理学探新,23(2),39-44.
舒华.(1994).心理与教育研究中的多因素实验设计.北京:北京师范大学出版社.
郑昊敏,温忠麟,吴艳.(2011).心理学常用效应量的选用与分析.心理科学进展,19(12),1868-1878.
American Psychological Association.(1994).Publication manual of the American Psychological Association(4th ed.).Washington,DC:American Psychological Association.
American Psychological Association.(2001).Publication manual of the American Psychological Association(5th ed.).Washington,DC:American Psychological Association.
American Psychological Association.(2010).Publication manual of the American Psychological Association(6th ed.).Washington,DC:American Psychological Association.
Bakeman,R.(2005).Recommended effect size statistics for repeated measures designs.Behavior Research Methods,37(3),379-384.
Baker,M.(2016).1,500 scientists lift the lid on reproducibility.Nature,533(7604),452-454.
Cohen,J.(1988).Statistical power analysis for the behavioral sciences.Hillsdale,N.J:L.Erlbaum Associates.
Cumming,G.,& Fidler,F.(2009).Confidence intervals:Better answers to better questions.Zeitschrift Für Psychologie,217(217),15-26.
Cumming,G.(2014).The new statistics:Why and how.Psychological Science,25(1),7-29.
Francis,G.(2013).Replication,statistical consistency,and publication bias.Journal of Mathematical Psychology,57(57),153-169.
Franco,A.,Malhotra,N.,& Simonovits,G.(2015).Underreporting in psychology experiments:Evidence from a study registry.Social Psychological & Personality Science,7(1),1-5.
Glass,G.V.(1976).Primary,secondary,and meta-analysis of research.Educational Researcher,5(10),3-8.
Green,J.L.,Hedges,L.V.,& Levine,F.J.(2006).Standards for Reporting on Empirical Social Science Research in AERA Publications:American Educational Research Association.Educational Researcher,35(6),33-40.
Hedges,L.V.(1981).Distribution theory for Glass’s estimator of effect size and related estimators.Journal of Educational Statistics,6(2),107-128.
John,L.K.,Loewenstein,G.,& Prelec,D.(2012).Measuring the prevalence of questionable research practices with incentives for truth telling.Psychological Science,23(5),524-532.
Kendall,P.C.(1997).Editorial.Journal of Consulting and Clinical Psychology,65(1),3-5.
Keppel,G.(1991).Design and analysis:A researcher’s handbook.Upper Saddle River(NJ):Prentice-Hall.
Kirk,R.E.(1996).Practical significance:A concept whose time has come.Educational and Psychological Measurement,56(5),746-759.
Lawrence,M.A.(2011).ez:Easy analysis and visualization of factorial experiments.R package version 3.0-0.http://CRAN.R-project.org/package=ez
Levelt,P.,Noort,E.,& Drenth,P.(2012).Flawed science:The fraudulent research practices of social psychologist Diederik Stapel.Retrieved from http://www.tilburguniversity.edu/upload/3ff904d7-547b-40ae-85fe-bea38e05a34a_Final%20report%20Flawed%20Science.pdf
Miller,G.(2010).Cognition research.Investigation leaves field in the dark about a colleague’s work.Science,329(5994),890-891.
Murphy,& Kevin,R.(1997).Editorial.Journal of Applied Psychology,82(1),3-5.
National Center for Education Statistics.(2002).NCES statistical standards.Washington,DC:Department of Education.
Olejnik,S.,& Algina,J.(2003).Generalized eta and omega squared statistics:Measures of effect size for some common research designs.Psychological Methods,8(4),434-447.
Open Science Collaboration.(2015).Estimating the reproducibility of psychological science.Science,349(6251),aac4716.
Simmons,J.P.,Nelson,L.D.,& Simonsohn,U.(2011).False-Positive Psychology Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.Psychological Science,22(11),1359-1366.
Thompson.(1994).Guidelines for authors.Educational and Psychological Measurement,54,837-847.