Assessing second language oral proficiency development with holistic and analytic scales

Rosane Silveira; Thaisy da Silva Martins

doi:10.5007/2175-8026.2020v73n3p227

Authors

Rosane Silveira Universidade Federal de Santa Catarina/CNPq https://orcid.org/0000-0003-0329-0376
Thaisy da Silva Martins UFSC https://orcid.org/0000-0002-5638-9102

DOI:

https://doi.org/10.5007/2175-8026.2020v73n3p227

Abstract

The present study discusses how a group of experienced raters use different types of scales to assess the development of oral proficiency in English as a second language (L2). Raters assigned rates to speech samples first using a holistic scale (CEFR, 2018) and then assigning rates for pronunciation, vocabulary, grammar and fluency performance using individual scales. The speech samples were recorded by five Brazilians. There were two data collection sessions, with the second one occurring 7-8 months after the first one. The results indicate high levels of agreement among raters for all scales. Furthermore, the raters detected changes in speakers’ performance in four out of five scales: L2 oral proficiency, vocabulary, grammar, and fluency, and these differences in rates across time were significant for oral proficiency, vocabulary, and fluency. Thus, the different types of scale allow detecting L2 oral proficiency development.

Author Biography

Rosane Silveira, Universidade Federal de Santa Catarina/CNPq

Professora Associada da Universidade Federal de Santa Catarina

Coordenadora do Programa de Pós-Graduação em Inglês

Bolsista de Produtividade do CNPq

References

BENT, T., BRADLOW, A. The interlanguage speech intelligibility benefit. The Journal of the Acoustical Society of America, v. 114, n. 3, p. 1600-1610, 2003.

CEFR: Common European Framework of Reference for Languages: Learning, Teaching, Assessment, 2001. http://www.coe.int/t/dg4/linguistic/source/framework_en.pdf

De JONG, N. H., GROENHOUT, R. S., SCHOOEN, R., HULSTIJN, J H. Second language fluency: speaking style or proficiency? Correcting measures of second language fluency for first language behavior. Applied Psycholinguistics, v. 36, n. 2, p. 223–43, 2015.

DERWING, T. M.; MUNRO, M. J.; THOMSON, R. I. A Longitudinal Study of ESL Learners’ Fluency and Comprehensibility Development. Applied Linguistics, v. 29(3), p. 359–380, 2007.

DERWING, T., MUNRO, M. Accent, comprehensibility and intelligibility: Evidence from four L1s. Studies in Second Language Acquisition, v. 19, p. 1-16, 1997.

DERWING, T.; MUNRO, M. Pronunciation Fundamentals: Evidence-Based Perspectives for L2 Teaching and Research. Language Teaching Series, n. 42, 2015.

FERRARI, S. A longitudinal study of complexity, accuracy and fluency variation in second language development. In: HOUSEN, A.; KUIKEN, F.; VEDDER, I. Dimension of L2 Performance and Proficiency: Complexity, Accuracy and Fluency in SLA. Amsterdam/Philadelphia: John Benjamins, 2012. p. 277-298.

GASS, S., VARONIS, E. The effect of familiarity on the comprehensibility of nonnative speech. Language Learning, v. 34, n. 1, p. 65–89, 1984.

HAYES-HARB, R., HACKING, J. Beyond rating data: What do listeners believe underlies their accentedness judgments? John Benjamins Publishing Company, v. 1(1), p. 43-64, 2015.

HUANG, B. H., & JUN, S. A. Age matters, and so may raters: Rater differences in the assessment of foreign accents. Studies in Second Language Acquisition, vol. 37, no. 4, p. 623–650, 2015.

HULSTIJN, J. H. The construct of language proficiency in the study of bilingualism from a cognitive perspective. Bilingualism: Language and Cognition, v. 15(2), p. 422 – 433, 2012.

NEIDE, C. Familiaridade do ouvinte e inteligibilidade da pronúncia de aprendizes brasileiros de inglês. Revista Horizontes de Linguística Aplicada, v. 7, n. 1, p. 88-103, 2008.

NIVJA H. DE JONG; RACHEL GROENHOUT; ROB SCHOONEN. Second language fluency: Speaking style or proficiency? Correcting measures of second language fluency for first language behavior. Applied Psycholinguistics, v. 36(2), p. 223-243, 2015.

O’BRIEN, M. Methodological choices in rating speech samples. Studies in Second Language Acquisition, v. 38(3), p. 587-605, 2016.

ORTEGA, L., HAN, Z.(Ed.): Complexity Theory And Language Development: In Celebration Of Diane Larsen-freeman. John Benjamins, 2017.

SAITO, K., SUZUKIDA; Y., SUN, H. Aptitude, experience, and second language pronunciation proficiency development in classroom settings: A longitudinal study. Studies in Second Language Acquisition, p. 1-25, (to appear in 2019).

SAITO, K.; HANZAWA, K. Developing second language oral ability in foreign language classrooms: The role of the length and focus of instruction and individual differences. Applied Psycholinguistics, v. 37, p. 813–840, 2016.

SHINTANI, N., SAITO, K., KOIZUMI, R. The relationship between multilingual raters' language background and their perceptions of accentedness and comprehensibility of second language speech. International Journal of Bilingual Education and Bilingualism, p. 849-869, 2017.

XXXXXXX (2018)

XXXXXXX (2011)

XXXXXXX (2012)