Reliability in the evaluation of international and national judges in an artistic swimming routine
DOI:
https://doi.org/10.1590/1980-0037.2021v23e76587Abstract
Artistic swimming (AS) is a sport evaluated by fifteen judges in routine sessions. The athletes' goal is to achieve proficient motor patterns according to pre-established criteria. The present studied analyzed whether there is difference between the two groups of AS judges with different levels of experience. The group of the International Swimming Federation - FINA (IG) evaluates AS at national and international level and the non-FINA group (NG) evaluates AS only at national level. Twenty experienced judges were divided into groups, 10 IG judges and 10 NG judges. Thus, judges evaluated the item execution of three routines with five required elements. Cronbach's alpha coefficient showed high internal consistency in IG (a= 0.85 in T1 and 0.83 in T2). In NG, internal consistency was observed in T1 and low consistency in T2 (a= 0.82 in T1 and 0.39 in T2). Evaluation analysis between IG and NG was significant (p?0.0330) and reliability analysis (bias: -0.1266 95% agreement limit: -1.642 to 1.388) showed consistency and high degree of confidence in results. The findings suggest that the item execution of required elements showed high objectivity regarding judges with different levels of experience, IG and NG, regardless of categorization and time of practice. FINA has changed the number of judges and the number of items evaluated in routine sessions. It is suggested that the reduction of items has contributed in a positive way so that judges can focus more on evaluation itself.
References
Ponciano KR, Fugita M, Figueira Jr AJ, Silva CB, Meira Jr CM, Bocalini DS. Reliability of judge's evaluation of the synchronized swimming technical elements by video. Rev Bras Med Esp. 2018;24(3):182-185.
Fédération Internationale De Natation. Manual for synchronised swimming Judges, coaches & referees. FINA Office. Lausanne: Suiça; 2017.
Fédération Internationale De Natation. Manual for synchronised swimming Judges, coaches & referees. FINA Office. Lausanne: Suiça; 2009.
Flessas K, Mylonas D, Panagiotaropoulou G, Tsopani D, Korda A, Siettos C, et al. Judging the judges’ performance in rhythmic gymnastics. Med Sci Sports & Exerc. 2015;47(3):640-648.
Annett, J. Subjective rating scales: science or art? Ergonomics. 2002;45(14):966-987.
Williams AM, Davids K, Williams JGP. Visual percep action in sport. London: Routledge; 1999.
Bard C, Fleury M, Carrière L, Hallé M. Analysis of gymnastics judges' visual search. Res Q Exerc Sport. 1980;51(2):267-273.
Pizzera A, Möller C, Plessner H. Gaze behavior of gymnastics judges: Where do experienced judges and gymnasts look while judging? Res Q Exerc Sport. 2017;89(1):112-119.
Plessner H, Haar T. Sports performance judgments from a social cognitive perspective. Psychol Sport Exerc. 2006;7(6):555-575.
Ste-Marie DM, Lee TD. Prior processing effects on gymnastic judging. J Exp Psychol Learn Mem Cogn. 1991;17(1):126.
Ste‐Marie DM. Expert–novice differences in gymnastic judging: an information‐processing perspective. Appl Cog Psychol Off J Soc Applied Res in Mem Cogn. 1999;13(3):269-281.
Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297-334.
Hopkins WG. Measures of reliability in sports medicine and Science. Sports Med. 2000;30(1):1-15.
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327(8476):307-310.
Appelbaum LG, Erickson G. Sports vision training: A review of the state-of-the-art in digital training techniques. Int Rev Sport Exerc Psychol. 2018;11(1):160-189.
Kredel R, Vater C, Klostermann A, Hossner EJ. Eye-tracking technology and the dynamics of natural gaze behavior in sports: A systematic review of 40 years of research. Frontiers in Psychology. 2017;8:1845.
Mann DL, Causer J, Nakamoto H, Runswick OR. Visual search behaviours in expert perceptual judgements. In: Williams AM, Jackson RC. Anticipation and decision making in sport. New York: Routledge. 2019. p. 59-78.
Ponciano K, Miranda MLDJ, Homma M, Miranda JMQ, Figueira Jr AJ, Meira Jr CM, et al. Physiological responses during the practice of synchronized swimming: a systematic review. Clin Physiol Funct Imaging. 2017;38(2):163-175.
Gilis B, Weston M, Helsen WF, Junge A, Dvorak J. Interpretation and application of the laws of the game in football incidents leading to player injuries. Int J Sport Psychol. 2006;37(2/3):121.
Helsen W, Gilis B, Weston M. Errors in judging “offside” in association football: Test of the optical error versus the perceptual flash-lag hypothesis. J Sports Sci. 2006;24(05):521-528.
