Analysis of the ENADE items complexity based on the Revised Bloom Taxonomy: contributions to physics teaching

Abstract

The National Student Performance Exam (ENADE) is the main component of the National Higher Education Evaluation System (SINAES) that deals with the Brazilian Higher Education Degrees. This article deals with 120 (one hundred and twenty) objectives and discursive items from four ENADE editions for Teaching Physics Degree, applied in 2005, 2008, 2011 and 2014. We search for the Exam complexity throughout cognitive aspects mobilized to solve each item, according to the Bloom Taxonomy Revised (BTR). The BTR proposes a two-dimensional matrix classification that crosses a knowledge dimension, which means, “what” students must know in order to answer the questions, with the cognitive process involved in the task, reflecting “how” the problem was solved. According the BTR theory, the knowledge has been understood in the dimensions: effective, conceptual, procedure and metacognitive, which deals, respectively with basics terminology knowledge, concepts knowledge, procedure methodology knowledge and reflexive/analytical knowledge. The cognitive process described in verbs at the BTR presents the student´s skills in the resolutions ranked as: to remember, to understand, to apply, to analyze, to evaluate and to create. To classify discursive items, pattern answers were used, and to classify multiple choice items, our resolutions of the questions were used. The items were located at the two-dimensional BTR matrix, which gives us an Exam complexity overview. We observe that only 17 (14%) items are in the effective knowledge domain, while 103 (86%) items require the conceptual and procedure knowledge domain, which demands higher complexity. We consider these results compatible with the ENADE role as a Higher Brazilian Education quality referential. We intend to update Physics professors throughout Physics Teaching Examination in order to increase their commitment to the Higher Education Evaluation. We also believe that understanding the examination could help them to reconsider ENADE cognition aspects, based on the BTR perspective, incorporating new assumptions in their praxis.