Scenario-based Reading Comprehension Assessment: Case Analysis, PISA 2018

Abstract

The objective of this paper is to analyze one of the units in the reading area of the PISA 2018 test. The purpose was to assess the cognitive processes underlying the resolution of tasks under the Scenario-Based Assessment Design (SBA). The chosen unit, Rapa Nui, shows a scenario in which the student must prepare to attend a conference about the field work carried out by a professor on the island of Rapa Nui. The article uses the methodological strategy of the Inverted Pyramid (DIP), whose premises enable a qualitative analysis at a macro level (country) that is then triangulated to a qualitative analysis at a micro level (case-unit-scenario-reactive). For the latter, a Cognitive Route is used, that is, an analysis mechanism that favors the identification of the reading processes involved in solving tasks. The results show, at a macro level, that the majority of students (60.8%) are located in the lower levels of the PISA scale (1 and 2), which implies that their reading skills are basic. Students located at level 3 (17.5%) and those at level 4, 5 and 6 (only 6%) can face more complex tasks. Finally, at a micro level, the analysis of the Cognitive Route allows us to have a list of knowledge and skills that a student requires to be able to face a task such as the one analyzed in the article.