Dimensionality analysis of students’ performance in 2013 BGCSE agricultural examination: implications for differential item functioning

Abstract

Equity and excellence in assessing desirable changes in learners' behaviour are applied only if the measures in use provide valid outcome data for different subgroups. The deterioration of student performance in the Botswana General Certificate of Education (BGCSE) examination results is a disturbing trend that bothers parents, teacher, policy makers and government. This problem prompts this study on dimensionality analysis of students' performance in 2013 BGCSE Agriculture Examination, implication for differential item functioning, to determine its dimensionality and fairness to all learners. The population for the study was all the 12784 students' responses who sat for the 2013 BGCSE agriculture examination. The students‟ responses were analysed using factor analysis and IRT (1PL, 2PL and 3PL) models to examine the psychometric parameter estimates of the forty test items; dimensionality analysis and the chi square test for each test item that fitted in the three IRT models. Differential Item Functioning (DIF) analysis for each item also was done according to gender and location type using logits test for t-test significance (p < .05). The findings revealed that examination was not unidimensional. None of the items fit the 1PL. Only one fit the 2PL and 8 items fitted the 3PL. The findings from this study on gender based DIF indicated that twenty nine (29) out of the 40 items were DIF, seventeen (17) items favoured boys whereas twelve (12) favoured females. With location based, the DIF findings indicated that eighteen (18) were DIF, ten (10) favoured rural and eight (8) favoured urban students. In conclusion, the results of this study, as it explored the national assessment tool, showed that 2013 BGCSE Agriculture Examination was neither unidimensionality nor fair to all students. It was, therefore recommended that test developers and examination bodies should consider improving the quality of their test items by conducting IRT psychometric analysis for validation DIF purpose among others.