COMPARING CLASSICAL TEST THEORY AND ITEM RESPONSE THEORY FOR RESPIRATORY SYSTEM QUESTION INSTRUMENTS
Abstract
This study aims to compare the results of instrument testing methods between applying classical test theory and item response theory using the Rasch model in question instruments on respiratory system material. This study employed descriptive quantitative methodology, with a sample involving 36 students. The analyzed instrument consisted of 40 multiple-choice questions on respiratory system material. Instrument analysis utilized classical test theory with Microsoft Excel and item response theory with Winstep Rasch ver 4.5.2.0. The data analysis from classical test theory and item response theory offers slightly different interpretations but is mutually complementary. Both classical test theory and item response theory may assess the validity, reliability, distractor effectiveness, difficulty level, and discriminating power of questions. Item response theory provides a comprehensive analysis of test results through the use of the Wright map as a bar which helps determine a student's ability about the difficulty level of the question. Scalogram is used to identify patterns in students’ responses, allowing for the detection of cheating and inaccuracies in answering questions. Additionally, DIF items are employed to identify item bias. This study concludes that any developed instrument must possess the characteristics that meet the requirements to measure competency effectively. The requirements for an instrument can be analyzed using item response theory with the Rasch model, which provides in-depth interpretation.
Keywords: Classical test theory, item response theory, Rasch model, instrument
Keywords
Full Text:
PDFReferences
Anselmi, P., Colledani, D., & Robusto, E. (2019). A Comparison of Classical and Modern Measures of Internal Consistency. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.02714
Arikunto, S. (2012). Prosedur Penelitian. Jakarta: Rineka Cipta.
Au, M. L., Li, Y. Y., Tong, L. K., Wang, S. C., & Ng, W. I. (2023). Chinese version of Yoon Critical Thinking Disposition Instrument: validation using classical test theory and Rasch analysis. BMC Nursing, 22(1). https://doi.org/10.1186/s12912-023-01519-y
Azmi, M. P., & Salam, A. (2020). Pengembangan Instrumen Tes Kemampuan Komunikasi Matematis pada Materi Segi Empat. Journal for Research in Mathematics Learning) p, 3(3), 181–192.
Bichi, A. A., Embong, R., Mamat, M., & Maiwada, D. A. (2015). Comparison of Classical Test Theory and Item Response Theory: A Review of Empirical Studies. Australian Journal of Basic and Applied Sciences, 9(7), 549–556. https://doi.org/10.13140/RG.2.1.1561.5522
Boonee•, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch Analysis in the Human Sciences. New York : Springer Dordrecht Heidelberg.
Chitaree, D., Junpeng, P., Sonsilphong, S., & Tang, K. N. (2024). Using Machine Learning to Score Multidimensional Assessments of Students’ Skill Levels in Mathematics. Pertanika Journal of Social Sciences and Humanities, 32(1), 217–235. https://doi.org/10.47836/pjssh.32.1.10
Darmana, A., Sutiani, A., Nasution, H. A., Ismanisa*, I., & Nurhaswinda, N. (2021). Analysis of Rasch Model for the Validation of Chemistry National Exam Instruments. Jurnal Pendidikan Sains Indonesia, 9(3), 329–345. https://doi.org/10.24815/jpsi.v9i3.19618
Erfan, M., Archi Maulyda, M., Hidayati, V. R., Astria, F. P., Ratu, T., Studi, P., Guru, P., & Dasar, S. (2020). Analisis Kualitas Soal Kemampuan Membedakan Rangkaian Seri dan Paralel melalui Teori Tes Klasik dan Model Rasch. Indonesian Journal of Educational Research and Review, 3(1), 11.
Gronlund, N. E., & Waugh, C. K. (2013). Assessment of Student Achievement. New Jersey : Pearson Education.
Intasoi, S., Junpeng, P., Tang, K. N., Ketchatturat, J., Zhang, Y., & Wilson, M. (2020). Developing an assessment framework of multidimensional scientific competencies. International Journal of Evaluation and Research in Education, 9(4), 963–970. https://doi.org/10.11591/ijere.v9i4.20542
Isnani, I., Utami, W. B., Susongko, P., & Lestiani, H. T. (2019). Estimation of college students’ ability on real analysis course using Rasch model. REID (Research and Evaluation in Education), 5(2), 95–102. https://doi.org/10.21831/reid.v5i2.20924
Jumini, S., Madnasri, S., Cahyono, E., & Parmin, P. (2023). Analisis Kualitas Butir Soal Pengukuran Literasi Sains Melalui Teori Tes Klasik Dan Rasch Model. Prosiding Seminar Nasional Pascasarjana Universitas Negeri Semarang, 758–765.
http://pps.unnes.ac.id/pps2/prodi/prosiding-pascasarjana-unnes758
Mitchell-Parker, K., Medvedev, O. N., Krägeloh, C. U., & Siegert, R. J. (2018). Rasch analysis of the Frost Multidimensional Perfectionism Scale. Australian Journal of Psychology, 70(3), 258–268. https://doi.org/10.1111/ajpy.12192
Mohajan, H. K. (2020). Quantitative Research: A Successful Investigation in Natural and Social Sciences. Journal of Economic Development, Environment and People, 9(4), 52–79.
Nurjanah, W. L., Sari, I. M., & Saepuzaman, D. (2024). Analisis Pemahaman Konsep Peserta Didik Pada Topik Perambatan Kalor Menggunakan Rasch Model. Jurnal Kependidikan, 13(2). https://jurnaldidaktika.org
Pellegrino, J. W., Chudowsky, Naomi., Glaser, R., & National Research Council (U.S.). Committee on the Foundations of Assessment. (2001). Knowing what students know : the science and design of educational assessment. Washington, DC : National Academy Press.
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2). https://doi.org/10.1103/PhysRevPhysEducRes.15.020111
Popham, W. James. (2017). Classroom assessment : what teachers need to know. University of California, Los Angeles.
Rahayu, W., Putra, M. D. K., Rahmawati, Y., Hayat, B., & Koul, R. B. (2021). Validating an indonesian version of the what is happening in this class? (wihic) questionnaire using a multidimensional rasch model. International Journal of Instruction, 14(2), 919–934. https://doi.org/10.29333/iji.2021.14252a
Robinson, M., Johnson, A. M., Walton, D. M., & MacDermid, J. C. (2019). A comparison of the polytomous Rasch analysis output of RUMM2030 and R (ltm/eRm/TAM/lordif). BMC Medical Research Methodology, 19(1). https://doi.org/10.1186/s12874-019-0680-5
Septiliana, L. (2023). Analisis Item Soal dengan Menggunakan Rasch Model sebagai Ukuran Kualitas Madrasah Ibtidaiyah pada Mata Pelajaran IPA. Pionir: Jurnal Pendidikan, 12(2), 1–12.
Sugiyono. (2016). Metode Penelitian Kuantitatif, Kualitatif, dan R&D. Bandung : Alfabeta.
Sumaryanta. (2021). Teori Tes Klasik & Teori Respon Butir : Konsep & Contoh Penerapannya. Cirebon : Confident.
Sumintono, B., & Widhiarso, W. (2013). Aplikasi Model Rasch Untuk Penelitian Ilmu-Ilmu Sosial. Trim Komunikata Publishing House.
Syafrial, Ashadi, Saputro, S., & Sarwanto. (2022). Trend creative thinking perception of students in learning natural science: Gender and domicile perspective. International Journal of Instruction, 15(1), 701–716. https://doi.org/10.29333/iji.2022.15140a
Thomas, F. B. (2022). The Role of Purposive Sampling Technique as a Tool for Informal Choices in a Social Sciences in Research Methods. Just Agriculture, 2(5), 1–8. www.justagriculture.in
Wongpakaran, N., Wongpakaran, T., Pinyopornpanish, M., Simcharoen, S., Suradom, C., Varnado, P., & Kuntawong, P. (2020). Development and validation of a 6-item Revised UCLA Loneliness Scale (RULS-6) using Rasch analysis. British Journal of Health Psychology, 25(2), 233–256. https://doi.org/10.1111/bjhp.12404
DOI: http://dx.doi.org/10.22373/pjp.v13i3.25435
Refbacks
- There are currently no refbacks.
Copyright (c) 2024 Bowo Sugiharto, Adha Nisfatulsanah
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
