Assessing Literacy Competencies Among Polytechnic Lecturers in Malaysia Using a Rasch-Based Measurement Model
DOI:
https://doi.org/10.53840/e-jpi.v13i1.348Keywords:
Assessment Literacy, Outcome-Based Education, Rasch Measurement Model, Polytechnic Lecturers, MalaysiaAbstract
Assessment literacy is a critical component of effective teaching and learning within Outcome-Based Education (OBE) systems, particularly in polytechnic institutions where competency-based outcomes are emphasized. Despite ongoing assessment reforms in Malaysian polytechnics, concerns remain regarding lecturers’ literacy competencies, especially in aligning assessment practices with OBE principles. Conventional evaluation methods often fail to capture latent competency levels and item functioning, limiting the accuracy of assessment literacy measurement. There is a need for a robust and psychometrically sound measurement approach that can accurately assess lecturers’ literacy competencies and reveal underlying cognitive demands associated with assessment practices. Employing Rasch measurement addresses this gap by providing interval-level estimates, enabling more precise evaluation of lecturer competencies and supporting evidence-based professional development and policy decisions in Malaysian polytechnic education. This study adopted a quantitative research design using a Rasch-based measurement model to assess literacy competencies among polytechnic lecturers in Malaysia. A structured assessment instrument focusing on Outcome-Based Education and assessment literacy was administered to a sample of 33 polytechnic lecturers. Rasch analysis was conducted to estimate item difficulty, evaluate item targeting, and examine construct representation. Descriptive statistics and Rasch logit measures were used to analyze item functioning and respondent performance. The Rasch analysis revealed a well-ordered item difficulty hierarchy ranging from −0.835 to +1.987 logits, indicating strong discrimination across varying competency levels. Items assessing foundational assessment knowledge were identified as easy, while those related to formative assessment and core OBE concepts were significantly more difficult. Although some items recorded high correct response rates (up to 87.9%), Rasch logit estimates highlighted latent conceptual complexity not captured by classical statistics alone. The scale demonstrated effective item targeting, minimal item clustering, and adequate construct coverage, supporting its measurement validity. The findings confirm the effectiveness of Rasch modeling in assessing literacy competencies among polytechnic lecturers and reveal critical gaps in advanced OBE-related assessment literacy. The study underscores the importance of using robust measurement models for evaluating educational competencies and provides empirical evidence to inform targeted professional development and assessment policy enhancement in Malaysian polytechnic institution
Downloads
References
Ab Rahman, R., Ahmad, S., & Hashim, U. R. (2018). The effectiveness of gamification technique for higher education students’ engagement in polytechnic Muadzam Shah Pahang, Malaysia. International Journal of Educational Technology in Higher Education, 15(1), 1-16.
Adams, D., Sumintono, B., Mohamed, A., & Noor, N. S. M. (2018). E-learning readiness among students of diverse backgrounds in a leading Malaysian higher education institution. Malaysian Journal of Learning and Instruction, 15(2), 227-256.
Adnan, N. A., Paimin, A. N., & Hasan, A. A. (2019). Readiness of Polytechnic Manufacturing Program in Malaysia towards Future Industry. International Journal of Engineering and Advanced Technology, 9(2), 1654-1659.
Agi, C. I., Aduloju, M. O., & Kpum, M. M. (2018). Beyond Cognitive Assessment: Teachers Attitudes towards Assessing Psychomotor and Affective Domains of Students’ Behaviors in Classrooms in North Central-Nigeria. JEP, 9(16).
Aleixo, A. M., Azeiteiro, U., & Leal, S. (2018). The implementation of sustainability practices in Portuguese higher education institutions. International Journal of Sustainability in Higher Education, 19(1), 146-178.
Amin, M. Y. M. (2023). AI and Chat GPT in Language Teaching: Enhancing EFL Classroom Support and Transforming Assessment Techniques. International Journal of Higher Education Pedagogies, 4(4), 1-15.
Asim, H. M., Vaz, A., Ahmed, A., & Sadiq, S. (2021). A Review on Outcome Based Education and Factors That Impact Student Learning Outcomes in Tertiary Education System. International Education Studies, 14(2), 1-11.
Azis, D. K. (2023). Developing self-assessment instruments of affective domain on belief and morality (Aqidah Akhlak) subject in Madrasah Tsanawiyah. Pegem Journal of Education and Instruction, 13(1), 193-201.
Azmi, T., & Salleh, D. (2021). A Review on TVET Curriculum Practices in Malaysia. International Journal of Education, Psychology and Counselling, 40, 35-48.
Breadmore, H. L., Vardy, E., Cunningham, A. J., Kwok, R. K., & Carroll, J. M. (2019). Literacy development: evidence review.
Brown, S. (2020). Learning, teaching and assessment in higher education. Learning, Teaching and Assessment in Higher Education, 1-232.
De Courcy, E. (2021). Faculty Perceptions on Contextual Factors as Affordances or Constraints to Innovative Teaching and Learning in Polytechnic Institutions (Doctoral dissertation, University of Toronto (Canada)).
Doyle, T. (2023). Helping students learn in a learner-centered environment: A guide to facilitating learning in higher education. Taylor & Francis.
Gyimah, N. (2020). Assessment of Technical and Vocational Education and Training (TVET) on the development of the World's Economy: Perspective of Africa, Asia and Europe. Asia and Europe (February 19, 2020).
Henriques, P. L., Matos, P. V., Jerónimo, H. M., Mosquera, P., da Silva, F. P., & Bacalhau, J. (2018). University or polytechnic? A fuzzy-set approach of prospective students' choice and its implications for higher education institutions' managers. Journal of Business Research, 89, 435-441.
Herd, P., & Moynihan, D. P. (2019). Administrative burden: Policymaking by other means. Russell Sage Foundation.
Ibrahim, W., Sweedan, A., Ibrahim, H., Marzouk, S., Ibrahim, W., & Zoubeidi, T. (2022). Embedded Course Level Assessment for Effective Assessment of Program Learning Outcomes.
Joshi, A., Vinay, M., & Bhaskar, P. (2021). Impact of coronavirus pandemic on the Indian education sector: perspectives of teachers on online teaching and assessments. Interactive technology and smart education, 18(2), 205-226.
Kadhila, N., & Nyambe, J. (2021). Barriers to the Quality of Emergency Online Pedagogies in Higher Education during the COVID-19 Pandemic: A Case Study from the University of Namibia: A case study from Namibia. Journal of Learning for Development, 8(3), 516-531.
Kianto, A., Shujahat, M., Hussain, S., Nawaz, F., & Ali, M. (2019). The impact of knowledge management on knowledge worker productivity. Baltic Journal of Management, 14(2), 178-197.
Koh, K., Burke, L. E. C. A., Luke, A., Gong, W., & Tan, C. (2018). Developing the assessment literacy of teachers in Chinese language classrooms: A focus on assessment task design. Language teaching research, 22(3), 264-288.
Levi, T., & Inbar-Lourie, O. (2020). Assessment literacy or language assessment literacy: Learning from the teachers. Language Assessment Quarterly, 17(2), 168-182.
Marzuki, M. A. B., Yusof, R., Ishak, R., & Musa, K. (2022). Evaluating the TVET Financial Allocation Based on The Polytechnics and Community Colleges Students’ Enrolment: A Preliminary Analysis. International Journal of Academic Research in Progressive Education and Development, 11(2), 1267-1284.
Meijer, H., Hoekstra, R., Brouwer, J., & Strijbos, J. W. (2020). Unfolding collaborative learning assessment literacy: a reflection on current assessment methods in higher education. Assessment & Evaluation in Higher Education, 45(8), 1222-1240.
Mellati, M., & Khademi, M. (2018). Exploring teachers' assessment literacy: Impact on learners' writing achievements and implications for teacher development. Australian Journal of Teacher Education (Online), 43(6), 1-18.
Mohamad Taib, S. (2021). Development of Innovation Acceptance Model for Wearable Computing: A Study of Users’ Technology Acceptance in Malaysia (Doctoral dissertation, Brunel University London).
Mustapha, R., & Hussain, M. A. M. (2022). Vocational Education and Training in Malaysia. In International Handbook on Education in South East Asia (pp. 1-28). Singapore: Springer Nature Singapore.
Nalathambi, D. K., Salleh, K. S. M., Noh, S. H. M., Solaiman, H. S., & Jayaraman, R. (2023). Effort of Politeknik Malaysia as TVET institute in Attaining Sustainable Development Goals (SDGs) Through Twelfth Malaysia Plan. Borneo Engineering & Advanced Multidisciplinary International Journal, 2(01), 37-46.
Nalathambi, D. K., Salleh, K. S. M., Noh, S. H. M., Solaiman, H. S., & Jayaraman, R. (2023). Effort of Politeknik Malaysia as TVET institute in Attaining Sustainable Development Goals (SDGs) Through Twelfth Malaysia Plan. Borneo Engineering & Advanced Multidisciplinary International Journal, 2(01), 37-46.
Naqvi, S. R., Akram, T., Haider, S. A., Khan, W., Kamran, M., Muhammad, N., & Nawaz Qadri, N. (2019). Learning outcomes and assessment methodology: case study of an undergraduate engineering project. The International Journal of Electrical Engineering & Education, 56(2), 140-162.
Nwosu, K. C., Okoyoe, C. C., & Onah, U. H. (2018). An Interpretive Descriptive Study of Factors Affecting Academic Achievement of Underachieving Student Teachers in Nigeria. Journal of At-Risk Issues, 21(2), 20-29.
Popham, W. J. (2018). Assessment literacy for educators in a hurry. ASCD.
Pratiwi, D., Arief, M., & Churiyah, M. (2018). The Development of Performance-based Model Authentic Assessment on Archival Subject. KnE Social Sciences, 150-166.
Race, P. (2019). The lecturer's toolkit: A practical guide to assessment, learning and teaching. Routledge.
Rao, N. J. (2020). Outcome-based education: An outline. Higher Education for the Future, 7(1), 5-21.
Raschwahungu, D. K., Wawire, V., & Kirimi, F. (2023). Strategies for aligning institutional engineering technical vocational education and training practices with industry skills requirements in Kenya. Reviewed Journal International of Education Practice, 4(1), 96-116.
Razalli, N. M., Ahmad, Z. A., Abidin, S. Z. Z., & Musa, H. (2023)Updated Academic Performance Audit (APA) Standards: Implications for Malaysian Polytechnic and Community Colleges.
Ridzuan, M. R., & binti Abd Rahman, N. A. S. (2022). The Analysis Of The Government Policy On Technical And Vocational Education And Training (Tvet) And The Predicaments Of Tvet In Malaysia. International Journal of Humanities Technology and Civilization, 53-58.
Sa, N. T. (2021). The impact of outcome-based philosophy and practice on a group of efl teachers’beliefs and actions. Journal of science and technology-iuh, 52(04).
Saleem, S. M. G., & Gouse, H. S. M. (2019). Comprehensive course-embedded assessment system (CCEAS) to measure program learning outcomes (PLO): Measures that matters. Health Sciences, 8(4), 107-116.
Saleem, S. M. G., & Gouse, H. S. M. (2019). Comprehensive course-embedded assessment system (CCEAS) to measure program learning outcomes (PLO): Measures that matters. Health Sciences, 8(4), 107-116.
Saraya, A., Mardhatillah, A., Fitriani, E. N., & Adiyono, A. (2023). The Role Of Islamic Religious Education Teachers: Problems In Evaluating Student Learning Outcomes In Junior High Schools. Dharmas Education Journal (DE_Journal), 4(2), 565-572.
Schilke, O., Hu, S., & Helfat, C. E. (2018). Quo vadis, dynamic capabilities? A content-analytic review of the current state of knowledge and recommendations for future research. Academy of management annals, 12(1), 390-439.
Seifert, T., & Feliks, O. (2019). Online self-assessment and peer-assessment as a tool to enhance student-teachers’ assessment skills. Assessment & Evaluation in Higher Education, 44(2), 169-185.
Setiawan, D., Sitorus, J., & Natsir, M. (2018). Inhibiting Factor of Primary School Teacher Competence in Indonesia: Pedagogic and Professionalism. Asian Social Science, 14(6), 30.
Setiawan, D., Sitorus, J., & Natsir, M. (2018). Inhibiting Factor of Primary School Teacher Competence in Indonesia: Pedagogic and Professionalism. Asian Social Science, 14(6), 30.
Shen, X., Wang, X., Zhang, Z., & Fei, L. (2021). Does non-conforming urban development mean the failure of zoning? A framework for conformance-based evaluation. Environment and Planning B: Urban Analytics and City Science, 48(5), 1279-1295.
Suo, Z. (2023). Reform Policy on Internationalization of Higher Education in Malaysia:--Analysis based on Malaysia Education Blueprint 2015-2025 (Higher Education). Journal of Education and Educational Research, 5(1), 154-158.
Tan, K., Chong, M. C., Subramaniam, P., & Wong, L. P. (2018). The effectiveness of outcome based education on the competencies of nursing students: A systematic review. Nurse education today, 64, 180-189.
Tenedero, E. Q., & Pacadaljen, L. M. (2021). Learning experiences in the emerging outcomes-based education (OBE) curriculum of higher education institutions (HEI’S) on the scope of Hammond’s evaluation cube. Psychology And Education, 9(58), 2.
Yahia, F. B., Essid, H., & Rebai, S. (2018). Do dropout and environmental factors matter? A directional distance function assessment of tunisian education efficiency. International Journal of Educational Development, 60, 120-127.
Yang, P., Lai, S., Guan, H., & Wang, J. (2022). Teaching Reform and Practice Using the Concept of Outcome-Based Education: A Case Study on Curriculum Design for a Microcontroller Unit Course. International Journal of Emerging Technologies in Learning (iJET), 17(3), 68-82.
Zaki, N., Turaev, S., Shuaib, K., Krishnan, A., & Mohamed, E. (2023). Automating the mapping of course learning outcomes to program learning outcomes using natural language processing for accurate educational program evaluation. Education and Information Technologies, 1-20.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 e-Jurnal Penyelidikan dan Inovasi
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
