|Year : 2022 | Volume
| Issue : 3 | Page : 161-164
The necessity, barriers and strategies to overcome the barriers in evaluation in medical education
Saurabh RamBihariLal Shrivastava1, Prateek Saurabh Shrivastava2
1 Deputy Director – Academics, Sri Balaji Vidyapeeth – Deemed to be University, Medical Education Unit Coordinator and Member of the Institute Research Council; Department of Community Medicine, Shri Sathya Sai Medical College and Research Institute, Ammapettai, Nellikuppam, Chengalpet District, Tamil Nadu, India
2 Department of Community Medicine, Shri Sathya Sai Medical College and Research Institute, Sri Balaji Vidyapeeth – Deemed to be University, Ammapettai, Nellikuppam, Chengalpet District, Tamil Nadu, India
|Date of Submission||16-Aug-2021|
|Date of Decision||01-Sep-2021|
|Date of Acceptance||03-Sep-2021|
|Date of Web Publication||26-Dec-2022|
Dr. Saurabh RamBihariLal Shrivastava
MD, Professor, Department of Community Medicine, Shri Sathya Sai Medical College and Research Institute, Sri Balaji Vidyapeeth (SBV) – Deemed to be University, Thiruporur - Guduvancherry Main Road, Ammapettai, Nellikuppam, Chengalpet - 603 108, Tamil Nadu
Source of Support: None, Conflict of Interest: None
The curriculum delivered to medical students as a part of their training program has to be multi-faceted, dynamic and should have the component of quality assurance and continuous quality improvement to ensure that competent medical graduates are produced. This review has been carried out to understand the necessity, acknowledge the presence of barriers, and identify the strategies that can be planned to ensure systematic evaluation in medical education. An extensive search of all materials related to the topic was carried out in the PubMed search engine and a total of 15 research articles were included in the review. Keywords used in the search include evaluation and medical education in the title alone only. There are no doubts that evaluation of medical education is an indispensable component of the learning cycle. Nevertheless, we cannot ignore the fact that the overall process of evaluation is complex, predominantly because of the long time interval between the introduction of the educational intervention and the learning outcome. We realize that there can be multiple barriers in the evaluation process in the field of medical education, but we have to overcome these challenges to ensure that the process of evaluation eventually benefits the administrators, teachers and the students. In conclusion, evaluation in medical education is an indispensable component of the training process. However, considering that multiple factors can determine the quality of evaluation, it is essential that specific steps are taken to eliminate these factors and ensure that the findings are valid and that they can be used for remedial actions.
Keywords: Evaluation, medical education, questionnaire
|How to cite this article:|
Shrivastava SR, Shrivastava PS. The necessity, barriers and strategies to overcome the barriers in evaluation in medical education. J NTR Univ Health Sci 2022;11:161-4
|How to cite this URL:|
Shrivastava SR, Shrivastava PS. The necessity, barriers and strategies to overcome the barriers in evaluation in medical education. J NTR Univ Health Sci [serial online] 2022 [cited 2023 Feb 7];11:161-4. Available from: https://www.jdrntruhs.org/text.asp?2022/11/3/161/365007
| Introduction|| |
The curriculum delivered to medical students as a part of their training program has to be multi-faceted, dynamic and should have the component of quality assurance and continuous quality improvement to ensure that competent medical graduates are produced. Medical education is not only about the delivery of the syllabus to the students, but essentially includes training on non-cognitive attributes (viz. professionalism, altruism, ethics, teamwork, leadership, communication skills, etc.) and introduction of new initiatives in the field of teaching-learning and assessment. Apart from the various quality control initiatives that can be planned by the institution, it becomes really essential to periodically evaluate the program to ascertain the advantages and limitations of the ongoing initiatives., This review has been carried out to understand the necessity, acknowledge the presence of barriers and identify the strategies that can be planned to ensure systematic evaluation in medical education.
| Methods|| |
An extensive search of all materials related to the topic was carried out in the PubMed search engine. Relevant research articles focusing on patient involvement in medical education published in the period 2006 to 2021 were included in the review. A total of 17 studies similar to current study objectives was identified initially, of which, 2 were excluded due to the unavailability of the complete version of the articles. Overall, 15 articles were selected based upon the suitability with the current review objectives and analyzed. Keywords used in the search include evaluation and medical education in the title alone only (viz. evaluation [ti] AND medical education [ti]). All the articles published in the English language were only selected for the review [Figure 1]. The collected information is presented under the following sub-headings, namely Necessity of evaluation in medical education, Potential barriers in evaluation, Strategies to improve evaluation, Implications for practice and Implications for research.
Necessity of evaluation in medical education
There are no doubts that evaluation of medical education is an indispensable component of the learning cycle. This is essential as it is quite difficult to implement any initiative in the same fashion as it has been planned during the planning stage., The potential change during the implementation phase can result because of lack of communication of the complete protocol, ambiguity in the assigned roles, interference from the administrators, lacunae in enforcement, logistics concerns, the absence of financial support, etc.,, It won't be wrong to note that in many settings, evaluation is being conducted promptly, just to highlight the fact that their initiative has demonstrated better outcomes. Furthermore, it is quite possible that based on the results of the evaluation, appropriate remedial measures can be taken to improve the learning outcomes.,,,
Potential barriers in evaluation
Nevertheless, we cannot ignore the fact that the overall process of evaluation is complex, predominantly because of the long time interval between the introduction of the educational intervention and the learning outcome. At the same time, there are multiple challenges/barriers in the evaluation, such as using a tool that can be easily used (viz. using a questionnaire instead of a qualitative method that definitely provides in-depth information, but is time-consuming and require more efforts), high rates of non-response from the stakeholders (which tends to have an impact on the remedial measures), exclusion of participants from analysis, selection of an inappropriate instrument for evaluation, and absence of a long-term follow-up.,,,
Moreover, the evaluation process can be significantly affected during the process of data collection (viz. lack of participation from everyone, poor maintenance of records, high dropout rate, ambiguous questions in the tools, asking more than one question in a single question, etc.).,, In addition, it is quite important to ensure the reliability of the instrument, as a tool that is not reliable, we cannot draw generalizable conclusions. Further, there can be even ethical issues, wherein responses or opinions of people who are in the minority is ignored or giving undue weightage to the opinions of people who occupy important positions, or adhering to only the views of the administrators, even when the available responses suggest otherwise., Also, there can be issues with regard to failure to interpret the results of evaluation, thereby making the entire process not useful.,
Strategies to improve evaluation
We realize that there can be multiple barriers in the evaluation process in the field of medical education, but we have to overcome these challenges to ensure that the process of evaluation eventually benefits the administrators, teachers and the students., The process of evaluation has to begin with the identification of the clear purpose of evaluation and the various elements that need to be considered during the same.,, A committee needs to be formulated that will carry out the evaluation process and the roles and responsibilities of each and every member have to be explicitly stated. [2,3] These members should be trained in all the domains of evaluation, so that they can discharge their roles independently. It is quite essential that the expected outcomes (both measurable and unmeasurable) need to be defined and that appropriate tools for evaluation should be selected. [1,2]
Further, the selected tool has to be validated and if required, pre-testing can also be done. The evaluators should ensure that the selected tool (viz. questionnaire, feedback form, case study, etc.) of evaluation is reliable. In addition, it is essential that all the concerned stakeholders are involved in the entire process of evaluation and a specific amount of budget should also be allocated to warrant that the evaluation can be executed without any interruptions., It is a good strategy to carry out periodic evaluation during the process of implementation itself. Moreover, different tools should be used to ensure 360-degree evaluation and opinion from all the stakeholders should be obtained.,, For better and complete responses, teachers can be trained in preparing an effective survey form.,, Finally, we must realize that evaluation should not end with obtaining feedback, and has to be coupled with follow-up actions to ascertain whether the program has been adequately modified or not.,,, In a study to evaluate the Canadian undergraduate ophthalmology medical education in the third year at Western University, it was reported that 88% of students completed the pretest, post-test, and Ophthalmology Education Survey. Further, it was observed that the additional ophthalmology lecture series proved to be fruitful in not only enhancing the knowledge of the students, but even resulted in an improvement in the desire of the students to learn better.
Implications for practice
The process of evaluation is an indispensable component of medical education delivery and becomes essential in the vision to continuously improve the quality of curriculum and its delivery. At Shri Sathya Sai Medical College and Research Institute, a constituent unit of the Sri Balaji Vidyapeeth, Deemed-to-be-University, Puducherry, significant attention has been paid to the process of evaluation. This evaluation has been carried out by the Feedback Committee, constituted under the directions of the Internal Quality Assurance Cell, wherein feedback have been regularly obtained from different stakeholders (viz. feedback from student on curriculum, feedback from student on teachers, feedback from teachers on curriculum, etc.), and depending on the received feedback, a specific Action Taken Report has been prepared under the directions of the Dean. The entire process has been carried out with an aim to improve the curriculum delivery for the subsequent batches and thereby benefit the students.
Implications for research
As quality improvement has to be a continuous process, there is ample amount of scope for research in the evaluation of medical education. The planned research can target obtaining feedback from different stakeholders for improving the overall process of planning, implementation and curriculum mapping. There is always a scope to conduct mixed methods or purely qualitative study (using approaches like focus group discussions or in depth interviews or key informant interviews, etc.) to get deep insights about the overall process and therefore plan for remedial measures accordingly.
| Conclusion|| |
In conclusion, evaluation in medical education is an indispensable component of the training process. However, considering that multiple factors can determine the quality of evaluation, it is essential that specific steps are taken to eliminate these factors and ensure that the findings are valid and that they can be used for remedial actions.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Sandars J. It is time to celebrate the importance of evaluation in medical education. Int J Med Educ 2018;9:158-60.
Yang XF, Talmy T, Zhu CH, Li PF, Wang W, Zhang P, et al
. Evaluation of teaching and learning: A basis for improvement in medical education. Chin Med J (Engl) 2017;130:1259-60.
Barteit S, Guzek D, Jahn A, Bärnighausen T, Jorge MM, Neuhann F. Evaluation of e-learning for medical education in low- and middle-income countries: A systematic review. Comput Educ 2020;145:103726.
Crampton P, Mehdizadeh L, Page M, Knight L, Griffin A. Realist evaluation of UK medical education quality assurance. BMJ Open 2019;9:e033614.
Schiekirka S, Feufel MA, Herrmann-Lingen C, Raupach T. Evaluation in medical education: A topical review of target parameters, data collection tools and confounding factors. Ger Med Sci 2015;13:Doc15.
Lurie SJ, Mooney CJ, Lyness JM. Commentary: Pitfalls in assessment of competency-based educational objectives. Acad Med 2011;86:412-4.
Mirzazadeh A, Gandomkar R, Hejri SM, Hassanzadeh G, Koochak HE, Golestani A, et al
. Undergraduate medical education programme renewal: A longitudinal context, input, process and product evaluation study. Perspect Med Educ 2016;5:15-23.
Dreiling K, Montano D, Poinstingl H, Müller T, Schiekirka-Schwake S, Anders S, et al
. Evaluation in undergraduate medical education: Conceptualizing and validating a novel questionnaire for assessing the quality of bedside teaching. Med Teach 2017;39:820-7.
Wasfy NF, Abouzeid E, Nasser AA, Ahmed SA, Youssry I, Hegazy NN, et al
. A guide for evaluation of online learning in medical education: A qualitative reflective analysis. BMC Med Educ 2021;21:339.
Shrivastava SR, Shrivastava PS. Exploring the data required for evaluation of a competency based medical education curriculum in India. Indian J Med Spec 2019;10:114-5. [Full text]
Oandasan I, Martin L, McGuire M, Zorzi R. Twelve tips for improvement-oriented evaluation of competency-based medical education. Med Teach 2020;42:272-7.
Musick DW. A conceptual model for program evaluation in graduate medical education. Acad Med 2006;81:759-65.
Perkins SQ, Dabaja A, Atiemo H. Best approaches to evaluation and feedback in post-graduate medical education. Curr Urol Rep 2020;21:36.
Li B, Curts D, Iordanous Y, Proulx A, Sharan S. Evaluation of Canadian undergraduate ophthalmology medical education at Western University. Can J Ophthalmol 2016;51:373-7.
Jebraeily M, Pirnejad H, Feizi A, Niazkhani Z. Evaluation of blended medical education from lecturers' and students' viewpoint: A qualitative study in a developing country. BMC Med Educ 2020;20:482.