1. Introduction

Over last 75 years medical education has evolved a lot. More and more focus has been on providing standard and quality medical education. The quality process in medical education are yet to be established, specially in developing world such as India. Through out the world these processes are different and mostly not imparted on regular and continuous manners, although basic standardisation has been taken care by the respective councils across the globe. The exit mechanisms for quality control are also well in place in many educational environments. Quality Assurances during the programs are not yet established.

One radical thought which has incorporated in educational thinking is that students are either a product or a producer, and in both cases the quality management process are required to be stringently enforced. The quality assurance which need to be internal institutional process are not in places, in the desired manner, as per the learners viewpoint. And that could be the reasons why perceptions and expectations gap are huge in nature between the learners and education managers.

Similarly the feedbacks for quality assurances are not yet established in the medical education and that too specifically in India. Such quality assurance should ensure that a standard educational environment is created and nurtured. But unfortunately it’s very difficult to cater to, especially in the medical education because of variety of reasons including absence of a well-conceived program, which incorporates the steps relevant.

The Academic Appraisal Program (AcAP) at J.N. Medical College DMIMS (DU) may be the break through program for quality assurance fulfilling the most of the components of quality assurance cycle including - primary stakeholder continuous feedback on conduciveness of educational environment as a whole, being bench marked for standardisation. All students participate in this program. The faculty plays facilitator’s role as education lead and academic lead supporting, managing and supervising the quality of learning process in their environment. Three reports are essentially generated -Pre-term, Midterm and Post-term. They are reviewed and monitored by administrators and policy makers, at College council, Joint College Council and Academic Council. The shortfall on any benchmarked objective are identified for suitable remedial actions and followed up with corrective measures.

The objective is to study the Academic Appraisal Program on undergraduate medicine course. To study the role of learner, faculty and education managers/administrators and its efficacy as quality assurance program towards the teaching-learning processes in medical education.

It is a prospective study undertaken between 15th May 2013 to 31st Dec 2015, at J.N.Medical College, Datta Meghe Institute of Medical Sciences (Deemed University), Wardha, involving all the undergraduate students pursuing MBBS (Undergraduate) course.

2. Material and Method

The Academic Appraisal Program as implemented for MBBS course at J.N. Medical College and students pursuing the same during the Academic year 2014-15; and the data extracted from Input forms of Students, Education leads on marker points pertaining to all Major subjects; Pre-term reports, mid term, post term and annual reports as submitted to College councils, Joint College council, Academic council with remedial actions undertaken; Annual reports of -2010,2011,2012,2013,2014 ; Results of summative examination of MBBS Course- Ist year, II year, III rd Part I & Part II.

Intervention tool: The Academic Appraisal Program (version 2.0)

Inclusion: All major subjects (directly evaluated) being taught in MBBS course - reports where the academic appraisal program is implemented and the marker point’s evaluation took take place between 1st July2014 to 30th June 2015.

Exclusion: Reports pertaining to minor subjects (indirectly evaluated).

“Data is statistically analysed towards Objective wise Quantative & Qualitative Program outcomes by linear-correlations such as odds ratio, and paired observations by T test / student ratio and other quality analysis marker as deemed appropriate. Significance will be obtained at 5% level.

3. Observations and Results

Table 1A. Subject wise Ave.MP Scores for Each key Objectives

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View next table

Table 1B. Key Objective Scores

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Table 1C. Subject wise Marker Point analysis

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Table 2A. Subject wise Summative results

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Table 2B. AcAP Score*, Good quality result** and overall passing %

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Table 3. Qualitative Analysis of Program as a whole Average Score on individual objectives and key Appraisal

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Table 4. Key corrections - 5 years (objective -wise)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Total Key Corrective actions: 57 actions; Corrective Actions taken at Departmental Level: 18/57, Action taken At Institutional Level: 16/57 and Action taken at University Level: 23 /57.The corrective action required to be taken were almost uniformly spread across all the levels.

Table 5. Quality Impact

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

4. Discussion

The learning environment is basically spread over 4.5 years with 4 main batches. It is predominately dominated by females with M:F ration of 43:57%. Learners are the key to give and appraise the learning environment. They have been traditionally participating in quality control processes and now also in quality assurances [20-26]^[20]. In this study too the learner have regularly participated in the program. No Marker point was abandoned due to non participation of students. They have been critically able to appraise on each aspect of Learning’s, giving a score on Likart scale of 5 for each objective .They have been categorically able to point the gaps, where-ever learning have not gone well . The Scores reflect that learners have participated in the program with full vigilance and not casually. The learners are wise enough to correlate the deficiency which has been a hindrance to their overall meeting of learning objectives. Various studies echoes the same and very categorically have roles in the learners to give them feedbacks for gaps in their learning environments ^{[27, 28, 29]}. They have been able to identify the difficulties in logistic implementation of the program and have suggested to it to be undertaken as Online program. The “learner centricity” in quality assurance is essential. There are many studies which have been bringing to forth the significance an impact of students participation, feed back and proactive involvement in the educational environment ^{[4, 5, 6, 7, 21, 22, 29, 31, 33, 35, 37, 40, 41, 47, 48, 49, 50, 51]}. This study too reflects and echoes the same. The academic appraisal program has made them to lead the through put - learning process, in very structure red and definite way, assuring that quality is achieved through out the curriculum dispensation.

Role and participation of Faculty: The faculty is very actively involved in the program as education-lead and academic lead. The education lead have regularly participated in the program. They have been able to place the Marker points on the curriculum & syllabus as to be dispensed for the forth coming terms. They have taken up the role regarding timely appraisal of the Marker points and further analyzed the scores highlighting the deficiencies in learning environment, as pointed out by learners. They too have identified the logistic difficulties and suggested for Online Version. The Academic Lead have timely analyzed the reports as submitted by the education leads in a detailed manner, and put forth few actions for policy decisions. The Academic lead has analyzed further and presented in the monthly college councils characterizing the deficiencies/ gaps for remedial action at appropriate levels.The academic lead have taken appropriate actions including urgent actions along with HODs, for immediate intervention. The role of faculty is like a facilitator - guide and supervise both, towards effective dispensation of learning processes, helping to overcome every barrier as pointed out by the learners in attainting their objectives. The same is repeatedly reflected by World Federation of Medical Education ^{[41, 42, 43, 44, 45]}. They have been categorically stating that roles are changing and learners need facilitators to help, guide and achieve the goals and objectives of medical education. The teachers should now merge into such roles.

Role and participation of Administrators /education managers and Policy makers.: The Policy makers have been actively involved in reviewing and monitoring the academic appraisal program. The administrative adherences to the program has been observed to the extent of 98%. The program has never been felt under any threats or have suffered in terms timely dispensations of review on gaps, planning remedies and taking corrective actions. The Academic Council, Joint College Council and College Councils along with HOD, HOI, University officials, Key administrators and Policy makers have intervened in desirable manner including Enhancements in Infrastructure, Curriculums, Syllabus, ICT facilities etc.

The basic onus of Fostering the conduciveness of learning environment towards preservation, sustenance and enhancements; through out the program lies on all the stakeholders and nobody is above a conducive learning environment. The three main pillar of learning environments Learners, Faculties and education keepers all have to come on a uniform platform with uniform concerns, accepting the gaps with big hearts, setting up a monitoring & review processes and prioritizing the corrective actions implementations. If it happens in a structured manner, quality assurance of learning process is ensured ^{[1, 2, 10, 16, 18, 19, 25, 27, 28, 32, 34, 39, 43, 44, 45, 46, 47]}.

The teaching environment planned at J.N.Medical College, is well conceived and in alignment with the Medical Council of India ^[14]. The number of hours are almost 10%-15% more. This is possible because at the DMIMS(DU) has more number of academic days, than as minimum prescribed by University Grant Commission which is 240. In the academic year 2014-15 the number of academic days were 282.This helped in conduction of remedial and extra teachings, further helping in sustaining the academic environment. The classes were held with regularity to the extent of average 99.13%. The main reason for the Classes not able to be held as per plan was mainly, due to the teachers inability. Being in medical field with patient as priority, this was mostly justified due to contingencies rehiring emergency care. Though there is a provision of imposing fine on teachers and students for their unjustified absence. Only in very few instances this was needed to be done. The classes not held were adequately compensated by taking remedial classes. These remedial classes also included those which failed to get a quality approval by not meeting the benchmarked standard. Such events was .08% (7 / 8303). The additional classes including both remedial and capsulated teaching program were held to the extent of 2.25%, hence adequately compensating activities as per the plan but missed or needed reinforcements. The marker points adequately covered the whole syllabus, evenly spread out across, covering the themes and topics as per the needs as per the retirement of the course. The attendance of learners have correlated directly with their performance in exit examination. More they attended the classes more the chances of passing was observed. Many other studies have also concluded the same, that as medical learning’s are more training based, hence the participation of learner is essential for good learning. ^{[7, 8, 12, 13, 23, 53, 54]} The quality assurance measure undertaken to ensure the adequate participation and monitoring through attendance cell is definitely helping the learners to regularly attend the classes.

The results of summative examinations also correlates directly with the quality assurance through AcAP, in almost all the subject with exception of 2 subjects in which despite the low score the result was above the average. These low scores were well above the benchmarked score. This ensured that if a learner assures of having met the learning objectives as per the bench marks in the AcAP than perhaps he has learnt well and will turn out to be a useful product in the community.

The assessments are very important part of the quality control process ^{[17, 26, 27]}.

Exit examinations evaluate the learners against their acquisition of knowledge, its a quality control activity than quality assurances; also its not possible to evaluate the learners against all the knowledge gain opportunity through exit examinations. But its very important aspect as along with the internal quality assurance, the quality control activity complete quality cycle is achieved With Quality assurance like AcAP, the learners led process; and exit assessments which are faculty led processes, a quality circle is created along with counter check mechanism. It is extremely necessary to realign the assessments with the teaching learning process as well ^[24].

The AcAP has been able to bring forth the quality assurances issues very categorically. The key objectives as identified are main focus units on which the learners gains are based. They were able to identify these gains or loss at times and reflect through AcAP to indicate that things are dwindling and should be urgent corrected.

The scores on all the key areas where above the benchmarked standards, indicating that learning environment from learners view point had been conducive to make meaningful gains.

How meaningful where these gains is evolved by looking at outcomes of summative result, as its the program exit point and after that the learner as a product of the institute starts providing services to community. The results corroborate with the findings of AcAP. The GQR is almost mimicking the scores of AcAP. The 4 subjects with highest GQR more than 75% are Microbiology, Ophthalmology, ENT and Paediatrics. In all 3 of them the AcAP score is more than 4.0 and in one 3.98. hence in these subjects a quality enhanced environment is developed and is well reflected in the results too. This validates that AcAP which is learner led and exit examination which is teacher led both concludes almost similarly.

The Average AcAP score for the institution is 3.99 / 5, which is just short of academically enhanced learning environment. The major quality assurance had been that the 64% of learners exited the course as good quality result category. Overall passing rate had been 97.57%, which is strongly correlating that meeting of benchmarked AcAP scores is not a coincidence, learners where very well able to work towards the preservation, sustenance and enhancement. assuring that learning quality is taken care of, throughout the course dispensations. (Figure 1).

Download as

• PPT
  PowerPoint Slide
• PNG
  Larger image(png format)

Veiw figureFigures index

•  NEW
  View larger figure in new window

Figure 1. Key Objectives and Qulaity Impact

The Academic appraisal program is a quality assurance initiative led by learners and evoking all stakeholders -teachers and education managers participation towards developing a conducive academic environment in Medical education.

5. Limitations

The academic appraisal program is an effort for learner led quality appraisal. The outcome and impact is entirely dependent on effective & timely participation of the learners Though mostly its implemented in the manner and with dispositions as its supposed to, but certain limitations has been observed: As the program is learner led. It becomes the major focus for corrections by all other stakeholders and perhaps other angles which may be as important for learning environment shifts to low priority. The objectives may not include all the components of learning environment, including an essential component of evaluation. As examination/evaluation and assessments are the priorities of students, there focus on topics which are more important from that view points take larger concerns than the whole curriculum towards acquiring their learning and competencies.

6. Conclusion

The academic appraisal program is an effective tool towards controlling the learning environment protected zealously by the leaners (product). It has been able to create a platform dedicated towards preservation, sustenance & enhancement of learning environment by all stake holders including Students, Faculties, Administrators, Managers and Policy makers; with learners leading from front. It is a well conceived, planned and implemented Program towards Academic Quality Assurance.

“The Academic Appraisal Program is quality assurance activity, empowering learners to control the learning environment towards effective dispensations as meant & perceived by them.”

References

[1]	M Best and D Neuhauseri: W Edwards Deming: father of quality management, patient and composer Health Care. 2005 Aug; 14 (4): 310-312.
	In article		View Article
[2]	Weiner N,The human use of human being in cybernetics and society, Roston, Houghton Miffin Co. 1950 -71
	In article
[3]	Thompson JL, Thompson AM, McCain FE Jr Effects of optional laboratory and lecture attendance in a medical physiology courseJ Med Educ. 1974 Oct;49(10): 963-72.
	In article		View Article
[4]	Eichna L W : Medical -school education 1975-79, A students perspective, N.Engl J Med. 1980; 303; 727-734.
	In article		View Article  PubMed
[5]	Jack Ende Feed back in clinical Medical Education, JAMA 1983 250, 777-781.
	In article		View Article  PubMed
[6]	L W : Medical -school education 1975-79, A students perspective, N.Engl J Med. 1980; 303; 727-734.
	In article		PubMed
[7]	Glanz K, Fiel SB Effect of attendance at lectures on medical student performance. J Med Educ. 1984 Jun; 59(6): 516-8.
	In article		View Article
[8]	Flournoy DJ, Hyde RM. Lecture attendanceJ Med Educ. 1986 Aug; 61(8): 705.
	In article		View Article
[9]	Ramsey PG, Gillmore GM, Irby DM Evaluating clinical teaching in the medicine clerkship: relationship of instructor experience and training setting to ratings of teaching effectiveness. Gen Intern Med. 1988 Jul-Aug; 3(4): 351-5.
	In article		View Article
[10]	Holm HA1, Tidsskr Nor Laegeforen, Quality assurance of medical education--a prerequisite for good medical practice, 1992 Nov 20; 112(28): 3565-9.
	In article
[11]	Ramsbottom-Lucier MT1, Gillmore GM, Irby DM, Ramsey PG.,Evaluation of clinical teaching by general internal medicine faculty in outpatient and inpatient settings. Acad Med. 1994 Feb; 69(2): 152-4.
	In article		View Article  PubMed
[12]	Hammen CS, KellandJL.Department of Zoology, University of Rhode Island, Kingston 02881Attendance and grades in a human physiology course Am J Physiol. 1994 Dec;267(6 Pt 3):S105-8.
	In article		PubMed
[13]	Riggs JW, Blanco JD Is there a relation between student lecture attendance and clinical Science subject examination score? Obstet Gynecol. 1994 Aug;84(2):311-3.
	In article		PubMed
[14]	Graduate Medical education regulations, 1997 (published in PartIII, section 4, of the gazette of India dated 17th May 1997 (amended unto February 2012) available at website- https://www.mciindia.org/RulesandRegulations/Graduate MedicalEducationRegulations1997.aspx.
	In article
[15]	SteinertY.University of Texas Medical Branch, Division of Gastroenterology, Galveston, TX 7755-0764, USA. Using student focus groups to improve faculty performance. Acad Med. 2001 May;76(5):574-5.
	In article		View Article  PubMed
[16]	Biggs J. The reflective institution: Assuring and enhancing the quality of teaching and learning. Higher Educ. 2001;41:221-38.
	In article		View Article
[17]	ElzubeirM, Rizk.D Evaluating the quality of teaching in medical education: are we using the evidence for both formative and summative purposes, Med teach.2002 May ;24 (3);313-9.
	In article		View Article  PubMed
[18]	Sallis E. Total Quality Management in Education. London: Kogan Page Ltd; 2002.
	In article
[19]	LyntanGray. Marketing Education 1991 Publisher: Open UnivPr (September 1991) ISBN-10: 033509676X ISBN-13: 978-0335096763.
	In article
[20]	Stark P.Teaching and learning in the clinical setting: a qualitative study of the perceptions of students and teachers. Med Educ. 2003 Nov;37(11):975-82.
	In article		View Article  PubMed
[21]	Darlaston Jones D, Pike L, Cohen L, Young A, Haunold S, Drew N. Are they being served? Student expectations of higher education. Issues Educ Res. 2003.
	In article
[22]	Breen, L., Cohen, L., Haunold, S., Pike, L., Young, A. (2001, August). Peer mentoring for university students: Is group mentoring as effective as one-to-one mentoring?. Paper presented at 16th Annual WAIER Research Forum, Perth, Australia. ( cross reference).
	In article
[23]	DhaliwalU. Absenteeism and under-achievement in final year medical students. Natl Med J India. 2003 Jan-Feb;16(1):34-7.
	In article		PubMed
[24]	Bloomfield L, Harris P, Hughes C School of Public Health and Community Medicine, Faculty of Medicine, Curriculum Unit, University of New South Wales, Sydney, Australia. L.Bloomfield@unsw.edu.auWhat do students want? The types of learning activities preferred by final year medical studentsMed Educ. 2003 Feb; 37(2):110-8.
	In article
[25]	Lockwood JH, Sabharwal RK, Danoff D, Whitcomb ME.Quality improvement in medical students' education: the AAMC medical school graduation questionnaire.Med Educ. 2004 Mar;38(3):234-6.
	In article		View Article  PubMed
[26]	Mayya S, Roff S. Students' perceptions of educational environment: a comparison of academic achievers and under-achievers at kasturba medical college, India. Educ Health (Abingdon). 2004; 17(3): 280-91.
	In article		View Article  PubMed
[27]	Savage C, Uhre S, Kovać K, Wennekes V, Jensen-Dahm C. Quality Assurance in Medical Schools, Moving from Quality Assurance to Quality Improvement Quality Assurance Workshop EMSA/IFMSA, Copenhagen, Denmark July 6-10, 2005 version 4.3. Available from: https://wiki.ifmsa.org scome/index.php?title = Quality_Assurance_in_Medical_Schools
	In article
[28]	United Nations Educational Scientific and Cultural Organization. Paris: 2005. Guidelines for Quality Provision in Cross-border Higher Education.
	In article
[29]	Grime P, Williams S, Nicholson S.Medical students' evaluation of a teaching session in occupational medicine: the value of a workplace visit.Occup Med (Lond). 2006 Mar;56(2):110-4. Epub 2005 Dec
	In article		View Article  PubMed
[30]	Steinert Y1, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, Prideaux DA systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8.Med Teach. 2006 Sep;28(6):497-526.
	In article		View Article  PubMed
[31]	Masić I1, Dedović-Halilbegović G, Novo A, Izetbegovíc S Quality assesment of education at Faculty of Medicine University of Sarajevo. Med Arh. 2006; 60(6):396-400.
	In article		PubMed
[32]	Karle H. Global Standards and Accreditation in Medical Education: A View from the WFME” Supplement on the occasion of the ECFMG 50th Anniversary Invitational Conference “Impact of International Medical Graduates on US and Global Health Care” Acad Med. 2006; 81(Suppl): 43-8. New Manual for Assessment and Accreditation – For Health Science Institution.
	In article		View Article  PubMed
[33]	Zuberi RW, Bordage G, Norman GR Validation of the SETOC instrument -- Student evaluation of teaching in outpatient clinics.AdvHealthSciEduc Theory Pract. 2007 Feb; 12(1):55-69. Epub 2006 Dec 8.
	In article		View Article  PubMed
[34]	Masić I, Novo A, Deljković S, Omerhodzić I, Piralić A. How to assess and improve quality of medical education: lessons learned from Faculty of Medicine in Sarajevo. Bosn J Basic Med Sci. 2007 Feb;7(1):74-8
	In article		PubMed
[35]	Van de Ridder JM1, Stokking KM, McGaghie WC, ten Cate OT What is feedback in clinical education? Med Educ. 2008 Feb;42(2):189-97.
	In article		PubMed
[36]	Kebriaei A, Akbari F. Quality gap of educational services at Zahedan University of Medical Sciences, Iran. Bangladesh Med Res Counc Bull. 2008; 34(3):76,26–80.
	In article
[37]	Aghamolaei T, Zare S. Quality gap of educational services in viewpoints of students in Hormozgan University of medical sciences. BMC Med Educ. 2008;8:34.
	In article		View Article  PubMed
[38]	BerkRA.Using the 360 degrees multisource feedback model to evaluate teaching and professionalism.Med Teach. 2009 Dec; 31(12):1073-80.
	In article		View Article  PubMed
[39]	Elam P, Scales M, Pearson D Do visits help improve the quality of teaching within undergraduate teaching practices? Educ Prim Care. 2009 Nov;20(6):429-34.
	In article		View Article  PubMed
[40]	Brochado A. Comparing alternative instruments to measure service quality in higher education. QualAssur Educ. 2009; 17(2):174-90.
	In article		View Article
[41]	Bell L, Neary M, Stevenson H. The future of higher education: policy, pedagogy and the student experience. London: Continuum International Publishing Group; 2009.
	In article
[42]	Aghamolaei T, Fazel I. Medical students' perceptions of the educational environment at an Iranian Medical Sciences University. BMC Med Educ. 2010; 10:87.
	In article		View Article  PubMed
[43]	WFME Global Standards for Quality Improvement in Continuing Professional Development CPD of Medical Doctors. Updated 2011 Jul 14. Available from: https://www.wfme.org/standards/cpd
	In article
[44]	WFME Global Standards for Quality Improvement in Postgraduate Medical Education. Updated 2011 Jul 14 Available from: https://www.wfme.org/ standards/pgme.
	In article
[45]	WFME Global Standards for Quality Improvement in Basic Medical Education BME Updated 2011 Sep 13,. Available from: https://www.wfme.org/standards/ bme.
	In article
[46]	Promoting excellence: standards for medical education and training; https://www.gmc-uk.org/education/standards.asp; accessed on 3rd Feb2016.
	In article
[47]	Medha A. Joshi Quality assurance in medical education Indian J Pharmacol. 2012 May-Jun; 44(3): 285-287.
	In article		View Article  PubMed
[48]	VarunKohli† and UpreetDhaliwal*Medical students’ perception of the educational environment in a medical college in India: a cross-sectional study using the Dundee Ready Education Environment questionnaireJ EducEval Health Prof. 2013; 10: 5.Published online 2013 Jun 30.
	In article
[49]	Ong WM, Nankervis A. Service quality in higher education: Students’ perceptions in Australia and Malaysia. Rev Integr Bus Econ Res. 2012; 1(1):277-90.
	In article
[50]	Bahram Nabilou and Davoud Khorasani-Zavareh, The Bridge Between Real and Ideal: Students Perception on Quality Gap in Reality and Their Educational Expectations Iran Red Crescent Med J. Sep 2014; 16(9): e14254.Published online Aug 10, 2014.
	In article
[51]	Rahim Khanli M1, Daneshmandi H2, ChoobinehA The students' viewpoint on the quality gap in educational services. J Adv Med Educ Prof. 2014 Jul; 2(3):114-9.
	In article		PubMed
[52]	MCI Vision 2015 (2011) [Last cited 2011 Dec 21]. Available from: https://www.mciindia.org/tools/announcement/MCI_booklet.pdf.
	In article
[53]	Tripti K Srivastava, Sandeep Shrivastava, Lalitbhusan S Waghmare, Impact of Attendnace Policy in rural medical school: a 3 year observational study Indian Journla of applied research Vl:4,issue 6 june2014 Page59-62, Issn2249-555X.
	In article
[54]	Sandeep Shrivastava, Relevance of “Quality Assurance” in Education Environment” Editorial, JHSE Vol 2 No 2.
	In article