Overview 

The MD Program relies on various sources of information to provide feedback on the quality of the program as a whole, on individual components including courses, and on individual teachers. This feedback enables evidence-based, continuous quality improvement of the program and student experience. It is also a core element of a faculty member’s teaching dossier, which is used for promotion and related purposes. One of the chief sources of such feedback is data obtained from students via teacher assessments and course evaluations.

Curriculum leaders should work with student course representatives to ensure that the importance of timely evaluation completion is well understood. Course directors should communicate their evaluation expectations to students at the beginning of the course and at subsequent points as necessary. 

Principles 

1. One of the most powerful and effective tools used to assess the quality and effectiveness of the MD Program curriculum and its teachers is constructive student feedback.
2. Students in the MD Program are in training to enter a profession that relies to a considerable extent on collegial critique for self-improvement. Giving effective feedback and responding to feedback are competencies essential for effective self-regulation.  
3. The MD Program endeavours to educate medical students in a manner that fosters the development of competencies essential for effective self-regulation.  
4. Students are essential partners in the education program and should contribute to the planning and implementation of a reasonable, required program of course evaluation and teacher assessment.
5. The time required to complete assessments of teachers and evaluation of courses should be minimized by ensuring:
   1. That the process of completion of forms be as easy as possible, including:
      1. That the forms be concise and only include essential information.
      2. That whenever possible, dedicated time be set aside during school hours for students to complete course evaluations and teacher assessments.
      3. That the forms be available for completion on a variety of technological platforms.
   2. That the number of students required to complete the forms be determined with regard to statistical principles. 
   3. That reminders to complete any forms be limited to no more than once per week.

Expectations

1. In light of the preceding principles, students are to evaluate all activities, faculty teaching events and faculty/residents interactions where they have substantial contact. The minimum number of clinical teacher assessments should be no less than three. In courses where there is substantial contact with residents it is also expected that students will complete no less than three resident assessments. Students are also expected to complete all end of course evaluations.
2. Students are expected to complete all evaluations forms upon receipt of the request and will receive reminders every two weeks. Evaluations forms must be submitted prior to a cut-off of 15 days from the time of receipt of the original request. The cut-off date is intended to ensure that feedback data remains valid, is not unduly influenced by recall bias, and is available in a timely manner to facilitate quality improvement activities.
3. Completion of course evaluation and teacher assessment forms will be monitored by the central MD Program administration. Clerkship students who have not completed the end of course evaluation will not have electronic access to assessments completed on MedSIS pertaining to their own performance until they have submitted the required evaluations in that course. If a student does not meet this requirement (completing end of course evaluation) they may still access their own assessment by scheduling a meeting with the course director at which time they should be prepared to discuss why they have not completed their evaluations as requested.
4. If students encounter a technical difficulty that hinders the completion of an evaluation form, it is their responsibility to bring this problem to the attention of the course administrator, course director, or technical staff in a timely manner.

Standards for the timely release of teacher assessment scores and feedback

The MD Program places great value on the commitment of the many teachers who contribute to the education of our students. In recognition of their efforts, student assessment of teacher effectiveness scores and other formal feedback will be made available to teachers within two months of the end of the course (in Foundations) and within two months of the end of the academic year (in Clerkship). The MD Program will facilitate the provision of each teacher’s student assessment of teacher effectiveness scores to the relevant University Department Chair(s).

Teacher assessment data will, however, only be released when a minimum of three assessments have been received for a given teacher for each learning activity in order to protect the confidentiality of the students who provided the feedback.

Courses that run for a prolonged period of time (particularly the entire length of the academic year) and courses with multiple rotations are encouraged to share interim or informal feedback earlier when this can be done without compromising student anonymity.

Failure to meet the two-month deadline will be brought to the attention of the Foundations Director or Clerkship Director as appropriate, and if necessary the Associate Dean, MD Program and/or the relevant Department Chair.

Standards for the use of teacher assessment scores and feedback

Student assessment of teacher effectiveness scores and other evaluation feedback about individual teachers must not be disclosed to those outside of the MD Program, nor to individuals within the MD Program, who do not have the authority to access that data. The only exceptions are when the disclosure is required by official MD Program business, by University policy, or by law.

Letters of reference or external award nominations written by MD Program leaders for teachers must not contain student assessment of teacher effectiveness scores or student comments retrieved from evaluation forms without the specific consent of the teacher.

Individuals aware of inappropriate disclosure of teacher assessment information outside of the MD Program should inform the Associate Dean, MD Program as soon as possible.

Teacher assessment appeals process

MD Program teachers have the right to request an appeal of their teacher assessments. Included below are guidelines for appeal requests and the adjudication appeal requests, including the reporting process.

Appellant Responsibilities:

1. Appeal requests are to be directed to the attention of the Director Program Evaluation, Medical Education (md.oae@utoronto.ca) and copied to the appellant’s Clinical Chief and Departmental Chair/Divisional Head, and the Course Director.
2. Appeal requests must be submitted no more than one year after the release of the assessments in question.
3. Notices of such requests are to provide a rationale for such requests.

Process & Reporting:

1. The teacher assessment in question as well as all other relevant teacher assessment records are compiled by the Office of Assessment and Evaluation for review by an ad hoc four-member Appeals Committee, chaired by the Director of Program Evaluation, Medical Education and includes both faculty and student representatives. This committee convenes as required.
2. Reviews are limited to appeal requests submitted by the deadline indicated above, and which pertain to teaching within the immediately preceding academic year unless more than one year of data was required to reach an aggregate of three assessments. 
3. Teacher assessments are treated as a single unit (quantitative and qualitative). If successful, the outcome of the appeal will include the elimination of the assessment in question.
4. All outcomes are considered final and are reported to the appellant and copied to the appellants’ respective Clinical Chief and Departmental Chair /Divisional Head as either supported or denied.
5. Students will not normally be notified when an appeal is made, nor will they be notified regarding the outcome of the appeal.
6. A summary of all appeals and their outcomes will be provided to the Associate Dean, MD Program on a yearly basis.

Standards & Guiding Principles:

To ensure uniformity and fairness, the committee relies on standards in its adjudication process that may include:

1. Face validity:
   1. A presentation of reasonably refuting evidence.
   2. Whether the feedback provided refers to the rotation or program rather than to the specific faculty member.
   3. Obvious transposition of scale ratings or mistaken identity.
2. For assessments in question, additional considerations may include:
   1. There is clear retribution by a trainee (e.g., the comments given by the trainee refer directly to the scenario in question; the comments given by a trainee align timing-wise with feedback they received from the faculty in question; there is a larger pattern of retaliatory assessments from a student directed at multiple faculty).
   2. There is clear evidence of discrimination by a trainee (e.g., the comments given by the trainee refer to one of the prohibited grounds under the Ontario Human Rights Code directly or to attributes/behaviours whose mention is likely related to those grounds, e.g., physical appearance).
   3. The degree of contact between Teacher and Trainee is reasonable for purposes of rendering an assessment of Teaching Effectiveness.
   4. There are personal issues arising between faculty and a learner leading to conflict, which may influence the learner’s assessment of the teacher.
   5. There is substantiation of low scores (1 or 2) by narrative comments.

In circumstances where arguments for and against upholding an appeal are balanced, the resolution will be to favour the appellant.

System error teacher assessment appeals process:

Instances may arise where a Course or Component Director identifies that the teacher assessment was submitted erroneously (due to a systems error beyond the teacher's control).  Examples of a system error include receiving low ratings as a result of miskey issues (ratings do not correspond with comments), or being assessed for the incorrect teaching activity. Included below are guidelines for appeal requests and the adjudication of appeal requests, including the reporting process.

Course or Component Director Appellant Responsibilities:

1. The Course or Component Director who has identified the incorrect assessment will formally notify the Director Program Evaluation, Medical Education by email (md.oae@utoronto.ca). There is no need to copy in the Clinical Chief and Departmental Chair/Divisional Head for the teacher in question.
2. Appeals requests must be submitted no more than one year after the release of the assessment in question.
3. The submission should include: i) the name of the teacher who received the low assessment, ii) the name and date of the session, and iii) a description of the system error.

Process & Reporting:

1. Teacher assessment records related to the teacher who has received the incorrect assessment are compiled by the Office of Assessment and Evaluation. The records are analyzed alongside the submitted material by the Office of Assessment and Evaluation Program Evaluation Director, Senior Analyst, and Manager, to verify that the low or incorrect assessment resulted from a system error.
2. If the outcome of the appeal is determined to be a system error, the assessment in question will be eliminated. 
3. All outcomes are considered final and are reported to the Course or Component Director who submitted the appeal and the teacher whose performance will be revised. 
4. Students will not normally be notified when the appeal is made, nor will they be notified regarding the appeal's outcome. 
5. A summary of the appeal case and the submitted documentation will be kept as part of the Office of Assessment and Evaluation records.

Low-Score Evaluations With No Comments Appeals Process:

This appeals approach is to address teacher assessments with NO substantiation of low scores (1 or 2) by narrative comments. Assessments with no comments do not offer insights into how the teacher can improve, and it is difficult to discern if there is any evidence for the low score provided.

Course or Component Director Appellant Responsibilities:

1. The Course or Component Director who has identified the low assessment with no substantiation by narrative comments will formally notify the Director Program Evaluation, Medical Education by email (md.oae@utoronto.ca). There is no need to copy the Clinical Chief and Departmental Chair/Divisional Head for the teacher in question.
2. Appeals requests must be submitted no more than one year after the release of the assessment in question.
3. The submission should include: i) the name of the teacher who received the low assessment with no comments, ii) the name and date of the session.

Process & Reporting:

1. The OAE will compile all teacher assessment records for the teacher who has received the low score to verify that no comments were provided despite the low score.
2. If successful, the outcome of the appeal will include the elimination of the assessment in question.
3. All outcomes are considered final and are reported to the Course/Component Director who submitted the appeal and the teacher whose performance was revised.
4. Students will not normally be notified when the appeal is made, nor will they be notified regarding the appeal’s outcome.
5. A summary of the appeal case and the submitted documentation will be kept as part of the OAE records.