Open ended questions that result in long or short written comments are an important part of the TCE system. The standard written comment "core" questions in the TCE system ask what students liked best about the course and what they would suggest to improve the course. They are intended for the instructor's benefit and results are returned directly to the instructor (online questions are reported verbatim in a PDF transcript).
Departments can request addtional open-ended questions in online TCEs. Departmentally added open-ended question results are disclosed to both the instructor and the department.
The use of written comments should be approached with several important considerations regarding their uses.
Student Written Comments in Faculty Teaching and Course Development:
Formative evaluation is a process which aims to use feedback solely for development or improvement of teaching skills or course design. The OIA encourages faculty to share student written comments with faculty peer mentors and/or teaching improvement consultants (e.g. OIA consultants).
Student Written Comments in Official Faculty Performance Evaluation:
Consequential performance appraisal is the most common and important example of summative evaluation of faculty. Annual review, merit review, special reviews, and promotion and tenure are each examples of consequential performance appraisal. Arguably, the stakes are highest for pretenure and adjunct faculty as well as graduate teaching assistants. Thus the role of student comments here is simultaneously the most intuitively appealing since the meaning of words may seem more accessible than numeric summaries and the riskiest in terms of drawing valid inferences about teaching performance or course quality. The greatest liability in the use of comments is that a single articulate (anonymous) student comment is given unsupported credence and/or undue weight.
In the literature of faculty evaluation research and practice, there are essentially two areas of best practice regarding the use of student written comments. Based on that, the OIA makes the following recommendations.
- First, the OIA recommends faculty use well-selected comments to illustrate or bring home points that are supported by other, valid and reliable data when writing their own reports (or teaching porfolios). For example, if TCE quantitative results show one's average ratings to be significantly above those in comparable courses, quoting well-selected eloquent student comments, or report the results of a systematically performed content analysis of comments overtime is an excellent way to demonstrate the intersection of one's teaching philosophy, teaching methods, and its impact on students.
- Second, the OIA recommends that departments using written comments in consequential performance appraisal take care to use empirically valid, systematic content analysis that accounts for sampling and uses a common rubric to intepret and score comments. One characteristic of student written comments widely observed in the research literature is that students with the most extreme opinions, pro or con, tend to write more comments with the majority of students writing very few comments, making the comment data poorly sampled at best. Another characteristic is to respond broadly on a wide range of topics, again making the comments difficult to generalize from concerning any one finding.
- Departments should also make explicit disclosures to faculty and student who offer comments concerning who will read the comments and how those comments will be used.
Without these special precautions, the use of student comments in performance appraisal by departments is strongly discouraged.