Improving the Evaluation of College Teaching

Source: Paige Herrboldt, used with permission

Colleges and universities all over the United States are striding through the fall term. Some are already at midterms, six weeks in. Others have just started within the last week. Millions of students are sitting in classes (yes, in person after some turbulent pandemic years) being taught by thousands of educators. Most of them probably share their views of class with friends and family. Whereas I touched on the history of evaluating teaching in a post here earlier this summer, this is a good time to take a further look at college teaching.

There is a very common way colleges and universities measure teaching. Most discussions revolve around the use of student evaluations of teaching (SETs). For readers outside of academia, note that when an instructor who is hired in a tenure-track job is up for promotion or tenure, external letter writers receive dossiers of information, also looked at by internal committees. For fixed-term instructors and for yearly evaluations of teaching, SETs are often the key. To fully capture the hard work that is teaching, we need to change how we evaluate and reward teaching.

What is the purpose of evaluating teaching?

Teaching is primarily discussed (when discussed) in the context of determining tenure or merit raises. Some faculty handbooks suggest evaluators complement SET scores with peer observation reports, self-reflections, and often a range of course materials.

What many of these processes miss, is that a good evaluation should serve as a vehicle for improvement. The process should help the instructor improve, help them help their students improve, and can help the department and university better fulfill their charge.

When one considers this major purpose of evaluation, one sees that overreliance on SETs, like overreliance on grades in the measurement of learning, is misguided. Alternatives to grading are receiving much-needed attention, as the “ungrading” revolution indicates (see Blum, 2021). It is time for the same revolution in teaching evaluation.

What constitutes effective teaching?

Most faculty handbooks describe what teachers should be doing. The language and aspirations are commendable, but they belie the fact that most faculty do not receive training in how to be effective teachers or how to document effectiveness.

Studies of exceptional teachers (Bain, 2004) and detailed examinations of the evidence of model teaching show that the fundamental hallmarks of effective teaching are clear: strong course design (assessments and course activities that map onto explicit student learning outcomes); clear, student-centered syllabi; instructor knowledge of content; the use of effective instructional methods (e.g., fostering active learning); and inclusive teaching practices. Some of these hallmarks of effective teaching can be demonstrated by a collection of course materials showing evidence of the practices used. Missing are adequate pedagogical training for these areas and effective ways to document them.

One feature insufficiently documented is student learning. While some handbooks may note that where obtainable, evidence of student learning enhances evaluation, this prescription rarely makes its way into the documentation process.

How do you measure effective teaching?

There is no one gold standard to measure effective teaching. While this may seem like bad news, it provides both faculty and administrators with the opportunity to focus first on what they find most important and then on how to assess it. Unfortunately, because there is no set standard, it is easy to overly rely on what is most commonly used to measure teaching (SETs).

SET scores are exceedingly easy to generate; perhaps one key reason they are so ubiquitous in higher education. There are also fraught with problems. While some SETs suffer from significant scale construction, validity, reliability, and response rate issues, it is also clear that a number of factors—such as course difficulty, the instructor’s race and gender, the instructor’s presentation style, and even chocolate—can influence them (Boysen, 2016; Carpenter & Witherby, 2020).

Though most universities still rely heavily on quantitative data from SETs, there are many ways to capture effective teaching (Bernstein et al., 2006). This said, it is rare to see universities have consistent (i.e., across schools and departments) multi-faceted measures of teaching. It is easy to understand why: Holistic pictures of teaching take time to put together and take time to evaluate. Often the knowledge of how best to do both is lacking.

Key realities and solutions

Reality: Learning is complex. Students’ perceptions of their learning are also biased. Learning is difficult to measure as it is biased by a wide host of factors related to the student, the instructor, and the course. Instructor demographics and teaching behaviors and practices can easily influence perceptions of learning and instruction. This said, “teaching occurs only when learning takes place” (Bain, 2004, p. 173), so including measures of learning when evaluating teaching is critical.

Education Essential Reads

Which Trendy Teaching Techniques Actually Work?

Why Students Should Resist Using ChatGPT

Solution: Provide faculty with assessment know-how, and support reporting student learning outcome achievement, changes, and levels (Suskie, 2018).

Reality: Teaching excellence is contextual. What works at one university, in one discipline, for one level (first year, senior year), and for one group of students may not work elsewhere. This makes “best practices” a misnomer as practices may not be “best” for every context.

Solution: Provide faculty with course design know-how, and support modifying assignments and using different instructional methods.

Reality: Teaching excellence is not a fixed entity. Effective teachers need to be ready to change their practices and evolve to address different pedagogical challenges and external uncontrollable events (e.g., pandemics). This means it is unreasonable to set numerical quantitative benchmarks to assess teaching.

Solution: View teaching effectiveness holistically, providing faculty with ways to document their efforts and track and reflect on changes in student learning over time (see Bernstein et al., 2006).

Reality: Capturing effective teaching is challenging. It would be nice to have a quick, effective, cheap measure of teaching but it is difficult to get all three at once. Good measurement takes time and is not always easy. Faculty need to be given the time, resources, and incentives to engage in evaluation as effective evaluation benefits from training.

Solution: Give faculty funding to participate in workshops on good evaluation, and support them with well-staffed centers for teaching and learning.

Measuring effective teaching

A first step in the better evaluation of teaching is to reorganize our priorities for measuring teaching or, alternatively, be clear on all the benefits of measurement. If the goal of higher education is to help students be lifelong learners and gain the skills and knowledge to be happy, healthy, and responsible citizens (albeit only one set of aspirations), we need to help teachers help students learn.

Measures need to capture the fundamentals of effective teaching while providing easy ways to scale up the level of detail and complexity for those who opt for it. Most measures are self-reports where a faculty member reflects on their own knowledge, skills, and abilities, or can also be completed by students. Providing faculty with checklists of the fundamentals gives them a clear set of goals and benchmarks with which they can track their own progress and development.

Higher education needs to develop a culture of teaching excellence on campus. The effort to be an effective teacher is easier to expend when teaching is valued, rewarded, and seen as part of the fabric of the university. Some keys:

1. Be clear about why you are assessing teaching. It is easier to invest effort in documenting teaching if it is clear why the evaluation is taking place. Evaluating teaching helps establish knowledge and use of fundamental evidence-informed practices provides benchmarks for self-improvement and can gauge student learning.
2. Make it easy to describe and assess. Provide efficient ways to elucidate pedagogical activities and knowledge, and provide guidance on how to evaluate the same using a developmental growth (reflect, modify, and aim to improve) rather than a threshold (hit this number) approach.

Quality teaching is critical to student learning. Faculty need support, training, and development to be effective educators. Let's do more to help them.

A longer version of this post was published in the Teaching Professor.