When districts strive to provide great training for their administrators on doing high-quality observations and evaluations, I’m delighted.
But there’s one goal the process can never achieve, and it bothers people to no end.
The unreachable goal? Calibration.
“But without calibration, how can teachers be evaluated fairly?” the concern goes. “If one administrator’s assessment would be different from another’s, how is this a defensible system?”
Of course we don’t want teacher evaluations to be capricious or unreliable. And using multiple evaluators for high-stakes evaluations isn’t a bad idea at all.
But what most districts try to do doesn’t work.
Calibration Training
Here’s how calibration training typically works: One school (or a vendor) provides a video of a lesson, and all the administrators watch and do a practice write-up.
Then, the calibration begins.
“Why did you give her a 3 for ‘monitoring student engagement’?”
“I gave him a 2 for ‘having a clear objective for the lesson.'”
And the debate begins…but it never goes anywhere.
And it can’t, because the observers don’t have the information they need.
Missing Context
I don’t believe teaching is like auditioning for a musical, where you can do a scene or a number and give the casting director a good sense of whether you’d be right for the show.
When you teach, you’re not just showing up and doing a little song and dance. Some of the most important work happens behind the scenes, when you’re phoning parents, planning lessons, reviewing student work, collaborating with other educators, and doing the million other things that go into great teaching.
Good evaluation frameworks account for this behind-the-scenes work, but even when it comes to understanding what’s happening during a lesson, there’s essential context that a stranger won’t know.
As a supervisor and colleague, you know things about your teachers and their students that are essential for conducting a fair evaluation, and even for truly understanding what’s happening in a lesson.
Ten Things Administrators Know About Their Own Classrooms
Here’s what’s missing when we do practice observations from videos, or even when we observe other schools’ teachers during site visits. We don’t know:
- What the teacher taught yesterday, and what’s happening tomorrow
- What went well and what didn’t go as planned yesterday
- Which students are having a hard time lately
- Which students have IEPs, behavior plans, or other systems in place
- What the teacher’s team decided jointly to do for this unit
- When the teacher’s dog died (true story…)
- What routines and procedures the teacher has in place
- How today compares to the typical day in this classroom
- What the teacher learned about students’ understanding yesterday
- How today’s lesson connects to upcoming plans
You can’t know this context if you’re watching a video.
But you also can’t know it if you aren’t in classrooms regularly, and if you aren’t involved in planning and collaboration meetings.
(If you’re ready to develop the habit of visiting classrooms regularly, so you know what’s going on, check out the Instructional Leadership Challenge, which currently has been taken by more then 10,000 participants worldwide).
The bottom line? Evaluation can only be done well in the context of a teacher-supervisor relationship. It can’t be simulated, at least not very well. And it can’t be calibrated, except among administrators who actually work in the same school and have the same information.
How To Help Administrators Do Better Observations
If you want to do an exercise of this type, forget about calibration, and focus on the quality of the written evaluation.
Don’t worry if one principal thinks the lesson was great and another thinks it was terrible. Focus on the quality of evidence the principal provides in the written report.
Share good and bad examples, talk about the difference, and give feedback on how to make the final product stronger.
As a side benefit, principals will develop a clearer understanding of the evaluation criteria you’re using, and this will directly aid the calibration goal.