Balancing the Scorecard with Student Voices and other Measures of Teaching Effectiveness

Here’s news that seems to be surprising some people: what students think about their teachers’ teaching is accurate and relevant.

Any and all efforts to improve schooling for our fast-changing era must, I believe, respect, honor and care about what students tell us.    Let’s use the evidence of our students’ perceptions alongside of test scores and alongside of student publications, website production, exhibitions and presentations, to evaluate more fully and more effectively the success of learning in our schools.

The New York Times reports on the Gates Foundation research:

Teachers whose students described them as skillful at maintaining classroom order, at focusing their instruction and at helping their charges learn from their mistakes are often the same teachers whose students learn the most in the course of a year, as measured by gains on standardized test scores, according to a progress report on the research..   After comparing the students’ ratings with teachers’ value-added scores, researchers have concluded that there is quite a bit of agreement.

Classrooms where a majority of students said they agreed with the statement, “Our class stays busy and doesn’t waste time,” tended to be led by teachers with high value-added scores, the report said.   The same was true for teachers whose students agreed with the statements, “In this class, we learn to correct our mistakes,” and, “My teacher has several good ways to explain each topic that we cover in this class.”

The best news for those of us who care deeply about learning that is deeper and richer than memorizing, drilling, and test prep comes in this part of the report:

One notable early finding, Ms. Phillips said, is that teachers who incessantly drill their students to prepare for standardized tests tend to have lower value-added learning gains than those who simply work their way methodically through the key concepts of literacy and mathematics.

Teachers whose students agreed with the statement, “We spend a lot of time in this class practicing for the state test,” tended to make smaller gains on those exams than other teachers.” “

Teaching to the test makes your students do worse on the tests,” Ms. Phillips said. “It turns out all that ‘drill and kill’ isn’t helpful.”

The balanced scorecard concept isn’t new: it is a famous tool from Harvard Business School, twenty years ago, arguing that corporations disserve their long-term success when they measure, report, and respond only to data about stock prices and profits.    Instead, they must balance the scorecard with data on R&D, professional development, and other factors.

Now we must apply it to our schools.   We will care about our measurable results, but let’s measure widely.   Let’s survey students, not just about the questions referenced here but also about whether they have the opportunity be creative in class, whether they can feel they can use technology to solve problems in school, whether they are asked questions for which there is no simple or single answer, whether they are learning to collaborate effectively.   The High School Survey of Student Engagement provides all these data.  Let’s put these data alongside our test scores, our graduation rates, our attendance and graduation rates, our graduate’s success in the schooling beyond graduation.

Then, next to the data, let’s put the demonstrations.   Exhibit student work, and organize it into easily viewed digital portfolios.   Show students work responding to authentic, real-world situated, challenging problems, and ask the adults on boards, accreditors, and the media to evaluate: is this effective problem-solving?  Would the skills, thinking, and communication demonstrated be useful in the world we adults inhabit?  Does it demonstrate knowledge and understanding of the deeper issues underlying the problem?

I recognize of course how controversial the Gates Foundation is in the circles of educational innovation in which I travel, but one of the most consistent themes and goals of my blogging is trying to learn how we can use data the right way, in a balanced, multiple measures methodology which informs judgement, not replaces it.

The work of the Gates Measures of Effective Teaching  Project seems to me to represent this approach.  From the project’s overview page:

Composite measure of teacher effectiveness: An accurate measure or predictor of effective teaching must be multidimensional.

Researchers are collecting five types of data:

  • Student achievement gains on state standardized assessments and supplemental assessments designed to measure higher-order conceptual thinking.
  • Classroom observations and teacher reflections
  • Teachers’ pedagogical content knowledge
  • Student perceptions of the classroom instructional environment
  • Teachers’ perceptions of working conditions and instructional support at their schools

This is great to me.  Standardized test scores are just one leg of five, and even that one includes higher order thinking assessment.  Balancing out this scorecard are other things that really matter: student perceptions, teacher perceptions, teacher knowledge.

There may be two camps out there: those who wish we we used data to evaluate teacher effectiveness more than we do and those who think we already do so too much.  I think both camps should be happy about these reports and this research project, the first camp because we are only going to get better informed in how we collect and use such data, and the second because this movement has so much movement that it is going to continue to grow whether it is liked or not, and if it is going to grow anyway, let’s support efforts which channel it in more meaningful, balanced scorecard ways.

One Comment

Comments are closed.