4 March 2010

Measuring Students and Teachers

Filed under: Culture, Psychology — ktismatics @ 6:39 am

Appertaining to yesterday’s post about workplace audits, the Gates Foundation conducted a survey of 40 thousand American teachers in primary, middle, and secondary schools. The results have just been released. From the summary of findings (and link to the full report) of respondents’ opinions:

In improving students’ academic achievement, establishing clear standards for students is deemed more important than making the standards tougher.

In measuring students’ academic achievement, ongoing assessments, class participation, and performance on class assignments are judged far more important than standardized tests. Teachers say they use results of student achievement measures to adjust the curriculum, to identify students needing special attention, and differentiate teaching for the learning needs of specific students.

In identifying the most important goal of education, preparing students for careers was rated far more important than preparing students for college, learning life skills, or graduating from high school. (Learning the material for its own sake was not included among the choices.)

In engaging students in learning and facilitating academic achievement, digital resources, magazines, and books other than textbooks are considered to be far more effective than traditional textbooks.

In evaluating teachers’ performance, students’ growth over the year and their engagement in learning are the most important indicator, whereas evaluations by parents, students, and the principal are least important.

In retaining good teachers, supportive leadership is most important whereas performance-based pay is deemed least important. Also rated highly important: collaboration among teachers, good teaching resources, clean/safe schools, professional development, higher salaries, collegiality. The more experienced teachers are less likely to support performance-based pay.

As always, the kinds of questions asked affect the kinds of answers generated. It seems, though, that teachers regard ongoing evaluation of student performance as important both for improving student achievement and for evaluating teacher performance. But “alignment of incentives,” in which pay is tied to performance, is regarded as not a very good management tactic. Not addressed in this survey is whether teachers think that students’ incentives should be aligned, such that kids who get the best scores on ongoing evaluations should get rewarded by placement in the best universities and jobs.



  1. Ha. Nice.

    As an experienced teacher, I’d say I don’t favor performance-based pay for four main reasons, which may well parallel incentive-alignment for students. One, rewarding people for past performance is not a reliable way to guarantee future performance. Two, I don’t trust the people devising the performance standards to do so intelligently. Combine one and two for the third reason. Four, performance pay is perhaps the single best way to rupture collaboration and collegiality and to create poisonous, festering pockets of entitlement and bitterness.

    I picked at some of this stuff here a while back.


    Comment by Carl — 8 March 2010 @ 3:51 pm

  2. Thought this was interesting from the Harvard Ed. magazine article (underlying the third prp link):

    “Teachers and many others are particularly offended by the underlying assumption of merit pay: namely, that teachers would work harder if only they were rewarded with even a minor financial bonus (the pay differential in most plans is typically quite small, only a few thousand dollars tops.) Most people who go into teaching are motivated by intrinsic rewards — the value of the work they do — rather than extrinsic motivators, such as money, many educators believe. ‘There’s an assumption under this that teachers would try harder if they were paid more,’ says Gratz. ‘The corollary is that they’re not trying hard enough now, which means they care more about money than kids. Frankly, teachers find that insulting.'”


    Comment by Carl — 8 March 2010 @ 8:08 pm

  3. Curiously, teachers in NC and SC express the greatest levels of enthusiasm about performance-based pay, so here’s another indicator that you’re alienated from your local environment.

    Performance-based pay is kind of old-school management. The continuous quality improvement gurus aren’t high on performance incentives in part because of your first criterion: a lot of individual variation is random, so both the high and low outliers in the distribution are likely to regress to the mean in subsequent measurements. They’re more interested in identifying work processes that high outliers as a group share, then teaching those processes (aka best practices) to everyone else, such that the rising tide raises all ships etc. This management style first became popular in Japan, where there’s arguably more of a collective orientation to doing well societally.

    K-punk seems to regard this whole idea of internalizing the indicators of excellence as an insidious method for management to control workers. One would hope that managers and workers alike would share a commitment to doing the job well. This should be especially true in education, where good teaching has societal benefits (as opposed to, say, a shared commitment to selling the most widgets). K-punk also regards measurement as an incentive to standardization and therefore an obstacle to innovation. Again, per CQI logic you’d think that excellence across the board is the most important objective. You’d want to create incentives for individual innovators to share what they know, rather than hoarding that knowledge so they can get their performance bonuses.

    Regardless of compensation issues, I’m curious about how one might assess student academic improvement over the course of the year, which is what teachers themselves regard as the best indicator. Standardized tests are rated very poorly in this regard, presumably because they don’t accurately evaluate what the teachers are actually trying to teach. I don’t know if they feel the same about AP and IB tests, which presumably do evaluate pretty directly what’s being taught. Still, we’d be trying to evaluate not actual scores but improvements, so somehow you’d need to adjust for student preparedness at the beginning of the school year. The easiest method would be to look at improvement of students’ grades from the prior year, but of course we’ve got grade inflation to contend with.

    Even if teachers dismiss standardized test results, parents take them seriously. The obsessive ones will actually move into school districts that get high standardized test scores. This increases demand for housing in the district, which raises housing costs, which further stratifies access to these presumably best schools.


    Comment by john doyle — 8 March 2010 @ 8:32 pm

  4. So if money won’t get teachers to do a better job, what will? That’s assuming we can agree on what “better job” means in fairly concrete terms, and that there is room for improvement.

    I know more about doctors’ performance measurement than about teachers. Doctors too are presumably intrinsically motivated to do a good job, plus they get paid extremely well. There are agreed-upon measures of excellence, which include both outcome measures (low death rates for surgeries, low blood sugar rates for diabetes patients, etc.) and process measures (e.g., treat high cholesterol with drugs if not responsive to diet/exercise after 6 months). Still, there’s a lot of variation between doctors, clinics, cities, etc. on these measures. And this variation has remained stubbornly resistant to significant improvement over the past couple of decades. Meanwhile, cost of healthcare has gone up at an alarming rate over this same time period. So you’ve got escalating costs, stagnant quality, and probably deteriorating population health.

    I read on your post your argument that money is a proxy for respect, but I’m not persuaded. Attempts to demonstrate respect that don’t turn into money are regarded with suspicion as a manipulative move. Regardless of whether you or I individually think money is particularly valuable, I find that a lot of people really do value it. And unlike respect or love, money is intrinsically quantifiable. I am persuaded by your arguments that incentive-based pay differential is a bad idea, and that it further stratifies pay scales between workers and executives.


    Comment by john doyle — 8 March 2010 @ 8:55 pm

  5. The other thing: not everyone is equally competent, equally impassioned, equally responsive, equally inspiring. Whether differential pay is used as an incentive to do better or as a reward for already doing well, the link of money to individual performance is problematic in fields like education and healthcare where excellence at the societal level is what we’re aiming for.


    Comment by john doyle — 9 March 2010 @ 4:47 am

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: