24 June 2010

The Students Make the Teacher

Filed under: Culture, Psychology — ktismatics @ 3:17 pm

In my last post I wrote about the difficulty of distinguishing effective from ineffective teachers. Training, certification, experience, adherence to best practices, student achievement — none of these variables stands up to empirical validation. In a recent discussion on Jon Cogburn’s Blog we agreed that college students’ evaluations of their teachers isn’t the way to do it either.

Maybe we’re looking at the whole teaching thing backward. A good teacher doesn’t make good students; good students make the teacher good.

It’s not like this is some profound insight I’ve had. I suspect that most teachers would agree that they do a better job and enjoy themselves more when they get to work with a classroom full of smart, interested, open-minded, curious, creative, outspoken, serious students. If the teachers don’t have that good fortune, then they teach to the handful of bright and engaged and enthusiastic students who shine forth in an otherwise mediocre class. Tenured university profs typically choose to teach the upper-level classes, which are populated by students who have already demonstrated their scholarship and commitment in the lower-level classes that winnowed out their slow and indifferent colleagues.

Studies typically find that the highest-rated teachers teach in schools where the students’ average aptitude scores are higher. It’s usually assumed either that these schools attract better teachers, or that the students would be high achievers regardless of the quality of their teachers. Maybe though it’s the bright students bringing out the best in their teachers.

It’s been shown that students’ ratings of teachers correlate with students’ grades in the class. Often this result is interpreted as kiss-ass teachers inflating their students’ grades as a crass tactic for bumping up their own “customer satisfaction” scores. But what if good students are generally happier with their teachers than are bad students? Our daughter, a junior in high school, offers personal testimony supporting this hypothesis. She says that in her classes the students who are doing poorly blame the teacher, whereas the students who are doing well blame the low-performing students’ own inattentiveness and laziness. She contends that this effect is robust across classes: kids with higher GPAs generally rate their teachers highly, while kids with low GPAs don’t. This contention could be put to the empirical test — maybe it already has: do students’ overall GPAs predict their teacher ratings for the following school year? I bet yes.

Students don’t just make the teachers; they make the schools. For the past several months our mailbox has been flooded with brochures from universities and colleges trying to persuade our daughter to apply. I asked her how she’d characterize these mailings: “propaganda” was her immediate reply. How so? The schools all pitch the same features to the prospective students. Part of it is the attractiveness of the campus, but mostly it’s the attractiveness of the students. Glossy photos of energetic, ethnically diverse students, mostly in groups, smile out at you on every page. There’s the occasional nod to faculty excellence — so many Nobel laureates, highly ranked research programs — but mostly what’s pitched is the possibility of actually getting to know your professors. These schools are selling an attractive lifestyle, and that lifestyle is a communal one.

The best students compete for acceptance by the best schools, but how do the students identify the “best”? For the select few — Harvard, Stanford, MIT — the reputation for scholastic excellence is universally acknowledged. For most schools though it’s the measurable quality of the students: their SAT scores, their high school GPAs and class ranks. These metrics are readily available for all colleges/universities on the Internets. Somewhere back in history the first cadre of excellent students was attracted by the quality of the faculty at some previously ordinary school, but by now the excellence is self-perpetuating: this year’s good students go where last year’s good students went. The profs too compete for the opportunity to work at these high-reputation colleges. Part of it is the attractiveness of associating with prestigious colleagues, but I’m sure it’s also the appeal of teaching those bright, creative, high-performing, highly motivated students who enroll year after year.



  1. My comment may be stepping back even further. In ancient Greek polis society, the assumption was that the state was responsible for making an individual into a good citizen. The collective society considered it their duty to produce good individuals. The individual is formed by the society.

    I can’t help but think that our society is as influenced by Protestant theology as it ever was, because we tend to think that some people/students/children are just not as good as others, or that some are just born winners and others are just born losers. I see this attitude all the time in my line of volunteer work. I go in and out of the jail every week, and I read the writings of inmates. They consider themselves “white trash.” They are the born losers who can’t get their act together. I try to ask them to go deeper: what forces influences and shaped you along the way–think back and think deeply. Write about your past with honesty and grace.

    So, I lean more toward the Greeks at this point. There is certainly a range of aptitude and giftedness that is genetic, but this does not preclude society as a whole from taking some collective responsibility for forming students. At this point, I think it is reasonable to assume that our Western culture still views some souls as inherently flawed and in need of saving.

    Well, I may be a bit off topic perhaps. My apologies.

    Comment by Jonathan Erdman — 24 June 2010 @ 5:42 pm

  2. The topic is broader than one post, Erdman, and you’re on it. You should have a look at this post from a week ago. In it I review two studies which demonstrate that growing up in poverty inhibits the expression of kids’ genetically-inherited aptitude for school learning. In a relatively affluent place like Boulder, by the time kids hit high school the differences is SAT scores and so on are accounted for almost entirely by genetics and practically not at all by environment, including parents and schools. So for middle class and up the innate individual differences do make a difference, and its these upper classes that put such a strong emphasis on individual competence. But it’s this very stratification of individuals that prevents the children of the poor from actualizing their individual genetic intellectual potential.

    The Greek polis was a democracy, but one in which only the elite had full citizenship rights. I’m pretty sure the children of slaves weren’t schooled to be productive individual citizens, nor were the girls. Still, learning in order to contribute to the common good as a citizen is very different from learning in order to get the most lucrative career for yourself as a member of the plutocracy.

    Comment by ktismatics — 24 June 2010 @ 6:10 pm

  3. I can share the last year of my own adventures in teaching. The situation is pretty clear in the Netherlands: depending on what class you belong to, you either go to a shit school, or to a good school. At the shit school, the teachers are being fucked from the top by overpaid managers who introduce self-assessments in order to put all the responsibility on the teachers (of course), and by the students themselves, who wouldn’t be sitting in such a school if they had any brains. How much shit you can take depends entirely on whether you’re Dutch or a foreigner, the Dutch having a much better chance to land a job in a decent school. The good schools I think are still relatively socialist and good compared to the kind of a situation you have in the States, where only ivy league schools really matter. This because every Dutchman til the age of 34 has the right to completely free study financing, and merit therefore does play a role in getting accepted by a good school. My concrete impression from teaching is first of all that this is work for people who didn’t accomplish themselves, so they ”help” other people accomplish themselves (sort of like social work) and if in all this you can find satisfaction, then I assume you should be a teacher, but it won’t be ME. Most teachers I met are in it because their own children have Autism or Asperger’s so they identify with the kind of problematic youth that ttends the shitty school. In conclusion I think the listed problems are still more severe in America, where the socialist buffering we still luckily have here, probably isn’t present to the same extent.

    Comment by Post-Continental Satyr — 24 June 2010 @ 6:23 pm

  4. Free schooling until age 33? Pretty great. According to a study I just looked at, the Netherlands has half the percentage of people living in poverty as does the US, plus there’s considerably more income mobility from one generation to the next. So if the Netherlands looks classist, the US is even more so.

    In Canada, France, and I think also in Britain most of the universities aren’t particularly selective, which is also true of most government-run universities in the States. There are the top tier schools too, as you know: Oxford-Cambridge, the Grandes Ecoles, which like the Ivy Leagues are ruling elite training grounds. The US also has a wide array of quite selective private universities in addition to the Ivy Leagues, which attract very good students and are quite expensive. These schools do have quite generous alumni, however, who provide quite substantial financial aid to lower-income kids if they’re good enough students to be accepted. For the most part the low-income kids can’t afford to live on campus, so they attend commuter schools — either 4-year universities or 2-year community colleges — and live at home. I think it’s possible to get a good education from these schools, but students do have to pay to attend, so it’s often difficult for the poor kids to finish the course.

    Comment by ktismatics — 24 June 2010 @ 10:48 pm

  5. John,

    Yes. I followed those posts, and I found them very enlightening.

    Comment by Jonathan Erdman — 25 June 2010 @ 8:44 am

  6. The old joke is “Harvard, where the best students in the country go to get the worst education in the country.” What this means I think is that the Harvard faculty do not ‘teach’, they model the cutting edge life of the mind (as you suggested in a previous post). This works great with smart, motivated, well-prepared students in self-teaching packs, and washouts are easily replaced.

    Comment by Carl — 26 June 2010 @ 3:17 pm

  7. My dad went through college on the GI Bill, which took him from VMI to Oklahoma State to Bradley and eventually to Northwestern, from which he graduated. Northwestern is easily the most prestigious of the bunch, but he thought that Bradley’s teaching was far superior. Because Bradley was a smaller school, he had more interaction with the teachers, who actually wanted to teach. Did my father learn more from these better teachers at the school with the less impressive rep? It’s hard to say.

    As to whether students make the teacher, I just read a study of Tennessee primary school kids’ 1-year gains in achievement test results. Kids’ baseline scores were highly significant statistically; i.e., their scores from the prior year were significantly correlated with their level of improvement in the subsequent year. Oddly, though, the pattern of findings wasn’t consistent. I.e., in half the analyses it was the lowest-scoring students who showed the most improvement, in a quarter it was the middle group, and in a quarter it was the highest scorers who improved the most. This inconsistency suggests that significant results may have been an artifact of the large sample sizes (over 10,000 kids in each analysis), where even very tiny differences become statistically significant. Sorting classes by ability had no significant effect on test score improvement, which might imply that sorting students into universities by ability level might not have much impact on how much they learn.

    It should also be noted that in this particular study the teacher effect was the strongest across the board. Again, however, there’s no indication of effect size. Also, unlike Goldhaber’s studies from the preceding post, this one looked at only one year’s worth of data for each teacher. In other studies the magnitude of the teacher effect tends to wash out over repeated years; i.e., the best-performing and worst-performing teachers in one year typically regress to the mean in subsequent years. Further, since I’ve been making a rhetorical case for the irrelevance of good teachers to good learning, I would be forced to discount these findings for one reason or another regardless of methodological rigor or magnitude of effect ;)

    Comment by ktismatics — 26 June 2010 @ 4:19 pm

  8. The main justification for sending the best students to the best colleges is if those colleges’ courses are tougher, so the students need to be better in order to succeed. I wonder: does it take more intelligence and harder work to get A’s at Harvard versus A’s at some more ordinary school? I’ve always presumed so. On the other hand, practically everyone who enrolls at Harvard graduates, so maybe it’s piece of cake.

    Comment by ktismatics — 26 June 2010 @ 5:20 pm

  9. John, I’ve been meaning to say that I really admire and appreciate the work you’re doing to research and analyze actual data on this question. It elevates the discussion dramatically. That said, I realize that part of why I’m not participating more enthusiastically in the discussion is because I just don’t trust that the right things are being measured. I know for a fact that my students would do poorly on the available standardized history tests – my colleagues and I took one recently out of curiosity and we all did poorly on it. It was a collection of trivial pursuit questions most of which were out-of-field for each of us. In fact, by that measure I strongly suspect our students would be shown to regress between high school and college. This is because we’re not primarily teaching content but critical processing.

    Comment by Carl — 27 June 2010 @ 9:03 am

  10. @8, there’s always some kind of flap about grade inflation at Harvard. But in principle if you get the very best students in the country together in one place they all ought to excel. Sort of puts George W.’s ‘gentleman’s C’s’ at Yale in perspective, huh.

    Comment by Carl — 27 June 2010 @ 9:09 am

    • Well yes, that’s my assumption too, but empirical evidence doesn’t necessarily support the assumption. Pretty much every study shows that clumping grade-school kids by ability level has no effect on individual learning outcomes, however they’re measured. I’d have expected that the clumping would have resulted in greater increases in learning for at least the high-ability kids, but no. Again, maybe they’re measuring the wrong things. It’s also clear that, in American schools, clumping of students is confounded with differences in curriculum, such that the high-ability kids are put on an accelerated track. So in part the smart kids excel because higher demands are placed on them by these higher-powered courses.

      One wonders whether the lower-ability kids might also rise to the expectations if they had the chance. I’m reminded of the programs in which high school kids at high risk of dropping out are more likely to do well and to graduate if they’re enrolled simultaneously in the accelerated track that gets them a 2-year associate degree at the end of high school.

      Comment by ktismatics — 27 June 2010 @ 9:55 am

    • “Pretty much every study shows that clumping grade-school kids by ability level has no effect on individual learning outcomes, however they’re measured.”

      Just to be clear, I’m not saying that clumping them has any effect on learning outcomes. I’m saying that smart capable kids are likely to do well, separately or together as the case may be. Presumably they’re already doing well for the study to be able to sort them by ability level, and this does not change.

      Comment by Carl — 27 June 2010 @ 6:35 pm

  11. It’s mostly a pragmatic thing: achievement test results are readily available for doing these sorts of studies on existing data. Kids do on average improve on these tests from one year to the next, so it’s not implausible that teaching might have some impact on amount of improvement. Still, I have no vested interest whatsoever in affirming the validity of these tests.

    If you teach critical processing, then I presume you evaluate students’ performance on critical processing, yes? So what you teach isn’t in principle impossible to measure, since you measure it yourself for each student. At issue is whether students get better at critical processing from the beginning to the end of the year, and how much of this improvement can be attributed to teaching. Based on your description of the AP essays, it seems that they evaluate critical processing at least to some extent. To do the kinds of improvement studies I’ve been reviewing in these posts, you’d need to administer a standardized essay requiring critical processing to kids upon enrollment in the AP class, then compare those results to how they do on the AP test at the end of the class.

    I’m sure it’s possible to develop age-specific evaluations of critical processing. In fact, I wouldn’t be surprised if critical processing subscales could be built from some subset of questions on the standardized achievement tests that already exist.

    For me, though, perhaps the biggest problem is the presumption of the importance of teaching without evidence to substantiate the claim. So you get degrees and certifications and tenure and raises in pay and concomitant raises in taxes/tuition, all premised on purported importance and excellence that everybody tacitly agrees on but that no one can demonstrate empirically. And my sense is that the teachers actively resist this empirical analysis, cooperating only reluctantly when administrators or politicians or researchers try to do it. As far as I can discern so far, none of the methods of distinguishing good from bad teaching/teachers is effective. So what presumption do you hold: that all teachers are equally effective, or that they’re all equally ineffective? Neither is a very satisfying answer to those inside the profession. The better solution is to mystify excellence, to claim that it transcends measurement of any kind, to assert that only those in the guild are qualified to evaluate their own excellence. I think the era of professional preciousness may be nearing an end.

    Comment by ktismatics — 27 June 2010 @ 9:44 am

    • I’m aware of the danger of handwaving and preciousness; it’s real and I’m not immune. But I do think that the sort of emotional/intellectual dispositions and skills that go into effective critical processing are pretty complicated to acquire (and presumably to teach), much more so than short-term recall of content, and don’t necessarily align quickly into effective performance. They’re not that hard to test, but their development is not necessarily linear from test to test. Also when I say ‘not hard to test’ I mean this in terms of test design, not actual testing, which as I know from both my own work and the AP reading is wicked human-resource-intensive, in contrast to a scantron.

      Comment by Carl — 27 June 2010 @ 6:55 pm

    • As a historian it’s sort of funny to see teaching in general questioned for empirical evidence of effectiveness. Teaching is a contender for world’s second-oldest profession, so if it’s not getting something done that’s a hell of a lot of jerking off. I’m open to suggestion that the something is not education, of course.

      Comment by Carl — 27 June 2010 @ 6:59 pm

  12. One of the more startling recent findings in psychology is how little measurable effect parenting seems to have on individual development, with genetics being the dominant predictor followed by miscellaneous cultural factors. Still, no one is claiming that you could leave kids to raise themselves and they’d turn out fine, because demonstrably they wouldn’t. What the results imply is that pretty much any non-abusive, non-neglectful adult within a given social class could serve the parental role and the kid would turn out more or less the same. I suspect this is true of teaching as well: kids would spool out there genetic intellectual potentials within the constraints imposed by their culture regardless of who their teachers are, but that’s not to say that they need no teaching. Rather, as long as they’re not abusive or neglectful, teachers are probably pretty much interchangeable over the long run. So my bet is that regardless of what sorts of educational outcomes are measured, differences between teachers will prove minimal.

    If empirically that’s the case — which it has been so far — then attempts to improve teaching quality via selective hiring/firing or further training or financial incentives will prove a waste of time. On the other hand, the cost of education continues to escalate, and so people try to figure out cost-saving approaches. Though teachers’ pay is tied to years of experience, there is no evidence that experience is related to outcomes, except for some small gains in the first 3 years or so. Though schools and parents regard student-teacher ratio as an important indicator of school quality, empirical studies consistently find no effect of class size on learning outcomes. Does the impact of teaching on learning deteriorate if it’s done electronically, via video and internet and so on? While there’s anecdotal evidence especially from teachers that e-teaching compromises learning, I’ve not seen empirical evidence.

    And then we get to college-age students. If student-teacher ratio doesn’t affect learning in primary and secondary school, why should it matter in college? Compared with primary and secondary school, why does college cost twice as much per student for half as much in-class teaching? I don’t believe it’s just managerial insensitivity to teachers and students that motivate these empirical investigations. However, the managers do seem willing to use the numbers insensitively and inappropriately in order to hit their numbers. It would be best if the teachers themselves looked for ways to make their work more cost-effective in order to keep good education universally accessible, rather than always reacting against what gets thrown at them from outside the guild.

    Comment by ktismatics — 28 June 2010 @ 1:23 am

    • “kids would spool out there genetic intellectual potentials within the constraints imposed by their culture regardless of who their teachers are, but that’s not to say that they need no teaching. Rather, as long as they’re not abusive or neglectful, teachers are probably pretty much interchangeable over the long run. So my bet is that regardless of what sorts of educational outcomes are measured, differences between teachers will prove minimal.”

      This feels like the payoff of the analysis you’ve been developing, and I’m cautiously on board with it. I do think this is going to be especially true the more you standardize curriculum and delivery, which lops off both the pernicious bottom and the inspired top. That is, you can get all teaching to work the same by requiring all teaching to work the same, which is the current regime and has been in one permutation or another for a long time. Ecologies of education will work the same too, of course, without policy intervention, and outliers are always sanctioned (they even routinely get scrubbed from the data before anyone gets to see it). But for most people most of the time, these considerations are irrelevant – they grind their way to what they need to know for the lives they’re going to live, one way or another.

      Comment by Carl — 28 June 2010 @ 10:54 am

  13. I don’t believe that quantitative interchangeability necessarily implies standardization though. Take parenting: just because a variety of parents might get the same results for the same kid doesn’t mean they would all go about it the same way. Kids probably follow certain preset genetic trajectories regardless of who’s doing the day-to-day parenting, but the qualitative differences of growing up with one set of parents rather than another will build a reservoir of memories and relationships that are unique for each person. I think this qualitative distinctiveness is even more likely to shine if parents aren’t too concerned about being really good at their parenting, as if they can either shape their kids to certain specifications if they do it right or impose permanent barriers on their kids’ futures if they do it wrong. Again, abuse and neglect really can scar a kid, but beyond that it’s a matter of what Winnicott and Betelheim separately called “good enough parenting.” And if the parents enjoy the parenting for the most part and regarding the kid as the main factor in his/her own development, then some of these qualitative factors can shine forth as good memories and good relationships.

    Maybe these same principles apply in teaching: be a good enough teacher, rather than one who’s too caught up in performance anxiety and delusions of massive impact on kids’ lives. Enjoy the job, recognizing that ultimately it’s the kids’ job to develop and to learn. Then relax, have some fun, honor the kids’ autonomy, let your own personal style shine forward, and the teacher and the kids might actually enjoy the ride together. I suspect this isn’t far from your personal pedagogical philosophy, Carl.

    So with respect to standardized teaching methods, I’d say forget it. Teachers are important, but “good enough” is enough. Let teachers find their own style; let qualitative differences play out, since they’re not going to affect the quantitative outcomes much one way or the other. But I do lean toward broad general agreement as to what kids are expected to learn as they progress through school, including both content and skills like critical processing, mostly so that no kid, especially a socioeconomically underprivileged one, finds him/herself behind the 8-ball when it comes time to move on to a higher level, from grade school to high school to college. Maybe give up on the annual standardized achievement tests too: again, empirical evidence demonstrates that results on these tests are not going to lead to any better teaching or learning, and the tests probably measure the wrong things anyway.

    Comment by ktismatics — 28 June 2010 @ 11:22 am

  14. […] or any other teacher variable. Generously, John’s conclusion in the most recent post, “The Students Make the Teacher,” is that “kids would spool out their genetic intellectual potentials within the […]

    Pingback by Are teachers like coaches? « Dead Voles — 28 June 2010 @ 1:09 pm

  15. […] or any other teacher variable. Generously, John’s conclusion in the most recent post, “The Students Make the Teacher,” is that “kids would spool out their genetic intellectual potentials within the […]

    Pingback by Are teachers like coaches? | Attention Surplus — 7 December 2012 @ 3:49 pm

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: