Teaching to the test

Posted by Marc Hodak on July 29, 2010 under Patterns without intention, Unintended consequences | Be the First to Comment

A frequent complaint about standardized tests as a measure of scholastic achievement is that teachers, who know the general content, can simply “teach to the test,” i.e., they will focus on those content areas to the exclusion of others in order to maximize the performance of their students so that they, as teachers, look good.  This is not good.  It limits the range of inquiry to those that are bureaucratically mandated, and can actually inhibit real learning.

If the teachers know the particular content of a test, then you get a double distortion:  On top of the inhibition on real learning, you will now also get artificially high scores that don’t reflect any learning at all.  And if the teachers are being paid or selected based on their students achievements on such tests, then the teachers must teach to the tests as a matter of personal career survival–a real and legitimate sore point for teachers and their unions.

The problem is that we can’t generally know when a teacher is teaching to the test.  In certain extreme cases, one can use statistical analysis to see if a teacher is actually cheating.  But generally, it’s hard to see in a sea of “gains” how illusory those gains are, and how much of them were the result of teaching to the test rather than real learning, even learning limited to the content of the content areas to be covered by the test.

Well, now we know the answer for New York State.  By slightly increasing their standards, proficiency in English went statewide dropped from 77% to 53%.

In New York City, the number of students scoring proficient in English fell to 42% this year from 69% in 2009. In math, 54% of city children scored proficient this year, down from 82%.

A decline in passing scores is the natural outcome of raising the bar.  But one would expect student scores to be randomly distributed on a bell curve if teachers weren’t teaching to the test, even to the point of cheating.  If teaching to the test were the norm, however, we would expect a very different pattern of scores, i.e., scores bunched up around the “passing” mark.  If scores are bunched up around that mark, then a slight increase in standards would create a huge decline in passing scores.  That is exactly what we saw in New York.

So, what are the reactions to this steep drop in proficiency?

[State officials] were careful not to assign blame for the previously low standards, saying that the tests had become too predictable and tested too narrow a range of knowledge, thus becoming increasingly easier year after year.

Well, yeah.  The question is:  what’s going to be done to prevent that from happening again?  Raising the standards may help make some kids get better at what’s being tested, but it won’t cure either of the problems mentioned above.

The clearest evidence of all that the “teaching to the test” theory is hurting minority students the most:

David Steiner, the state education commissioner, said the reason was that a disproportionate number of black and Hispanic students were clustered around levels that had just made the proficiency level, so they were disproportionately affected when the proficiency level was increased.

NYU professor Robert Tobias pulls these results into the bigger picture:

“Just admitting it is a major improvement and a step in the right direction,” said Robert Tobias, a professor at New York University’s Center for Research on Teaching and Learning and the former testing chief in New York City.

But he warned that there may be other problems with the test that cannot be fixed by simply changing the point at which a student is termed proficient. “Unless you address those issues, you’re proceeding in ignorance,” he said.

Amen.