C O V E R S T O R Y
Does One Size Really Fit All?
An Educational Philosophy Face-Off: Michael Farris Rebuts Marc Tucker at the NAGB Hearing
Public Transcript of the National Assessment Governing Boards Hearing on Voluntary National TestingWednesday, April 7, 1999
Although a permanent prohibition on national testing was imbedded in the General Education Provisions Act passed by Congress last fall, NAGB still retains some authority over developing national voluntary tests (see the November/ December 1998 Court Report, page 6). According to NAGB Chair, Wilmer Cody, Commissioner of Education in Kentucky and formerly the superintendent of Montgomery County, Maryland, the purpose of the hearings is to provide the Board with information for determining the purpose and intended use of the proposed voluntary tests, a definition of the term voluntary, and the means for reporting the results. (NAGB Voluntary National Test Hearing Transcript, April 7, 1999, page 5, lines 12–18.)
Federally mandated one-size-fits-all education or locally controlled individualized instructionthats what it boils down to.
Home School Legal Defense Association President Michael Farris testified on April 7, 1999, at a National Assessment Governing Board (NAGB) hearing on the development of voluntary national tests. Mikes testimony followed Marc Tuckers, a proponent
of national testingnot to mention a national curriculum, OBE, mastery certificates, and Goals 2000.
Taking advantage of this unique opportunity, Mike decided to set aside his prepared testimony and respond to Marcs ideology, offering an analysis of the paradigm underlying the agenda of those who support national testing and believe that big brother knows best.
Marc Tuckers Argument for a National Test
Mr. Cody: Marc Tucker, the President of the National Center for Education and the Economy. Welcome, Marc. . . .
Mr. Tucker: . . . Members of the Governing Board, my name is Marc Tucker. Im President of the National Center for Education and the Economy and co-director of New Standards.
The National Center is [a] not-for-profit organization, headquartered here in Washington, and a national leader in the standards-based education movement. New Standards is a consortium of states and school districts, directed jointly by myself and by Lauren Resnick, director of the Learning Research and Development Center at the University of Pittsburgh.
The New Standards Consortium has been a pioneer in the development of academic performance standards and examinations to meet those standards. The New Standards performance standards and assessments have been widely used in recent years by a number of states and school districts as a benchmark against which to judge the quality of their own standards and examinations.
In 1992, when we launched New Standards, we had a dream. That dream colors my views of the national test, which is why I need to share it with you.
Suppose we said there is one clear high standard for all students, set to the standards in place in those nations where student performance is highest. Suppose there are examinations that accurately measure student progress toward those standards, and suppose that texts and other curriculum materials are available that are matched to the standards, with teachers well trained to use those materials with the students from a variety of backgrounds.
And lastly, suppose that entrance to college and access to good jobs depend on studying those materials and passing the exams that are set to the standards. That actually is the system in most of the rest of the developed world. It is most especially the system in those parts of the developed world that have the highest consistent performance among their students, that is, the smallest range between the top performance and the bottom performance.
So we said if that happened, then all students have a strong incentive to take tough courses and to study hard, which is what we found in the countries that we studied. Students would know just what they have to know and be able to do and how well they have to do it. The materials used in their classrooms would be designed expressly for the purpose of helping them to know what they have to know to reach the standards. Their teachers would be trained to get students, just like them, to reach these high standards, and, in fact, the whole school would be geared up for that purpose and that purpose alone.
Our program, New Standards, did the international benchmarking needed to find out what the standards are in countries with consistently high levels of achievement; and, based on that research, weve built a set of standardsin English language arts, mathematics, science, and applied learning at grades four, eight, and ten. Those standards were unique in the United States in that they were performance standards. That means that they were built around examples of student work, real student work, that meets the standards.
Why was that important? Because without the examples of student work that meets the standards and a commentary showing in detail what it is about those pieces of work that makes them meet the standards, the student really has no idea what he or she has to do to meet the standard. And the teacher has only the foggiest idea as to whether that student is doing work that meets the standard.
If neither the student nor the teacher knows whether a particular piece of student work meets the standard, in our view, there is no standard.
This is a crucial point for the argument that Ill make in a moment. Our dream was a dream about a strategy for raising the performance for all children to world-class levels. If your objective is only to measure student performance, it doesnt matter whether the student knows with any precision what the standard is. It really doesnt. But if your objective is to raise that students performance, then it is absolutely essential that the student know just what the standard is because the student cant hit a target that he or she cannot see.
When most people think about standards for tests, they think about cut points, meaning that a passing score is a score of at least a certain numberso many items right out of so many items in the test; or a certain percentage correct; or, when all the scores from all the students are in, a score that falls above a certain point on the curve on which the test was graded. So when the student asked what he or she has to do to pass the test, the answer comes back in the form of [G]et at least a certain score. But for many students, especially those students who are furthest down on the curve, that answer might just as well be in Greek. I dont have anything against Greek, by the way, but in any unintelligible language.
When they are told that they have to get a certain percentage right, they have no idea what to do to improve their performance. If theyre told that they have to get at least a certain score on a test graded on a curve, it is even worse, because not only do they have no idea what they have to do to win, they are also being told they have to do better on the test than many other students who take that test. Its a very different thing to work to a standard.
Working against a standard, one defined by examples of student work that meets the standard, the student has a very concrete idea of what has to be done and knows that every student who produces work like that win[s], even if that means that all the students who took the test and met the standard are there. Everybody can win if youre working against standards.
What Ive tried to do here is share with you a set of criteria for a system designed to use standards and assessment for the specific purpose of improving student performance.
Now, lets see how the proposed national tests stack up against those criteria.
First, a test set . . . that makes it clear to students what they have to do to meet the standard. When we ask whether the national tests have been set to standards, the answer comes back that theyre set to the NAEP standards. But the NAEP standards were never designed to be standards that teachers teach to or that students study for.
The opposite is actually true. NAEP was designed to be the nations report card to report on the state of learning among Americas students no matter what curriculum they had studied, no matter what local tests they had taken. No student was supposed to take the whole test. It was, therefore, impossible for the test to count for anything for the individual student, and therefore, pointless for the student to prepare for the test.
The result is that students who take NAEP and, by extension, the national tests, will have only the foggiest idea what they have to do to produce student work that meets the standard. The only way to create a test that is set to explicit performance standards is to build the standards first, including and especially the examples of student work that meet the standards, and then write the test specificationsthe instructions, that is, to the item writerson the basis of the standards. This wasnt done with NAEP, nor was it done with the national test. This is, in my judgment, a fatal error for the national tests.
The only reason to add another test to the great burden of tests that American students now take is to improve their performance, not simply to measure it. That wont happen unless the students know what kind of work they have to do to meet the standards and have a chance to study a curriculum that will enable them to do such work.
Two. A test that teachers can teach to. If you want a test that students can study for, then you also want a test that teachers can teach to. One element in this criterion is the same as it was for the students, that is, that it be clear one kind of student work meets the standard. Another is that the curriculum that must be studied to do well on the test fits into the time available to teach it.
It is not at all clear to me that the NAEP standards were created with an eye to this criterion at all. One of the useful ways to [decide] whether a test is designed to be taught to is to find the course of study guide for itif you will, the syllabus that lays out what topics are to be studied, what books should be read, what standard-meeting student work looks like, what work . . . does not meet the standards looks like, the rubrics that scorers will use to decide what score to give a particular piece of student work and so on.
To my knowledge, no such guide or syllabus is available for the national tests, nor are there any plans, to my knowledge, to produce one.
A test that will get studentsthis is the third point . . . to take tough courses and to work hard in school. Students take tough courses and study hard from Japan to Germany when they think that something they really care about depends on how well they do in school. Its hard to imagine why students from kindergarten through grade twelve are going to feel compelled to take tough courses and to study hard because of an English test they take in the fourth grade and a math test they take in the eighth grade.
What will make them take tough courses and study hard in school is an entire system of tests, culminating in a set of school-leaving tests they take at the end of high school, the scores on which play a major role in determining whether they go to college and what kind of job theyre likely to get when they leave high school if theyre not immediately going on to further education. One would have to argue that these two tests will somehow be expanded into such a series to make the case that the national tests will prove a powerful motivator of improved student performance. But I havent heard that argument made, and I dont see how it would work.
Lastly, instructional materials to match the test and the standards. I gather that the Department of Education has mobilized to identify resources that teachers can use to teach classes that are likely to improve student performance on the national tests, based on the best research available. This is, I think, a very important thing to do. But our experience suggests this isnt likely to be very effective until there are grade-by-grade standards, accompanied by materials matched to those grade-by-grade standards all the way through high school. This kind of resource is important for everyone, but it is absolutely essential for students who come from low-income and minority backgrounds that make them particularly dependent on the education they get in school. But there is actually much more to it than that.
The materials sent to those of us invited to testify in these hearings, by NAGB, include the statement that the national test is notN-O-T is capitalizedtied to a preferred curriculum, teaching method, or approach. That statement may represent a political asset, but that political asset is a very important pedagogical problem, especially for children from low-income and minority families with a history of poor performance in school.
Its almost certainly true that the most admired school tests in the United States and probably the most expensive are the College Boards advanced placement tests. It is very important to note that these tests are, in fact, tied to a known curriculum, issued by the College Board; and assistance is provided by the Board to teachers who want to know about the preferred method for teaching the courses that students take to prepare for the tests.
One of the most interesting facts in American education is that students who take these courses, especially students from families whose circumstances would predict low performance, often perform much better than those same students perform on the Scholastic Achievement Tests, now called the SAT-1. The Scholastic Achievement Tests, unlike the advanced placement examination, are tests one cannot study for, that is, there is no approved curriculum. Whereas, one is supposed to study an approved curriculum for the advanced placement tests.
The message here is that improving the performance of students who traditionally do not perform well depends very heavily on having an approved curriculum and teaching it to those students. This is what happens all over the world where the performance of the lower half of the distribution of performance is much higher than it is in the United States. And the scores that you see on the SAT-1s versus the advanced placement tests make it plain as day what is going on.
This shouldnt come as a surprise. Students from more advantaged backgrounds pick up a great deal of background information on . . . which they can draw . . . when taking what is really a general purpose intelligen[ce] test, like the SAT-1. Less advantaged students who dont have that knowledge nor any practical means of acquiring it quickly will do worse, even if equally intelligent. But give that disadvantaged student a curriculum that is focused on the right material, teach it well, and youve leveled the playing field in the most important way possible.
The prohibition against defining the curriculum to which the national tests are set is, in my view, a crucial obstacle to their potential for raising the performance of children from low-income and minority families.
An integrated system of standards, assessments, and instructional materials and techniques. This point really is an extension of the last one. The national tests are not the only entry in the national assessment sweepstakes, as weve just heard here in this room. Every state hasbut onenow has state standards, and a growing number have their own assessments, some matched to their standards and many not. Few have curriculum materials that are matched to either their standards or their assessments. There are, in addition, many standardized off-the-shelf tests in use by the schools. The demand is rapidly growing for systems of tests that are related to standards. As I just pointed out, the national tests are two tests. They are not a system of tests. Unless one imagines that these two tests are the nose of federal government camel under the national tent, its hard to see how they can greatly influence the outcome in an environment in which state after state is moving to grade-by-grade testing closely matched to the standards.
If that doesnt happen, then they will not greatly influence the companies that publish the textbooks, which means that few students will have an opportunity to study a curriculum that will enable them to do well on the tests, which, as I have just said, is, in my view, the only reason to have them.
So what to do? My bottom line is that national tests that arent set to national performance standards arent worth having. So the question is how to get national performance standards? My view is they wont come from the federal government because the country isnt ready for the federal government to produce them. It might be ready for the federal government to organize a conversation about such standards, though even that is not clear. Its much more likely that some or all of the states will organize such a conversation and lead it to a successful conclusion. The examinations should follow the standards, and close behind the standards and exams what will be most needed will be instructional materials and systems that are matched to the standards and exams. The federal government has, since the 1950s, had a mandate for funding the production of such materials and systems and that mandate has only to be renewed.
And, in fact, only the federal government has the resources to conduct the research and development programs needed to produce the instructional materials. In the meantime, I dont think the national tests will do either very much harm or very much good. Thanks.
Michael Farris Rebuttal
Mr. Cody: . . . Mike Farris, President of the Home School Legal Defense Association. Welcome, Mr. Farris.
Mr. Farris: After listening to the last couple of speakers, Im going to deviate from my written testimony, although I will turn it in as prepared. Just simply to say that I agree with much of what Marc Tucker just said, although I reach opposite conclusions based on the agreements. I dont think that a national test that is not tied to national standards will accomplish very much. But Im adamantly opposed to national standards and national tests.
Moreover, I want to question the whole concept of standardizing children. Where do we have research or where do we have any intuitive evidence that standardization works to achieve what we really want to achieve and that is maximizing the learning by each and every child[?]
My father is a retired elementary school principal, and, as I was growing up and studying education policy under his guiding hand and was being trained to be a lawyer for school districts in the early 1970sI switched sides after I saw what was happening to children . . . my dad told me the more and more standardization you see coming from central offices, from state departments of education, and ultimately from the federal government, the worse things will get for children. And thats exactly what we see.
Why do we think that education goods and services are any better or any different than consumer goods and services[?] Weve just seen the collapse of the Soviet Empire . . . where there [were] centralized national norms for how many refrigerators you were going to produce. The result was . . . they produced terrible refrigerators and not in sufficient quantity to meet the demands. But where the system of economics in consumer goods is allowed to run freepeople can produce whatever they want and meet the demands of the economythen we get very good consumer goods.
Why do we think standardized, centralized control of education will have any different result than standardized, centralized control of consumer goods and services[?] The Soviet system has failed, and the American experiment in dabbling in standardization and centralization is dragging our public school system into chaos, into academic failure. Theres no reason to believe that the long-term results will be any different.
On the other hand, the most free thinking, the most non-standardized system of education in America, works quite well, thank you very much. And thats home schooling. There are no standards whatsoever in this country for home schooling. Home schoolers do not have to meet any of these state standards. They do not have to meet any local standards.
In most states, they dont even have to take a standardized test, but they do anyway, so we can get comparison scores. It doesnt matter if you are a poor child, it doesnt matter if you are a black child, it doesnt matter if youre a Hispanic child, an Asian child, a child from a rich family, a child from a well educated family, none of those factors will diminish the basic fact that home school students on all of those segments will score 15 to 20 percentile points higher than their public school counterparts in every grade, in every subject.
Wheres the evidence that suggests that standardization works? The most non-standardized group in America is the highest performing group in America, in every grade, in every subject, in every income category. Now, thats a phenomenon that should be studied rather than experimenting and dabbling in the socialization and the further regimentation and totalitarian nature of the standards movement.
What Marc Tucker has proposed for you is an absolutely totalitarian scheme. And thats really the two choices: you either need to go to Marc Tuckers scheme or we should turn education back to teachers and parents and school districts and let them run them for themselves and the federal government should get out of the business altogether. Those are really your two choices. Were just messing around with the in-between and really messing things up and wasting a lot of money.
If theres evidence that the increased centralization of education that weve had over the last 30 years in the educational field has produced better schools, then lets have more centralization. But the fact of the matter is as centralization has gone up, educational performance in all measurable ways has gone down. At least in America, where we have traditions of individualism and not traditions of totalitarianism, like Germany and other places, where were trying to become like them, I dont frankly want to become like Germany or Japan, where kids are committing suicide over test scores.
Id like to see excellence, but not at the price of regimentation and totalitarian and groupthink kind of methodologies. Id like to see excellence based on individual freedom and individual initiative and hard work and things that cut across race and economic barriers.
And Id offer you all an experiment thats ongoing. About a million and half kids are being home schooled in this country today. Thats about the size of the public school system in the seventh or eighth largest state in the country. Collectively, its a pretty good sized experiment, and were getting very good results. I would urge you to pursue the course of non-standardization, where we dont worry about all children fitting into one standardized mold, but rather we worry about each child and maximizing each childs potential. And the only way an each child system will work is where the people who are making the decisions know that individual child.
People in Washington, DC, dont know the individual children. They dont love the individual children; . . . teachers know those children. They love those children. The parents know those children, and they love those children. When those people are the real decision makers, education flourishes. When those people are stripped of any real power as the standardized movement doesit strips the people of real power, it strips the parents of real powerthen education will be the one-size-fits-all pablum that . . . will continue to drag our nation down. Its only where teachers are empowered, parents are empowered, do students become empowered because education works for each child. It never works for all children.
Its for those reasons I would urge this board to abandon the folly of any sort of national test. It is just simply a waste of time, a waste of moneyunless were going to go to the totalitarian system proposed by Marc Tucker. And I, for one, would simply not think that that would be in the best interest of our nation, our traditions. If we want to produce free thinking and imaginative people who run a country based on freedom, we dont want people who have been trained in the regimentation of standards and totalitarian kind of thinking. Thank you very much. Ill give you copy of my results as well as the recent study of home schooling achievement.