Still Raising the Scores, Still Ruining the Schools

‘Standardised testing has swelled and mutated, like a creature in one of those old horror movies, to the point that it threatens to swallow our schools whole.’

Alfie Kohn, 2000

This was the dramatic – some might argue hyperbolic – opening to American academic Alfie Kohn’s ‘The Case Against Standardised Testing‘ (sub-title ‘Raising the Scores, Ruining the Schools’), published in the USA as long ago as the year 2000, but for those who accused him of scaremongering, and for the Scottish Government, which recently pledged to re-introduce standardised testing at regular intervals in the school-life of every young person growing up in Scotland, it is worth considering 15 years down the line whether Kohn’s fears have been vindicated, or whether the focus on tests really has improved the school experience, and performance, of young Americans.

standardized-test-cartoon-pictureFirst of all, let me summarise what I believe to be the main reasons for Kohn’s opposition to standardised tests, although I should point out that while he believes standardised testing to be a thoroughly bad idea, some forms of standardised testing are regarded as slightly less bad than others. I would also acknowledge that in summarising his position, one runs the risk of over-simplifying the case. As always, there is no substitute for buying the book and reading it in full, including the list of references and the research behind his conclusions.

  1. Standardised tests create the ‘illusion’ of objectivity. The results of the tests may sound scientific, since they are assigned a numerical score, but the reality is that they are set by adults who have an assumed ‘correct answer’ in mind, and taken by children with hugely differing experiences and attitudes, even on test day. It is not possible to remove subjectivity from the process.
  2. Standardised tests are no indicator of ability. If the justification for standardised tests is that we need to know what someone is capable of doing, there are very few less reliable ways of measuring that than a paper-and-pencil test, where the tasks are kept secret until the last minute. It is difficult to find examples of this kind of test being replicated in real life situations.
  3. Standardised tests tell us what we already know. The main thing standardised test scores tell us is how big students’ houses are. Research tells us that socio-economic factors (the amount of poverty in communities where schools are located) is the biggest factor in the variation of test scores from one area to another. To suggest therefore that standardised test scores are going to close an ‘attainment gap’ is demonstrably false.
  4. Standardised tests are mainly a test of memory. In the worst kind of standardised tests – those where children are asked to choose the right answer from a selection of possible answers – choosing the right answer gives no indication of understanding. Most standardised tests take no account of how an answer was arrived at, and bear no resemblance to problems faced in the real world.
  5. Standardised tests are designed to separate children into categories. The ultimate goal of standardised tests is not to evaluate how children have been taught, or how well they have learned. If a certain question is included in a trial paper and almost everyone gets it right – or if almost everyone gets it wrong – it will almost certainly be chucked out. Remember, the goal is not to test what has been learned, but to separate and categorise.
  6. Standardised tests teach kids (and teachers) the wrong lessons. When tests are given a status above all else in the education system, they contribute to the ‘already pathological competitiveness’ of the culture. The process of schooling becomes more about winning than learning, and we see others as barriers to our own success. In addition, an emphasis on remembering facts encourages a ‘pub quiz’ view of intelligence that confuses being smart with knowing loads of stuff.
  7. Standardised tests encourage the view that learning is something you do on your own. Tests are given to individuals, and supporting each other is known as ‘cheating’. In real life, learning is something we do with (and for) each other. Standardised tests don’t measure co-operation, collaboration, effort, empathy……..
  8. Standardised tests have inaccuracies built into them. Even when they are scored correctly, and meet the required standards for reliability, many children end up being ‘misclassified’ because of the limits of test accuracy.
  9. Standardised tests do not lead to greater accountability. A common justification for using standardised tests is that there are poor teachers out there and we need to find out who they are. This is based on a flawed logic. First of all, even if you believe that teachers are responsible for their students’ results, it would be irrational to hold a teacher responsible for the results of children who have recently arrived in his or her class. Secondly, and paradoxically, the test-driven teaching which results from the introduction of standardised tests actually reinforces what the worst teachers have been doing all along.
  10. Standardised tests stifle creativity. In an environment where high-stakes testing prevails, teachers become defensive and competitive, making sure everyone knows that low test scores were not their fault. Teaching to the test becomes the norm, and activities which don’t appear to contribute to test preparation are curtailed.
  11. Standardised tests narrow the conversation about education. The more that scores are emphasised, the less discussion there is about the goals of education. The content and the pedagogy of the school are adversely affected; the tests effectively become the curriculum. Spontaneity is discouraged, interesting pathways ignored. Children’s social, moral and intellectual development is put on hold.
  12. Standardised tests are educationally damaging. As teachers are encouraged not only to spoon-feed students the facts they will need to pass the tests, but to provide them with ‘test-taking’ skills, such as skimming a text rather than reading it deeply and reflectively, they spend less time helping them to become ‘critical, creative, curious thinkers’.
  13. Standardised tests don’t ‘raise standards‘. When teachers and students are forced to focus on only those things which can be reduced to numbers, such as how many grammatical errors are present in a piece of writing, the  process of thinking has been effectively relegated to a lesser importance. As the saying goes, we are then valuing what we can measure, rather than measuring what we value.
  14. Standardised tests discriminate against poorer children and parents. When the stakes are high, parents and schools use whatever means they can to achieve better results, which usually means buying more and better test preparation materials, or access to tutors and extra tuition. When schools decide to buy ‘reading schemes’ for example, as a quick fix, it is often at the expense of more exciting and interesting books and materials. The result is a narrowing of the learning experience generally for children in deprived areas.

kohn‘Testing allows politicians to show they’re concerned about school achievement and serious about getting tough with students and teachers. Test scores offer a quick-and-easy – although, as we’ll see, by no means accurate – way to chart progress. Demanding high scores fits nicely with the use of political slogans like ‘tougher standards’ or ‘accountability’ or ‘raising the bar’.

Alfie Kohn, 2000

Conventional wisdom used to have it that top U.S. students did well compared to their peers across the globe, when adjustments were made for higher poverty levels and racial diversity, but even allowing for these factors the latest available PISA test results, released in December 2013, showed that the best-performing U.S. students were falling behind even average students in Asian countries (or sub entities), which now dominate the top 10 in maths, reading and science. (source). In other words, even in the ‘pro-testers’ world’ and using the success criteria preferred by the pro-testing lobby, the relentless focus on testing does not appear to help kids perform better in standardised tests! It is of little surprise therefore that many leading academics are now questioning the validity of The PISA tests themselves, and the propensity for governments around the world to use them in determining educational policy (source). The key findings of that 2013 report demonstrate that not only were the serially-tested American youngsters failing to make any headway in global comparisons, but that the testing regime was having a damaging effect on their ability to think for themselves and apply their learning in real-life situations.

PISA 2012 Key Findings USA

  • Among the 34 OECD countries, the United States performed below average in mathematics in 2012 and is ranked 27th (this is the best estimate, although the rank could be between 23 and 29 due to sampling and measurement error). Performance in reading and science are both close to the OECD average. The United States ranks 17 in reading, (range of ranks: 14 to 20) and 20 in science (range of ranks: 17 to 25). There has been no significant change in these performances over time.
  • Mathematics scores for the top-performer, Shanghai-China, indicate a performance that is the equivalent of over two years of formal schooling ahead of those observed in Massachusetts, itself a strong-performing U.S. state.
  • While the U.S. spends more per student than most countries, this does not translate into better performance. For example, the Slovak Republic, which spends around USD 53 000 per student, performs at the same level as the United States, which spends over USD 115 000 per student.
  • Just over one in four U.S. students do not reach the PISA baseline Level 2 of mathematics student proficiency – a higher-than-OECD average proportion and one that hasn’t changed since 2003. At the opposite end of the proficiency scale, the U.S. has a below-average share of top performers.
  • Students in the United States have particular weaknesses in performing mathematics tasks with higher cognitive demands, such as taking real-world situations, translating them into mathematical terms, and interpreting mathematical aspects in real-world problems.
  • Socio-economic impact has a significant on student performance in the United states, with some 15% of the variation in student performance explained by this, similar to the OECD average. Although this impact has weakened over time, disadvantaged students show less engagement, drive, motivation and self-belief.
  • Students in the U.S. are largely satisfied with their school and view teacher-student relations positively. But they do not report strong motivation towards learning mathematics: only 50% of students agreed that they are interested in learning mathematics, slightly below the OECD average of 53%.

This week, the first signs appeared that America is about to admit that it got it wrong with George Bush’s inappropriately named ‘No Child Left Behind‘ reforms, when President Obama called for a reduction in testing in American schools (New York Times story), and a warning is issued today to the Scottish Government in the form of a report for the newly-formed left-wing political alliance, RISE. ‘Placing Our Trust in the Teaching Profession: The Case Against National Standardised Testing‘ uses several international studies to show that, far from reducing the attainment gap in education, the introduction of high-stakes national tests may well have the exact opposite effect.

Similarly, in its ‘Book of Ideas‘, the Scottish independent ‘think and do tank’ Common Weal had this to say to politicians seeking election to Holyrood next May:

‘But education should, at heart, be about improving our quality of life. This can mean many things. It can mean exposingideas ourselves to ideas and thoughts which expand how we see ourselves and our lives. It can mean learning coping skills to help us respond positively to the things that happen to us throughout our lives. It can mean giving us the skills to do the things we enjoy. It certainly means making us feel good about ourselves as valuable members of society. It certainly shouldn’t mean creating a system driven by the need to pass exams as a means of avoiding a bad life. The cycle of pressure and anxiety that an educational regime driven by testing exerts has been shown to change the brain chemistry of children and can affect them throughout their lives. You cannot test a child into being a happy, constructive and productive citizen.’

We have a government in Scotland which is enjoying unprecedented popularity, and which has worn its ‘progressive’ label as a badge of honour when others have sought to use it as a term of abuse. As far as the education system is concerned, the next few months will certainly put that commitment to progress to the test.

Further Reading:

The Guardian: Obama Calls For Cuts to Schools’ Standardized Testing Regimens

Diane Ravitch: The Badass Teachers Association Respond To Testing Announcement

Andy Hargreaves and Dennis Shirley: The Coming Age of Post-Standardization

Advertisements

Testing Time For Teachers?

“In defining literacy for the 21st century we must consider the changing forms of language which our children and young people will experience and use. Accordingly, the definition takes account of factors such as the speed with which information is shared and the ways it is shared. The breadth of the definition is intended to ‘future proof’ it. Within Curriculum for Excellence, therefore, literacy is defined as:

the set of skills which allows an individual to engage fully in society and in learning, through the different forms of language, and the range of texts, which society values and finds useful. “

Scottish Curriculum for Excellence: Literacy and English Principles and Practice

When ‘Teaching Scotland’s Future’, the Scottish Government report on the findings of the Review of Teacher Education, was published in 2011, one of its more controversial recommendations was the introduction of literacy and numeracy assessments for aspiring teachers, a strange suggestion – at least to my mind – in a country which already has an all-graduate profession, and where the overwhelming majority of new entrants has a Higher English qualification. Just over two and a half years later, the tests, which appear to be voluntary, have just been published on an Education Scotland website and greeted with predictable  media headlines such as this one in the Scotsman newspaper –  ‘TEACHERS TO BE GIVEN TESTS IN STANDARDS DRIVE‘ – thereby cementing in the collective consciousness an assumed relationship between the introduction of a test and the raising of those elusive ‘standards’. There already exists a set of ‘Standards for Registration’ for anyone entering the teaching profession in Scotland, and very good they are too. In fact they were revised recently, and you can find them on the General Teaching Council for Scotland’s website, which is where I would have thought anyone aspiring to a teaching career in Scotland, and with a modicum of ambition, would look first.

And so to the tests themselves (raising along the way the question of whether it is ever acceptable to begin a sentence with a conjunction). Despite the very broad definition of literacy in Education Scotland’s own CfE literacy framework document (see above), the new literacy tests consist of a very small number of questions on spelling, punctuation and grammar, such as this one, where you are asked to choose the correctly punctuated version of a short piece of prose.

Test.jpg

When you have clicked on your answer, the following text appears:-

Answer.jpg

As you can see, in this case the correct version of the sentence is number 3, for the reasons given. Except that it isn’t. None of the sentences is correct. The comma before the direct speech would suggest that the prefect was instructing the class to whisper the words ‘This noise is unacceptable’, which I don’t think she was. The comma in fact should be a full stop. You see, the problem with this kind of test is that what you are testing is pretty complex, and even when you think you have nailed it you are never quite sure what you are actually testing. Then of course a decision has to made about what percentage of correct answers makes a person ‘literate’ enough. At the end of another section of the tests – ‘confusing words quiz’ – for example, where you are invited to choose the correct version in context between two homonyms or similarly spelled words, a score of three from seven is deemed to be ‘reasonable’. How reasonable do you think 3 correct answers out of  7 is?

If this is a diagnostic tool to help aspiring teachers  judge for themselves which aspects of their language skills they need to work on – and the fact that the tests are voluntary and tucked away on a website which took me more than half an hour to find would suggest that it is – then all well and good. It may prove to be a useful resource in addition to those which are already out there. Unless I am missing something, the fact that the tests are not compulsory would also suggest that the government has rejected the recommendation to introduce some kind of additional ‘entry level’ examination, and decided on a different approach. If so, all credit to them – the hazardous nature of setting such a test has been demonstrated above. Standards will not be raised by introducing more tests, but through an understanding on the part of anyone entering the profession that their first commitment is to their own continuing programme of learning, and an acceptance of three basic principles:

  • We are learners first, teachers second.
  • Good communication is at the heart of all learning and teaching.
  • We are all learning to be more literate.

“Candidates for teaching should undertake diagnostic assessments of their competence in both literacy and numeracy. The threshold established for entry should allow for weaknesses to be addressed by the student during the course. A more demanding level should be set as a prerequisite for competence to teach.”

Teaching Scotland’s Future: Report of a review of teacher education in Scotland, 2011


Homework

To try the tests for yourself go to to the Aspiring Teachers website.

To read more about the vagaries of grammar teaching and testing visit this excellent blog by language expert David Crystal.

For a list of free websites to help with the development of English language skills click here.

Big Apple For The Teacher

kindlenytimesJust as Scotland’s teachers are digging deep into their final reserves of energy and ingenuity this week in the run-up to the long summer holiday, their efforts received a boost from the other side of the Atlantic in an article in the New York Times, which praises the different approach the country has taken to curriculum design from those in the rest of the UK – as well as the US – an approach which places less emphasis on standardised testing, has lighter-touch inspections, gives greater autonomy to teachers in their classrooms,  and has focused on a re-alignment of the balance between knowledge and skills. The key to this progress (I hesitate to use the word ‘success’ prematurely) has been a general consensus among the general public, the government and the professional teaching associations  – rarely referred to as ‘unions’ these days – as to the kind of educational system we want in a modern-day Scotland.

“In the same week that Britain’s (sic) education minister, Michael Gove, announced yet another measure to make the national exams taken by high school students in England more rigorous, their counterparts in Scotland were taking a curriculum in which national exams for 16-year-olds had been abolished……….

In 2005, Scotland introduced the Curriculum for Excellence. While education in England became increasingly prescriptive — with public debate on precisely what students were expected to know and whether, for example, there ought to be a greater focus on kings and queens, or the history of the British empire — the Scottish decided to pay more attention to how subjects were taught.”

Scottish Schools Focus On More Than Just Tests, New York Times, June 23, 2013

It is worth reminding ourselves how we came to this parting of the ways. In 2002 the then Scottish Executive undertook the most extensive consultation ever of the people of Scotland on the state of school education, through the National Debate on Education. Through that debate, most stakeholders – pupils, parents, teachers, employers and others – said that they valued and wanted to keep many aspects of the current curriculum, especially those principles which had a long tradition in this country stretching back to the introduction of public schools, and including:

  • the flexibility which already existed in the Scottish system – no one argued for a more prescriptive ‘national’ system
  • the combination of breadth and depth offered by the curriculum
  • the quality of teaching
  • the comprehensive principle (privately-funded schools account for around 5% of schools in Scotland)

Some also made compelling arguments for changes which would ensure all our young people achieved successful outcomes and were equipped to contribute effectively to the Scottish economy and society, now and in the future, changes which would:

  • reduce over-crowding in the curriculum and make learning more enjoyable (the implication being that it wasn’t enjoyable enough!)
  • better connect the various stages of the curriculum from 3 to 18
  • achieve a better balance between ‘academic’ and ‘vocational’ subjects and include a wider range of experiences
  • make sure that assessment and certification support learning (rather than lead learning as had been the case prior to the introduction of CfE)
  • allow more choice to meet the needs of individual young people

piperA key element of the changes has been the replacement of 5-14 national tests with The Scottish Survey of Literacy and Numeracy (SSLN), a national sample-based survey which monitors performance in literacy and numeracy in alternate years at P4, P7 and S2, and involves only a handful of randomly chosen young people from each school. Information from the survey is also used to inform improvements in learning, teaching and assessment within the classroom; it has been aligned with Curriculum for Excellence and includes written, online and practical assessments.

There is still much work to be done, especially I believe in respect of the last two objectives on that list of changes which people wanted to see, but as another academic year draws to a close, it is reassuring to know that what we are attempting to do here is attracting some admiring glances from other, bigger nations, who have perhaps found themselves seduced into thinking that more and better tests were the answer to better learning, only to discover that the two are only loosely connected.

The Tyranny of the Test

Proof yet again this week from the USA –  if more proof were needed – of the flawed logic of equating improvements in test scores with improvements in literacy, or indeed of believing that literacy can be improved by legislation.  According to a CNN report, one of the net effects of George W Bush’s flagship education act, No Child Left Behind,  is actually a lowering rather than a raising of standards. The act states that every child must be proficient in reading and maths by 2014, and schools which fall short of that target are subject to financial penalties. What would you do in that situation, faced with cuts in what is already a meagre budget, especially if your school was in one of the more deprived areas of the country? Exactly. In almost a third of states, the test score required for “proficiency” was lowered to the point where almost every student was able to pass, and since states are responsible for setting and assessing their own tests, this was not difficult to achieve. The end result was that in one state the score required for proficiency was 70% of that required in a neighbouring state.

reading

Source: New York Public Library: 1920s

What I find quite depressing about this story is not just the scramble to improve test scores, the desperation of governments and politicians to be seen to be improving standards, or the schools’ attempts to massage the figures and hang on to their budgets, but the fact that the most immediate concern of the CNN reporter, assuming to speak on behalf of parents if not the nation, is to find a way of making the test scores more reliable, robust and “standardised”, rather than engaging in a genuine debate about what it actually means to be proficient in reading, why it is necessary, and how it might be achieved for all young people.

It couldn’t happen here, could it?

Related Articles

Read the full CNN report by Randy Kaye here

Read Jaye Richards on Future Models of Assessment here