Still Raising the Scores, Still Ruining the Schools

‘Standardised testing has swelled and mutated, like a creature in one of those old horror movies, to the point that it threatens to swallow our schools whole.’

Alfie Kohn, 2000

This was the dramatic – some might argue hyperbolic – opening to American academic Alfie Kohn’s ‘The Case Against Standardised Testing‘ (sub-title ‘Raising the Scores, Ruining the Schools’), published in the USA as long ago as the year 2000, but for those who accused him of scaremongering, and for the Scottish Government, which recently pledged to re-introduce standardised testing at regular intervals in the school-life of every young person growing up in Scotland, it is worth considering 15 years down the line whether Kohn’s fears have been vindicated, or whether the focus on tests really has improved the school experience, and performance, of young Americans.

standardized-test-cartoon-pictureFirst of all, let me summarise what I believe to be the main reasons for Kohn’s opposition to standardised tests, although I should point out that while he believes standardised testing to be a thoroughly bad idea, some forms of standardised testing are regarded as slightly less bad than others. I would also acknowledge that in summarising his position, one runs the risk of over-simplifying the case. As always, there is no substitute for buying the book and reading it in full, including the list of references and the research behind his conclusions.

  1. Standardised tests create the ‘illusion’ of objectivity. The results of the tests may sound scientific, since they are assigned a numerical score, but the reality is that they are set by adults who have an assumed ‘correct answer’ in mind, and taken by children with hugely differing experiences and attitudes, even on test day. It is not possible to remove subjectivity from the process.
  2. Standardised tests are no indicator of ability. If the justification for standardised tests is that we need to know what someone is capable of doing, there are very few less reliable ways of measuring that than a paper-and-pencil test, where the tasks are kept secret until the last minute. It is difficult to find examples of this kind of test being replicated in real life situations.
  3. Standardised tests tell us what we already know. The main thing standardised test scores tell us is how big students’ houses are. Research tells us that socio-economic factors (the amount of poverty in communities where schools are located) is the biggest factor in the variation of test scores from one area to another. To suggest therefore that standardised test scores are going to close an ‘attainment gap’ is demonstrably false.
  4. Standardised tests are mainly a test of memory. In the worst kind of standardised tests – those where children are asked to choose the right answer from a selection of possible answers – choosing the right answer gives no indication of understanding. Most standardised tests take no account of how an answer was arrived at, and bear no resemblance to problems faced in the real world.
  5. Standardised tests are designed to separate children into categories. The ultimate goal of standardised tests is not to evaluate how children have been taught, or how well they have learned. If a certain question is included in a trial paper and almost everyone gets it right – or if almost everyone gets it wrong – it will almost certainly be chucked out. Remember, the goal is not to test what has been learned, but to separate and categorise.
  6. Standardised tests teach kids (and teachers) the wrong lessons. When tests are given a status above all else in the education system, they contribute to the ‘already pathological competitiveness’ of the culture. The process of schooling becomes more about winning than learning, and we see others as barriers to our own success. In addition, an emphasis on remembering facts encourages a ‘pub quiz’ view of intelligence that confuses being smart with knowing loads of stuff.
  7. Standardised tests encourage the view that learning is something you do on your own. Tests are given to individuals, and supporting each other is known as ‘cheating’. In real life, learning is something we do with (and for) each other. Standardised tests don’t measure co-operation, collaboration, effort, empathy……..
  8. Standardised tests have inaccuracies built into them. Even when they are scored correctly, and meet the required standards for reliability, many children end up being ‘misclassified’ because of the limits of test accuracy.
  9. Standardised tests do not lead to greater accountability. A common justification for using standardised tests is that there are poor teachers out there and we need to find out who they are. This is based on a flawed logic. First of all, even if you believe that teachers are responsible for their students’ results, it would be irrational to hold a teacher responsible for the results of children who have recently arrived in his or her class. Secondly, and paradoxically, the test-driven teaching which results from the introduction of standardised tests actually reinforces what the worst teachers have been doing all along.
  10. Standardised tests stifle creativity. In an environment where high-stakes testing prevails, teachers become defensive and competitive, making sure everyone knows that low test scores were not their fault. Teaching to the test becomes the norm, and activities which don’t appear to contribute to test preparation are curtailed.
  11. Standardised tests narrow the conversation about education. The more that scores are emphasised, the less discussion there is about the goals of education. The content and the pedagogy of the school are adversely affected; the tests effectively become the curriculum. Spontaneity is discouraged, interesting pathways ignored. Children’s social, moral and intellectual development is put on hold.
  12. Standardised tests are educationally damaging. As teachers are encouraged not only to spoon-feed students the facts they will need to pass the tests, but to provide them with ‘test-taking’ skills, such as skimming a text rather than reading it deeply and reflectively, they spend less time helping them to become ‘critical, creative, curious thinkers’.
  13. Standardised tests don’t ‘raise standards‘. When teachers and students are forced to focus on only those things which can be reduced to numbers, such as how many grammatical errors are present in a piece of writing, the  process of thinking has been effectively relegated to a lesser importance. As the saying goes, we are then valuing what we can measure, rather than measuring what we value.
  14. Standardised tests discriminate against poorer children and parents. When the stakes are high, parents and schools use whatever means they can to achieve better results, which usually means buying more and better test preparation materials, or access to tutors and extra tuition. When schools decide to buy ‘reading schemes’ for example, as a quick fix, it is often at the expense of more exciting and interesting books and materials. The result is a narrowing of the learning experience generally for children in deprived areas.

kohn‘Testing allows politicians to show they’re concerned about school achievement and serious about getting tough with students and teachers. Test scores offer a quick-and-easy – although, as we’ll see, by no means accurate – way to chart progress. Demanding high scores fits nicely with the use of political slogans like ‘tougher standards’ or ‘accountability’ or ‘raising the bar’.

Alfie Kohn, 2000

Conventional wisdom used to have it that top U.S. students did well compared to their peers across the globe, when adjustments were made for higher poverty levels and racial diversity, but even allowing for these factors the latest available PISA test results, released in December 2013, showed that the best-performing U.S. students were falling behind even average students in Asian countries (or sub entities), which now dominate the top 10 in maths, reading and science. (source). In other words, even in the ‘pro-testers’ world’ and using the success criteria preferred by the pro-testing lobby, the relentless focus on testing does not appear to help kids perform better in standardised tests! It is of little surprise therefore that many leading academics are now questioning the validity of The PISA tests themselves, and the propensity for governments around the world to use them in determining educational policy (source). The key findings of that 2013 report demonstrate that not only were the serially-tested American youngsters failing to make any headway in global comparisons, but that the testing regime was having a damaging effect on their ability to think for themselves and apply their learning in real-life situations.

PISA 2012 Key Findings USA

  • Among the 34 OECD countries, the United States performed below average in mathematics in 2012 and is ranked 27th (this is the best estimate, although the rank could be between 23 and 29 due to sampling and measurement error). Performance in reading and science are both close to the OECD average. The United States ranks 17 in reading, (range of ranks: 14 to 20) and 20 in science (range of ranks: 17 to 25). There has been no significant change in these performances over time.
  • Mathematics scores for the top-performer, Shanghai-China, indicate a performance that is the equivalent of over two years of formal schooling ahead of those observed in Massachusetts, itself a strong-performing U.S. state.
  • While the U.S. spends more per student than most countries, this does not translate into better performance. For example, the Slovak Republic, which spends around USD 53 000 per student, performs at the same level as the United States, which spends over USD 115 000 per student.
  • Just over one in four U.S. students do not reach the PISA baseline Level 2 of mathematics student proficiency – a higher-than-OECD average proportion and one that hasn’t changed since 2003. At the opposite end of the proficiency scale, the U.S. has a below-average share of top performers.
  • Students in the United States have particular weaknesses in performing mathematics tasks with higher cognitive demands, such as taking real-world situations, translating them into mathematical terms, and interpreting mathematical aspects in real-world problems.
  • Socio-economic impact has a significant on student performance in the United states, with some 15% of the variation in student performance explained by this, similar to the OECD average. Although this impact has weakened over time, disadvantaged students show less engagement, drive, motivation and self-belief.
  • Students in the U.S. are largely satisfied with their school and view teacher-student relations positively. But they do not report strong motivation towards learning mathematics: only 50% of students agreed that they are interested in learning mathematics, slightly below the OECD average of 53%.

This week, the first signs appeared that America is about to admit that it got it wrong with George Bush’s inappropriately named ‘No Child Left Behind‘ reforms, when President Obama called for a reduction in testing in American schools (New York Times story), and a warning is issued today to the Scottish Government in the form of a report for the newly-formed left-wing political alliance, RISE. ‘Placing Our Trust in the Teaching Profession: The Case Against National Standardised Testing‘ uses several international studies to show that, far from reducing the attainment gap in education, the introduction of high-stakes national tests may well have the exact opposite effect.

Similarly, in its ‘Book of Ideas‘, the Scottish independent ‘think and do tank’ Common Weal had this to say to politicians seeking election to Holyrood next May:

‘But education should, at heart, be about improving our quality of life. This can mean many things. It can mean exposingideas ourselves to ideas and thoughts which expand how we see ourselves and our lives. It can mean learning coping skills to help us respond positively to the things that happen to us throughout our lives. It can mean giving us the skills to do the things we enjoy. It certainly means making us feel good about ourselves as valuable members of society. It certainly shouldn’t mean creating a system driven by the need to pass exams as a means of avoiding a bad life. The cycle of pressure and anxiety that an educational regime driven by testing exerts has been shown to change the brain chemistry of children and can affect them throughout their lives. You cannot test a child into being a happy, constructive and productive citizen.’

We have a government in Scotland which is enjoying unprecedented popularity, and which has worn its ‘progressive’ label as a badge of honour when others have sought to use it as a term of abuse. As far as the education system is concerned, the next few months will certainly put that commitment to progress to the test.

Further Reading:

The Guardian: Obama Calls For Cuts to Schools’ Standardized Testing Regimens

Diane Ravitch: The Badass Teachers Association Respond To Testing Announcement

Andy Hargreaves and Dennis Shirley: The Coming Age of Post-Standardization

Found In Translation

One of the highlights of last week’s Scottish Film and Learning Festival was Rob Smith’s presentation ‘Using Film in the Classroom‘, which you can hear (though unfortunately not see) on the Radio EduTalk website by clicking here, and I would suggest that one of the reasons Rob’s workshops and Literacy Shed website are so popular, is that he is thoroughly convincing when he argues that using film in the classroom is the key to unlocking creativity, especially when it comes to the quality of children’s writing. And that is the point. Reading or watching film is often seen as an alternative to using printed texts, which leads to a polarised debate about the relative merits of films and books. ‘Books allow you to use your own imagination, while in a film the director has done all the work for you’, the argument goes, ‘and surely the only way to improve writing skills is by studying WRITTEN texts?’

If you listen to Rob, you will discover the fallacy of both statements, and if you accept that using books and using film in the classroom are not mutually exclusive, you will have made the problem disappear. Keep in mind also that there are many ways to create texts, and the written word is only one of them. Which is why one of my Ten Tools For Reading Film is the grandly titled ‘Generic Translation’, an approach which allows teachers and students to experiment with media and come to understand the possibilities each of them presents. Take this example of a short animation, based on the Charles Bukowski poem ‘The Man With The Beautiful Eyes’. What better way to develop an understanding of metaphor than by studying the printed text and the animation side-by-side.

You will find more detailed suggestions on how to use this film in the classroom, as well as many others, at the Moving Image Education website by clicking here.

To listen to more talks from the Scottish Film and Learning Festival see previous post.

A Feast of Film

Intros

Introducing the first Scottish Film and Learning Festival

You know that feeling you get when you have been planning a big event for the past few months and suddenly it’s over? That’s how I’m feeling today, after a truly fantastic day at the first Scottish Film and Learning Festival in Glasgow on Saturday. The area around George Square really did have a festival feeling, as the Great Scottish (Children’s) Run was also in town, but not even the samba band immediately outside one of the conference venue’s seminar rooms could dampen the enthusiasm inside. For those of you who were not able to attend this time, here is the complete list of presenters and presentations. If you click on the title of the presentation it will take you to some further information or resources related to the speaker and/or the presentation topic. A big thank you to John Johnstone from Radio EduTalk who came along and captured some of the presentations, which you can hear by going to the EduTalk website.

John Murray – Reading Explorers

Jo Hall – BBC L.A.B.

Sarah Wright – The Show-Stopping Toolkit

Rob Smith – Using Film in the Classroom

Mark Reid – Cinematheque Francaise and Understanding Cinema

Tim Flood – Draw What You See

Jonathan Charles – Using Storyboards to Develop Visual Literacy

Claire Docherty – Using the Scottish Film Archive in the Classroom

Bill Boyd – Ten Tools for Reading Film

Sarah Derrick – Discovery Film Festival DCA

Athole McLauchlan – Film Studies in Social Studies

David Griffith – From Shots to Sentences

Barbara Hill and Gordon Brown – SQA and the Scottish Survey of Literacy and Numeracy

Jo Spence – Into Film Programme for Schools

Craig Steele – Movie Mashup

Jennifer Jones – Commonwealth Digital Project

Glow Scotland – Using Glow to Enhance Visual Literacy

Bruce Eunson – Film and the Scots Language