Measure then fix – why 107 academics are wrong

Posted byLance Wiggs 42 Comments on Measure then fix – why 107 academics are wrong

A group of academics signed off on a letter against school league tables. The stated logic may work in an academic research setting but is inappropriate to apply to the real world. We should instead publish the measurements, improve the measurements and their context over time and, most importantly, focus energy and resources on understanding the issues and helping the schools at the bottom of the league.

Let’s look at the reasoning used in the media briefing note:

1. National Standards data are unsuitable for comparing schools The performance of schools cannot meaningfully be compared with each other unless it can be demonstrated that assessment measures, processes and moderation have been used consistently across schools. <snip>

The argument is that schools have high variability between each other and across years. It’s a combination of measurement error based on inconsistent and low samples and the national standards only measuring numeracy and literacy and not more holistic skills.

However to improve something we first need to measure it, and if we can’t measure it accurately then an approximation will do. In business that means using surveys of customers that have clear sampling bias, reacting more to customers who complain and even believing what we read in the papers. We know all of these sources are incomplete and have bias, but we can account for it somewhat, and are much improved by using the input. The online advertising industry is a lovely example, using a system for measurement that is clearly wrong to measure traffic, but while it is wrong, it is wrong for everyone, and it’s only the starting point for a conversation.

It’s far easier to start a conversation about the quality of a school when confronted with a combination of the socieoeconomic data about the catchment area and the National Standards results over time.

2. The contextualising data are incomplete

Many elements of the school’s local community context affect teaching and learning processes and children’s achievement. These include socio-economic and other intake differences (such as ethnicity, student transience rates, the proportion of English language learners or children with special needs) and other school and area characteristics (local labour market, urban/rural location, popularity compared to surrounding schools).

There are also internal school contexts, such as past leadership or reputational issues, significant staffing changes or schools being damaged.

Many attempts at comparing school performance do not even try to use the best available statistical methodologies. Instead the school decile rating is typically used as a proxy for all these contextual indicators. 

I agree. The National Standards data are only one piece of the puzzle, and the puzzle needs to be completed.

However we need to start somewhere, to create a minimum viable product and steadily improve it over time. While many criticized the early versions of iPhone, Xero and even Powershop, the steady improvement in functionality and usability were what won consumers over time. It’s the same with a measurement system that relies on a variety of data. Some of the early data will be wrong, and some of the things measured will be missing, but we should accept that and move to steadily improve the quality and context over time. If we don’t have the right socioeconomic data, for example, then someone will find it and mash it up with the National Standards data. The 107 academics are ideally placed to perform this work.

The reference source of information on schools will be the website (no doubt) that combines the highest quality information in a way that is meaningful to parents, teachers and students. Releasing the data in an open form is the first step towards creating  complete school reports across a broad spectrum of facets.

I understand the natural academic reluctance to never release data that is potentially wrong, and I see that in business sometimes where companies do not want to release an imperfect product. But while they are polishing the bezels yet again competitors are releasing their inferior but higher selling versions. Similarly we should release the data, and call on the power of academics, hundreds of thousands of parents and even students to provide both sunlight as a disinfectant and the right context.

3. League tables are educationally harmful

The compilation and release of achievement data in league tables to enable comparison of schools has the potential to cause harm: to learners, teachers, schools and local communities.

These harmful behaviours include: ‘teaching to the test’ and ‘narrowing of the curriculum’; valuing of some students over others because of their ability to perform and to conform; prioritising the teaching and other support given to some students over others in order to maximize the numbers that ‘reach the standard’; and damaging effects on students’ anxiety levels and conceptions of themselves as learners – ‘I’ll be below standard’.

All systems can be gamed, and business is no exception. Larger businesses often make decisions based on their corporate structure and internal politics rather than on the facts at hand. Inside a business the ‘wrong’ people may be promoted as they are better at understanding their boss’s requirements than their harder working colleagues.

However it’s a lot easier to understand where there are performance issues when we have at least some degree of measurement. Early warnings in business are easy in some areas, like sales departments who have targets, and much harder  in others such as marketing. But it’s common to have at least some concrete measurements on your personal scorecard.

The really smart businesses understand that people will game the system to over-achieve on their personal scorecards, and make sure that the personal goals are 100% aligned with the corporate goals. Nobody complains about the sales guy who brings in another $1 million of business by pushing harder at the end of the quarter – in fact it’s common to see spiky sales results based on timing of bonus periods. Harmful gaming, such as stealing customers, is deadly however, and jumped on very quickly.

It’s can be the same with schools. No doubt the current system, whatever it is, is gamed. Some schools will get more resources than others simply because they are better at working with the system or at fund raising through other sources. Point England school has done incredible things on a tiny budget, because they worked in a smarter way.

So any measurements must cascade from the goals of the education system, and we should fight to remove bias and the potential for harmful gaming. I would imagine that getting people through the National Standards is the ultimate purpose of the system, and that means increasing the quality of the standards themselves each year as well as helping students achieve. We need to be constantly aware of the potential negative impacts, and, quite simply, measure and set goals for them. If losing poor students early is a problem, then make student retention a critical measure of performance, and so on. It may become a real battle, but ultimately a catch-all “don’t game the system or else” will bring out the best side in our educators.

4. The political argument for league tables is weak

The argument that the Ministry of Education should release league tables in order to prevent the media doing so, does not address the problems that their effects will be damaging and the data used to compile the tables will be incomplete. <snip>

A long piece containing several arguments about why releasing the data is bad.

However while it might be considered bad by the academics, it is not by myself, and more pertinently, at least some parents. While even a small minority, and this is not a small minority, wants access to our data, New Zealand has a policy and obligation to provide it. Arguing against releasing data is quite remarkable for a group of academics. It should be easier to understand school performance than to read about individual student’s private lives on Facebook.

In particular, the moral principle of social justice demands that the situation of the most disadvantaged in our society should not be made worse through the release of official information. 

Businesses often ignore the fundamental problems, such as with their business model, until it is too late to do anything about it. As a consultant to businesses tiny to large I generally focus on these problems, often ignored before my arrival. A great reporting system which highlights emerging problems and a smart management team that follows up on them keeps the expensive consultants at bay.

It’s the same here – the moral principle of social justice demands that the situation of the most disadvantaged in our society be identified and fixed, and not hidden from public view. We can fix these broken schools, and we don’t have to look further than Wellington High or Pt England to find great examples.

I am somewhat dismayed at the attitude of the educators, although I do understand their reluctance to release what is seen as incomplete data from an academic study perspective.

From a society perspective there is at least some demonstrated demand from parents.

From a business perspective there are a number of businesses and individuals who would  love to mash this data up to create something new and useful.

But, most of all, from an educational perspective, releasing the data as a league table will allow us all to ask the hard questions of everyone involved – how are we going to help the schools at the bottom?

Published by Lance Wiggs

@lancewiggs

42 replies on “Measure then fix – why 107 academics are wrong”

  1. We’re talking about children. Not businesses or profits. New Zealand’s educators rank very close to the top of the table in performance, while being well down on funding and teacher:student ratios.

    To me this says we should direct more money to education and let it be used as the educators see fit.

    Would Point England be able to do the things they are doing now if their standard test results show they are low down the table? Or will they be spending time trying to improve their standard test results, like has happened to many schools in the USA?

    Like

    1. We are talking about children, and our future, and we need to do everything we can to help. I’m shocked at a movement to close off data about anything, especially at something as critical as this.

      So let’s help them get better. The bar is moving all of the time, and we can do a lot more. Some students, schools and educators are excellent, but others are not doing so well. So let’s do what we can to identify and emulate the best educators and schools, and help the laggards achieve much more.

      Student measurement, and change in student measurement over time in particular, is part, but not all of the solution, and certainly in needs to improve over time.

      Like

      2. Look, here’s a gap identified, with measurements, and suggestions on ways to close it: http://www.ero.govt.nz/National-Reports/Literacy-and-Mathematics-in-Years-9-and-10-July-2012/Findings/Setting-improvement-goals-and-targets-for-Years-9-and-10-students-in-literacy-and-mathematics

        Why, WHY are we throwing money and time at a solved problem Lance? Wouldn’t you counsel a business with a fantastic reporting system to carry on using it, rather than add more measures that will just burn time?

        Like

  2. If it’s numbers you’re after, we already have them. The OECD regularly compares schools and education across countries and there’s good news. New Zealand regularly ranks in the top ten. We beat out the UK, Australia, the US, Canada in almost every measure. We’re typically up there with the Scandinavian countries, some of the smaller city states and those countries that teach their kids how to pass exams, but not much else.

    http://www.oecd.org/document/61/0,3746,en_32252351_32235731_46567613_1_1_1_1,00.html

    We have one of the world’s best education systems yet spend relatively little on education. Again, the OECD data says we spend less per child than most countries in the world.

    We do indeed have a tail – Maori and Pacific Island children in particular don’t do as well as their European/Asian/Other classmates – but we’re addressing the problem and as with all educational issues it takes time to filter through. The good news is that in the time between 2009 when the main results from OECD came out and today, we’ve improved our tail dramatically (if tails can be said to improve).

    The problem with National Standards is that there really isn’t a national standard associated with it. We teach our children a standard curriculum, but oddly the curriculum is not the basis of National Standards, so in effect we’re teaching one thing and testing something else entirely. On top of that, our National Standards don’t line up with the testing that’s gone on before – that’s ok if the new test is better, but from what my own school has seen so far the new test is actually about 12 months behind the old. That is, we’re testing seven years olds on whether or not they can beat the tests they did last year. This way we’re sure National Standards will be a “success” because all the kids will pass. This is great if you’re implementing a new regime that needs results but lousy if you want children to grow and develop to their full potential.

    National Standards also test broadly across the whole school cohort (sorry, edu-speak – across the whole school population). That’s fine if all the kids are the same or come from the same background or have the same potential but here’s a rarely aired problem: not all kids are the same. Again, at my school we have a unit for kids with severe disabilities. There are a number of kids who will be in care for their entire lives – they are at school as much for the social element as for education. Yet they and the school that teaches them will be judged as failures because they aren’t “achieving”. That’s not fair to the children and it’s certainly not going to reflect well on the school that spends time, money and energy (so much energy) on these kids.

    As for league tables, I don’t know. Look at the Metro article on Auckland schools – it’s a nice, objective approach to the issue of ranking schools and the Catholic schools won through time and again. So does that mean they should teach all our kids? Well, possibly, but perhaps the data is skewed by selection. The Catholic schools take kids from a different set of criteria than state schools. State schools take everyone and anyone who applies from within their catchment area – they don’t get to say “Sorry but your child doesn’t appear to be willing/able/capable of learning to the required standard” because that’s not how state run schools (or hospitals) work. Much like home, these are places where when you have to go there, they have to take you in.

    Measuring student achievement at school isn’t a great way of telling whether the schools are doing well. It’s just not that simple. Rather, we should look at results like the PISA data that compares us with other nations and with as broad a range of outcomes as possible. On that basis, we should double our teachers’ pay and encourage our educators to teach teaching at an international level because we are world class. We should be exporting our learning (but not our people) to the rest of the world.

    Like

    1. In a good news story I stil find it strange how it’s ok to measure using OECD league tables, but not domestic ones. If those domestic numbers are broken then the disinfecting sunlight will make it plain for all to see, and that’s the point here.

      For sub-cohorts who are disadvantaged it’s clear that a 1% score is not useful, but the second part of an appraisal which talks about attitude and so on is. Why do we assume though that we will judge schools on the absolute score, when there are so many more useful things to measure over time, and by class as they progress? Moreover why are we not shouting to the rooftops about pockets of low scores and doing what we can (lower class sizes, special units, intervention) to fix the issue? All this just brings some of that activity into the public eye, and not before time.
      We should look at PISA results, although they are a lagging indicator, ad we should look at broader outcomes over time – a really lagging indicator. We should also look at as many other indictors as ew can, and we should seek to continuously improve the measurement systems as well as the results.

      Like

      1. “We should also look at as many other indictors as ew can, and we should seek to continuously improve the measurement systems as well as the results.”

        Which is *exactly* what we are doing, and what we have been doing, and why our education system is so great. Go and look up an ERO report on any school. We did when looking for a suburb to live in, the information is all there out in the open, ready to be disinfected.

        Why demand new numbers, that will take more time and effort to produce, and will (not might, will, based on international experience) result in time wasted, gaming, and poorer quality teaching?

        Like

    2. The Metro article took a small set of quantitative data then divided it by the school’s decile ranking. It’s a crackpot methodology that gave rise to one of the top schools (according to Metro) being one that is in danger of closing because of low enrolments. Insane.

      Like

  3. I agree with Ben. To say you need to measure something to fix things implies your metrics are infallible and that you know everything.

    They’re not and you do not. Children aren’t simple iDevices whose development can be charted and compared to others in some elitist vanity project.

    Adding further bureaucracy and performance angst to the educational system won’t help anyone, least of all the children.

    Thing is, learning is a life long process that depends on teaching being available. Tests and results are not anywhere near as important as that.

    Like

    1. > Thing is, learning is a life long process that depends on teaching being available. Tests and results are not anywhere near as important as that.

      Tests and results give one strong indicator of when teaching is not available or indifferent.

      Progress of students can, indeed, be charted over time and within and between cohorts. Refer to the body of work that has spun out of the US cohort datasets.

      Like

        1. An excellent article, though the author rushes to place the blame on testing without considering the massive other problems that the US educational system has. It’s funding, for example, is principally derived from local taxes on property. Poor areas have more kids, lower value property and therefore lower quality schools, which can get top-up federal (and state perhaps) funding. Wealthy areas have less kids, higher house values and are therefore generally taxed less as a percentage of the asset value and the schools are wealthier. It’s vicious.

          Nobody is suggesting that we abandon everything we know about how to teach in favour of a relentless testing regime – and suggesting that this is the case is alarmist. However we are missing a consistent fact-based progress report that can be cascaded up through teach, school and regional level. If teachers are not fans of the system, as some appear not to be, then let them teach as they do, and the kids will perform regardless. hat much is clear from the US article – it’s poor practice to teach to a test, far smarter to lead the voyage of discovery and check progress periodically with a test.

          Like

      2. And to be clear, yes, I *know* we need to measure people, and that measuring people results in datasets. What I’m saying is the way ERO measure schools, teachers, and students (by looking at them holistically) is a much better way than taking an umbrella number.

        Like

        1. Ben,

          If the ERO data is so good, allow me to issue a (tongue placed firmly in cheek) challenge:

          Create a site like the one Lance is describing using the ERO data. Allow me to compare and contrast without the need for expensive and potentially ineffectual measures. Then everyone wins – Lance has his metrics to improve upon and you have your qualitative measures :)

          Like

        2. I’m reading through the ERO reports for schools – and increasingly shocked. You absolutely have to read between the lines. Here’s one example:

          “Urenui School is a small, rural primary school located north of New Plymouth. At the time of this review in April 2012 it had a roll of 46 students, a significant decrease from the 77 enrolled at the time of the previous ERO review in 2008.”

          Wait – what? The school roll dropped by 40% and it took four years for the ERO reviewers to stop by?

          “Most students are positively engaged in their learning. Classrooms are well resourced. An ongoing focus for the principal and staff has been building consistency of teaching practice.”

          An ongoing focus… translates to me as there are real problems with some teaching practices – now is that in particular subjects, with particular teachers or what? There is no idea here.

          “ERO and the school are aware of ongoing issues involving relationships between the school and a section of the community. Resolving these issues is an agreed priority.”

          What issues? What section of the community? The mind boggles. I have absolutely no idea what this is from the report.

          “During the review, ERO checked the following items because they have a potentially high impact on student achievement:
          – emotional safety of students (including prevention of bullying and sexual harassment) physical safety of students
          – teacher registration
          – stand-downs, suspensions, expulsions and exclusions
          – attendance.”

          However the ERO report gives no outcomes for that checking.

          Like

        3. Thanks for all the links Ben. The ERO reports are far to generic to be useful for me – they leave me begging for both real data beyond ethnicity and decile, and wanting to read the report that is actually written and not the one released to the public. That’s assuming there is one – and if not then a 3 yearly review of that superficiality is poor.

          The intent of the ERO is good though – but wouldn’t it be bolstered by a bunch of hard numbers? They could use those as a starting point, much as they use the crude ethnicity and decile numbers as a starting point now. L

          Like

        4. So precisely how is standardised testing going to help here Lance? In what way will test scores tell you about ho Urenui is dealing with their local community? What will a number tell you about stand-downs, suspensions, expulsions and exclusions?

          Perhaps Urenui’s test scores will improve, because they’re excluding their worst students. Great result!

          What the ERO report tells you is that someone is monitoring the school.

          Like

        5. The point being made is that the ERO is inadequate. League tables showing vital stats including roll should raise red flags well before. Four years between reviews is clearly insufficient for this school.

          Like

  4. Lance,
    I think you get a side effect from this that is undesirable.

    As a guilty party of this behavour the problem you get is that rather than schools being improved those that can will direct their children to more “successful” schools. Zoning helps to some extent but this is still obviously worked around when you look at property purchases and rent values in specifc school zones.

    The issue is this then leaves the school with a lower ranking the chidren of those that cannot vote with their feet and the performance averages are often brought down further…

    Q

    Like

  5. Oh, and I forgot one small anecdote about the impact of labelling kids at an early age.

    My parents were born at the end of World War II. Both have had successful careers in medicine and teaching. My mother has an MBA, ran various hospital and public health units and was director of nursing before she retired, yet to this day remembers vividly (and is upset in a way nothing else affects her) by the memory of being a failure at school. She failed an exam and was made an example of in front of the whole school.

    She was eleven years old.

    My principal has just come back from a sabbatical trip to the UK – in London she visited a school that has hired a deputy principal whose sole job is to collate test data for the ministry. This DP doesn’t teach kids – all they do is check the test data and submit it for the various tables and charts that are made public. Oddly the principal says she would rather spend the money on a teacher who was allowed to teach but that, apparently, is not an option.

    Like

    1. Of course, Lance’s argument rests on the premise that it is possible to formulated standards such that “…any measurements cascade from the goals of the education system…” and that we “…fight to remove bias and the potential for harmful gaming.”

      I just can’t see it happening in such a highly politicised and emotionally charged arena. Unlike Lance, I see opposing national standards as an act of pragmatism. That might be perceived as *giving up*, but I would hope that the first lesson that business could teach education is learning from others mistakes.

      Like

    2. I’m sorry to hear about your mother’s experience; it was clearly traumatic but perhaps representative of the way children were treated in the 1950’s and 60’s. I don’t believe NS or league tables would necessarily give rise to the humiliation of individuals as happened to your mum.

      I think an important question though is: what happened next? Your mum is clearly a success both educationally and in her career (as well as a mother), despite this brutal slap in the face. Is it possible that the fear of failure, and a determination never to be seen to let the school down again, were drivers for her to succeed?

      Point 3 of the academic’s argument seems to be very defeatist, and ignores the basic human instinct and drive to succeed, to improve and to do better, Better than last year, better than the previous class in this year, better than the next school, better than the other Island. We should be fostering and encouraging the aspirational instinct, not decrying competition because you assume children will naturally think they’ll be “below standard”.

      Like

  6. I think you’re giving the Government too much credit when you talk about improving the measures over time. I also think you’re giving them too much credit with the comment that “The reference source of information on schools will be the website (no doubt) that combines the highest quality information in a way that is meaningful to parents, teachers and students”. I think what you’re more likely to see is the bare minimum spent on this piece of work, much like everything else in Government these days. It will be poorly implemented and rather than incrementally improved, it will be left to languish until a big bang project is needed. Rinse and repeat, it’s the Goverment way here in NZ.

    Like

      1. I don’t agree that *only* people with children are “qualified” to comment (that sort of argument is what fuels the Garth McVicar’s of the world).

        That said, you’ve pretty much admitted that any such programme would involve a period of gaming and manipulation that has been proven to have pretty terrible outcomes for children. After an indeterminate period of refinement and fixing, we hope that the new standards will then accurately and fairly reflect performance (as determined by…?), at which time we can then address the issues.

        A business owner can and should play the long game. But as a parent of a pre-schooler you’re asking me to risk the education of my child (in a system which, by international measures, appears to be pretty good) on the ability of government wonks to get this system right in time, against the advice of educational specialists. And if educational specialists, who presumably we should entrust with aligning national standards with educational outcomes, think it’s a terrible idea, how exactly are we supposed to achieve this?

        Like

        1. As someone who went through NCEA, I can tell you that the experimentation is definitely disruptive and disheartening for students (95% get Excellence on one standard, 95% fail the next!? What gives?)

          However, the most disruptive thing for my ongoing education was definitely the large number of teacher strikes…

          Like

      2. no, ad hominem would be calling you names. calling you out on a pretty major lack of experience in a given field is not. I have no kids either, but don’t presume to tell people with kids what to do.

        Like

        1. I presume to want to help people with kids, and kids themselves, make smarter decisions about which schools to attend. And presume to help New Zealand get better educated so that we can sustain an even better society.

          I’m astonished at how well we are doing – the schools of today are turning out amazing people, and what it means to be a New Zealander has forever changed. But progress is important, and “trust me, I’m an expert” is never good enough.

          Like

  8. An interesting take, but I disagree with the whole analysis and the business/technology analogy made me feel dirty. (I found it difficult to carry on reading after the words “minimum viable product” entered my brain, but I persevered manfully.) Then, this sprang to mind for some reason:

    Inchworm, inchworm
    Measuring the marigolds
    You and your arithmetic
    You’ll probably go far

    Inchworm, inchworm
    Measuring the marigolds
    Seems to me you’d stop and see
    How beautiful they are

    Like

    1. Minimum viable product referred to the statistical reporting and analysis not the the education process – but I’m sure you knew that.

      Employer employer Can I have a job? I thought that my school was great but now I’m stuck at home Employer Employer Don’t you think I’m great? I want to learn the facts for life I hope it’s not too late

      Like

  9. Lance, I’ve just read your bio. I note it does not include a formal, recognised qualifcation in education or teaching. I note you have never worked in an educational setting. I also noted that you have worked for the media outlet Fairfax, which said volumes to me.
    As a teacher, I am sick and tired of people from a non-education background telling educationalists we are doing it wrong and should do as we are told.
    Do you see teachers marching into a newspaper office and telling them how to do their jobs? (BTW, I’m often appalled by spelling, grammar and basic wrong details in many newspaper articles… it makes me wonder what the editors do all day!!)
    Do you see teachers marching onto farms telling farmers how to milk cows, dock lambs tails or what onions to plant?
    Do you teachers marching into banks and telling them how to lend money or sell credit cards to people who can’t afford them?
    As a teacher, any data we collect from children should be for the purposes of their learning, our planning and professional learning as teachers, reporting to the child’s parents, and for our school to use to evaluate our own performance as a school, where we should go next and what resourcing should be required.
    To have data released to the general population is in fact an invasion of each individual child’s privacy. I will be making a complaint to the privacy commissioner. I hear you asking why.
    I live and teach in a small, rural community where everyone knows everyone else. We have less than 50 children at my school. Infact, half of NZ schools have a roll of 150 students or less. I have one Year 8 student, one Year 7 student, seven students each in Years 5&6, and so on. Any data released to the public will publicly out where each of our students sit in regards to National Standards. That is demoralising. My Year 8 student will be below the standard, but these results will not show that this child a year ago didn’t read with accuracy or comprehension, but now does; didn’t write even two sentences with 50% of words spelt correctly, now wants to write a whole page and has 75% correct spelling and great approximations for mispelt words; didn’t get 100% for any NZCER spelling test, and now has the first four lists down pat and has made incredible progress in the next four lists; was turned off maths, now begs for maths; didn’t finish any work with any ‘fulness’, now finishes work, striving to do his best, full of thinking and asks for more.
    So I oppose your arguments, not just because I am a teacher, but because I know that educationally it will damage the most fragile learners to have their data exposed publically when it should be between the learner, the teacher, the parent and the school.
    BTW, I don’t see the data about your job being publically published in the paper. Can I see your latest performance review…. or is that sensitive?

    Like

  10. I was asked for an academic vision –

    In raw terms Heisenberg’s uncertainty principle states:

    “The more precisely the position of some particle is determined, the less precisely its momentum can be known, and vice versa”- borrowed from Wikipedia.

    A corollary to this is that gaining certainty by measurement in one dimension creates uncertainty elsewhere. Put another way, more measurement does not necessarily create clarity in all areas. This is the case with measurement and assessment in education. Stretching the point, more testing changes educational outcomes. Measuring any human changes them, measuring children can change them profoundly.

    What I know:

    The more children get measured during the course of education the more they think that the goal of education is to perform well in tests. This is a problem because it promotes effort-outcome behaviour that maximises test achievement for minimum intellectual effort. It gets worse when teachers experience the pressure of published tables. The desire to do well, serves to increase focus on testing and limits engagement with activities not directly linked to future testing. This is not “gaming” but an unavoidable consequence of measurement on pupils and teachers. The products of a well-tested educational system arrive at tertiary level very intelligent and with all the skills they have practised in school. Unfortunately the bulk of these skills amount to finding out the answer as quickly as possible and moving on to something that is more fun. Worryingly, more recent generations of students can skilfully find solutions to educational problems by-passing much of the understanding and will to “go the extra yard” desired by many stakeholders. To put this into context many students just don’t do enough homework and know only what passes the test. Therefore a consequence of seeking improvement in education via more measurement, is a very narrow approach to achievement and less educated students whose value to society is at best unclear.

    It is not necessarily the case that measurement is a bad thing, but the assumption that more measurement in education is a good thing would be untrue. Academics can refer back to forty years of their own research/homework to underline this point. My vision as requested, is one of more teaching, less testing and fewer league tables; a vision based on observation and evidence.

    Like

    1. thanks –
      “My vision as requested, is one of more teaching, less testing and fewer league tables; a vision based on observation and evidence.”

      The issue is that we need to understand what good teaching is, and what good outcomes are. An element of consistently applied testing, such as how the OECD does it, is required. Step one is get the consistant testing, step two is make it excellent. It’s hard to influence step 2 if you fight for your lives against the first.

      Like

      1. ‘Evaluation’ not ‘testing’.

        I want my children’s school to be evaluated against a set of broad standards not measured by a set of prescribed tests that the children are taught to pass so the school can elevate itself up a league table.

        Like

  12. Understanding good teaching is a longitudunal problem, not a cross-sectional one. Consequently comparing one child to another or one school to another is specious. If you cannot compare the same thing at the beginning or the end of the teaching enterprise you compare apples with bannana’s. It is not merely the test that has to be good, but the question and the method. Until the problems of testing/assessemnt at secondary level are resolved I see no manadate for extending the problem to elsewhere in the eductaional system.

    A useful perspective on these issues can be gained via Sport Talent Identification. Which is a failed and intellectually bankrupt enterprise despite extraordinary time, money and motivation devoted to the task.

    “Step one is get the consistant testing, step two is make it excellent. It’s hard to influence step 2 if you fight for your lives against the first.”

    I have no objection to seeking excellent testing; indeed testing that tells students what they are good at and why, is of considerable value. But this need not be conflated with more testing at the wrong time. Tests can focus attention and provide a viable challenge for the willing and able. However this is best explored at the end of an educational experience, not the start. I would argue that where pupils cannot understand either the consequences or likely benefits of testing it is as unethical as it is invalid. Outcomes come from processes and in this sense ERO has got it right. Look at what schools do and then at what happens later in the system. By all means refine the tests we have, but don’t add more.

    Like

  14. Pingback: Wiggs on League Tables | Kiwiblog

Comments are closed.