Assessment at the Edge 2: Clash of the Paradigms

(or Rumble in the Epistemological Jungle)

clashoftheparadigms

We operate, on the whole, within education systems that are based on a traditional linear, positivist, computational paradigm that has been the dominant scientific paradigm since Newton et al back in the 17th century. It is a paradigm in which education is perceived as a form of industrial or mechanised process.

It is, essentially, a closed system, which is the sum of its parts (learners, teachers, curriculum, content, delivery, technology, etc.). By controlling these parts, we can regulate the performance of the whole system. Educational systems design is the process of regulating these closed systems. It is a system in which human behaviour and performance are assumed to be predictable within known circumstances, and in which knowledge is assumed to be an external, quantifiable object that can be transmitted to and acquired by learners. This enables patterns of behaviour to be analysed and used to make judgements about how learners are thinking or what they have learned.

It is a system in which a ‘line of determination’ is assumed between cause and effect: for example – teaching predictably causes learning. These assumptions over-simplify the world and tend to reduce human learning, performance and achievement to a repertoire of manipulable behaviours. But learning is far more complex and much less certain than these assumptions infer.

In one corner we have the dominant Positivist or Quantitative Paradigm which is based on the epistemological belief that all true knowledge is ‘scientific’ knowledge. In this paradigm there is a single objective reality ‘out there’ that is orderly, predictable, and can be studied, captured and understood by amassing data and triangulating it (I shall return to the triangle).

The overarching aim is to achieve explanation and control, which is possible because knowledge is objective, measurable, value-free and a quantifiable object that is transmitted by the ‘teacher as expert’ to, and acquired by, learners. Rigour is achieved via the ‘holy trinity’ of validity, reliability and generalisability.

In the other corner we have the Interpretive or Qualitative Paradigm in which subjectivity is inherent and should be acknowledged because complete or pure objectivity is impossible and should never be claimed. For those in this corner ‘truth’ is a matter of consensus amongst informed and sophisticated constructors, not of correspondence with an objective reality. Furthermore, because all measurement is fallible, there is great emphasis on multiple measures and observations in order to able to claim authenticity, and for that authenticity to be recognised.

Those who operate within this paradigm understand that there are multiple realities and that knowledge is subjective, contextualised and value-dependent. They aim for understanding in order to enhance learning, they are openly self-questioning and self-critical, and they welcome scrutiny and debate. Importantly, they view students as co-constructors of their learning, and perceive themselves to be partners and participants in learning as well as guides and mentors. (That position, by the way, does not prevent them from also being experts!).

In order to find a way to deal with all of this epistemological complexity in relation to how we approach assessment, I’m suggesting that one way – and of course there are and will be others – is to approach assessment as a form a qualitative research instead of a quasi-scientific investigation. If we choose to follow the interpretive paradigm in relation to assessment then we need adjust our thinking and our language. Essentially we need to do a form of ‘Find and Replace’.

We need to replace :

Validity with Credibility, Coherence, Consistency, Trustworthiness, Authenticity
Certainty with Relativity
Generalised Explanation with Local Understanding
Source Data with Empirical Materials
“Is it true?” with “Does it work?”
Single Point Perspective with Multiple Perspectives
the Triangle with the Crystal

To be continued……

Next instalment coming soon: Assessment at the Edge 3: Triangles and Crystals

Back to Assessment at the Edge 1

Assessment at the Edge 1: Faultlines

I know I’m not alone in feeling – increasingly as the years roll by – that all too often the way we assess is at odds with the way our students (and we ourselves) actually learn and experience learning. While I and everyone else round the assessment board table is doing their very best to be professional, to ensure that procedures and regulations are followed, and taking great care to ensure that students are treated fairly and reliably….a bit of my brain is suffering from a form of cognitive dissonance and saying ‘This is nuts!’

There used to be a one of those car stickers that went something like ‘Do Not Adjust Your Mind…There Is A Fault With Reality’. And that’s how it feels. There seems to be a serious disjunction or faultline between what appears on the hundreds of assessment print outs – actual or virtual – and the actual day-to-day experience of learning and teaching, of creating work, of pursuing ideas, of encouraging and enabling students to really stretch themselves, to try out new things, to fail gloriously, to boldly go.

As teachers we need to – and are required to – ascertain, with as much validity, reliability and fairness as possible, what our students know and understand. For most of us, learning, teaching and assessment is a form of journey along the highways and byways of a particular subject. We, the guided and the guides, explore the landscape of the discipline. Our role as guides, more often than not, is to enable those we guide to understand the meaning and significance of what is seen, what is heard, what is felt, what is experienced.

Occasionally, because as guides we take our work seriously, and there are matters of accountability and responsibility that need to be attended to, we stop and check to see how much those who have entrusted their education to us know and understand, and what they can do.

In order to assess our students we stop acting as guides and essentially become researchers or purposeful explorers. We set out to discover what they know and understand, and what skills they possess. We ask them, demand of them, to demonstrate their knowledge, understanding and skills. We assess them, evaluate them, judge them, measure them against a set of standards or criteria.

If it’s a relatively simple matter of fact or basic competence then it is relatively straightforward to test it. The student either knows who, or what, or when or how…or they don’t. But the landscapes we explore in education are highly complex, intricate, shifting, multi-layered, multi-faceted. Simple straightforward answers and simple straightforward questions are hard to come by. The terrain does not reveal itself easily. Nor should it. In such a landscape meaningful assessment is also highly complex, intricate, shifting, multi-layered and multi-faceted.

If we consider the types of assessment that dot the landscape, we can see a veritable bio-diversity of assessment. But this diversity is also a challenge, and it is worth noting just how many of these types of assessment result in assessment ‘data’ that is qualitative rather than quantative in nature.

But there is may be a problem with this: the more assessment involves qualitative information, the more subjectivity is involved. Now this would be mitigated and we would have improved reliability if we had strict or stricter assessment criteria and also more structured and proscribed content. But, and this is a big ‘but’, if we had those it would obliterate the essence of qualitative assessment in terms of flexibility, personal orientation and authenticity. Which brings us, eventually, to the question of assessment paradigms and to the Clash of the Paradigms.

Next instalment: Assessment at the Edge 2: Clash of the Paradigms

‘See me. Feel me. Touch me.’ (Pt. 1)

Object lessons and reflections on the HEA Arts & Humanities conference 2016

Brighton-HEA3

Early March. Brighton is an alluring place, despite the chill in the air. The sun is shining, the sea is blue, the promenade and beach lie temptingly just across the road from the conference venue, and the esoteric shops, cafés and bars of The Lanes are just a couple of minute’s walk away. So it was a testament to the commitment of the participants and the quality of the many and varied sessions on offer that so many were able to resist the temptation to ‘skip school’.

While, in some sessions and in Jonathan Worth’s fascinating keynote on the second day, there was an inevitable and valuable focus on the digital and the virtual, the most powerful message – for me – was the extraordinary pedagogic power of the physical, tangible object. From Kirsten Hardie’s opening keynote with accompanying green plastic teapot, pineapple ice bucket and toilet brush, to the Lego sessions of Contemplative Pedagogies, by way of Simon Heath’s wonderful drawings (see image below) that captured the essences of the whole event, it was the object that held centre stage. And there were plenty more sessions that focused on making and doing as a pedagogic activity, not just a practical or physical one.

Photo left: Hannah Cobb @ArchaeoCobb

I have written elsewhere (‘On history and all that’ ) on the power of objects to engage the imagination, to generate stories and lines of enquiry, to provoke philosophical, political, ethical debates, and to provide learning experiences that really ‘stick’. I still recall clearly the ‘History of Decoration’ seminars from my art student days when ‘Simi’ (Ms. Simeon the lecturer) would enliven her lectures on, say, Ancient Egypt, by taking a vase or piece of jewellery or some other artefact out of the cardboard box she always brought. She would casually hand the object to someone to examine and then pass around the room with the words ‘Do try to be careful, dear, that’s three and half thousand years’ old’. This would be repeated every session, whether the topic was Ancient Rome (jewellery), Medieval Europe (a crucifix) or Tudor England (a lace ruff). I only realised what we had been passing round when I heard that, on her death , Simi’s large collection of “just something to look at while I’m talking” had been bequeathed to and enthusiastically accepted by the V&A museum.

What also became clear during the conference, is that ‘object lessons’ are not just the preserve of the creative arts community. Every discipline clearly has its associated artefacts which can be used not only to enhance the teaching of an ‘academic’ subject, but to act as foci for the characteristics and qualities of the sort of learning that Kirsten Hardie talked about: learning that engages, amazes, provokes, exhilarates, takes risk, liberates.

One of the things I remember from those, now distant, art history sessions is something I frequently refer to in my work on curriculum design and assessment. In one her first seminars, Simi passed round an Ancient Greek vase that was covered head to foot in decoration. The reason, she said, for filling every possible square inch was ‘horor vacui’ – fear of open space – because it was through open space that the ‘Evil Eye’ enters the world. That might well be one of the reasons (though I would avoid mentioning the ‘Evil Eye’ or the Devil in module specifications and handbooks) why we insist on filling our curricula with content: ‘Idle hands make the devil’s workshop’ and all that. But we also know that deep learning, creativity and innovation require time and space to incubate and develop.

Objects, importantly, enable us to slow down time: to observe, to really look, to touch, to feel, to explore. Simon Piasecki, at the conference, talked about how he gets his performance students to slow right down and focus on the minutiae of what they are doing, and the artist Marina Abramovich – one of whose concerns is the fact that we don’t stop to really look any more – has a number of exercises she uses with those who come to view her work to achieve the same slowing down. When I worked at the Liverpool Institute of Performing Arts (LIPA), one of the first year ‘options’ that I established – open to any student – was a traditional life-drawing class. All the students that participated in that quiet, contemplative two hours on a Wednesday evening, amidst an extraordinarily hectic timetable (‘horor vacui’!), reported that they understood that it wasn’t about being able to draw. It was about having the time and space to slow down and really observe not only the ‘object’ (usually another student) but also themselves….and to ‘take a line for a walk’ in Paul Klee’s famous phrase.

Ken Robinson, in his now famous TEDTalk on creativity and education, jokes about academics generally seeing their bodies as a form of transportation to get them to meetings. He, among others, stresses the importance of mind and body, the intellectual and the emotional, the psychological and the physiological. What came through so strongly at the HEA Arts and Humanities conference was that objects – in all their glorious variety – and our close interactions with them, provide a means to engage powerfully in deep, meaningful learning experiences. Objects both inhabit space and create space. We just need the space, the time and, impotently, the confidence to engage in our own object lessons.

Brighton- HEA2

Photos by Paul Kleiman unless otherwise stated

Conference Twitter hashtag: #HEAArts16

On beauty and elegance in education

In his book ‘Fearful Symmetry: the search for beauty in modern physics’, Anthony Zee describes how Einstein displayed a supreme disinterest in any proposed formula or solution, no matter how accurate it might be, that he considered ugly.

“As soon as an equation seemed to him to be ugly, he really rather lost interest in it and could not understand why somebody else was willing to spend much time on it. He was quite convinced that beauty was a guiding principle in the search for important results in theoretical physics.”

Today (13 Aug 2014), amongst the usual ugly headlines of death, destruction and disease, Maryam Mirzakhani is being celebrated as the first woman to have been awarded the prestigious Fields Medal – the Nobel Prize for mathematicians. Her work – as described by those who have some grasp of her achievements – has a “breathtaking scope, is technically superb and boldly ambitious”. She herself describes mathematics as full of “beauty and elegance”.

Now, I’d hate to think that beauty and elegance is the sole preserve of mathematicians dealing, like Mirzakhani, in esoteric fields such as complex geodesics, transcendental objects, and differential geometry. I’d argue that we all need at least a bit of beauty and elegance in our lives and work, and we certainly can see people striving for it (though many just don’t care) in many areas: whether it’s the presentation of food, the design of buildings and spaces, the arrangement of an exhibition, the movement of a dancer across the stage, the order and rhythm of words on a page.

So, why not strive for some beauty and elegance in education and in the curricula and learning experiences we design for our students? Yes, it’s often messy and a bit (or very) chaotic. But just as the mathematics of chaos have a certain underpinning beauty and elegance, we – as ‘architects of education’ – should strive to construct and compose learning and teaching experiences that flow and connect in ways that have a certain beauty and elegance about them. It’s not easy, but surely worth the effort.

The Elixir of Wonder

Yesterday I spent the morning discussing future strategies and scenarios in higher education. What struck me was that over the course of three hours the words education, teaching, and learning were never used.

In the evening I was a guest at the opening night of the Royal Northern College of Music’s production of Donizetti’s opera ‘The Elixir of Love’ to be sung in Italian and featuring – in the cast and orchestra – students from right across the undergraduate and post-graduate provision.

I have to admit that opera generally – and particularly early 19th century Italian comic opera – does not feature on my list of favourite ‘must see’ genres. In fact it’s probably well along the ‘must avoid’ end of the continuum. However, I’d been invited by colleagues I like and respect, and there was a pre-show dinner at the RNCM’s excellent restaurant.

There was a full house, so the building was humming. There’s something great about that pre-show foyer buzz as people arrive, meet, greet, drink, chat, etc. As Richard Schechner pointed out many years ago, the trouble with too many shows is that what happens in the foyer, bars and social spaces before, at the interval, and after a show is all too frequently the most interesting phenomena of the evening.

We took our seats as the orchestra tuned up. Again there’s that wonderful expectancy as the various instruments tune in to that plaintive A on the oboe.

Then it was curtain up and straight into what, from the start to the finish, was a hugely enjoyable, visually seductive and witty, brilliantly performed and played production. 24 hours later, as I write this, I am still smiling because of it.

However this is not a review of the production. But rather, thinking back to my rather dry and ‘education-free’ meeting – about higher education – in the morning, a reflection on how our much disparaged and ‘useless’ disciplines of dance, drama and music in education (all present in this production) provide opportunities for students and also staff to engage in the creation of truly wonderful work. This is something that too many of those who are in control of our education systems, with their obsessions with protocols and standardisation and compliance and conformity and league tables and graduate employability and sustainability and an infinite host of other -isations and -ilities, just don’t get!

I’ve written before how our obsession with the somewhat triumphalist notion of ‘excellence’ has blinded us to that which is so obvious about genuine, transformative education: it’s not excellence we should be pursuing…it is wonder-full education.

The RNCM show provided that Elixir of Wonder.