As a Jewish family, we don’t really do Christmas. Yes, we’ll gather together, have a big meal, play board games, and watch far too much television. But that’s because it’s a holiday, everything is closed, and there’s not much else to do. Coming up to Christmas that year we’d just had some building work done, and there was still loads of cleaning up, making good, re-decorating to do. So it was looking like a DIY Christmas and new year.
As it happens, the builders had found an old, crumpled newspaper stuffed into the gap between the window and the wall our bedroom. When they first found it, it reminded me of a dead bird.
On the afternoon of Christmas Eve I decided to have a go at ‘uncrumpling’ it using our steam iron. It turned out to be the Lancashire Post from 30th September 1943. The paper was very fragile – I suspect they didn’t use the best quality newspaper during the war years – and I became absorbed in the task of slowly but surely flattening the paper one careful centimetre at a time. I wasn’t even aware that the radio was burbling quietly away on the other side of the room.
As each piece revealed its flattened secrets, I read about the Fifth Army’s losses in North Africa, the Russian advances along the Dneiper, a successful bombing raid over Germany (only eight missing in action). I also read about a local woman fined for fiddling ration books, and a Polish aircraft man convicted of drunk driving.
There was also a small item about how the German U-boats had failed to sink a single ship in the previous month. 70 years later, on Christmas Eve, the newspapers and television were full of the news about the Royal pardon given to Alan Turing, the man who had ‘cracked’ the German Enigma code used by the U-boats and, by doing so, had in no doubt saved thousands of lives and perhaps helped to end the war. But Turing was gay at a time when it was illegal, and had been found guilty of gross indecency, jailed, chemically castrated, and forbidden to undertake any work linked to national security. He was and should have been hailed as a national hero. Instead, he committed suicide two years later in 1954, and full recognition has only come very recently.
Those U-boat failures, reported on the front page of that 1943 Lancashire Post, were a direct result of the work Turing had undertaken at Bletchley Park.
Absorbed in the task and on this fascinating window on history, I suddenly became aware of a beautiful, solo treble voice filling the room, singing “Once in Royal David’s City….”. It was three o’clock in the afternoon and the start of that wonderful, traditional Christmas Eve ‘Festival of Nine Lessons and Carols’ from King’s College Chapel in Cambridge. And as that pure, high voice soared…..I burst into uncontrollable sobs. A veritable tsunami of grief poured over, into and through me: disturbing and disruptive in its power and intensity.
It took me a while to realise what it was.
When my mother died, I cried no more than a couple of times, and that was around the time when it all happened and usually in response to an individual’s kindness and sympathy. But nothing like this.
Though, obviously, Jewish and very proud of her traditions, my mother also had a deep love for many things that were quintessentially English. Among them she had a particularly high regard for and interest in the great cathedrals and churches of England. After she passed away I discovered, amidst the thousands of documents she left, a neatly stacked pile of guides to virtually all the English cathedrals and some other churches, all annotated with her distinctive handwriting. I always smile when I see those guides as my father did not approve of my mother’s interest in Christianity, and I know he would not step inside a church. So I have a very clear picture of my mother wandering around inside with her guide, no doubt quizzing whoever she could find about the history of the building and making notes, while my father sat patiently on a nearby bench doing his Times crossword.
Just before three o’clock in the afternoon on Christmas eve, without fail, and though Jews – on the whole – don’t do Christmas, my mother would sit herself in her favourite chair, turn on the radio, quite loudly, and wait until that beautiful, soaring, solo treble voice filled the room, singing the first verse of the carol, followed by the choir and then the congregation. Earlier in the day she would have called me to remind me NOT to call her between 3pm and 5pm.
As that deep pang of grief and my sobbing subsided, and I was able to collect my thoughts, I wondered how that boy’s voice could trigger such intense emotion. I remembered how some of Mark Rothko’s last paintings, those gigantic fields of deep colour, have a similar effect on some individuals, and I recalled that in some therapeutic contexts music and song are used to enable individuals and groups to confront severe trauma.
In my case, that moment triggered an intense ‘remembrance of things past’ and a huge sense of both loss and love. I suspect we all have those triggers, those Proustian ‘Madeleine biscuit’ moments, when something – perhaps completely unexpected – plunges us into the deep well of memory, love and loss.
That afternoon, as I ironed that old newspaper (something my mother would have loved – the newspaper not the ironing!) and listened to that young boy singing those famous words, I probably missed her more than I’ve done at any other time.
After I wrote a piece on inspiring learning through objects and artefacts, I began to think a bit more about what makes that form of learning so powerful. I was walking the dog (I use it as a form of idea-generation therapy) wondering what might be the opposite or complementary term to ‘virtual learning’. Suddenly, as I walked past the butcher’s shop in the shopping precinct, the word ‘visceral’ fell into my head. Passers-by must have wondered at this figure muttering to himself and repeating the phrase ‘virtual learning, visceral learning’.
I began to like the idea of visceral learning, with its connotations of strong emotions and physical experiences (not to mention unmentionable bodily functions). I suspect, however, that we won’t be seeing the phrase ‘ visceral learning’ in our institutional mission statements and learning and teaching strategy documents. ‘Immersive’ is much safer, but doesn’t have the visceral heft.
Why visceral?
There is a phenomenon that has been occurring in the last few decades, particular in the arts and popular culture. Essentially it consists of a reaction to a world that, increasingly, is viewed and experienced via gazing at a screen – whether a TV screen or a computer monitor or laptop/tablet/phone screen. Once, audiences used to flock to the theatre to watch the ‘well-made play’. They would sit in the dark, in silence, watching the action on stage. Then TV came along. Similarly people used to flock to Working Men’s Clubs for a ‘good night out’. Then TV came along. Then computers came along, and now we’ve reached the point where a virtually infinite universe of entertainment and information can be accessed at the click of a mouse or, more recently, by tapping the screen or simply by asking Siri or Alexa or your favourite AI ‘friend’.
But there was a reaction to this sitting in front of a screen; and that reaction was to make performances more visceral. No longer was it sufficient to sit passively and watch. The relationship between the performer, the ‘text’, the audience and the environment became blurred, mutable, transactional. The veritable explosion of site-specific, immersive, interactive performances and performance experiences can be seen, in part, as a reaction to the relative passivity of just watching a screen. Audiences were engaged and involved: physically as well as emotionally. And that pattern can be seen in many fields beyond theatre.
Higher education has, perhaps, been a bit late to the visceral learning party. Perhaps it’s got something to do with the innate distrust of anything that is not focused on the mind and the intellect. If you want to put this to the test, try doing a simple, short physical warm-up exercise with a group of colleagues or students from non-performance based disciplines. The looks and expressions tell you that you might as well be asking them to stick needles in their eyes!
But there’s clearly a shift happening, though currently it tends to occur predominantly amongst the creative, educational ‘outliers’. But slowly, as in Wenger’s notion of legitimate peripheral participation, as more individuals and groups within that community of practice adopt and adapt the ideas, discourses and – importantly – the new or certainly different practices, the activity moves gradually from the periphery towards the centre of a particular community of practice.
The virtual and the visceral are the ying and yang of learning and teaching. It’s not either/or, but both/and. The more institutions focus on enhancing (and investing) in digital and virtual learning experiences, the more that needs to be complemented by enhancing (and investing in) visceral learning experiences. No longer should students be required to sit passively in the (lecture) theatre, listening to and watching the action on the stage. They can usually get that via clicking a mouse or tapping the screen and watching the video of the lecture on YouTube (or via the VLE). Visceral learning goes beyond ‘engaged’ learning. It involves immersing oneself intellectually, emotionally, physically and kinaesthetically in the learning experience. That learning experience needs to be designed skilfully to enable that immersion to occur, and it needs skill and confidence on the part of the teacher, who acts not as a transmitter of knowledge but as a guide, mentor and partner through the visceral learning journey.
12 years ago I came across and wrote a blog piece about Terran Lane, a tenured associate professor in the US, recently moved from academia to industry.
His move caused consternation amongst his friends and colleagues: “voluntarily giving up tenure is roughly akin to voluntarily giving up a lung”. On his blog – which went viral – he made a list of the “forces that are making it increasingly unpleasant to be an academic in the US right now”. Here’s that list, and it sounds remarkably familiar:
the difficulty of making a tangible, positive difference in the world;
struggles with workload and life balance;
increasing centralisation of power into university administrations and decreasing autonomy for academic staff;
a strained funding climate that is trapping academics between dwindling central funding and intensifying university pressure-to-be-funded (generate income);
specialisation, narrowness of vision and risk aversion within academic disciplines;
poor incentive structures;
moves towards mass production and automation of education;
salary disparities between the academy and industry;
the rise of anti-intellectualism and anti-education sentiment. The creation of that list “turned into not just a dissection of dissatisfactions with the system, but a cry for loss for a beautiful institution that I have loved and outrage at the forces that are eroding it”.
As the list of actual and proposed department closures and redundancies grows longer on a seemingly daily basis, those who work in higher education in the UK may well nod our head in agreement with most if not all of that list.
Meanwhile, as the juggernaut of centralised conformity, cost-cutting and bott0m-line accounting rolls inexorably onwards, on a day-to-day-basis many who work at the ‘chalk-face’ are creating, planning and delivering wonderful, creative, innovative, exciting, relevant learning experiences that defy the forces of ‘command and control’ and the stultifying blanket of conformity.
Also some years ago, before Brexit, at a European conference on the future of arts education, I happened to be standing in the coffee queue next to the German Federal Minister of Education who had just given a keynote address. After an exchange of introductions, and having established I was from the UK, he went on – in a light-hearted way – to list some of our structural problems (transport, health, etc….it was a time, admittedly, when nothing in the UK seemed to be working properly).
He then went on to say that he had a serious question: “For the last 30 years or so, until reunification, our economy was good and many good things both promoted and flowed from that. Yet, culturally and artistically we produced relatively little of world class. Over the same period, in the UK your economy has never been strong, yet you have consistently led the world in music, design, fashion, theatre, etc. My question is what is it that you are doing, or maybe NOT doing in your education system that has allowed you to achieve that?”
I didn’t – standing in that coffee queue – have a coherent, evidence-based answer. But I did say that perhaps it was to do with the fact that we have a long and honourable tradition of non-conformity in the UK combined with a high tolerance of eccentricity.
Is that true…and if so, does it still hold true? Or, in our education systems, have we lost – or are we losing and/or having taken away from us – the very attributes that enable us to lead the field in creative and cultural endeavour and achievement?
I once attended a conference on academic integrity. Full of academics and university managers all concerned about the rising tide of plagiarism, cheating and the exponential growth of services and products aimed at providing their users the means with which to ‘game’ the system.
As is customary at such gatherings, there was a keynote presentation from an expert in such matters. In this case it was a senior academic and university leader talking about the way their university had approached the ‘problem’. The presentation, accompanied by the inevitable Powerpoint slides of charts and bullet points, was going well and had my full attention until the first photographic image appeared. It was a stunning image of an iceberg, showing the relatively small, visible above water section in contrast to the immense invisible section below the water line, used to illustrate the point that what we see and catch in terms of plagiarism, cheating, collusion etc. is only the ‘tip of the iceberg’.
I immediately recognised the image, and if you Google ‘iceberg’ and select ‘images’ you’ll see it amongst the first set of images. It’s also – because it’s a great image – promiscuously plagiarised on dozens of websites and blogs. I recognised it, because I’ve also ‘stolen’ it for a satiric poster…but that’s another story.
But here we had a patently plagiarised image, with no acknowledgment as to its source, being used to illustrate a presentation on tackling and preventing plagiarism, given by an expert in the field at a large conference where the ‘fair use’ conditions don’t apply.
A wee bit ironic.
But the thing is, I’m sure we’ve all done it at some point. Let’s ‘spice up’ our otherwise visually boring Powerpoint presentations with some nice ‘visuals’. Isn’t that what we’re encouraged to do? And doesn’t something like Google images or Flickr make it so easy?
But is it so hard (and ignoring, for the moment, issues of copyright and fair use) to acknowledge one’s sources? There’s even less excuse when there are now several websites ( e.g. Unsplash, Openverse, Creative Commons, Freerange, Pixabay etc.) that provide access to millions of images that are free to use but usually request that you cite the source.
Perhaps, when it comes to plagiarism, we should try just a bit harder to practise what we preach when we present our work – whether to students or colleagues.
Image: “Iceberg in the Arctic with its underside exposed” by AWeith is licensed under CC BY-SA 4.0.
This is the full transcript (with slides) of my end-of-conference keynote address at the International Assessment in Higher Education (AHE) Conference held in Manchester, UK in June 2023. The keynote video is available here: https://youtu.be/nZbxDv3qqlA3 .
Thank you, for that kind introduction and thank you to the conference for the invitation to speak today. Given the many tremendous presentations and sessions over the past couple of days, there are a lot of very hard acts to follow!
Thinking about this presentation and trying to present something coherent has been like trying to build a house during earthquake. The tectonic plates on which higher education is built are moving dramatically, and is there any safe ground? So, what follows is, I realise, a rather non-linear, almost stream-of-consciousness set of thoughts and ideas about learning, teaching and, particularly assessment.
When I used to Chair an institution’s Exam Board, responsible for progression and awards, I regularly used to experience a sort of cognitive dissonance. Faced with page after page of student ID numbers, names, and a seemingly infinite array of rows and columns filled with numbers, I felt a deep sense of disconnect between what I was looking at and my day-to-day experience of working closely with those same students, discussing their work and their hopes and fears, their successes and disappointments.
Nowadays, when I explore assessment with colleagues I often start a session with finding out how they feel and where they are in regard to assessment, and I ask them to illustrate that by drawing or making marks on paper and then give them no more than 30 seconds to explain what they have drawn. The task not only energises the room at the start of an intense few hours but is also very revealing about attitudes towards and feelings about assessment. It also, puts paid to the “oh, I can’t draw” response. No matter which disciplines are represented by the participants, there is always an interesting, revealing and creative response.
These are just a couple that really stick in my mind and which, in a way, relate to my title and theme.
The individual who drew this said: “This is where I’m at regarding assessment. At its best, the work of my students is expansive! Multi-dimensional and multi-coloured. Dynamic and non-linear. This is how I sometimes feel when I come to quantify (assess) a student’s work – like I’m funnelling a wild and wonderful rainbow into a grid better suited to a game of noughts and crosses.”
And the individual who drew this said:
“This is my big black hole of confusion. It sits at the bottom of all my feelings of assessment. The deepest darkest part is numbers grades. I can write feedback articulating my views, but putting a number to it, seems almost impossible. Objectively, the better students should get a better grade. But what about learning journeys?”
So to begin….If a week is a long time in politics, then a year in higher education is an eternity. When I was invited, shortly after last year‘s conference, to give this keynote address or provocation we were sort of returning to what we might call the post-Covid ‘new normal’ in higher education. The idea that I presented to the conference committee was about how assessment had changed during Covid as we realised that assessment processes, protocols and procedures that had been assumed to be graven in stone, were, in fact, mutable “oh, we can do that!”.
Things changed and changed fast out of necessity. There was a veritable explosion of innovation and creativity in regard to assessment. Digital transformation occurred right across the sector during the pandemic. We learned important lessons about equity, about learning design and about interoperability. We saw success stories and consistently high levels of student attainment. Thankfully we moved away from traditional unseen exams to other forms of assessment that saw many students, particularly those from disadvantaged backgrounds, perform better.
The danger was always going to be, now that we were back in the “new normal“ was that we and our institutions would simply snap back to the old ways of doing things.
Paul Trowler writes about all institutions and organisations having ‘engrooved practices’. Those deeply embedded, social and cultural practices and norms that are so familiar and, to some extent, comfortable , that no one really challenges them. Occasionally, some new exciting innovation that looks like it may well enhance learning, teaching and assessment comes along, and enough people say let’s give a go’ and they give it a go.
The thing about engrooved practices is that they really are deeply embedded, not just in the social and cultural norms of the institution or faculty or department but often also in the minds and psyches of many people within the institution. It’s like they are part of the DNA of the institution, and we know how hard it is to change DNA. So something new, exciting, innovative comes along and some people get really excited about it. “This is the best pedagogic thing since sliced bread.” But unless this new exciting innovative thing becomes not only accepted but embedded within the systems and processes of institution, it remains ephemeral no matter how successful, and it can easily disappear as people snapback to the familiar and comfortable ways of doing things. Or the individual or individuals involved in it move on either within the institution or to another institution.
We have heard about and seen some wonderful, creative, innovative approaches to assessment these past two days….and I’m sure we’ll all leave here full of inspiration and good intentions….but…and it’s a very big BUT ….what happens when we return to our institutions and those pesky engrooved practices and attitudes?
So, that was essentially my starting point for what I was planning to say a year ago, and which I had titled “I am not a number!“; reassessing assessment“.
A year on, I have to say that while the reassessing assessment bit still holds, if I had been asked now to provide a title, it would have been something along the lines of “I am a not a stochastic parrot: reassessing assessment”.
I am, of course referring to last November and the appearance of ChatGPT (or, as an Australian colleague refers to it, ChattieG) and the academic and moral panic that ensued on its public release.
A glance at some headlines gives a flavour of the mood at the time:
ChatGPT is making universities rethink plagiarism!
Chat GPT: a tool for teaching or cheating?
Universities fear cheating epidemic
Chat GPT has universities in emergency mode to shield academic integrity.
EXCLUSIVE: ‘Half of school and college students are already using ChatGPT to cheat’: Experts warn AI tech should strike fear in all academics
I was half expecting to come across a headline that went something like CHATGPT DRIVES A STAKE THROUGH THE HEART OF ACADEMIC INTEGRITY
And we have seen universities reacting in very different ways. Like any crime prevention strategy, the pendulum has been swinging wildly between, at one end, a punitive hang ‘em and flog ‘em type of approach to cheating, at the other end, let’s understand it, let’s acknowledge it, let’s use it genuinely to enhance teaching, learning and assessment.
And if you’re wondering about the ‘Stochastic Parrot’ bit, it comes from the title of an influential and controversial 2021 paper by Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell,
A stochastic parrot is an entity “for haphazardly stitching together sequences of linguistic forms … according to probabilistic information about how they combine, but without any reference to meaning.” The paper covered the risks of very large language models, regarding their environmental and financial costs, their inscrutability leading to unknown dangerous biases, the inability of the models to understand the concepts underlying what they learn, and the potential for using them to deceive people. The paper, controversially, resulted in Gebru and Mitchell losing their jobs at Google….but that’s a story for another day. Though the parrot may appear later.
As an aside, a couple of descriptions of ChatGPT or ChattieG, I’ve come across.
This by Prof. Inger Mewburn in her definitely worth reading blog The Thesis Whisperer:
Generally, the best way to use ChattieG is to imagine it as a talented, but easily misled, intern/research assistant who has a sad tendency to be sexist, racist and other kinds of ‘isms’.
And this, by a Senior Lecturer colleague I interviewed recently:
It’s by far the best thing since the internet! The benefits to me and my students are incalculable….and it really levels the playing field for disadvantaged students.
And if you think that generative AI is all about the written word, think again! As we’ve seen, images, artworks, music, posters, powerpoint presentations can all now be produced by Artificial Intelligence…..though I have heard calls for it be called Augmented Intelligence or Assisted Intelligence.
Going back to my original theme, Covid demonstrated conclusively that we can adapt and change our approaches to assessment. The extent to which those changes have been adopted across the board and embedded within institutional systems and protocols is a moot point.
One thing that changed quite dramatically, as we shifted to permanent online, was the relationship with students. There were some very interesting papers and ideas around what was termed “the intimacy of Zoom…or Teams”.
Larry DeBrock and colleagues wrote;
“ that student, who is sitting far enough away in the lecture hall that you can’t quite read her expression amid the proverbial, sea of faces? When you call on her in a live zoom session, she pops up right in front of you, one on one, looking you straight in the eye. There is no backseat in online education. Every student is in the front row.“
There’s a funny thing about front rows. If there’s a choice, no one likes to sit in the front row. Whether it’s a classroom, a lecture theatre or a conference presentation.
When I worked in touring theatre, doing performances in all sorts of venues like community centres and village halls, we used to put out a false front row of seats, knowing that nobody would sit in them. Just as everyone was seated and the show was about to begin the stage crew would remove the front row of chairs – much to people’s shock and amusement -to ensure that we did actually have people sitting in the front row.
But I digress…well, a bit.
The point is, and you may well object to the comparison, the one thing that a performance to an audience in a village hall has in common with teaching, learning and assessment is that it is about relationships, establishing a rapport with others, the very human act of engaging with and communicating with another person or persons.
Those of you familiar with my work will know that I have banged on for years about the word assessment deriving from the Latin ‘Ad Sedere’ – to sit together. Involving students actively in assessment, not just viewing them as objects of assessment, but as agents of and in their ownassessment.
And I’ll be coming back to this later.
But we still hold on to that idea of sitting together at PhD level but have lost it at undergraduate level, due to the massification of higher education and the quasi-industrial/ commercial/ marketised model that has developed as a consequence in which students are regarded as fee-paying customers and given a customer number, (we’re back to “I am not a number!”) but actually are treated more like units to be acquired, processed and produced, with rigorous quality mass testing at every stage of production.
The way we assess and how we assess is all part and parcel of the capitalized, marketized version of higher education that is now so dominant that it’s hard to imagine that there might be an alternative. It is an engrooved model.
It’s a model in which students are both units of production and objects of assessment, replete with individual ID numbers and batch numbers for their particular year group, and it should come as no surprise that students understand – either explicitly or, more likely, implicitly – their role and function inside that model: which is to succeed. And it really should come as no surprise that within a model of higher education predicated on a sort of ‘winner takes all’ version of success that students will be tempted to do whatever it takes to ensure that success. So we enter this eternal game of, on the one side, ever more sophisticated ways of cheating and, on the other side, ever more sophisticated and expensive ways of preventing or spotting the cheats.
Covid severely disrupted that model, but didn’t fundamentally change it. Students could no longer be regarded as a class (pun intended) or units or a set of ID numbers. No longer could we require students to gather together en masse outside and inside examination halls. One of the ironic consequences of lockdown was that it forced us to regard and deal with students as individuals – all sitting in the front row – all with their individual needs, hopes and fears but, at the same time, in regard to assessment trying to meet the requirements of validity, reliability, inclusivity and equity etc.
Then, just as we’re settling back into normal educational service is resumed, we hit another massive interference: this time it’s Generative AI in the shape of ChattieG and its fellow travellers. And now assessment is really in trouble!
I suppose the big question is…is the HE model that currently reigns supreme sustainable?
The famous lines from W.B. Yeats comes to mind:
Things fall apart: the centre cannot hold (from ‘The Second Coming’)
There is also this quote from Antonio Gramsci:
The crisis consists precisely in the fact that the old is dying and the new cannot be born; in this interregnum a great variety of morbid symptoms appear (from ‘Prison Notebooks’ – Notebook 3)
I trained and worked as a designer before stumbling into teaching, and one of the things one learns as a designer – often very painfully – is that there comes a point where it is futile trying to further change, adapt, tinker with an existing design. You’ve got to have to courage to scrap it and start again or, at the very least, undertake a fundamental re-design. Of course, for very understandable reasons, we are really bad at doing either of those things in higher education. Even if we had the will, we rarely, if ever, have the time or resources to engage in such an undertaking.
If, like me, you happen to own a boat – in my case it’s an old narrowboat – that’s it in the photo – you will know that even the best designed vessel, after weeks and months in the water, gathers barnacles and other things around its hull.
Eventually what originally cut through the water and was a pleasure to sail or steer becomes heavy in the water, difficult to steer, a makes slow progress. The same applies to what we do. Even the best designed learning, teaching and assessment strategy gathers ‘barnacles’ over time: a new module here, new content there, new assessment changes everywhere, etc. It gets heavy in the pedagogic waters. Sally Everett, yesterday, talked about wading through treacle.
In the case of a boat , the answer is to take it out of the water, put it into dry dock, scrape off the barnacles, undertake a complete overhaul and, if necessary, refit.
But we can’t do that with our curricula and our learning, teaching and assessment strategies. We don’t have the time or resources between one cohort leaving and the next cohort starting. So often what we do – through programme review – is tinker…..with the result that often we end up not only moving the barnacles around instead of removing them but actually adding more!
So what MIGHT we usefully and positively do in the face very real, clear and present dangers?
I use the word MIGHT deliberately instead of what WILL we do?
A great teacher and mentor I had the privilege of meeting and observing used to say that the most powerful word in education is ‘Might’. If you ask “What is the answer to this problem?” or “What will you do about that?” that implies there is only one correct answer or one possible course of action and a host of incorrect ones. But if you ask “What might be the answer?” or “What might you do?” it opens up the curiosity, the creativity, the possibilities instead of certainties.
We need, somehow, to shift out of our familiar, engrooved discourses and practices and ask how and why students are assessed in the first place. We could also usefully ask why are students motivated to cheat or cut corners?
We actually know why…
High stakes assessment
Assessment overload
The feeling that it’s just another hoop to jump through
Lack of time and space
Lack of intrinsic motivation
Lack of creativity
Lack of inspiration
Lack of meaningfulness – a snapshot in time seemingly unrelated to anything else.
Lack of engagement
Lack of a sense of belonging
Lack of learning!!
You could probably add more.
When I go into institutions to help them solve or resolve a particular issue, I often ask the senior management: “Do you want evolution or revolution?”.
The answer, of course, is usually Evolution, but occasionally, actually very rarely, they bite the bullet and say something along the lines of: ‘We prefer Evolution but if it takes Revolution then we’ll support it’.
I have a fond memory of presenting a proposal for a new, radical approach to the assessment of creative practices to an institution’s senior management team where we proposed moving away from learning outcomes, and the Director of Academic Studies raising his arms in the air and shouting “Hallelujah! Finally!”.
And that’s what we did, and I am pleased, and relieved, to say it works. We have the data and the feedback. But that’s another story. But I wrote a case study which I wanted to title ‘We Don’t Need No Learning Outcomes’ in homage to the famous Pink Floyd song, but the publisher insisted it was changed to ‘We Don’t Need Those Learning Outcomes’ as the original was deemed grammatically incorrect.
What this is all leading to, as you’ve probably guessed, is that I am sure I am not alone in this room in thinking that we are now the stage – in regard to assessment – where REVOLUTION is required. “The centre cannot hold”! We’ve moved beyond adapting, amending, adjusting.
So, in a spirit of revolution, what might we usefully do.
If you look at the various lists of ‘Skills for the 21st century’ you find the same dozen or so attributes appearing in various combinations. These are the skills that students and graduates need to keep up with the lightning-pace of today’s world. Each skill is unique in how it helps students, but they all have one quality in common. They’re essential certainly for today and probably tomorrow.
Here’s one version of them:
Now, let’s for a moment imagine those skills are assessment criteria, and what if we were to assess our own assessments, the things that we require students to do, what if we were to assess them against those criteria? I suspect we’d manage to tick perhaps one or two – including critical thinking. But as for the rest?.
So, here are some thoughts on what we might do:
Create Space. Stop Filling the Void.
Like many, I am fascinated by Japan and Japanese culture. (I did say this was somewhat non-linear!) That fascination, in my case, goes back a long way. I recently watched James Fox’s series of documentaries about Japanese art and culture and also Monty Don’s programmes about Japanese gardens and garden design. Both presenters commented on the importance of the Japanese idea of ’Ma’ – often translated as ‘negative or empty space’ but it is much more than that. One way of understanding ‘ma’ is as the space between tangible things that gives those things meaning. It is not so much empty or negative space, but rather it is a space full of energy, potential and promise. The character for “Ma” (間) combines the character for “gate” 門 with the character for “sun” 日 – an image of light beaming through the empty space of a doorway.
In the western tradition and culture we have nothing like the idea of ‘Ma’. Instead, we dislike a void, and tend to fill it. There’s actually a Latin t erm for it “horror vacui”. One of the few things I remember from my student art history days are the large, ancient storage jars called Attic Vases. They are often covered head to toe in decoration. The reason for that, so we were informed, was the belief that the Evil Eye enters through empty space. Perhaps that notion is still hidden deep within our Western psyche?
What is that proverb? The devil makes work for idle hands.
When I started working in higher education I was immediately struck by the fact of just how busy our curricula, our timetables, our workloads, our assessments are…and it’s got much worse. It’s as if we are afraid of leaving ‘empty space’. Why? In case students and staff get up to ‘mischief’?
Rather than filling the curriculum, timetable and workload voids, what if we designed them incorporating the idea (and actualité) of ‘Ma’. Designing in those spaces that help to make sense of the whole, providing the time and space that allows us and our students to step back, to think, to reflect, to make, to create, to experiment….and perhaps to have those conversations that enable us and our students to get a good and genuine sense of how they are doing, what they might need to do, what they might want to do?
Create a Skunkworks.
Every HE institution, and possibly every faculty, needs a Skunk Works.
How many people here are familiar with the term?
A Skunkworks is essentially a group within an organization given a high degree of autonomy and unhampered by bureaucracy, with the task of working on advanced, experimental projects.
The term goes back to World War Two, Lockheed and America’s need for a new fighter jet as soon as possible, and the group ‘the original Skunkworks’ who managed to get it from drawing board to runway in record time.
So if you’re interested in moving to some of the creative, innovative approaches we’ve heard about these past two days: unessays, collaborative annotation, using ‘ungrading’ approaches to assessment, or designing and implementing genuine authentic assessment, then having an equivalent of an officially sanctioned Skunkworks within your institution or faculty may provide some significant gains and rewards for both students and staff.
Change the Language Around Teaching, Learning and Assessment.
If you visited a National Trust property here in the UK, you would often see signs saying things like Keep off the Grass, Don’t Touch the Plants, Keep to the Path. It was all in the negative and proscriptive. But, as an experiment, they kept the signs but changed the language. Now it’s Dos instead of Don’ts accompanied by some encouraging words. Rules are there for a reason, but rather than focus only on what people can’t do, try to point them in the direction of what they can do. If you need to impose a restriction zone, for example around a fragile object, then simply explain why and direct visitors to where they can take a closer look at the detail (for example, online). People tend to be more relaxed and understanding when they feel informed and can make a choice.
There was a recent short exchange on Twitter with an HE colleague looking for better word or words than ‘Delivery’ in regard to teaching and learning. And I distinctly remember the late Ken Robinson wondering in regard to the obsession with ‘delivery’: “When did education become a branch of FedEx?”
In my own research into creativity in higher education, when I asked colleagues from across a whole range of disciplines, for the words and phrases they used to describe creativity or being creative in regard to learning and teaching, the top twenty words and phrases contained words that never appear in programme or module specifications or Teaching, Learning and Assessment strategies.
Words like joy, play, fun, passion, excitement, adventure and let’s admit they sit alongside anxiety, stress, disorientation which are also part of learning.
Learning and studying should involve all of those…..and so should assessment.
And instead of hitting students as soon as they start with dire warnings about plagiarism and cheating, let’s talk about integrity, trust, responsibility, partnership, collaboration, and so on.
I’d also like to suggest that we stop using the word failure. It’s such a loaded word. Much better, in my own mind and practice, to be able to say to a student: “OK, that didn’t work, and here’s why, but what have you learned from the experience? And design an approach to assessment that rewards the learning instead of penalising the so- called failure.
One of our engrooved or deep-seated beliefs is that grades are important because they motivate students to do the work. Take them away, and students won’t do anything.
But oddly, for a discipline that says it relies on evidence-based research, there is little to no evidence or research that demonstrates that grades make students learn more or work harder. In fact, there is ample evidence that grades actually do the opposite: They hurt academic motivation and inhibit learning.
We’ve known for a long time, well before Covid, that the way we do assessment is damaged and creaking at the seams. Perhaps Covid and now GenerativeAI can finally provide the impetus we need to let go of outdated, obsolete practices that are well past their sell-by date, and embrace those that are fit for purpose to meet the challenges we and our students face
What we do know is that students – and we are all students, lifelong learners – work harder, learn more and are much more likely to thrive and achieve when we are intrinsically motivated. When we have some real autonomy, real choices. When we feel we are in control of our learning. It means being given meaningful choices and engaging, authentic tasks to choose from. It means feeling empowered to choose, as students, where to invest our time and energy. It also means feeling encouraged and supported even if that means, receiving feedback that is uncomfortable but honest and that comes from a good place.
Autonomy also means that our own autonomy, our own academic identity has to shift, from the keepers and transmitters of knowledge to facilitators of learning
Also, as students we like to feel we’re continually growing, improving, developing new skills and understandings. Our own students are no different, so the question for us as teachers and assessors is how best can we focus both our and our students skills, time and energy on helping them build the skills they are motivated to learn?
A sense of relatedness, a sense of genuine belonging is also critical. Somehow we need to find ways of enabling our students to understand they are not just a number, not just cogs in a vast machine but valued as individuals and as part of a larger community… that they matter more than their grades. And they will respond and realise they don’t need the carrot and stick of grades to care about their learning.
So…let’s leave grading to recede in the rear-view mirror, and focus on the road ahead and where that might lead.
For example, there’s a growing body of evidence that pass/fail, scaffolded by clear and coherent explanations, expectations, exemplars etc. shifts the student focus away from grades towards a focus on learning.
For example, move to some of the approaches that have flowed out of the ‘Ungrading’ movement, eliminating or at least greatly minimizing the use grades, focusing instead on providing frequent and detailed feedback to students on their work, in relation to the course learning goals.
For example, move away from atomised learning outcomes and the atomised, tick-box assessment practices that often accompany them towards a much more holistic approach and setting high expectations.
As Chickering and Gamson wrote back in the 1980s,
“Expect more and you will get more. High expectations are important for everyone – for the poorly prepared, for those unwilling to exert themselves, and for the bright and well motivated. Expecting students to perform well becomes a self-fulfilling prophecy when teachers and institutions hold high expectations of themselves and make extra efforts.“
It’s beginning to look like the only way to guarantee -and I use that word very advisedly – some sort of academic integrity involving an encounter with what a student genuinely understands, what they know, what they think and what they can do is, to meet them face-to-face and to interrogate them – in a rigorous but empathetic way.
Many ago, when I was doing some research around assessment, I came across an article in the Harvard Education Review. The article was called “Assessment at a Crossroads: Conversations” (Plus ca change!) and it was a verbatim record of a panel discussion between some of the editors of the Review. It was a really interesting piece, but one comment particularly struck a chord and stuck in my mind. And it’s been a sort of touchstone for me ever since. It was a remark by Walt Haney, who was at that time Professor of Education at the Center for the Study of Testing, Evaluation and Educational Policy in Boston. He said this:
“You’ve got to involve students actively, not just view them as objects of assessment, but as agents of assessment. This can be done in many ways. One is that you ask students systematically what they have learned. It’s a simple idea; it’s rarely done…..You find that students say some remarkable things.”
And students DO say some remarkable things, and not only is it a journey of exploration and discovery for them, it is also a journey of exploration and discovery for us. It’s a dialogic process.
And I imagine some of you sitting there thinking, that’s all very well with cohorts of 20 or 25 students. But what if you’ve got ten times that number. It’s impossible!
It isn’t, as Erik Driessen demonstrated 20 years ago in his work on portfolio-based assessment in Medical Education. That involved 247 students meetings regularly with a mentor, discussing their work, what they’ve learned, what should go in the portfolio and why. Of course, portfolios are not unproblematic, but those very real, person-to-person interactions, transactions and conversations cannot be faked.
So, to wind up….
My partner and I both have theatre backgrounds, and over dinner a couple of nights ago she asked how this keynote was going. I said, as you do, that it was going ok and that, but I needed a really, positive ending before the curtain comes down on the conference. Then, across the lemon ricotta pasta and spinach, she said: “Why don’t you ask Chat GPT?”.
I said “Really?” and I got that ‘I dare you’ look that’s hard to resist.
So I did, and asked Chattie G to provide an inspirational end-of-conference speech for an international conference on assessment in higher education. Full disclosure: what follows is entirely generated by the Stochastic Parrot, unexpurgated.
“As I stand before you today, inspired by the collective brilliance and unwavering dedication of this esteemed gathering, I call upon each of you to embrace an audacious vision for the future of assessment. We find ourselves at a critical juncture, where the very essence of assessment is under siege from numerous threats, both internal and external. But let us not succumb to fear or despair. Instead, let us rise as fearless champions of change and architects of a new era in assessment.
The threats we face are formidable, but they are not insurmountable. The first threat lies within ourselves—the shackles of complacency and conformity that confine us to the stagnant realms of the past. We must break free from these chains and embrace innovation with unyielding determination. Let us dare to dream of an assessment landscape that defies convention, where imagination knows no bounds and boundaries are mere stepping stones to greatness.
But we must also confront the external threats that loom over us. The threat of exclusion, where assessment becomes a gatekeeper, perpetuating inequity and stifling the voices of the marginalized. We must dismantle this fortress of exclusivity and construct a new paradigm that celebrates diversity and inclusivity. Let our assessments be instruments of empowerment, enabling every individual, regardless of their background or circumstances, to unleash their true potential.
Another menacing threat we face is the commodification of assessment—a perilous path where education becomes reduced to a mere transaction, and students are reduced to data points and metrics. We must resist this dehumanizing force and reclaim the true essence of assessment. Let us infuse it with purpose and meaning, recognizing the inherent worth of every learner and valuing their holistic growth and well-being above all else.
Lastly, the encroachment of technological advancements poses both promise and peril. As we navigate this digital frontier, we must wield technology as a powerful ally, not as an all-knowing, all-seeing entity. Let us ensure that technology serves as an enabler, enhancing our capacity to assess with accuracy, efficiency, and integrity. But let us never forget that behind every data point, there lies a human story—a story that must be honored and cherished.
So, my fellow trailblazers of assessment, let us unite in a resounding call to action. Let us challenge the status quo, disrupt the norms, and shape a future where assessment transcends the boundaries of tradition and rises to its full transformative potential. Together, we can forge a future where assessment becomes a catalyst for growth, a beacon of equity, and a testament to the power of human potential.
As we embark on this audacious journey, let us remember that the future is not something that happens to us; it is something we create. It is in our hands to mold and shape, to inspire and empower. So, let our legacy be one of courage, innovation, and unwavering commitment to the cause of assessment.
With hearts afire and minds alight, let us go forth and pioneer a future where assessment becomes the cornerstone of a just and enlightened society. May our resolve be unyielding, our endeavors boundless, and our impact everlasting.
Thank you, and together, let us transcend the boundaries of what assessment can be!”
And now, in my own words: we have a lot to do, colleagues…tremendous challenges….but as these past two days have demonstrated, we have incredible knowledge, skills, understanding, creativity, motivation, commitment to achieve what needs to be done. Thank you.
References
‘Assessment at a Crossroads: Conversations’ (1996). Transcript of panel discussion. Harvard Educational Review, Fall 1996.
Bender, E., Gebru, T., McMillan-Major, A., & Mitchell, M. (2021) “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” FAccT ’21, March 3–10, 2021, Virtual Event, Canada https://dl.acm.org/doi/10.1145/3442188.3445922 (accessed July 2023)
Chickering, A. and Gamson, Z. (1987 ‘Seven Principles for Good Practice in Undergraduate Education’. AAHE Bulletin p.3-7.
DeBrock, L., Scagnoli, N., Taghaboni-Dutta, F. (2020) ‘The Human Element in Online Learning’. Inside Higher Education, 17 March 2020.
Gramsci, A. (1930) in Selections from the Prison Notebooks of Antonio Gramsci, ed. and trans. Quintin Hoare and Geoffrey Nowell-Smith (London: Lawrence & Wishart, 1971), 276.
Trowler, P. (2020) Accomplishing Change in Teaching and Learning Regimes: Higher Education and the Practice Sensibility. Oxford University Press
Stumbling with Confidence 