curriculum – Stumbling with Confidence

F-AI-L: forget about assessment?

You can’t move these days in Higher Education without reading about how AI has completely compromised most, if not all, the approaches to assessment that require a student to produce an artefact e.g. an essay, a poster, a portfolio, etc. It would appear that the only way to ensure ‘assessment security’ is to, first, thoroughly body scan and search each student for any devices – external or internal – that might be used to ‘assist’ them to pass the exam and then lock them in a Shielded Room or Faraday Cage under very strict and tight surveillance.

Not exactly an enlightened or constructive educational experience.

Having focused on assessment for much of my career, I’m beginning to wonder whether we should give up on assessment altogether. But there is one secure form assessment, one that we have practiced for centuries, that goes back to assessment’s Latin root ‘assidere (to sit together or beside)’: the viva. Simply sitting down with a student and asking them or observing them rigorously and systematically in regard to what they know, what they understand, what they have learned, and what they can do.

There are, essentially, two questions that a university has to answer in regard to their graduates: 1) does the student possess sufficient knowledge, understanding and skills in their particular discipline in order to be awarded a qualification? and 2) does the student possess sufficient knowledge, understanding and skills to stand a good chance of successfully navigating their way through an uncertain and complex world?

Given that many graduates (an old figure in the UK was c. 48% but I suspect it hasn’t changed much) end up working in a field unrelated to their course of study or discipline, the efficacy of the first question is…..questionable (except in the case of professional qualifications e.g. doctors, engineers etc.). As to the second question there’s an assumption they are ready to go out into the ‘real’ world, but that remains an assumption in many cases.

So…..what if we scrap assessment as we know it?

One thing we do know about assessment is that the inter-rater reliability of a group of experts observing, discussing and assessing a student’s work tends to be higher than when those same experts assess against a set of criteria or rubrics.

I’m suggesting we scrap the inordinate amount of time and energy spent on marking and grading assessments that are now insecure and unsafe and replace them with regular ‘assessment tutorials or mentoring’ with at least two members of staff.

In order to do that properly those tutorials must be built into the teaching/contact hours. Teaching is no longer about ‘delivery’ i. e. a teacher-centred and content-oriented paradigm, and shifts to a student-centred, learning-oriented paradigm in which assessment is about sitting down with a student and discovering the answers to those two questions.

^{About me:}

^{I am an independent higher education consultant and co-founder of the educational consultancy Ciel Associates specialising in organisational and transformational change and enhancing learning, teaching and assessment. I have over thirty years experience in a wide range of higher education providers both as a consultant and, before that, in a number of management, leadership and strategic roles. My work is informed by my belief in the transformational power of education and a commitment to enabling institutions to create meaningful and sustainable change. Contact me at Ciel Associates} .

Adventures in HE: time for a Skunk Works?

A few years ago, a UK university established a new initiative with funding from central government. The initiative was one of about 70 such initiatives across the HE sector which were established in the hope that, within the five years of the funding, the universities involved would have taken risks and pioneered innovative learning approaches that could be adopted and/or adapted across the sector.

The university in question established a project with the aim of developing more practice-based arts provision within the university, establishing more links between the university and external artists and arts organisations, and to increase the links between the subject areas within the creative and performing arts programmes within the university. The project rapidly established a reputation within the university for taking risks and for having a different way of looking at things. It was also known for having a different way of talking about things and even a different way of naming things, for example, modules. A critical factor in the project’s ability to forge an innovative path was that its director had been brought in from outside academia, and was relatively and refreshingly unencumbered by things like history and precedence and the engrooved customs, practices and discourses of the institution.

The director had created a Skunk Works.

Skunk Works are, typically, small and loosely structured groups of people who research and develop a project for the sake of innovation. There are several important and essential features of a Skunk Works: one is that it operates independently of the normal research and development operations of the company or organisation; another is that, in order to achieve unusual results, the people working on the project work in ways that are outside the normal ways of working.

One of the major projects initiated by the director was the development of a pilot MA in interdisciplinary arts practice. The aims of the course were to explore and extend the boundaries of various arts disciplines, and all aspects of the course were designed to be innovative and radical – the curriculum, the delivery, and the assessment. As with all courses, the proposal had to go through the university’s validation process and there were intense discussions about what might or might not be acceptable, especially given the relatively traditional academic culture of the university. The director stuck to her guns and, much to the surprise and, indeed, chagrin of some colleagues, the university validated a course that included modules called ‘Adventures in Interdisciplinarity’ and ‘Further Adventures in Interdisciplinarity’ in which the course participants– a mixture of recent graduates and mature, part-time students– gathered on a Friday and worked intensively through the weekend on various experimental projects and exploratory assignments with artists, directors, film-makers and composers.

These ‘Adventures’ were designed to provide unusual, typically hazardous (creatively, intellectually, etc.), experiences or activities. Calling the modules ‘Adventures’ was a very deliberate move and was highly resonant of Ronald Barnett’s call to educators to ‘hang onto a language of delight, wonder, care, excitement, fun, engagement and love– the language in which a student is caught and even entranced’ (Barnett, 2008).

At a time of particularly rapid change with a host of serious challenges facing higher education, perhaps establishing a Skunk Works and investing in the space to breathe, to invent, to imagine, to innovate, and to create something that is different, exciting, and genuinely transformational might be a very good and beneficial move.

^{Barnett, R. 2008. A will to learn. London: Open University Press.}

^{Image created by AI}

^{About me:}

Assessment: paradigm shift required?

There has been a slew of recent journal articles, blogs, podcasts etc. on the challenges posed by Generative AI in higher education and, particularly, the threat GenAI poses to assessment integrity and security along with possible approaches to mitigate the threat.

The picture that emerges is one in which none (perhaps bar one: the viva?) of the traditional forms of assessment are secure – were they ever? It seems clear that not only does a new assessment paradigm need to emerge but also a new paradigm for learning and teaching. As Gramsci wrote in the 1930s: ‘The old is dying and the new is yet to born: in the interregnum, all kinds of morbid symptoms appear’.

The challenge we face in regard to somehow ensuring or at least maximising assessment integrity and security in the face of GenAI is that, essentially, assessment – in the age of mass higher education – has become an economic and logistical issue, not a pedagogical one. An industrialised process of mass production that relies on the production-line workers (lecturers) doing much of the quality control i.e. assessment and marking, in their spare time in order to meet the production deadlines

What has largely disappeared from the way we assess students is the idea of assessment, inherent in its Latin root ’assidere’ (to sit together or sit beside), as a dialogic process. What hasn’t disappeared, though somewhat diminished, is the dominance of the teacher-centred, curriculum-focused paradigm rather than a student-centred, learning-focused paradigm.

In a recent and worth reading GenAI and assessment-focused journal article by Guy J. Curtis*, he wrote: ‘Students can reasonably expect to have their learning assessed on what they have been taught.’ He’s right but, just as important, students should also expect to be assessed on, or at least have an intensive discussion about, what they have learned. This may well (and should) extend beyond disciplinary specificity as they head towards an increasingly complex and uncertain future in which, whether we like it or not, AI will play an integral role.

So, while we struggle to find ways to make assessment secure without resorting to unacceptable and detrimental levels of surveillance, we might usefully attempt to answer the question: how might we best assess students in ways that reveal what they actually know, what they have learned and what they can do? The pedagogical answer may well be by sitting beside them and questioning them rigorously and systemically. But that begs another question: is higher education willing and able to shift to a new assessment paradigm?

* Guy J. Curtis article

Image created by AI

^{About me:}

Design for Learning: a case for detangling education?

Academics, eh? We may be great teachers. We may be great researchers. We may even be great managers and administrators. But that does not make us great educational designers….and I write this as someone who trained and worked as a designer before I stumbled into teaching design and some other subjects in higher education. Once inside academia I was immediately struck by the fact that a great number of things just didn’t seem to work very well. I was surrounded by talented, skilful, intelligent, committed and passionate colleagues who appeared to spend an inordinate amount of time and energy just getting things to work for them and their students.

It didn’t take me long to work out that one of the primary causes of all this inefficiency and waste and the bang-head-on-desk frustration that resulted, was frequently and simply a tangle of poor design. The plethora of complex systems, labyrinthine processes and perplexing protocols that extend to every corner of our educational endeavours all too often have been created by individuals and groups who – with the best will in the world – are not well acquainted, if at all, with the basic principles of good design.

An awareness and appreciation of concepts such as ‘good design enhances the users’ experience’, or ‘good design is logical e.g. form follows function’, or ‘good design is minimal design e.g. as little as possible and only as much as absolutely necessary’, or ‘good design is consistent right down to the fine details’ was and is often entirely lacking.

There is also an aesthetic quality to good design, but I’ve yet to hear or read that word, or anything similar, when it comes to discussing the crucially important task of designing the educational experiences of students.

If we are to be architects of educational experiences, then we must accept that not only do we need to embrace the principles and practices of good design, but – crucially – we either need to become skilled educational architects and designers ourselves or ensure that at least some of us have or develop those skills so that we can help our colleagues and institutions not only detangle the knots that bind us but also, and more importantly, to create and support the wonderful educational experiences that our students truly deserve.

Paul Kleiman Design for Learning

^{About me:}

We’d never get away with it now!

6 problems and developing creative confidence in students

LIPA (Liverpool Institute for Performing Arts) welcomed its first students in 1996. In those early days, as a very rare brand new higher education institution, LIPA had many of the typical characteristics of a ‘start-up’: exciting, risk-taking, a bit of a roller-coaster. Looking back at that time, we were certainly operating on the edge of chaos.

I was Head of Performance Design and had designed and written most of the Performance Design curriculum. One of the things I was really keen on was the idea of ‘de-schooling’: getting the new first year students out of the more creativity inhibiting habits and expectations they had arrived with.

So, during induction week, alongside the usual introductory sessions, the handing out of timetables, module handbooks, etc., we set the students six visual ‘problems’ that they had to solve before the end of that first semester.

There were some interesting strings attached to that assignment:

the students were on their own, they could not discuss the problems with their fellow students or with any of their tutors
their work would not be assessed
all their ‘solutions’ would be shown in an exhibition at the end of the semester which would have a proper opening and an invited guest list.

A number of colleagues, when they heard about this, thought we were mad, confident that the students would never do it as it wasn’t being assessed.

The six problems included the following:

Create a self-portrait in any medium.
Create a map of how you get from your bed to the studio in the morning
Take three matchboxes and create an object or objects using all the contents and the boxes themselves
The Black Square Problem: using six black squares against a white ground illustrate a series of words e.g. chaos, love, kindness, growth etc.

(To be honest, nearly thirty years on, I can’t remember the other two problems! One, I think, was something to do with delineating space, and I have no idea what the sixth one was. I was hoping it might come to me as I was writing this).

The day the students came in to set up the exhibition was an extraordinary day. As each student brought in their work, their peers gathered round to see and discuss what was before them. The sense of collegiality and excitement was palpable as was the sense of creativity and imagination at play.

Some of those solutions remain clear in my mind to this day. Here are just three:

J., a German student who had grown up in East Germany before unification, had taken the three matchboxes. She had flattened out the three drawers and stuck them together to make a flat sheet. On the sheet, she had drawn three musical staves (five lines each) using the ends of some burnt matches. Then, using the matches as musical notes, she stuck them along the staves in such a way as to create the opening bars of Beethoven’s ‘Ode to Joy’. The remaining matches were stuck around the sheet to make a frame. Two of the outer matchbox containers were used as labels for the work. But where was the third? It was stuck on the back of the sheet and used to hang the sheet onto a hook on the wall.

(I know I have a photograph of it somewhere!)

C. had attached a very large, rectangle of black card on the wall. It was in landscape format and across the middle, horizontally, ran a c.4cm strip of paper which was covered in dozens of identical, narrow vertical stripes of red, orange, light blue and dark blue. It looked like a beautiful spectrogram. In the bottom right hand corner was written ‘C’s map’. I remember staring at this with a colleague and wondering what did this map represent. Eventually, we called C. over and asked her to explain. It turned out that she lived about 2 miles from LIPA and it was a very cold day when she started making the map. What she had done was to walk from her warm bedroom, through the cold streets to LIPA taking a temperature reading every 20m or so. The different colours represented different temperatures. Red/Orange = warm, Light Blue/Dark Blue=cool/cold. When we asked what the warm strip right in the centre was, C. answered that her measured steps had taken to her to a hot air outlet of a shop.

T. had really struggled with the Black Square problem. Just couldn’t get his head around it. Sitting at breakfast a day or so before the exhibition opening, he was pondering the problem to the extent that he forgot he’d put a slice of bread in the toaster. Suddenly out popped a square of blackened toast. It was one of those Eureka moments. T. had found his solution to the black square problem. His solution consisted of illustrating the words using a series of four pieces of square black toast stuck to the windows of the exhibition space.

The exhibition opening was a great success. All the students managed to produce a creative solution to each of the problems…some of them truly extraordinary. There were 2D works, 3D works, video, audio, animation. Some made one laugh. Others, particularly some of the self-portraits, were rather disturbing. The strange, bloodied doll in the box which had a peep hole haunts me still!

Art work. Wooden bok with peep hole. Through the peephole you see a doll sitting in the corner of the box, with a dribble of blood coming from her mouth.

All the art works were presented and lit beautifully. The purpose of the whole enterprise was, of course, nothing to do with the actual problems or the exhibition of work. It was all about confidence, enabling the students to realise that not only did they have some wonderful creative ideas but they could work on their own and create and produce wonder-full work.

The offer of the exhibition was key to getting the students motivated and committed to the project despite there being no assessment. The exhibition made it clear that we valued their ideas and the work they produced. Too often, work into which students have poured their heart and soul is simply handed in or submitted electronically without even a ‘thank you’. I recall one department in another institution where essays and dissertations were simply posted into a box fixed to the wall near the departmental office.

The six visual problems project demonstrated that if you create interesting assignments and make it clear that you really care about the work students produce, assessment isn’t a given. In recent decades, higher education has developed to the point where the culture is one of ‘if it moves, assess it!’. The solving and exhibiting of the six visual problems proved that need not, necessarily, always be the case.

Could we get away with it now?