Education – Page 2 – Stumbling with Confidence

F-AI-L: forget about assessment?

You can’t move these days in Higher Education without reading about how AI has completely compromised most, if not all, the approaches to assessment that require a student to produce an artefact e.g. an essay, a poster, a portfolio, etc. It would appear that the only way to ensure ‘assessment security’ is to, first, thoroughly body scan and search each student for any devices – external or internal – that might be used to ‘assist’ them to pass the exam and then lock them in a Shielded Room or Faraday Cage under very strict and tight surveillance.

Not exactly an enlightened or constructive educational experience.

Having focused on assessment for much of my career, I’m beginning to wonder whether we should give up on assessment altogether. But there is one secure form assessment, one that we have practiced for centuries, that goes back to assessment’s Latin root ‘assidere (to sit together or beside)’: the viva. Simply sitting down with a student and asking them or observing them rigorously and systematically in regard to what they know, what they understand, what they have learned, and what they can do.

There are, essentially, two questions that a university has to answer in regard to their graduates: 1) does the student possess sufficient knowledge, understanding and skills in their particular discipline in order to be awarded a qualification? and 2) does the student possess sufficient knowledge, understanding and skills to stand a good chance of successfully navigating their way through an uncertain and complex world?

Given that many graduates (an old figure in the UK was c. 48% but I suspect it hasn’t changed much) end up working in a field unrelated to their course of study or discipline, the efficacy of the first question is…..questionable (except in the case of professional qualifications e.g. doctors, engineers etc.). As to the second question there’s an assumption they are ready to go out into the ‘real’ world, but that remains an assumption in many cases.

So…..what if we scrap assessment as we know it?

One thing we do know about assessment is that the inter-rater reliability of a group of experts observing, discussing and assessing a student’s work tends to be higher than when those same experts assess against a set of criteria or rubrics.

I’m suggesting we scrap the inordinate amount of time and energy spent on marking and grading assessments that are now insecure and unsafe and replace them with regular ‘assessment tutorials or mentoring’ with at least two members of staff.

In order to do that properly those tutorials must be built into the teaching/contact hours. Teaching is no longer about ‘delivery’ i. e. a teacher-centred and content-oriented paradigm, and shifts to a student-centred, learning-oriented paradigm in which assessment is about sitting down with a student and discovering the answers to those two questions.

^{About me:}

^{I am an independent higher education consultant and co-founder of the educational consultancy Ciel Associates specialising in organisational and transformational change and enhancing learning, teaching and assessment. I have over thirty years experience in a wide range of higher education providers both as a consultant and, before that, in a number of management, leadership and strategic roles. My work is informed by my belief in the transformational power of education and a commitment to enabling institutions to create meaningful and sustainable change. Contact me at Ciel Associates} .

Adventures in HE: time for a Skunk Works?

A few years ago, a UK university established a new initiative with funding from central government. The initiative was one of about 70 such initiatives across the HE sector which were established in the hope that, within the five years of the funding, the universities involved would have taken risks and pioneered innovative learning approaches that could be adopted and/or adapted across the sector.

The university in question established a project with the aim of developing more practice-based arts provision within the university, establishing more links between the university and external artists and arts organisations, and to increase the links between the subject areas within the creative and performing arts programmes within the university. The project rapidly established a reputation within the university for taking risks and for having a different way of looking at things. It was also known for having a different way of talking about things and even a different way of naming things, for example, modules. A critical factor in the project’s ability to forge an innovative path was that its director had been brought in from outside academia, and was relatively and refreshingly unencumbered by things like history and precedence and the engrooved customs, practices and discourses of the institution.

The director had created a Skunk Works.

Skunk Works are, typically, small and loosely structured groups of people who research and develop a project for the sake of innovation. There are several important and essential features of a Skunk Works: one is that it operates independently of the normal research and development operations of the company or organisation; another is that, in order to achieve unusual results, the people working on the project work in ways that are outside the normal ways of working.

One of the major projects initiated by the director was the development of a pilot MA in interdisciplinary arts practice. The aims of the course were to explore and extend the boundaries of various arts disciplines, and all aspects of the course were designed to be innovative and radical – the curriculum, the delivery, and the assessment. As with all courses, the proposal had to go through the university’s validation process and there were intense discussions about what might or might not be acceptable, especially given the relatively traditional academic culture of the university. The director stuck to her guns and, much to the surprise and, indeed, chagrin of some colleagues, the university validated a course that included modules called ‘Adventures in Interdisciplinarity’ and ‘Further Adventures in Interdisciplinarity’ in which the course participants– a mixture of recent graduates and mature, part-time students– gathered on a Friday and worked intensively through the weekend on various experimental projects and exploratory assignments with artists, directors, film-makers and composers.

These ‘Adventures’ were designed to provide unusual, typically hazardous (creatively, intellectually, etc.), experiences or activities. Calling the modules ‘Adventures’ was a very deliberate move and was highly resonant of Ronald Barnett’s call to educators to ‘hang onto a language of delight, wonder, care, excitement, fun, engagement and love– the language in which a student is caught and even entranced’ (Barnett, 2008).

At a time of particularly rapid change with a host of serious challenges facing higher education, perhaps establishing a Skunk Works and investing in the space to breathe, to invent, to imagine, to innovate, and to create something that is different, exciting, and genuinely transformational might be a very good and beneficial move.

^{Barnett, R. 2008. A will to learn. London: Open University Press.}

^{Image created by AI}

^{About me:}

Assessment: paradigm shift required?

There has been a slew of recent journal articles, blogs, podcasts etc. on the challenges posed by Generative AI in higher education and, particularly, the threat GenAI poses to assessment integrity and security along with possible approaches to mitigate the threat.

The picture that emerges is one in which none (perhaps bar one: the viva?) of the traditional forms of assessment are secure – were they ever? It seems clear that not only does a new assessment paradigm need to emerge but also a new paradigm for learning and teaching. As Gramsci wrote in the 1930s: ‘The old is dying and the new is yet to born: in the interregnum, all kinds of morbid symptoms appear’.

The challenge we face in regard to somehow ensuring or at least maximising assessment integrity and security in the face of GenAI is that, essentially, assessment – in the age of mass higher education – has become an economic and logistical issue, not a pedagogical one. An industrialised process of mass production that relies on the production-line workers (lecturers) doing much of the quality control i.e. assessment and marking, in their spare time in order to meet the production deadlines

What has largely disappeared from the way we assess students is the idea of assessment, inherent in its Latin root ’assidere’ (to sit together or sit beside), as a dialogic process. What hasn’t disappeared, though somewhat diminished, is the dominance of the teacher-centred, curriculum-focused paradigm rather than a student-centred, learning-focused paradigm.

In a recent and worth reading GenAI and assessment-focused journal article by Guy J. Curtis*, he wrote: ‘Students can reasonably expect to have their learning assessed on what they have been taught.’ He’s right but, just as important, students should also expect to be assessed on, or at least have an intensive discussion about, what they have learned. This may well (and should) extend beyond disciplinary specificity as they head towards an increasingly complex and uncertain future in which, whether we like it or not, AI will play an integral role.

So, while we struggle to find ways to make assessment secure without resorting to unacceptable and detrimental levels of surveillance, we might usefully attempt to answer the question: how might we best assess students in ways that reveal what they actually know, what they have learned and what they can do? The pedagogical answer may well be by sitting beside them and questioning them rigorously and systemically. But that begs another question: is higher education willing and able to shift to a new assessment paradigm?

* Guy J. Curtis article

Image created by AI

^{About me:}

Design for Learning: a case for detangling education?

Academics, eh? We may be great teachers. We may be great researchers. We may even be great managers and administrators. But that does not make us great educational designers….and I write this as someone who trained and worked as a designer before I stumbled into teaching design and some other subjects in higher education. Once inside academia I was immediately struck by the fact that a great number of things just didn’t seem to work very well. I was surrounded by talented, skilful, intelligent, committed and passionate colleagues who appeared to spend an inordinate amount of time and energy just getting things to work for them and their students.

It didn’t take me long to work out that one of the primary causes of all this inefficiency and waste and the bang-head-on-desk frustration that resulted, was frequently and simply a tangle of poor design. The plethora of complex systems, labyrinthine processes and perplexing protocols that extend to every corner of our educational endeavours all too often have been created by individuals and groups who – with the best will in the world – are not well acquainted, if at all, with the basic principles of good design.

An awareness and appreciation of concepts such as ‘good design enhances the users’ experience’, or ‘good design is logical e.g. form follows function’, or ‘good design is minimal design e.g. as little as possible and only as much as absolutely necessary’, or ‘good design is consistent right down to the fine details’ was and is often entirely lacking.

There is also an aesthetic quality to good design, but I’ve yet to hear or read that word, or anything similar, when it comes to discussing the crucially important task of designing the educational experiences of students.

If we are to be architects of educational experiences, then we must accept that not only do we need to embrace the principles and practices of good design, but – crucially – we either need to become skilled educational architects and designers ourselves or ensure that at least some of us have or develop those skills so that we can help our colleagues and institutions not only detangle the knots that bind us but also, and more importantly, to create and support the wonderful educational experiences that our students truly deserve.

Paul Kleiman Design for Learning

^{About me:}

Notes from the edge: piano lessons

I aim to play the piano most days, if I am near one. I work from home a great deal of the time, and I do a lot of writing. The work often involves some complex problems – large and small – that need to be addressed. When I am stuck, simply fed-up and frustrated or just need a break, I’ll go to the room with the piano and play for 10, 20, maybe 30 minutes. When I sit at the piano, I might choose to run through one or two of the classical pieces I’ve learned to play reasonably well over the years. Or I might choose a jazz or popular standard that I’ve picked up by ear, which involves a bit of improvisation in that sense of working relatively loosely within a recognised framework. I never play the same tune in exactly same way: but then, who does?

Usually I just place my hands on or over the keys, and I wait to see what happens. I have no idea of what is going to happen before it takes place. Something stirs. Something starts. A note or a chord is played. And off I go. Or off ‘it’ goes, because I feel I’m not in conscious control of my fingers. I am, of course, but it doesn’t feel that way.

David Sudnow, in his now classic work Ways of the Hand (1978) which is a remarkable insider’s account of learning to improvise jazz piano that was based mostly on his own introspection, describes having the most vivid impression of his hands making music by themselves. Sometimes, for me, it feels a bit awkward, as I travel down some musical cul-de-sac or find myself in a particular and sometimes too-familiar groove. Other times it just flows, I’m ‘in the zone’, and I know, especially when it really flows, that it clears and refreshes not only my mind but also my spirit.

In relation to ‘flow’ and Czikszentmihalyi’s influential work on that topic, we know that performances that combine flow states with a degree of risk taking might hold the key to achieving optimal levels of musical communication in improvisation. Being in the flow or ‘groove’ sometimes enables experienced improvisers to move beyond or extend their previous cognitive limits.

Sudnow uncovered many principles about learning, and particularly, what we might call embodied learning. His analysis and observations resonate powerfully with the work and research around creativity in learning and teaching, and what might be referred to as learning or teaching at the ‘edge of chaos’.

_{Photo by Paul Kleiman}