Tag Archives: TEF

From ‘evidence-based’ to ‘post-truth’: is this a trend in higher education?

Is there a trend within higher education that parallels the general trend in society, from ‘evidence-based’ to ‘post-truth’? There has been a trend (that I have been aware of for several months, though it has probably been going on for very much longer) of a move away from research and data towards a justification of claims in the media by using statements such as, ‘a lot of people think that’. This trend has been played out very publicly in elections in the UK and in the US in the past year, where it seems that if you say something often enough and loud enough, then it will be accepted as part of the canon. Maybe that has always been so? But when we have Government ministers on the TV telling us that we shouldn’t listen to experts because sometimes they can get things wrong, it does sound like Homer-Simpson-reasoning.

We seem to be witnessing a similar trend in higher education where ideas seem to be distorted to fit political and economic aims. If you are really cynical, you might go back through press cuttings and see a move from ‘evidence-based’ to ‘student-centred’ to ‘post-truth’. I am not arguing against student-centredness here, but I am aware of the ways that is can be miss-represented so that the phrase ‘but the students want it’ seems to trump other arguments without any real analysis of what or why. But there is a question (probably many) here about what students want, which students want it and why students want it – whatever ‘it’ might be. There also seem to be a number of contradictions in what ‘students want’. We are told that students want more online learning. So it seems sensible to capture lectures and allow students to review the content in their own time. All very sensible. However, I have been told by a lot of people (in a post-truth sense) that if we insist that lecturers are filmed teaching, then they adopt a more conservative approach in the classroom for fear of being ‘You-tubed’. This might lead to increasingly teacher-centred, didactic lectures – after all, discussion and dialogue don’t always play well in recordings of lectures. But, hang on. I am also told that students want more engagement in class – something that might be inhibited by lecture-capture. So the students want it both ways? Problem.

In the media there seems to be an apparent polarisation of the community in which elements are now referring to students as a ‘snowflake generation’, who we cannot challenge or upset, for fear of unleashing their displeasure as costumers. Such an approach to students seems to be a device to increase the distance between teachers and students. And when I have interacted with students recently, they actually seem to want challenge in their education. So again, what is it that students want? We seem to be in danger of assuming there is a single ‘student voice’ that is truly representative. But ‘a lot of people think’ there are actually a lot of different voices within the student body, and among the academics. As the philosopher said, ‘all generalisations are incorrect’. Perhaps we should be looking at ways to exploit diversity rather than seek homogenisation?

Politicians seem to be in the same position of power over universities as the students. After years of criticism about the NSS and the way it informs us (or not) about teaching quality, we are now set to employ selected elements of it in the TEF to evaluate the teaching quality of institutions. By referring to this as a ‘metric’, we have managed to confer some level of credibility to the numbers generated so that interested parties may call the whole process a rigorous and tested procedure. In the face of such post-truth pronouncements, universities seem to have rolled over and accepted their fate – ready to be measured-up (either for their new ball gown or their coffin, depending upon which axe you are interested in grinding).

There may also be a difference between what students want and what students need. To take a health analogy here – over the years many patients have wanted (and got) antibiotics from their doctors because they are suffering from a virus. This is despite all the evidence that antibiotics do not have an effect on viruses. Any expert can tell you this. However, after years of overprescribing, we are now in the situation where antibiotics are becoming less effective against bacteria – bacterial resistance. This pandering to patients has had a harmful effect on the overall population. Without proper debate and analysis of issues, some colleagues view acknowledgement of the student voice as a similar kind of pandering – and such a dismissive view does not help the development of more informed pathways to student-centredness and student engagement.

So where are we and where do we want to go? It appears that we now live in a society where if we don’t provide evidence, then the fake news sources will make it up anyway. I am now wondering if I will soon read a research report that offers ‘post-truth’ as a theoretical framework to underpin conclusions. Perhaps peer review of research will become even more interesting in the coming years?

 

Advertisements

Christmas mindbender – “Learning Jounce”: real or imaginary?

 

Here’s an idea that may or may not make sense. Stemming from one of those after dinner conversations that ended up with a “what if … ?”. Starting from the idea that things move at a certain speed and in a certain direction – velocity. This has some resonance with current discourse on learning – ‘learning gain’ being the distance travelled by a student over a period of time. So students learn at a certain speed. So we can see an analogy between velocity and learning gain – see the left side of the figure below. Learning gain has been under-theorised up until now – so let us problematize!

Now we can consider things like accelerated learning – where there is a change in the velocity of learning. We could have learning acceleration as an idea. So far so good. So the 1st derivative of the position vector (of understanding) would be learning gain  and the second derivative of the position vector would be learning acceleration:

learning-jounce

PDF FIGURE:  learning-jounce

Accelerated learning is widely known, where students are taken along at a faster speed than is typically anticipated. So what if we push the analogy along? It might be more fun than trying to figure out the jokes in your Christmas cracker. And the discussion might make more sense after a glass or two of mulled wine! Anyway, here goes:

A change in acceleration is known as ‘Jerk’. So if the rate of acceleration increases or decreases we would have +ve or -ve Jerk. So if we had a change in the acceleration of learning we would have ‘Learning Jerk’. This is something that perhaps we could get our minds around. If learning gain has a ‘normal speed’ (e.g. one module per semester), then accelerated learning would have an increased speed (e.g. one module per semester, then two modules per semester, then three, and so on). A change in that rate of acceleration (e.g. suddenly back to one module per semester or up to four modules per semester) would be a ‘learning jerk’. So learning jerk would be variation in learning acceleration – a break in the smooth pattern of acceleration. If we then take the student voice into consideration, we could have ‘student-initiated acceleration’, where students felt they could move ahead more quickly, or ‘student-initiated jerk’ where the student body were allowed to vary the rate of learning acceleration at different points in their learning journey in response to changes in other factors.

For those who would like to apply the maths for Jerk:    j = \frac{da}{dt} = \frac{d^2v}{dt^2} = \frac{d^3x}{dt^3}   .

So logically (perhaps after another glass of wine), we should be able to proceed to a change in learning jerk – learning jounce. This would be a change in the change in the change of learning gain. In a student-led institution there will be variation in jerk across the student body and this will need a dedicated administrative team: possibly overseen by a new senior post (PVC – JOUNCE). Just imagine the learning analytics (Jounce analytics), wouldn’t they be fun. But what would learning jounce look like? And more importantly, what would be the metric that we could apply to TEF?

The ‘Jounced University’ would certainly be student-focussed, and may even come to the realisation that assessment inhibits  Jounce. Within the TEF, universities that acknowledge Jounce could be awarded a Bronze level in the TEF, those that implement ‘Assessed Jounce’ awarded a Silver, with Gold reserved for those institutions who manage to operate ‘Free Jounce’.

As an aside (and I think we need one here so we don’t get too serious), Jounce (the 4th derivative) is also known as ‘snap’. So can you guess what the 5th and 6th derivatives are called?  Yep – ‘crackle’ and ‘pop’. Wouldn’t it be great if we could have metrics for snap, crackle and pop with which to confuse our political masters? The Boxing Day game here is therefore to imagine your university league table for 2017, ranking institutions for “snap, crackle and pop”.

Merry Christmas!

 

Reference:

MrReid.org at: http://wordpress.mrreid.org/2013/12/11/jerk-jounce-snap-crackle-and-pop/

Student evaluation of teaching: are we reaching for the wrong type of excellence?

Over twenty years ago Carr (1994: 49) wrote:

 ‘It is a shallow and false view of education and teaching which takes it to be a matter of the technical transmission of pre-packaged knowledge and skills in the context of efficient management’

However, it seems that this false view is still able obscure a more contemporary and research-informed views of teaching. The on-going drive for ‘teaching excellence’ still seems to focus on actions of the teacher that promote Carr’s ‘shallow view’. That is not to say that the student voice is not important, but we need to ensure that students are asked the right questions so that we do not promote student passivity as learners and do not subvert the student voice for purely political ends.

Fitzgerald et al (2002) wrote, ‘I value student perspectives in thinking about my practice. However, the institutional instrument designed to assess student perspectives focuses on a form of practice that ill fits my own values. Each semester students have been asked to rate the course and instructor on a nineteen-item rating scale. Many of the items are consistent with a teacher-directed pedagogy and linear information-processing model of learning (for example, Objectives are Clear, The Instructor Enhanced Knowledge of Subject, Organised Class Sessions Well, Demonstrates Knowledge of the Subject). When I first came to Uni, I was intimidated by these student evaluations because I believed them to be overly focused on clear objectives and a class structure predicated on teacher control of the classroom. These survey items do not adequately capture what I hope to accomplish in the classroom, and I correctly anticipated receiving mixed review on this measure. Students may desire and need clearly presented knowledge, attained in a highly structured teacher-directed context, but educational opportunities are impoverished if this is the only form of pedagogy provided. A rather monovocal assessment tool inscribes a particular vision of education, and fails to provide useful feedback to educators who teach in alternative ways. This narrow representation of education limits our vision of what ‘good’ education might be, and privileges a particular mode of learning.’

So are we still promoting ‘monovocal assessment tools’ ? If so why? Many commentators ask why those who purportedly revere the power of critical thinking go on to employ simplistic, quantitative tools to ‘measure’ teaching quality. Clearly, in the UK, the Government’s agenda to assess teaching is pushing things along with a single purpose in mind. Katzner (2012) has asserted that in their quest to describe, analyze, understand, know, and make decisions, western societies have accepted the myth of synonymy between objective science and measurement. He comments that what we cannot measure gets demoted as ‘less important’. If it has been measured, it must be ‘scientific’ and ‘rigorous’ – especially if we can apply statistical analysis that the common man/woman will not understand.

So we go from monovocal to monocular (possibly also myopic). A system in which ‘ideas diversity’ and ‘methodological variation’ (typically seen as indicators of health for an academic community) are apparently no longer valued. We then end up with a ‘hard core’ set of unquestioned statements and assumptions that are not supported by evidence. The result is that we have an academic community that will survive by ‘maintaining their autonomy and academic freedom through demonstrating symbolic compliance or pragmatic behaviour’ (Teelken, 2012: 287). This could result in innovative teaching being driven underground, like a resistance movement – a situation that is likely to promote pedagogic frailty (Kinchin, et al, 2016). An outcome that is the opposite of that intended. Fitzgerald et al (2002) talked about values as the underpinning concept that drives things forwards with any meaning. I wonder if the explication of values (particularly shared values, rather than any spurious mission statement placed on a web site) by universities will form part of the TEF, which will inform teaching evaluations in the UK over the coming years. Or is teaching supposed to be ‘values-free’ in the modern era? I didn’t get that memo.

 

References:

Carr, D. (1994) Educational enquiry and professional knowledge: Towards a Copernican revolution. Educational Studies, 20(1): 33 – 54.

 

Fitzgerald, L.M., Farstad, J.E. & Deemer, D. (2002) What gets ‘mythed’ in the student evaluations of their teacher education professors? In: Loughran, J. and Russell, T. (Eds.) Improving teacher education practices through self-study. London, Routledge/Falmer (pp. 203-214).

 

Katzner, D. W. (2012). Unmeasured information and the methodology of social scientific inquiry. Springer Science & Business Media.

 

Kinchin, I.M., Alpay, E., Curtis, K., Franklin, J., Rivers, C. and Winstone, N.E. (2016) Charting the elements of pedagogic frailty. Educational Research, 58(1): 1 – 23.

 

Teelken, C. (2012) Compliance or pragmatism: how do academics deal with managerialism in higher education? A comparative study in three countries. Studies in Higher Education, 37(3): 271 – 290.

 

“It’s the wrong discourse, Grommit!”

In the animated film, “The Wrong Trousers”, fans of Wallace and Grommit will remember how the situation develops in which Wallace is wearing a pair of mechanical trousers that are taking him in various directions that he doesn’t want to follow – controlled by the evil Feathers McGraw. As he is taken off down the road he shouts, “It’s the wrong trousers, Grommit!”.

So, I am asking if universities have found themselves wearing the metaphorical ‘wrong trousers’, following a path that perhaps they don’t want to follow – with someone else at the controls. Rather than trousers, I am asking if universities are racing along in a direction that is guided by the wrong discourse?

There is so much discussion of elements of the instructional discourse at the moment. Colleagues are concentrating on issues such as assessment & feedback and curriculum processes in order to enhance the student experience. Now they probably also have half an eye on the ways in which the UK Government might be heading (Wallace-like) to evaluate these through a set of metrics that might constitute a ‘Teaching Excellence Framework’.

But what about the other side of the picture:

discourse cmap

 PDF of discourse cmap

What of the values, theories and beliefs that underpin all this procedural stuff?  Are we convinced that we have got that covered? Are we all in agreement about everything and so it can just be a ‘given’?  I am not convinced.

Looking at the educational research literature, especially that which tries to visualise the qualitative structure of understanding (as I do), perhaps we have brought some of this monocular view on ourselves? If you look through the concept mapping literature, most of the concept maps that feature in publications tend to have focussed on content. This is ironic given that much of that literature is critical of teaching that is content-focussed. But there are relatively few concept maps that concentrate on the left hand side of the figure above.

If you used the map in the figure here as a guide, I wonder how much of it could be annotated by looking at university web sites that proclaim the distinctiveness of institutions and the quality of student experience that is offered. Within those pages you could probably find evidence to build a picture of the content to be taught, the assessments that will be administered and the curricula that will be delivered (the right hand side). But how much evidence is there of the underpinning values of the institution, the educational theories that inform teaching or the beliefs that guide decision-making processes (the left hand side) ?

I wonder if anyone can identify an institution that has an explicit ‘binocular view’, as expressed in the figure above? And I wonder if the TEF will demand a binocular view, or if it is being driven down a monocular view by a real ‘Feathers McGraw’ ? Let us hope that in 18 months time we are not all screaming, “It’s the wrong framework, Grommit!”.

References

The Wrong Trousers:  https://en.wikipedia.org/wiki/The_Wrong_Trousers

Teaching Excellence Frameworkhttps://www.timeshighereducation.co.uk/news/jo-johnson-commits-teaching-excellence-framework