Cancel the Teaching Evaluations Too!

In a recent piece for CNN about universities adopting a pass/fail policy for students during the COVID-19 crisis, David Perry added a few thoughts about faculty as well:

Faculty also need to be judged on a pass/fail basis. Online teaching is uniquely hard and cannot be mastered as a skill in a couple of hours or days. Student evaluations — which for pre-tenure or pre-promotion faculty are vitally important to their career advancement — will be useless in this new arrangement, as will any evaluations of research productivity.

Perry’s comments are spot on and worth serious consideration. While much of the conversation in higher ed at this time has rightly focused on students–because of the major disruption caused by the coronavirus, the emotional and psychological turmoil brought on by the crisis, and the sudden transition to a new learning environment–faculty are facing all of this too. Universities should grant faculty the same grace they are extending to students and administrators should not evaluate their teaching in the same way they would in other semesters.

There are several options for how we might proceed with respect to student evaluations of teaching (SETs for short; some people also call them student ratings of instruction, or SRIs, but I’m not splitting semantic hairs in this post) at this time, and I’ve divided those options into four categories below:

Category 1: Change nothing about the current process of collecting SETs and using them for decisions that affect a person’s career.

Analysis: To my mind this is an unjustifiable approach at this time. As I noted above, faculty are working in an unprecedented situation where they are being asked to do so much with tools, parameters, and environments that are–in many cases–unfamiliar. They’re doing amazing work, but it is unfair to evaluate them as if they were teaching in traditional circumstances.

Category 2a: Collect SETs but do not use them for reappointment, promotion, and/or tenure.

Category 2b: Collect SETs but allow faculty to choose whether or not to include the results when they are up for reappointment, promotion, and/or tenure.

Analysis: Both of these options look reasonable on the surface, but they share a common flaw. If the results are out there, then it is possible that people other than the faculty member will see the results. This may lead to implicit bias down the road when faculty are being evaluated for reappointment, promotion, and/or tenure. Similarly, I think the choice option is the least desirable. Imagine a situation where two faculty are going up for tenure at the same time. One has chosen to include results from the COVID-19 affected semester and one has not. How will both of these faculty be evaluated at that time? Also, will faculty feel undue pressure to make a choice they are uncomfortable with? These options simply open up too many potential scenarios that undermine their utility.

Category 3: Design a new SET that focuses more on learning than on teaching behaviors.

Analysis: Great idea; it’s never going to happen given the compressed timeframe.

Category 4: Do not collect SETs during COVID-affected semesters.

Analysis: This, to me, is the most equitable model. This is the option with the least possibility for misuse (because there will not be anything to misuse) and it is the only one that truly levels the playing field.

I know what you’re thinking: but, Josh, don’t we want student feedback on their experience this semester? Yes, we do, but we want different kinds of feedback. Just like with any semester, those faculty who want individualized, formative feedback from their students should be encouraged to create a short survey in Survey Monkey, Qualtrics, etc. to ask their students these questions, or they could work with their friendly neighborhood teaching center to develop such a survey. This is right in our (meaning those who work in teaching centers) wheelhouse!

Secondly, we also want student feedback on the transition process–glitches in the technology, workload across the board, etc.–independent of what is happening with individual courses. We need to develop mechanisms to get this feedback sent directly to IT and the Provost’s Office in ways that are anonymized and that have no impact on a person’s career.

In short, this is a tumultuous semester for everyone. Faculty should not have their careers negatively affected by this disruptive transition, just as students should not have to worry about the effect of all of this on their grades. SETs have an impact on faculty of every stripe–adjuncts, full-time NTT faculty, and tenure-stream faculty. We need to look carefully at how we can address their use in the weeks and months ahead.

 

Using Educational Research to Inform Our Daily Practice: Caveats and Opportunities

“Educational research is the beginning of a conversation, not the end of one.”

I’ve been saying that phrase quite a lot these days, both in my daily work at the University of Mississippi and in the workshops I have been facilitating at other campuses.

Although I’m not often given to pithy maxims, this one seems to resonate with folks. Here is what I mean:

  1. Educational research (e.g., research on the effectiveness of teaching strategies) can only ever provide us with a limited perspective on one aspect of a key question. No paper is *ever* going to have all the answers to our teaching questions or even a single, complete answer to our inquiries about teaching practices simply because there are too many variables when it comes to the classroom. It would be unreasonable to expect any single paper or even a cluster of papers to provide this for us.
  2. No study is perfect. Even good papers can have methodological oversights. Even great papers will be limited in scope (see #1).
  3. No matter how much we would like to have neatly packaged answers about pedagogical conundrums, the truth of the matter is that such solutions simply don’t exist. Even the debate about lecturing vs. using active learning in the classroom is far more nuanced than we often admit.
  4. Do numbers 1, 2, and 3 mean that we should discount the value of educational research for informing our teaching practices? Absolutely not!

If you are a faculty member or graduate student for whom educational research is the primary means of scholarly communication, promotion, tenure, and professional advancement, then by all means dissect methods, stats, p-values, and much more. This work is very valuable, and I don’t want to discount it. I enjoy talking to my fellow medievalists about which manuscript of The Canterbury Tales is more representative of Chaucer’s vision for the project. At the same time, it’s important to acknowledge that all of this is inside baseball. When I think about what I want colleagues and students outside of Medieval Studies to know about The Canterbury Tales, there are far more relevant questions I want them to consider.

If, on the other hand, we are thinking about an instructor of a college-level course in her or his discipline who is seeking out more effective ways of teaching, I would say that our concerns are a bit different. What does this colleague need to see in a study to give her or him confidence not that an effect is statistically significant but instead enough confidence to give the strategy a try in the classroom, to get feedback from students, to refine and try again? Must the bar for educational research be so high that it prohibits experimentation in our courses? I don’t think it needs to be, but conversations in the higher ed press and on social media sometimes set it at a place where it can make the work sometimes feel unapproachable or perhaps even unusable for those who are doing the work of teaching day in and day out. I think that’s a missed opportunity on many levels and it can lead to an over-reliance on intuition and assumption rather than evidence when choosing pedagogical strategies.

This is not to say that each and every study rises to the level of confidence I just described. What I’m proposing here is that we might advocate for one use of educational research as providing us with evidence for testing something out in our classrooms, and to make this determination we will still need to interrogate the studies, just with a slightly different eye.

These issues came into sharp focus two weeks ago when a study was published in Proceedings of the National Academy of Sciences arguing that a) students learned less in a class session taught entirely through lecture than they did in a session taught through active, engaged pedagogies, and b) they felt like they learned more in the lecture session. Immediately the lines in the rhetorical sand were drawn. Eric Mazur, Harvard professor and developer of the pedagogical strategy called Peer Instruction, said “This work unambiguously debunks the illusion of learning from lectures.” Others joined this chorus. Of course that’s not true. No single study could *unambiguously debunk* anything.

On the other side of the conversation, some researchers looked very closely at the methodology and the statistical analysis in the paper. Clarissa Sorensen-Unruh wrote a very thoughtful blog post about some of the issues she found at this level of analysis. I learned a lot from her and from others parsing the study with this degree of detail, but at the same time I can’t help but think that we see more of the trees than the forest when we do this. What is there for individual teachers to take away from a study when we move in this direction? If every paper received this level of scrutiny, the conversation would either proceed slowly or not at all. To be clear, serious mistakes should rule a study out of our consideration, but–beyond that–how do we decide whether something is worth trying?

What do we make of all this? If we think of educational research as the beginning of a conversation rather than the end of one, then we’ll think about whether or not we are convinced that a given strategy might be something we could try out in our courses. Then we’ll look at our own results, compare them to other research that comes out, and continue the conversation from there. For the study I’ve just been describing, I’m intrigued enough to test out a similar approach and to suggest that others consider trying it too. We need to take what we find valuable and give it a go. Regardless of whether something is shown in a study to work well or not as well, it may work (or not work as the case may be) in our own courses, and that is something worth exploring.

The same, by the way, is true of larger research programs, like the kind that led to models such as Carol Dweck’s mindsets and Angela Duckworth’s grit. These frameworks were all the rage in educational circles for years, but recent meta-analyses have called into question the efficacy of growth mindsets and the predictive power of grit. Both threads of research are important. Do we throw grit and mindsets out wholesale because of this new work? No! But we approach them with caution, take the pieces of them that may be valuable for our students, and we evaluate them for ourselves.

As someone who has benefited greatly from the many outstanding research studies on teaching and learning in higher education, I hope that this post will be taken in the spirit of debate. We need more discussions of teaching in higher ed, not fewer, and I hope educational research can play a major role in these conversations.

**I’m grateful to Sarah Rose Cavanaugh, whose Twitter thread on the PNAS study spurred me on to write this post.

 

A Response to Richard Utz: On Inclusivity in Medieval Studies and the International Congress on Medieval Studies

[Note: This post appeared as a Twitter thread and as a comment on insidehighered.com earlier today.]

I have a lot of respect for Richard Utz, and the International Congress on Medieval Studies at Western Michigan University has meant a lot to me personally and professionally for the last 18 years, but I think the article in today’s Inside Higher Ed misses the point about inclusivity at the conference and in the field more generally.

First, it positions the debate as the BABEL Group versus the world, and this is simply untrue. Yes, it was the BABEL steering committee that authored the letter, but the hundreds of additional signatures indicate that the concerns raised in the letter are shared by many. Second, the article suggests that the issue of inclusivity is limited to an inclusion of areas of study and/or viewpoints on the field. This is certainly one dynamic, and I want to address it before moving on.

To demonstrate that the ICMS really is inclusive of different fields, Utz first cites the many (and diverse) types of traditional sessions that the ICMS has offered in the past, which have been sponsored by groups like the Pearl Poet Society and Cistercian Studies. He then suggests that the ICMS has embraced more recent areas of study by saying, “The 2018 program, for example, features the term ‘race’ nine times, ‘disability’ nine times and ‘gender’ and ‘feminism’ 48 times.”

Here’s the problem I see with this logic: I took his cue and counted the terms myself. I actually counted 12 instances of the term “disability,” but most of these occur in ONLY 2 SESSIONS both sponsored by the Society for the Study of Disability in the Middle Ages. There are 2 other papers beyond these sessions that mention the term “disability,” and then the rest of the occurrences appear whenever the SSDMA is mentioned. That’s 4 out of 600 sessions that touch on disability in any way. As someone who has worked in the area of Medieval Disability Studies for over a decade, I am grateful for the support of the ICMS over the years, but I would hardly count the number of appearances of the term as evidence that the area has been broadly included. The same is true for other terms cited in the article. The mere appearance of the terms is not indication that these fields are “included” in a truly meaningful way. Ask me how many times I presented on a disability-related panel with 10 or fewer in the audience.

Honestly, though, the bigger issue with respect to inclusion is one that the article barely even addresses, which is the degree to which scholars from traditionally underrepresented groups have felt included in both the ICMS and Medieval Studies. The call for ICMS to include more sessions about the state of the field is directly related to this larger point about inclusivity. It isn’t just a push for “progressivism” for its own sake but is a response to structures that have pushed people to the margins.

If you want evidence of that, you need only read this powerful blog post by @Nahir_Otano: http://medievalistsofcolor.com/uncategorized/lost-in-our-field-racism-and-the-international-congress-on-medieval-studies/ …

Or listen to the loud calls coming from many angles by medievalists of color, LGBT scholars, early career scholars, and more. Or listen to the whisper networks that have always existed. But we must listen. The project of inclusivity in Medieval Studies is a big one, and it will take a collective effort of all of us to make it possible. We cannot reduce the efforts to pat solutions like counting sessions, because the issues are structural and will take a lot of work. The work is essential, though, and we must attend to it carefully.

The Best Education Books I’ve Read (Since 2013)

‘Tis the season for best-of lists, I hear.  I don’t want to feel left out of the list-making, so I’ve decided to create one of my own.  As I have mentioned many times on this blog, I have been writing a book for the last few years, and I have read a lot (and I mean A LOT) of material in the process of doing research for the project.  I thought, then, that I might make a list of the best education books I have read in that period of time.  Hence, this is not a list of my Absolute Top Picks of All Time, although some of the books I discuss below would make that list as well.  Instead, it is a recent best-of list.  Let me know what your own favorites are in the comments!

Here they are, in alphabetical order:

Susan Debra Blum, “I love learning; I hate school”: An Anthropology of College.  I learned a lot from this book.  Blum brings her skills as an anthropologist to bear on our institutions of higher education and forces us to ask essential questions about the work we do.

Susan Engel, The Hungry Mind.  This is an amazing book!  I recommend it so often that I think people are getting tired of me.  Engel provides an in-depth study of curiosity:  why it matters, how essential it is for child development, how it gets lost in our educational systems, and what we can do about it.

Alison Gopnik, Andrew N. Meltzoff, and Patricia K. Kuhl, The Scientist in the Crib.
I was late to the party on this book (it was published in 1999), but it forever changed my view on learning.  Just read it; you’ll see.  I have to thank my friend and colleague Robin Paige for recommending the book to me.

Anya Kamenetz, The Test. Kamenetz’s criticisms of the standardized test movement in America are powerful. She clearly shows the damage they have done and offers important suggestions for a way forward.

Jessica Lahey, The Gift of Failure.  This is not strictly a book about education–it covers parenting, as well–but Lahey is a middle school teacher, and so much of the book connects to our work in the classroom.  It’s a vital read for lots of reasons, but my favorite quote from the book says a lot about its importance:  “We have taught our kids to fear failure, and in doing so, we have blocked the surest and clearest path to their success. That’s certainly not what we meant to do, and we did it for all the best and well-intentioned reasons, but it’s what we have wrought” (xi).

James Lang, Small Teaching: Everyday Lessons from the Science of Learning.  As someone who works in a Center for Teaching Excellence, this book has been a gift.  Lang’s easy-to-implement suggestions pair efficiency with effectiveness and are supported by recent findings in cognitive science.

Claire Howell Major, Michael S. Harris, and Todd Zakrajsek, Teaching for Learning. I love this one!  It’s such a useful handbook of pedagogical topics and research-based strategies. Any question you might have about teaching is addressed to some degree in this book.

Tracey Tokuhama-Espinosa, Mind, Brain, and Education Science. Although I have some methodological quibbles with the self-described discipline of MBE Science, this book is a wonderful example of how to productively synthesize findings from science research and apply it responsibly to our work as educators.

Honorable Mention:  I am currently reading Sarah Rose Cavanagh‘s The Spark of Learning: Energizing the College Classroom with the Science of Emotion.  Because I haven’t finished it yet, I did not include it in the main list, but I fully expect it to make the next list I write (whenever that may be).

Why We Need to Stop Using the Phrase “Bulimic Learning”

A short post today, and one with a simple thesis:  It is time to remove the phrase “bulimic learning” from the lexicon of higher education.  The term, which as far as I can tell dates back to the early ’90s, is often used to discuss the common strategy of students cramming for an exam and then spilling as much knowledge as possible during test time.  The popularity of the phrase is evidenced by the fact that it is the subject of a recent book on teaching and learning, a peer-reviewed article from 2010, in educational workshops, and even on websites affiliated with university Centers for Teaching and Learning.  You’ll have to trust me when I tell you that there are many more examples in rather prominent places, which is exceedingly unfortunate.

The frequency with which “bulimic learning” is used is alarming, particularly in a field that strives for inclusivity and empathy.  Referring to any learning strategy in such an offhanded way, of course, minimizes the struggles and the suffering of those who are living with eating disorders.  In this country alone, “20 million women and 10 million men suffer from a clinically significant eating disorder at some time in their life.”  The fact that more women than men are diagnosed with eating disorders means that there is a gendered component to the phrase “bulimic learning, ” as well, which only adds to our list of reasons to find a different way to talk about the educational phenomenon of cramming for tests.  For example, we could simply say “cramming for tests.”  If we really need a metaphor here, what about the image of a toaster?  Bread goes in, and bread goes out.  The bread is of no actual value to the appliance.  For those who simply must have a biological analogy for studying (though I’m not sure why!), even “regurgitation” would not include as many traumatic connotations.  I still think it’s kind of gross, to be honest, but at least the word is not as stigmatizing as “bulimic.”

In short, let’s show respect and compassion to our students and our colleagues, some of whom may either have an eating disorder, had one in the past, or know someone who has/had one of these diseases.  Let’s find other ways to talk about student learning.

 

Guest Post: “A Plea for Research, Part 1” by Kisha Tracy

Today’s entry is the third in a series of guest-posts from the roundtable on “Teaching the Humanities in the Current Climate of Higher Education” that I organized for the recent International Congress on Medieval Studies in Kalamazoo, Michigan.  Kisha Tracy and I teamed up on a presentation called “A Plea for Research” as a way to encourage our fellow medievalists (and humanists more generally) to engage in the scholarship of teaching and learning.  What follows is Kisha’s part of the presentation.

Kisha’s Bio:  Kisha Tracy is an Assistant Professor of English Studies, specializing in early British and world literatures, and Co-coordinator of the Center for Teaching and Learning at Fitchburg State University. She received her Ph.D. in Medieval Studies from the University of Connecticut in 2010. She is currently working on two book projects: Why Do I Have to Take This Course? Theory and Practice of Student Investment in Learning and Sins of the Past: Remembering, Forgetting, and Confessing in Middle English Literature. The former considers how we can encourage our students to be more invested in their courses, and the latter explores how the traditional medieval relationship between memory and confession provides a valuable framework for understanding the employment of recollection in various Middle English literary texts.


A Plea for Research, Part 1

Several years ago, a group of colleagues and I secured a grant to study the effect of embedded librarians in first-year composition courses. We were surprised at the definitiveness of the data we collected and decided it was worth publishing. At this time, I was relatively new to the concept of the scholarship of teaching and learning, but this was an excellent opportunity and introduction to this type of research. We wrote it up and were quite pleased with the outcome. Then came the most difficult question and the most enlightening for me: in what order did our names go? We simply stared at each other in confusion. Being mostly humanities scholars, we weren’t used to multi-authored scholarship. We wrote alone or, at the most, with only one other colleague. We had no idea in what order to write our names.  We did eventually figure it out!

For me, this was a moment that brought home the idea that we in the humanities do not work in collaboration nearly as often as we should.  It is time for some transparency with our humanities colleagues. One way is in collaborating on disciplinary research, but collaborating on the scholarship of teaching and learning provides additional opportunities. It makes clear to ourselves and our colleagues what we do in our classrooms, how we teach our subjects. That kind of communication can only improve teaching on a broad scale. Transparent collaboration of this nature also creates clear and consistent messages about the value of studying the humanities and the direct impact that we have upon what students are learning.