Using Educational Research to Inform Our Daily Practice: Caveats and Opportunities

“Educational research is the beginning of a conversation, not the end of one.”

I’ve been saying that phrase quite a lot these days, both in my daily work at the University of Mississippi and in the workshops I have been facilitating at other campuses.

Although I’m not often given to pithy maxims, this one seems to resonate with folks. Here is what I mean:

  1. Educational research (e.g., research on the effectiveness of teaching strategies) can only ever provide us with a limited perspective on one aspect of a key question. No paper is *ever* going to have all the answers to our teaching questions or even a single, complete answer to our inquiries about teaching practices simply because there are too many variables when it comes to the classroom. It would be unreasonable to expect any single paper or even a cluster of papers to provide this for us.
  2. No study is perfect. Even good papers can have methodological oversights. Even great papers will be limited in scope (see #1).
  3. No matter how much we would like to have neatly packaged answers about pedagogical conundrums, the truth of the matter is that such solutions simply don’t exist. Even the debate about lecturing vs. using active learning in the classroom is far more nuanced than we often admit.
  4. Do numbers 1, 2, and 3 mean that we should discount the value of educational research for informing our teaching practices? Absolutely not!

If you are a faculty member or graduate student for whom educational research is the primary means of scholarly communication, promotion, tenure, and professional advancement, then by all means dissect methods, stats, p-values, and much more. This work is very valuable, and I don’t want to discount it. I enjoy talking to my fellow medievalists about which manuscript of The Canterbury Tales is more representative of Chaucer’s vision for the project. At the same time, it’s important to acknowledge that all of this is inside baseball. When I think about what I want colleagues and students outside of Medieval Studies to know about The Canterbury Tales, there are far more relevant questions I want them to consider.

If, on the other hand, we are thinking about an instructor of a college-level course in her or his discipline who is seeking out more effective ways of teaching, I would say that our concerns are a bit different. What does this colleague need to see in a study to give her or him confidence not that an effect is statistically significant but instead enough confidence to give the strategy a try in the classroom, to get feedback from students, to refine and try again? Must the bar for educational research be so high that it prohibits experimentation in our courses? I don’t think it needs to be, but conversations in the higher ed press and on social media sometimes set it at a place where it can make the work sometimes feel unapproachable or perhaps even unusable for those who are doing the work of teaching day in and day out. I think that’s a missed opportunity on many levels and it can lead to an over-reliance on intuition and assumption rather than evidence when choosing pedagogical strategies.

This is not to say that each and every study rises to the level of confidence I just described. What I’m proposing here is that we might advocate for one use of educational research as providing us with evidence for testing something out in our classrooms, and to make this determination we will still need to interrogate the studies, just with a slightly different eye.

These issues came into sharp focus two weeks ago when a study was published in Proceedings of the National Academy of Sciences arguing that a) students learned less in a class session taught entirely through lecture than they did in a session taught through active, engaged pedagogies, and b) they felt like they learned more in the lecture session. Immediately the lines in the rhetorical sand were drawn. Eric Mazur, Harvard professor and developer of the pedagogical strategy called Peer Instruction, said “This work unambiguously debunks the illusion of learning from lectures.” Others joined this chorus. Of course that’s not true. No single study could *unambiguously debunk* anything.

On the other side of the conversation, some researchers looked very closely at the methodology and the statistical analysis in the paper. Clarissa Sorensen-Unruh wrote a very thoughtful blog post about some of the issues she found at this level of analysis. I learned a lot from her and from others parsing the study with this degree of detail, but at the same time I can’t help but think that we see more of the trees than the forest when we do this. What is there for individual teachers to take away from a study when we move in this direction? If every paper received this level of scrutiny, the conversation would either proceed slowly or not at all. To be clear, serious mistakes should rule a study out of our consideration, but–beyond that–how do we decide whether something is worth trying?

What do we make of all this? If we think of educational research as the beginning of a conversation rather than the end of one, then we’ll think about whether or not we are convinced that a given strategy might be something we could try out in our courses. Then we’ll look at our own results, compare them to other research that comes out, and continue the conversation from there. For the study I’ve just been describing, I’m intrigued enough to test out a similar approach and to suggest that others consider trying it too. We need to take what we find valuable and give it a go. Regardless of whether something is shown in a study to work well or not as well, it may work (or not work as the case may be) in our own courses, and that is something worth exploring.

The same, by the way, is true of larger research programs, like the kind that led to models such as Carol Dweck’s mindsets and Angela Duckworth’s grit. These frameworks were all the rage in educational circles for years, but recent meta-analyses have called into question the efficacy of growth mindsets and the predictive power of grit. Both threads of research are important. Do we throw grit and mindsets out wholesale because of this new work? No! But we approach them with caution, take the pieces of them that may be valuable for our students, and we evaluate them for ourselves.

As someone who has benefited greatly from the many outstanding research studies on teaching and learning in higher education, I hope that this post will be taken in the spirit of debate. We need more discussions of teaching in higher ed, not fewer, and I hope educational research can play a major role in these conversations.

**I’m grateful to Sarah Rose Cavanaugh, whose Twitter thread on the PNAS study spurred me on to write this post.

 

A Response to Richard Utz: On Inclusivity in Medieval Studies and the International Congress on Medieval Studies

[Note: This post appeared as a Twitter thread and as a comment on insidehighered.com earlier today.]

I have a lot of respect for Richard Utz, and the International Congress on Medieval Studies at Western Michigan University has meant a lot to me personally and professionally for the last 18 years, but I think the article in today’s Inside Higher Ed misses the point about inclusivity at the conference and in the field more generally.

First, it positions the debate as the BABEL Group versus the world, and this is simply untrue. Yes, it was the BABEL steering committee that authored the letter, but the hundreds of additional signatures indicate that the concerns raised in the letter are shared by many. Second, the article suggests that the issue of inclusivity is limited to an inclusion of areas of study and/or viewpoints on the field. This is certainly one dynamic, and I want to address it before moving on.

To demonstrate that the ICMS really is inclusive of different fields, Utz first cites the many (and diverse) types of traditional sessions that the ICMS has offered in the past, which have been sponsored by groups like the Pearl Poet Society and Cistercian Studies. He then suggests that the ICMS has embraced more recent areas of study by saying, “The 2018 program, for example, features the term ‘race’ nine times, ‘disability’ nine times and ‘gender’ and ‘feminism’ 48 times.”

Here’s the problem I see with this logic: I took his cue and counted the terms myself. I actually counted 12 instances of the term “disability,” but most of these occur in ONLY 2 SESSIONS both sponsored by the Society for the Study of Disability in the Middle Ages. There are 2 other papers beyond these sessions that mention the term “disability,” and then the rest of the occurrences appear whenever the SSDMA is mentioned. That’s 4 out of 600 sessions that touch on disability in any way. As someone who has worked in the area of Medieval Disability Studies for over a decade, I am grateful for the support of the ICMS over the years, but I would hardly count the number of appearances of the term as evidence that the area has been broadly included. The same is true for other terms cited in the article. The mere appearance of the terms is not indication that these fields are “included” in a truly meaningful way. Ask me how many times I presented on a disability-related panel with 10 or fewer in the audience.

Honestly, though, the bigger issue with respect to inclusion is one that the article barely even addresses, which is the degree to which scholars from traditionally underrepresented groups have felt included in both the ICMS and Medieval Studies. The call for ICMS to include more sessions about the state of the field is directly related to this larger point about inclusivity. It isn’t just a push for “progressivism” for its own sake but is a response to structures that have pushed people to the margins.

If you want evidence of that, you need only read this powerful blog post by @Nahir_Otano: http://medievalistsofcolor.com/uncategorized/lost-in-our-field-racism-and-the-international-congress-on-medieval-studies/ …

Or listen to the loud calls coming from many angles by medievalists of color, LGBT scholars, early career scholars, and more. Or listen to the whisper networks that have always existed. But we must listen. The project of inclusivity in Medieval Studies is a big one, and it will take a collective effort of all of us to make it possible. We cannot reduce the efforts to pat solutions like counting sessions, because the issues are structural and will take a lot of work. The work is essential, though, and we must attend to it carefully.

The Best Education Books I’ve Read (Since 2013)

‘Tis the season for best-of lists, I hear.  I don’t want to feel left out of the list-making, so I’ve decided to create one of my own.  As I have mentioned many times on this blog, I have been writing a book for the last few years, and I have read a lot (and I mean A LOT) of material in the process of doing research for the project.  I thought, then, that I might make a list of the best education books I have read in that period of time.  Hence, this is not a list of my Absolute Top Picks of All Time, although some of the books I discuss below would make that list as well.  Instead, it is a recent best-of list.  Let me know what your own favorites are in the comments!

Here they are, in alphabetical order:

Susan Debra Blum, “I love learning; I hate school”: An Anthropology of College.  I learned a lot from this book.  Blum brings her skills as an anthropologist to bear on our institutions of higher education and forces us to ask essential questions about the work we do.

Susan Engel, The Hungry Mind.  This is an amazing book!  I recommend it so often that I think people are getting tired of me.  Engel provides an in-depth study of curiosity:  why it matters, how essential it is for child development, how it gets lost in our educational systems, and what we can do about it.

Alison Gopnik, Andrew N. Meltzoff, and Patricia K. Kuhl, The Scientist in the Crib.
I was late to the party on this book (it was published in 1999), but it forever changed my view on learning.  Just read it; you’ll see.  I have to thank my friend and colleague Robin Paige for recommending the book to me.

Anya Kamenetz, The Test. Kamenetz’s criticisms of the standardized test movement in America are powerful. She clearly shows the damage they have done and offers important suggestions for a way forward.

Jessica Lahey, The Gift of Failure.  This is not strictly a book about education–it covers parenting, as well–but Lahey is a middle school teacher, and so much of the book connects to our work in the classroom.  It’s a vital read for lots of reasons, but my favorite quote from the book says a lot about its importance:  “We have taught our kids to fear failure, and in doing so, we have blocked the surest and clearest path to their success. That’s certainly not what we meant to do, and we did it for all the best and well-intentioned reasons, but it’s what we have wrought” (xi).

James Lang, Small Teaching: Everyday Lessons from the Science of Learning.  As someone who works in a Center for Teaching Excellence, this book has been a gift.  Lang’s easy-to-implement suggestions pair efficiency with effectiveness and are supported by recent findings in cognitive science.

Claire Howell Major, Michael S. Harris, and Todd Zakrajsek, Teaching for Learning. I love this one!  It’s such a useful handbook of pedagogical topics and research-based strategies. Any question you might have about teaching is addressed to some degree in this book.

Tracey Tokuhama-Espinosa, Mind, Brain, and Education Science. Although I have some methodological quibbles with the self-described discipline of MBE Science, this book is a wonderful example of how to productively synthesize findings from science research and apply it responsibly to our work as educators.

Honorable Mention:  I am currently reading Sarah Rose Cavanagh‘s The Spark of Learning: Energizing the College Classroom with the Science of Emotion.  Because I haven’t finished it yet, I did not include it in the main list, but I fully expect it to make the next list I write (whenever that may be).

Why We Need to Stop Using the Phrase “Bulimic Learning”

A short post today, and one with a simple thesis:  It is time to remove the phrase “bulimic learning” from the lexicon of higher education.  The term, which as far as I can tell dates back to the early ’90s, is often used to discuss the common strategy of students cramming for an exam and then spilling as much knowledge as possible during test time.  The popularity of the phrase is evidenced by the fact that it is the subject of a recent book on teaching and learning, a peer-reviewed article from 2010, in educational workshops, and even on websites affiliated with university Centers for Teaching and Learning.  You’ll have to trust me when I tell you that there are many more examples in rather prominent places, which is exceedingly unfortunate.

The frequency with which “bulimic learning” is used is alarming, particularly in a field that strives for inclusivity and empathy.  Referring to any learning strategy in such an offhanded way, of course, minimizes the struggles and the suffering of those who are living with eating disorders.  In this country alone, “20 million women and 10 million men suffer from a clinically significant eating disorder at some time in their life.”  The fact that more women than men are diagnosed with eating disorders means that there is a gendered component to the phrase “bulimic learning, ” as well, which only adds to our list of reasons to find a different way to talk about the educational phenomenon of cramming for tests.  For example, we could simply say “cramming for tests.”  If we really need a metaphor here, what about the image of a toaster?  Bread goes in, and bread goes out.  The bread is of no actual value to the appliance.  For those who simply must have a biological analogy for studying (though I’m not sure why!), even “regurgitation” would not include as many traumatic connotations.  I still think it’s kind of gross, to be honest, but at least the word is not as stigmatizing as “bulimic.”

In short, let’s show respect and compassion to our students and our colleagues, some of whom may either have an eating disorder, had one in the past, or know someone who has/had one of these diseases.  Let’s find other ways to talk about student learning.

 

Guest Post: “A Plea for Research, Part 1” by Kisha Tracy

Today’s entry is the third in a series of guest-posts from the roundtable on “Teaching the Humanities in the Current Climate of Higher Education” that I organized for the recent International Congress on Medieval Studies in Kalamazoo, Michigan.  Kisha Tracy and I teamed up on a presentation called “A Plea for Research” as a way to encourage our fellow medievalists (and humanists more generally) to engage in the scholarship of teaching and learning.  What follows is Kisha’s part of the presentation.

Kisha’s Bio:  Kisha Tracy is an Assistant Professor of English Studies, specializing in early British and world literatures, and Co-coordinator of the Center for Teaching and Learning at Fitchburg State University. She received her Ph.D. in Medieval Studies from the University of Connecticut in 2010. She is currently working on two book projects: Why Do I Have to Take This Course? Theory and Practice of Student Investment in Learning and Sins of the Past: Remembering, Forgetting, and Confessing in Middle English Literature. The former considers how we can encourage our students to be more invested in their courses, and the latter explores how the traditional medieval relationship between memory and confession provides a valuable framework for understanding the employment of recollection in various Middle English literary texts.


A Plea for Research, Part 1

Several years ago, a group of colleagues and I secured a grant to study the effect of embedded librarians in first-year composition courses. We were surprised at the definitiveness of the data we collected and decided it was worth publishing. At this time, I was relatively new to the concept of the scholarship of teaching and learning, but this was an excellent opportunity and introduction to this type of research. We wrote it up and were quite pleased with the outcome. Then came the most difficult question and the most enlightening for me: in what order did our names go? We simply stared at each other in confusion. Being mostly humanities scholars, we weren’t used to multi-authored scholarship. We wrote alone or, at the most, with only one other colleague. We had no idea in what order to write our names.  We did eventually figure it out!

For me, this was a moment that brought home the idea that we in the humanities do not work in collaboration nearly as often as we should.  It is time for some transparency with our humanities colleagues. One way is in collaborating on disciplinary research, but collaborating on the scholarship of teaching and learning provides additional opportunities. It makes clear to ourselves and our colleagues what we do in our classrooms, how we teach our subjects. That kind of communication can only improve teaching on a broad scale. Transparent collaboration of this nature also creates clear and consistent messages about the value of studying the humanities and the direct impact that we have upon what students are learning.

Guest Post: “‘So Are You Going to Open a History Store?’ Explicit Professionalization and the Undergraduate Humanities Major” by Leigh Ann Craig

Today’s entry is the second in a series of guest-posts from the roundtable on “Teaching the Humanities in the Current Climate of Higher Education” that I organized for the recent International Congress on Medieval Studies in Kalamazoo, Michigan.  Here we have the insightful remarks from Leigh Ann Craig.

Leigh Ann’s Bio:  Leigh Ann Craig is an associate professor of history at Virginia Commonwealth University in Richmond, VA.  She is the author of Wandering Women and Holy Matrons:  Women as Pilgrims in the Later Middle Ages (Brill, 2009), the associate editor of The Encyclopedia of Medieval Pilgrimage (Brill, 2009), and is currently in progress on a manuscript entitled Deprived of Sense and Intellect: Demons, Humors, and Diagnoses of Loss of Mind in Medieval Europe, 12140-1500.  She has also been deeply involved in curriculum revision at VCU and is the recent winner of the VCU College of Humanities and Sciences Distinguished Teaching Award.


“‘So Are You Going to Open a History Store?’  Explicit Professionalization and the Undergraduate Humanities Major”

We have all heard the same derisive questions from “realists” about the value of the undergraduate degree programs our disciplines offer.[1] My favorite iteration came from a grad school colleague, whose roommate’s father asked him, upon learning he was a History major, whether he planned to one day open a history store.[2] Meanwhile, we have all seen, and many of us have written for, the stream of apologia and outcomes research that seeks to correct these misapprehensions by revealing the flexibility and broad applicability of undergraduate training in the humanities, and the employability and career success of our graduates.[3] We nonetheless meet the same questions in an endless loop. This is both wearying and worrying, especially amid the climate of budgetary scarcity and gently declining enrollments that seem presently to be typical of state institutions like my own.[4] Narrative, as we all know, has a way of becoming reality when it is repeated often enough.

I would like to propose today that our responses to that cultural meme need to be curricular, as well as rhetorical and academic: that we must show rather than tell, and we, the subject-area experts, must do that showing ourselves. Two curricular trends already exist which may be instructive, both of which are usually, in higher-ed circles, tagged as ‘professionalization’ efforts. The first of these is the ‘professionalization’ curriculum which is more and more coming to be appended to graduate programs in the humanities. These programs teach, in a systematic way that was unavailable to me two decades ago, the nuts-and-bolts skills of academic career and project management, and discuss our specialized professional writing formats such as grant applications and job market materials.[5] The second is the ‘professionalization’ curriculum enmeshed within degrees in fine arts, including BFA and MFA programs, whose graduates require a broad set of career and business management skills in order successfully to pursue studio art, professional theatre, and the like.[6] But at the same time, each of these professionalization curricula are apt to consider in a transparent fashion how the skills of a highly specialized area of study might translate to work outside of the pure disciplinary focus of the academy or the art studio.[7] This kind of coursework is now beginning to appear, as an elective offered through careers centers, for use by undergraduate humanities majors, in places like University of Texas at Austin and the University of Arkansas.[8]

My own department is currently debating the possibility of piloting an undergraduate course on Professionalization for the History Major. The goals of the course would be threefold: first, to make explicit connections between the goals and objectives of the History major and the interests and professional goals of the student;[9] second, to facilitate active careers exploration (inclusive of a broad range of speakers from the community); and third, to offer students the skills they need to build the interface between their training and career paths of interest. This would include coursework designed to teach specialized writing skills (such as skills-based resumes, cover letters and personal statements explaining the applicability of their specific education and experience, public history writing, writing for online forums and social media), but also service-learning requirements which would foster interpersonal skills in public speaking, networking, and interviewing, via work in support roles for on-campus events.

(As an aside, I note that it would be hard to simply add this to other standing courses in our current curriculum. While some of these goals might be served in our semester-long internship capstones, but a majority – the critical self-marketing skills, especially – are not. Nor do our internships, grounded in one institution, directly support a broad process of career exploration. Indeed, students might be far better prepared to choose and to apply for internships if professionalization were introduced first. We also lack space for this in our writing-intensive gateway course; I have taught it for a decade and cannot see how this could be embedded within it, given time constraints and its more purely disciplinary goals.)

I have come to ponder whether it might be wise for us to consider not just offering, but requiring such a course for all History majors at VCU, for reasons that are philosophical and pedagogical rather than merely economically pragmatic. I am the parent of a high school junior, and based on her experience I can say with some confidence that by the time they reach us, our students have jumped through many a hoop, badly designed by those far from the classroom. They have rightfully come to understand the process of education, and especially of assessments of learning, to be devoid of applied purpose more often than not. By contrast, I have found that transparency about goals, and about the utility of specific classroom methods I use, is a reliable way to elicit cooperative, and even wry-but-cheerful, engagement from students, even in exercises with the worst reputation for mindless drudgery (see: How to Write A Footnote.) If we can use transparency to our advantage on the level of an individual assignment, it seems to me that a similarly nuts-and-bolts discussion of the professional value of the entire curriculum might help engage students in the process in all of their coursework to a helpful degree. While I cannot guess whether this might lead to improvement in retention and graduation rates, my instincts suggest that it would at very least improve engagement in ways less susceptible to statistical analysis, yet substantively important in the classroom.

And this, I think, is also the reason why humanities faculty should consider offering such courses in our home departments in cooperation with other kinds of specialists, rather than farming the issue out entirely to campus careers centers. We are our own smartest and best advocates, when it comes to explanations of the value of the curriculum we design and implement. The other edge of that sword, of course, is that we are trained medievalists rather than trained professionalization coaches. I myself have not spent a single year outside of academic spaces since I turned four, and I did not apply my two undergraduate humanities degrees outside of structures that were dedicated to pure disciplinary focus. But it seems to me that at that point we must reach for the vaunted autodidacticism and adaptability of the humanities-trained, reach out to our networks to gain the outside expertise we may need, and put it to work. As we consider this possibility at VCU, I would be very interested in the experience of any of my colleagues here with the development or implementation of this type of curriculum, and of discussion of its value.

 

[1] For a more prominent example, see the comments of Sir James Dyson in 2012. http://www.thetimes.co.uk/tto/education/article3596125.ece. A broader overview may be found in this article from Paul Jay and Gerald Graff’s 2012 article in the Chronicle of Higher Education: https://www.insidehighered.com/views/2012/01/05/essay-new-approach-defend-value-humanities.

[2] With thanks to Dr. Marc Horger, Department of History, The Ohio State University.

[3] See for example Katherine Brooks, “Why Major in History?,” Psychology Today, August 2012, https://www.psychologytoday.com/blog/career-transitions/201208/why-major-in-history; or Jeffrey Dorfman’s analysis on the excellent return on investment in humanities degrees in Forbes from Nov. 2014, http://www.forbes.com/sites/jeffreydorfman/2014/11/20/surprise-humanities-degrees-provide-great-return-on-investment/#14e395c094a9.

[4] On the nationwide numbers and decline in enrollments, see http://www.historians.org/publications-and-directories/perspectives-on-history/november-2014/drilling-down-into-the-latest-undergraduate-data.

[5] See curricular info from UCSD at http://humctr.ucsd.edu/graduate-professionalization-series/; see infro from Loyola Chicago here: http://luc.edu/history/stories/archive/professionalizationworkshops.shtml.

[6] See the recent discussion by Eliza Lamb, “Best Practices for Career Preparation in Four Undergraduate Art Programs” (Ed.D. thesis, Teacher’s College, Columbia University, 2015).

[7] Lamb, ch. 4, passim; Brooks, “Why Major in History?.”

[8] Brooks’ course at UT Austin has been running in brick-and-mortar space for a decade and is now available online: http://news.utexas.edu/2010/10/07-0. The University of Arkansas course through their Department of English, taught by Dr. Lissette Lopez Szwydky, includes a collaborative blog available at https://theadaptivehumanist.wordpress.com/about/.

[9] For further discussion of this portion of things, see the AHA Tuning Project: https://www.historians.org/teaching-and-learning/tuning.