Estimated reading time: 5 minutes, 25 seconds
I was right and wrong. I was right that I didn’t really need to read the post by Lawrie Phipps and Dave Cormier on narratives vs analytics in education and beyond (I like how they connected it to politics but that’s secondary to my post). Because it’s already a worldview I subscribe to, so really, what more was there to learn from a blogpost version of what I had been understanding deeply for years? Sure I tweeted it out coz it’s a brilliantly summarized version of these ideas. Really brilliant. And I tweeted this:
"Data does not care about context…it:s raw and seen thru the lens of the analyst" amen – @Lawrie & @davecormier https://t.co/NSwhfP9hYX
— ℳąhą Bąℓi, PhD مها بالي 🌵 (@Bali_Maha) November 19, 2016
But I was wrong. Because I do have something to add to this conversation and it’s also related something I posted on Paul Prinsloo’s Delightful blogpost on decolonizing student data.
And I have two things to say. I actually say these very often but for some reason people continue to make what I consider a fundamental mistake in looking at research (I don’t mean anyone in particular here and I don’t mean they have that mistake in their heads but that they may have it in the way they express themselves) :
- It’s not a divide between quantiative and qualitative research METHODS. It’s a divide between positivst and interpretivist (also critical) paradigms of research. It is completely possible to do qualitative research and analyze it in a positivist manner. Meet Creswell. It is also possible to mix methods in a positivist or interpretivist manner
- Empowering the “subject” comes from participation rather than the act of making our methods quantiative or qualitative. Yes. Qualitative research gives people more leeway to express their context and clarify their own interpretation and understanding of their own lives and experiences. But none of this is decolonizing if a researcher does it TO the “other”. Ethnography used to have colonial purposes. Understanding the colonized in order to have more power over them. Even critical approaches to research that are usually more qualitative and interpretivist can sometimes fall into the trap of the researcher believing they have a more objective view of the world than the disempowered/oppressed others being researched.
Any act of a researcher selecting which stories to share and how to share and interpret and aggregate them is a potential act of violence on the lived experiences of those being researched. If the researched individual/community is not involved in making those decisions.
Collaborative Autoethnography is the closest methodology I could find so far where the researcher and researched are the same… But not working alone to the extent that it’s mere navel-gazing and autobiography. The communal aspect of it is not perfect but it helps.
The problem with data is not just that it is decontextualized (as important as that is) but also who gets to create, collect, interpret and report it. Which is worse for quantiative data but entirely possible with qualitative data as the researcher also can decontextualize it through postpositivist approaches to “coding”. And even intepretivist approaches can do violence by just not having enough participation by those the researcher is trying to represent.
I am not saying let’s not do any non-participatory research. I am saying let’s recognize the violence and limitations of even the most well-meaning research.
And stop talking about rigor as if you know how useful that is. The more “traditionally rigorous” research has the least usefulness to teachers in classrooms. For obvious reasons imho.
For background on this post, read my previous post.
I am also pasting my comment on Paul Prinsloo’s blogpost
I just realized I hadn’t commented here. This point really struck me:
“those who are on the receiving end of discriminatory practices and bias are often unheard, redlined and often excluded from access to the criteria being used to make decisions”
I suggest maybe it’s not just ACCESS that’s the problem. Not just access to data but also access to decision making. And so maybe each individual should have agency on how and when to give their data and how to interpret it and how those affect decisions that they themselves are at least involved in making, if not fully agent in making them.
What would be the Paulo Freire or bell hooks approach to learning analytics? If the data is collectible, how do we raise consciousness of learners that they figure out ways of using it that may benefit THEM. Give them opportunities to recognize what may be hindering their learning or promoting it, and to analyze the realities behind the numbers and act based on what they learn? This of course is still problematic if the collectors of data are the institution/colonizer because the categories chosen are external to the learner. But what if it were different and learners had choice in every step of the way? It reeks of self-monitoring that Foucault may loathe… But I think it’s more of a self-consciousness, a self-awareness that can be fostered in community to help learners benefit from each others reflections.It occurs to me that much ethnographic and critical research that isn’t participatory has the same issues. So maybe I am calling for a participatory learning analytics approach or a broader thing that encompasses all kinds of learning. A kind of auto-reflection on own learning and learning data – be it analytics or qualitative data. What do you think?
Added later. In response to Frances Bell comment below… I went back to blogposts on CAE (collaborative Autoethnography) I wrote in 2014/5. I skimmed them all and found one Frances hadn’t responded to and in which I think I was most critical/reflective. It’s long.
This blogpost specifically and this paragraph especially:
- I keep circling back to the same thing, right? There power questions, there are questions of who can tell whose story? There are multiple “others” in the “we” of autoethnography, and what do we do by telling our story and leaving out theirs? What about the people who do not wish to be researchers but whose stories are in the narratives? What about the people who didn’t contribute to the narratives but were rhizo14ers? What about the people who didn’t even blog visibly or at all, and so have no easy “trace” to find even if we wanted to incorporate their views?
If folks are interested, I have several publications using Collaborative Autoethnography (in my publications page).
Thanks for tagging me in a tweet leading me to this post. I am interested in your point 1 above about there being different divides ( not the word I would use) – methods, paradigms (and data). Paul Prinsloo is doing some excellent thinking on agency and ownership in learning analytics and I am interested in what you say about CAE. I first came across CAE a few years ago in the proceedings of a MOOC conference. My recollection was that there was lots of analytics number-crunching and categorisation, and then a bit of CAE. The narrative accounts were enlightening and very welcome but I wondered what might be missing. There is oodles of mixed methods, trans- and interdisciplinary research and elsewhere that can provide some ideas.
If we think of alternative perspectives as relational rather than vs or divides, as dualities rather than dualism then we can acknowledge the limitations of our roles and methods. You know that I have been involved in collaborative and participatory research that acknowledges its limitations and treats data ethics seriously. So can you answer me a question that I think about quite a bit- What (as well as its obvious benefits) are the limitations of CAE?
I have several blogposts about limitations of CAE and have mentioned them when doing CAE publications. CAE is exclusive. You miss out on experiences of those who didn’t join the CAE, for a variety of reasons including dissent, but also not feeling able to express oneself in this method and not having time. Some of our collaborative work in rhizo14/5 got creative and sometimes chaotic, some ppl didn’t find themselves in that form also. More than that, just collaborating doesn’t remove power hierarchy. Some ppl will have stronger voices by personality or by training/experience or by race, gender, position. Etc. I am one who has that dominating type of personality and can dominate if i don’t step back. That also means some ideas/voices dominate and this can lead to groupthink in CAE. I wrote all this over several posts that I think u may have read but it’s hard to remember of course! Can go back and link to them if you like and put them in the bottom of this blogpost as a P.S.?
In any case – the CAE of the oppressed/subaltern is more empowering for them than the interpretivist research of the dominant ABOUT them. Because no matter what it’s an external person setting the research question and agenda and interpretation. More problems happen when the subaltern does not participate. Because even if the dominant initiate then include the subaltern… Like profs including students in their research… Requires a heck of a lot of stepping back and supporting while allowing the less experienced to lead. Which maybe in education is a pedagogically good idea and of use…but not usually in the world.
And yet. Really. Who is any of us to think we know what is best for another? To set questions for another. This relates a bit to my OER17 keynote btw. Thanks
Thanks for linking to that blog post and for answering my question. I certainly don’t remember reading it at the time but who knows? I think we can ask others questions (that they are not obliged to answer) without knowing what is best for them. Qualitative researchers know the partial and provisional nature of their understandings and rarely come up with prescriptions. The question of who can tell whose story is an interesting one. I think that even when we tell our own story, we are inevitably including our interpretations of others’ stories. Participation and checking back our interpetations are good things to do but not a complete solution. The concept of acts of violence is very useful across the range of more and less participative research and practice. There are some good reasons seek qualitative data about things we haven’t or couldn’t participate in I think. I found some refs to share with you by email.
Another power issue that struck me and prompted my question about the limits of CAE is the issue of funding. I’d love to see a study on the distribution of funding in educational research and the nature of research funded. I know about some of the Learning Analytics funded projects, and the now completed Code Acts in Education project but I can think of relatively little qualitative research (unfunded – various MOOC CAEs by you and others, work done by Jenny Mackness, me and Mariana Funes and some by Lesley Gourlay but unsure of funded status) in comparison with very visible funded research in Learning Analytics.
I had co-submitted w folks in UK and others a project researching sthg from both ethnographic and learning analytics perspectives across countries. It didn’t go through…but I now think maybe the inclusion of learning analytics was added to enhance chances of securing funds? This is interesting to think about as a colleague and I are about to consider embarking on something similar
Good luck with that. Qualitative research (not just CAE) is ‘expensive’ in time and therefore money if it is funded and I worry if future decisions rely mainly on findings from LA-type research. In our third paper, we found the analytics data mainly useful in conjunction with qualitative inspection of archives.
I find combinations useful. It’s which interests drive the research that’s most important. I trust in work of people like you and Paul Prinsloo to let the more important questions drive research. Of course who cares whom I trust, right? I am only trusting ppl whose values i believe are in the right ethical place from my POV. So I believe all ur research that I have reae was done in an ethical manner w intent to shed light on what’s not obvious to others.
I do think research that is more action-oriented should be participatory as those implementing are the ones who will be taking action so should at least be involved with agency somewhat…not peripheral or silent. Even kids
P.S. I do a lot of research that is less participatory btw. For different reasons (as even the term violence is more meaningful if we are researching truly vulnerable populations like refugees, drug addictsx or the mentally ill and entirely exaggerated in terms of potential harm when we talk of adult learners in MOOCs – damage is possible but not anywhere near life-threatening then). And of course people aren’t always aware of their own situation. But they may also choose not to share w a researcher. Or the researchers questions may confuse them and lead them a certain path. We try, of course, to probe and ask open questions and follow up but it’s never enough. Even if research involves observation, not all behavior is observable. Today my kid didn’t finish her Arabic writing homework but she wrote the word in the air with her fingers and on a computer app. Visibly, u see the incomplete homework and u think sje isn’t engaged. But when she left the homework she took out an Arabic book and asked me to read it for her and to show her some words. That’s to me evidence of how assessment can go wrong by prescribing how learners express their learning
Yes! That was the potential that I saw when I got excited about learning analytics. I’d seen the power of participatory data analysis to challenge assumptions (false truths) – Data’s ability to empower *people*.
That is VERY different than another person or a black box algorithm making decisions on your behalf.