By Alex Ralph, Editor, Corruption in Fragile States Blog
At the end of 2021, I interviewed CJL’s co-founder, Cheyanne Scharbatke-Church. The occasion was to discuss her and our colleague Dhaval Khotari’s recent review of Monitoring & Evaluation efforts devoted to social norms change. Our wide-ranging conversation (edited for length and lightly for clarity) touches upon the ethics and importance of M&E but also, because I’m a fiction writer and the son of a therapist, begins with how Cheyanne came to her work. Discussions of Cheyanne’s childhood will, alas, have to wait for future interviews. (Though check out the portrait of the young practitioner on horseback.) Hope you enjoy the glimpse into some of what’s shaped Cheyanne and the work she does.
Alex Ralph, Editor, Corruption in Fragile States Blog
Alex: How would you characterize the purpose of your work?
Cheyanne: I'm really interested in situations where the political processes cannot contain the will of the people and that then most obviously manifest in violence. Over the course of my career working in conflict zones on peace and conflict issues, I realized that corruption is another way that political processes can't contain the opinions and voices of the people. It's about having a mechanism that those differences of opinions can come in and a reasonable solution can come out. When that doesn't happen, over time, you can go to conflict. War or corrupt processes start to take hold. And so all of my career has been around understanding those issues.
Alex: When did you know that this would be your life’s work? Was there an “aha” moment?
Cheyanne: Early on [in my 20s] I was working [on peacebuilding evaluation] in Northern Ireland, and I became really interested—and “interested” is the wrong word—perplexed by who was deemed to be doing good work. I was very influenced by a project with Protestant paramilitaries that was encouraging them to shift from violence to politics. As a pastime, they would make fun of all the “community-relations work,” which was the Northern Irish term for “peacebuilding.” And they would often make fun of the groups that the NGO world thought were the best. I was like, “Huh? How are they [certain NGO groups] thought to be the best, but the guys who are actually doing the blast bombs and the beatings just say, “It’s jobs for the boys,” meaning a way of getting other people jobs. So that led me to think about, well, what does success mean for us? Who matters in the decision-making? And how do you figure out success? Northern Ireland was absolutely pivotal [to me]. The time informed both tracks of my career—understanding effectiveness and success in peacebuilding and the critical nature of corruption. It became very clear that the Protestant paramilitaries, just like the IRA, partially funded themselves through illicit activities. Smuggling cigarettes, brothels, those kind of things. And corruption was one way that they enabled those illicit activities to happen. So that's where I first kind of stumbled on corruption’s role in conflict.
Alex: What led you to first begin looking at social norms?
Cheyanne: I was doing a lot of M&E advising work for a donor that funded anti-corruption projects. And these proposals were coming in and I started to realize that they were often missing the social dynamics. They were generally technical responses driven by a gap analysis approach where they say, “Here's everything we have in the West. Let's look to see what you have. Have it. Don't have it. Don't have it. Don't have it.”
And we're like, that's the wrong way to analyze [the context]. You have to analyze it to understand what's driving the corruption. What are the conditions in the environment that will allow it to happen? Not having a law doesn’t cause corruption. Not having a law enables it to happen. So what’s driving it? Because if you only go after the enablers and the drivers are still there, people are bloody creative. They’ll be like, “I still have whatever was driving me. [Be] it needs, survival, greed, status, political power or whatever was driving me is still there. So I’m just going to get more creative to do it.”
Cheyanne and CJL have written extensively about social norms from our blog’s 2016 beginning, be it a primer with social norm resources or arguing that social norms are a missing link in anti-corruption programming.
Right from the get-go, we said we have to rethink how we analyzed corruption in order to then figure out where to intervene. And one of the things that we need to analyze in addition to the legal structure is politics, and the social. And that’s right around where we started using the language “social norms.” But to be totally honest, we were using it like a lay person. We weren’t using it as the way we understand it as an academic construct now.
We were fortunate enough to secure our first grant from INL. It was to develop a different analysis process. And I thought that systems analysis, this way of causal loop mapping, was the way to do it because corruption never stands alone—corruption is always embedded in the health system, in the education system, in the justice system. So we need to figure out a way to analyze that embeddedness. I knew Diana Chigas and Peter Woodrow, both of whom worked at CDA at the time, were doing causal loop mapping in real deep dive ways for Conflict Analysis. So I called them up.
Alex: From your working paper, one thing that's interesting to me was the dearth of examples of M&E on social norms. Can you talk about the challenges of doing M&E on social norms? And what about these complexities has led evaluators to shy away from doing them?
Cheyanne: I don’t think there's a shying away. I don’t think there’s a lot of opportunity to dive in. What makes it challenging? If we’re trying to assess a behavior, like do we have a behavior change, there are a variety of data collection techniques, but typically it’s a singular construct. You might be looking at regularity or frequency of that change. But the thing you’re looking at is reasonably one dimensional. Social norms are a four-dimensional object. So there’s the two perceptions [descriptive and injunctive norms], there’s how they come together. There’s the influence of the group, which is situational. They’re not like values. My values are the same whether I’m in Canada or whether I’m in Boston or whether I’m in Congo. Social norms, though, are multifaceted. So it’s the difference between trying to measure a one-dimensional object versus a four-dimensional object. That’s conceptually challenging.
And then that pragmatically teases out into quantity of data. Academics in other fields have been doing evaluation work of social norm change. And they come at it from a research perspective, which is a cousin to evaluation, but not the same thing. And this can be great and really insightful but your average anti-corruption project is two years. You’re really tickled if you have three. So to have a formative evaluation process that takes months and months to produce conclusions doesn't do anything for you. And so one of the things Dhaval and I were trying to think about is, okay, that's how you would do it for research. Cool, good. But if we’re trying for M and E, which is to inform decision-making, how is that done? And I think that’s figuring out what corners can be cut with what consequences. That's the interesting challenge. I don’t mean that negatively like what corners can be cut. But of the ten variables that researchers want to collect information on, are there four that are essential?
Subscribe here to receive the Corruption in Fragile State Blog's biweekly posts. (Biweekly as in twice a month, not twice a week--because we at CJL want to be kind to your inboxes.)
Alex: What you’re getting at is one of the challenges within the anti-corruption field of social norms M&E. And that is that it has to be actionable. It has to be usable for the group doing it. And the traditional research way of doing M&E might come up with some interesting conclusions, but those organizations doing the work don't have the luxury of time or money to do those things. So to make it useful, we have to isolate and scale down to meet their needs.
Cheyanne: Useful and feasible. The other consequences of ten factors of data and quantitative analysis is money. Somebody's gotta hire those enumerators, hire those staff to gather that data, and to process it. Your Transparency International Mozambique with twelve staff and an annual budget of $120,000 USD—and I have no idea their precise resources, by the way—it’s not feasible, so they're just not gonna do it. We need those guys [academics] doing that research. We need to understand that bigger picture. That’s not the issue. It’s just this is a separate lane, like figuring out the feasible, useful way to get valuable data is a separate lane and needs to be done.
Alex: What do we not yet fully understand about social norms that we should? What’s the stuff that we're overlooking?
Cheyanne: From a pure M&E perspective, this notion of proxy. Can we say, look, if 90% of people, have this attitude, it's unreasonable to think that the social norm would still be dominating.
The notion of reference group is [also] just not well fleshed out. Again, a lot of paragraphs end in a sentence that says, “Of course the norm is held by a reference group.” Period. Move on. They tell us what reference groups are not bounded by. They’re not bounded by geography for instance. And “diaspora,” by the way, are a fascinating reference group. They're not bounded by size. So we know what they're not. It's very hard to then say, well, who are they and how would we figure it out?
"If we don’t figure out how to stop the habituated abuse of power for personal gain, you can’t get the water, you can’t get the health care, you can’t get the justice. And if you don't get those things, then you don’t get a chance to have a prosperous life.”
Alex: This is going to be a weird thing to say. The more you're talking, sometimes the less feasible all this seems to me. Because if we're now talking about the diasporic community, how do we do any of this stuff. What can we do?
Cheyanne: I hear you one hundred percent. It can be like, “Throw it up and walk away.” Like, “Pick potable water and work on that area.” But be it potable water or health care, if we don’t figure out how to stop the habituated abuse of power for personal gain, you can’t get the water, you can’t get the health care, you can’t get the justice. And if you don't get those things, then you don’t get a chance to have a prosperous life. I don’t mean “prosperous” meaning “rich.” Prosperous meaning where you have all your basic needs met. And some hope and opportunity, some chance.
It’s the trip cord in the basic services. All these groups spending millions of dollars trying to get healthcare improvements in Ugandan hospitals. Well, the staff are selling the medicines off the side. So the only way you get the meds is if you bribe. The only way you only get a bed is if you bribe. And why are they [the staff] demanding bribes? Well, partially because they don't regularly get paid, but partially because it is expected by the other staff, because if they don't, then they're going to be seen as questionable, disloyal, untrustworthy. And partially because their family back home is expecting it. Nurse Sally has a government job. And she has fifteen nieces and nephews who she has to help pay school fees for because that’s the expectation in the family—that she or he whom has access should be stepping up. So if we [those working on corruption] don't go after that stuff, we will never diminish child mortality.
Alex: What’s a take-away you want readers to know from your and Dhaval’s paper?
Cheyanne: I care about what's going to make a difference for people on the ground in their lives without also at the same time doing harm. So this social norms thing seems to be very clearly a missing piece. But anytime you do something new, you have to build in your own beta test. You have to build that in. And we haven't done that well as an international community.
I've had people say to me, “What if I’m doing M&E, and it proves it doesn't work?” Well, that’s a success too. It’s disappointing. But it's a success too, because then we don’t waste your time anymore, right?
Alex Ralph began serving as the Corruption in Fragile States Blog editor in June 2021. In addition, he is a lecturer at the Gerald R. Ford School of Public Policy at the University of Michigan, where he teaches expository writing and courses on storytelling through public policy. When not editing or teaching, he is at work on a novel set in 1970s Detroit. His reporting on mental health and addiction service providers has appeared in the Cincinnati Enquirer and other outlets. He received his MFA in creative writing from the University of Michigan and currently lives in Cincinnati, Ohio.
Cheyanne Scharbatke-Church is a scholar-practitioner with a lifelong interest in governance processes that have run amuck. She has significant experience working on issues related to peacebuilding, governance, accountability and learning across the Balkans and West and East Africa. For fifteen years, Cheyanne taught program design, monitoring and evaluation in fragile contexts at the Fletcher School. Prior to that she was the Director of Evaluation for Search for Common Ground and Director of Policy at INCORE. As part of her consulting practice, she has had the privilege of working in an advisory capacity with a range of organizations such as ABA/ROLI, CDA, ICRC, IDRC, UN Peacebuilding Fund and the US State Department. She can be commonly found in the Canadian Rockies with her fierce daughters and gem of a husband.