Mobilizing the 3Cs Impact Evaluation

Can a 2-day gathering help individuals take action on gnarly problems such as homelessness in Seattle?

Client: Journalism that Matters | In collaboration with Praxis Associates

Project Methods:

  • Participatory practices
  • Survey Design
  • Participant Recruitment
  • Interviews
  • Complexity-informed Evaluation
  • Data Analysis and Theming
  • Evaluation Report and Layout Design

The Challenge

Seattle’s housing crisis is dire, complex, and seemingly intractable. Some might think of it as a gnarly problem. There is great variance in understanding how to make a shift towards accessible and safe housing for all who need it. In an effort to root deeper, in October of 2018, a number of Seattle-based agencies came together in dialogue, inquiry, and imagination. Among many, this group included Journalism That Matters, The Evergrey, Real Change, Facing Homelessness, and the Impact Hub Seattle. These groups collaborated to design an experience that would center an effort to Mobilize Creativity, Compassion, and Community to end homelessness. 

A 2-day event was hosted and leveraged two facilitation methods, the World Cafe and Open Space Technology, to host the dialogue of 135 Seattle residents at Pioneer Square’s co-working venue, The Impact Hub Seattle. 

The Task

My involvement in this project initially began as a participant in the event, and, with my great interest in community equity and experience in hosting World Cafes, Peggy Holman, Mobilizing 3Cs project organizer and creator of Open Space Technology, invited me onto the team to help synthesize what emerged from the conversation. Six months after the event, I assisted in a follow-up research evaluation to learn, “What, if any, ripple effects of impact happened as a result of this 2-day event?”

Action

A breakout discussion on the second day of the event

The action is two parts, first understanding what we learned at the event itself and second, to learn, what if any, impact happened 6 months later. 

Part 1: Share out the learnings from the event

The event was held over two days in October 2018. Across the two days, there were pre-designed questions and questions generated in real-time to dig deeper into the issue and touch the shared knowledge of the community,

On the first day, these questions included:

  • Tell me about a time when housing made a difference in your life.
  • What does being at home mean to you? 
  • What moves you now?

And, on the second day included:

  • Vision: What would it look like if everyone was cared for?
  • What can I learn about homelessness in a 5 minute walk around Impact Hub?
  • What are the root causes of homelessness and what we can do to prevent them?
  • What have the homeless learned that EVERYONE should learn?
  • What if it is about creativity, community, and compassion? And what if it is us?
  • What’s working and how do we know?
  • How do we bring “anti-homeless” organizers into spaces talking about solutions?
  • How do we share belonging with our friends outside?
  • How might individuals support someone successfully toward housing stability?

Harvesting

In the World Cafe, the last phase of the dialogue process is the Harvest phase. It is through the Harvest that connections, patterns, and learnings are uncovered and shared. To assist in harvesting, the design group sent a survey to the participants. I took the survey summary and notes from the event and created a report that could be shared with those who attended and those who didn’t.

A social media post I created after the two-day event

Here are some feedback highlights post-event:

Quotes from the attendees:

“The socio/economic standing is a construct. It is not a real thing. At the community level, it doesn’t even exist. And yet it is so dividing and governs the way we think, act, and believe.”

“And we recognized that these all fit together: that human connection leads to valuing the worth of each person, which leads to self-led solutions, the will to fund to the level of need, and the will to create housing.”

Comments from the post-event survey:

“I learned that all types of people face housing security. People whom you might not have guessed.”

“I did not have any expectations of how I would stay engaged in the process. By the completion of the workshop, I found a much greater sense of commitment to the issue.

“I gained new perspectives about homelessness from others. Perspectives homeless people brought to the table. I always thought they should be included and now I’m certain.”

And some numbers:

  • 94% of participants reported that they would continue to stay engaged with the issue of homelessness. 
  • 94% of participants reported that they had made at least one new connection at the event.
  • 89% of participants reported that they had learned something new about the issue of homelessness. 
  • 67% indicated that they were planning to get more engaged in the issue after the event.

Read the recap report here.

Part 2: Find the ripple effects 6 months later

With 67% of participants of the post-event survey stating they would be more involved in this issue based on their experience, Peggy Holman of Journalism that Matters, was curious, “did anyone take additional action?” Myself, and lead-collaborator Yve Susskind of Praxis Associates, were hired for a full-service impact evaluation with recommendations to Journalism That Matters to help aid them in this question.

We determined 3 key evaluation questions:

  1. What made the gathering generative?
  2. What impact did the event have? 
  3. What else did we learn about how community conversation can help solve gnarly problems?

Survey

First, we conducted a survey, distributed via email, that went out to all the confirmed participants of the original event. Total survey participation was 21 people, which was ~25% of event attendees.

Closed ended survey questions focused on: 

  • Facts about their attendance (e.g., which days they had attended, prior experience with World Café or Open Space) 
  • Their personal experience of homelessness 
  • Their prior involvement on the issue of homelessness 
  • Actions they took as a result of their experience at M3Cs 

Open ended questions asked for detail about the experiences, successes, and challenges with the actions they took and how M3Cs influenced those actions.

Interviews

Second, we conducted in-depth follow-up interviews with those who were willing to be contacted further. 
We were able to reach and interview 10 participants and each interviewee was offered a $50 stipend for their time. Most of the interviews were conducted by a pair of researchers, one as primary questioner and the other as note-taker.

The in-depth interviews (six with individuals and two with pairs) focused on:

  • Their most significant or memorable moments at M3Cs.
  • The impact M3Cs had on them. 
  • What, if anything, they had done around homelessness since the event, and what, if any, connection they could trace between M3Cs and those actions.
  • Feedback on the event.
  • Anything else they wanted to bring up.

Analysis

Third, I transcribed the 10 interviews and assisted in analyzing for themes within and across interviews. Our evaluation team met with Peggy Holman to review the themes and distill big takeaways. We went back to the data to check themes and congruence between the takeaways.

The Result

An example of the high-level recommendations from the evaluation

As a result of this work, Yve and I produced a 36-page report with a summary, recommendations, and a detailed appendix. I designed the layout to be coherent with the first report, adding familiarity and brand to the effort that was previously not established. We presented the report to Journalism That Matters, and it was published on the Lenfest Institue blog in January 2020.

Here are a few key learnings:

1. Those with less prior involvement in the issue were more likely to take more and new action

While 65% of survey respondents had low levels of involvement before the event and they took 76% of the actions taken as a result of the event. Those with a higher prior level of involvement, representing 35% of respondents accounted for only 24% of the actions taken. 

2. Participants did look at homelessness in new ways but didn’t know what to do next

“I need to figure out what my relationship is to the problem. Like helping someone a person or individual it is just a long-term problem. So I don’t know if it’s like chipping in on one thing once a year like helping at St Mary’s place or doing furniture delivery. But it’s not a problem that will go away anytime soon. I haven’t figured out if it’s a money problem, or if we need to convince more people to do something. But it’s not clear to me yet.”

3. Don’t try to solve a gnarly problem all at once with a Goldilocks solution. Get support and learn from small experiments, especially empowering the local efforts of those most impacted. 

“The event could have been… like “check it off, I [did] my thing…with interesting conversations and now I am on my way to not doing very much.” But I think that the event really helped leverage my momentum and exploded it. Whatever I had that was a nascent energy around it, this event really helped it to develop.”

Want to read the full evaluation report? Check it out here.

Study Retrospective 

Study Weak Points

  • Data were collected 6 months after the event, so it was difficult for participants to remember all that happened at the event and how it directly connected to their actions. 
  • Had unreliable numbers of attendance and demographics at each event. 
  • Conducted research with a limited number of participants. (~25% took the survey, ~10% had an interview). 
  • We are only studying this off of one event, and not multiple one-time events. 

Study Strengths 

  • Sufficient time passed after the event to observe behavioral results (actions) 
  • Mixed methods of data collection. 
  • Tried to speak to diverse viewpoints represented. 
  • Two people present during most interviews, to help debrief and balance some biases

This project was done in collaboration with Praxis Associates and all rights belong to Journalism that Matters