Measuring Your Member’s Sense of Community

Measurement bedevils community managers. As practitioners of “soft skills” it can be difficult to know what to track and report. Luckily there’s a growing body of best practices to help guide you, like Saira Mortier’s practical tips on what to measure when tracking engagement (hint: it’s all about your community’s goals).

But sometimes you want to lean into the touchy-feely aspect of community and ask “what makes my members feel a sense of community?” Getting clarity on why your members participate in your community is important, because knowing motivations allows you to create a successful retention strategy. However, your members often aren’t articulate about what makes them feel like members, which makes surveying challenging.  So, if you want to dig into your members’ sense of community you’re going to need a good set of questions.

Good Questions Lead to Good Answers

I’ve been struggling to increase retention for the volunteers in my global network of #tech4good meetups. To help identify where I’m failing, I’ve run a survey for the last four years which revealed that that 80% of my event organizers are satisfied and that 97% believe they will still be hosting events in six months (definitely untrue! According to my retention reports the average term of a new organizer is 14 months.)

My questions were generating garbage answers that didn’t reflect reality, and the questions weren’t granular enough for me to determine how to make my event organizers. But if my questions were bad where would I find something better?

I turned to the CMX community (and academia!) for help.

At the 2018 CMX Summit Ben Leong shared the results of his forum surveys using the Sense of Community Index and it blew my mind – here was a battle-tested set of questions that explicitly set out to measure a community’s sense of belonging.

“The Sense of Community Index (SCI) survey is the most frequently used quantitative measure of sense of community in the social sciences. The survey is based on a theory...that stated that a sense of community was a perception with four elements: membership, influence, meeting needs, and a shared emotional connection. Results of prior studies have demonstrated that the SCI has been a strong predicator [sic] of behaviors (such as participation) and a valid measurement instrument.”

Via SenseofCommunity.com

Inspired by Ben’s experiments (download the slides here) and his nifty radar graphs, I decided to apply the approach to my own community. The standardized set of questions are broken into four categories, and gave me actionable results because it helped me pinpoint exactly where my community was feeling disconnected.

Here are the survey results with commentary on how I plan to act on my low scores. I’m sure you’ll also learn a few tips that will strengthen your community.

How Important Is it to Feel a Sense of Community?

The initial question, “how important is it to you to feel a sense of community with other community members?” gives me a benchmark for comparison when I run the survey again. Our academic friends have found that total sense of community is correlated with this question.

1. Reinforcement of Needs

The first set of questions is on “reinforcement of needs.” The lowest score is for “I get important needs of mine met because I am part of this community” – and maybe this isn’t surprising because I’m not sure I know my member’s needs. I’ve asked what being a member helps them accomplish, but this is different from their needs.

‍Next step: Start asking clear and explicit questions about member needs.‍

Lesson: You can’t meet your members needs if you don’t know what they are.

2. Membership

The membership responses show tremendous room for improvement. While each local event leader is effectively their own community manager, hosting monthly local events and acting as the hub for #tech4good conversations, it is worrying that most organizers feel they don't know each other. Members feel that each meetup is an isolated island, without shared rituals across the network.

‍Next step: Build personal connections between members with the launch of a Welcoming Team led by power-users that will foster relationships between chapter leaders.

‍Lesson: Don’t forget to embrace rituals, even if it reminds you of that time you got trapped in a hippie drum circle during college.

3. Influence

The Influence section scores high, but the top challenge is the feeling that event organizers have "little influence over what this community is like." This gap might be because our long-standing Ambassador program for super-users was put on hold last year. This closed our biggest channel for organizers to shape the network.

‍Next step: Open new channels for community feedback. Consistently report back the results and what actions I will take.

‍Lesson: Trust is the second step in the CMX Engagement Cycle and you can’t build a healthy community without your members being confident that their feedback is being heard (and acted on!).

4. Shared Emotional Connection

My members aren’t sharing important shared events or spending time together online (let alone in person). These problems mirror the sad state of ritual and member relationship identified earlier. The fact that these problems are showing up a second time in different clothing emphasizes that I need to pay more attention to this problem area.

‍Next step: Celebrating more community milestones and working with the Welcoming Team to foster personal conversations, rather than focusing exclusively on event planning logistics.

‍Lesson: We are all community managers in our network, but even community managers need community. I need to play the same supporting role for my event planners that CMX and the CMX Pro hub plays for me.

What Did I Learn?

Working in public can be humbling, but it also forces you to think more clearly because you have to explain your work to an external audience. The Sense of Community Index survey helped me identify the hidden gaps in my engagement plan for meetup leaders. But going through the process also highlighted key lessons that apply to the work of all community managers.

First, like attracts like and people want to work with others on shared tasks. My meetup hosts may be bringing people together in their own city, but they still hunger for peer relationships between hosts in other cities. How can you create peer-to-peer relationships where your members can learn from each other?

Second, are you absolutely certain you know your member’s needs? Don’t guess  – ask! But don’t forget to ensure you’re asking the right questions. I surveyed my meetup hosts annually, but eventually the questions stopped giving me useful answers. Do you still trust your questions or is it time to try another approach?Third, symbols and rituals matter when building community. Fly your flag! Celebrate your jargon! Take what’s unique about your community and lean into it. What are the natural symbols for your community?

Finally, we can’t work alone. Community managers need to a community of practice to keep learning. This whole survey journey began because I attended the CMX Summit and build on the work of my fellow community managers. How are you continuing to learn? Where do you go to be inspired by new ideas and approaches to community management?