How to Measure and Report Your Community’s Value: A Moz Case Study

Editor’s Note: You can learn more about community measurement strategy by enrolling in CMX Academy. Learn more here.

You spend a lot of time doing the hard work of community building. You plan events, rally contributors to write content, and talk to members on a regular basis. But if you can’t measure your effect on the business, you’re missing a key component of what will make your community successful in the long-run.

To get to that point, you have to be able to show your value in concrete business terms.In a 2014 blog post on community dashboards, Square Community Lead Caty Kobe explained: “Those who sign the checks tend to be higher up the corporate ladder, and the higher up the ladder you go, the simpler the objective is — hit the number.” Ultimately, that’s what this is all about.

So to get to that point, you can follow in the steps of others who have created community health dashboards that map back to your goals and who have created reports that show their clear business value.

Today, we sit down with two community experts who know the ropes of community measurement: Jennifer Sable Lopez, Senior Director of Community at Moz, and Megan Singley, Social Community Manager at Moz. Their reports have shown that Moz customers who start as community members stick around four times longer than those who come in through other channels. That’s the kind of measured value that their community team creates and that has allowed them to grow their team over the years.

Together, Jennifer and Megan’s experience provides a blueprint for measuring community health that maps back to business objectives. In this case study, they break down how they’ve done this as Moz’s community has been formalized and grown in the last 12 years since Moz was founded.

Key Terms:

Before we dive in, here are key terms to know as you read through this piece:

Community Health Dashboard: This is a dashboard that rolls up the activity inside your community programs. You can decide if you create this report daily, weekly, monthly, or quarterly. Just be consistent. This is a tool for your team to help you revisit tactics.

KPIs (a.k.a. “Key Performance Indicators”): These are the metrics that you will measure in order to determine the efficacy of your work. Harvard Business Review has some excellent resources like this to help you carve out yours. See below for what Moz uses and other suggestions.

Reports: You create community reports based on the insights that your dashboard provides. Reports are meant for those outside your team. You should include only those insights that map back to KPIs here.

Getting Started: Consistency and Organizational Objectives

When Jennifer was running the Moz community singlehandedly, she didn’t just start with a full-fledged community dashboard. She started with simple benchmarks. That’s why she emphasizes the importance of just starting somewhere.

“Start to gather data now, wherever you are,” Jennifer advises other community professionals. “It will allow you to benchmark and look back a few years once you’ve moved forward. So much of our job is just about consistency.”

When you get started, start somewhere and measure consistently against what your organization cares about. It will make the much-larger task of creating a full-fledged dashboard that much easier to wrap your head around.

Depending on the business value that you serve, you’ll want to start working towards measuring that value, even if you can only do so in a rudimentary way.

Here are sample metrics to guide you at this stage:

• Support: Measure activity in forums, reduction in email response time for support team, number of problems solved by community members
• Product: Measure number of ideas generated and implemented, measure improvements in supply chain
• Acquisition: Measure referrals from community members, lifetime value of customers who come in through the community
• Contribution: Measure contribution, engagement, and lifetime value of members in the community
• Engagement: Measure activity in the community space, measure retention and turnover of members

And how do you do this? Tools like Google Analytics (“UTM tagging has been key for us,” says Megan), social listening tools, SQL queries (back in the day, Jennifer used to run her own SQL queries to gather data until the data team helped her formalize the data-collection process), and custom Tableau dashboards now do the trick for Moz.

Breaking Down Health by Programs

Now that Moz’s team has grown, it has become vital for them to formalize their community health dashboards and to solidify their process of measuring success.

Moz’s community team sits under their marketing department. As such, they measure their data alongside all other marketing metrics in a large dashboard with tabs for each and every marketing program, including the main blog and YouMoz blog, all social channels, Q&A forums, and events.

In order to measure community’s contribution to marketing goals, Megan sets aside dedicated time each week to update their community dashboard. She works with the team to round up all the various programs that the community team leads.

“Every Monday, I pull the numbers,” she says. Each Monday, she breaks apart each program they run and measures them independently. Setting aside a day to focus on the health means that she can spend the rest of the week executing on new plans and programs.

At the end of the month, Megan takes all the weekly dashboard numbers and puts them into the monthly dashboard and report that then goes out to the entire company (more on that below). They measure things like:

Social

• Shares
• Likes
• Comments
• Conversation rate per channel
• Amplification rate and applause rate
• Site traffic coming from social and from specific campaigns

Moz uses True Social Metrics to obtain this data. Here’s a quick snapshot of the engagement metrics inside the marketing dashboard (top) and the social growth metrics inside the marketing dashboard (bottom). Each quarter, their priorities around engagement or growth shift, and they watch the numbers change so they can analyze the impact of their work.

Q&A Forums

The Q&A Forums are a premium feature and a serious value add for Moz members. The team ensures that the forums are useful to those members by tracking things like:

• Questions asked in the last 30 days
• Questions that remain unanswered
• The percent of questions that are unanswered
• The number of staff replies (at different times, the community team has focused on incentivizing Moz employees’ participation in the forums)

Moz uses their own in-house forum software to measure these metrics, though they are planning to change to a forum provider in 2016.

Blog

Moz has a user-generated blog as well as a company blog, and the community team keeps track of all the activity happening on that content. They focus on metrics such as:

• Posts
• Comments
• Favorites
• Conversations
• Applause
• Amplification

Reporting Based on Your Community Dashboards

Your C-Suite and other executives are not going to care about every community health datapoint that you cover in your dashboards. But that doesn’t mean that it isn’t important for you to have a pulse on them. The health dashboard will show large trends, good and bad, that you can then share with the executive team.

At Moz, their dashboards change occasionally, and their reporting and goals change at quarterly intervals depending on what the department’s goals are for that time period. The report that comes out of the dashboard is therefore a guiding document for future strategy and analysis. Here’s how that works at Moz:

Weekly Health Dashboard

The dashboard includes the nitty gritty details that guide Megan and the team on what is going on in the community. If they see dips in activity in the forums or an underperforming blog post, they gain insight into how to improve the experience the next time around. This is mainly for the community team and the wider marketing team.

The weekly health dashboard is for you, not necessarily for your executives. It rolls up into the monthly numbers you see above.

Monthly Community Report

At Moz, the Community Chronicle comes out once a month. This is the story that all the dashboard data tells, and it puts the community team’s work into context for the entire organization. Megan says that, most months, it prompts conversation internally about community programs and ignites new ideas. Again, this is not about analysis. It’s about presenting data with context that tells a story. It’s about raising awareness for what your team does — the big wins and the small ones.

Here’s a snapshot of Moz’s November monthly report.

Quarterly Reports

Each year, Moz sets four key objectives for the entire company, and that affects the yearlong goals that go into the quarterly reports. This is where analysis and goal-setting comes into play.

“Everyone’s priorities roll up to the key goals for the company,” explains Jennifer of how their quarterly analyses work. The quarterly reports are what travel up to your organization’s executive team. This is what really counts to your boss and your boss’s boss. 

Jennifer reports to the CMO, who then puts her quarterly report into her report to the executive team and the board. From there, they analyze the story the data tells and strategize new goals for the coming quarters and the community team performs the execution behind the larger goals.

Analysis and Goal-Setting Based on Reports

Measurement means nothing if you can’t act on what you learn from the data. Your dashboards and the resulting reports should prompt action.

It’s key to understand the difference between measuring community health, reporting on community success, and working with your team to analyze and re-align your strategy around what is happening inside the community. If you don’t, you risk sharing insights that your executives don’t care about or executing on strategies that no longer make a dent in business priorities over time.

The reporting includes pulling the data from your tools and the creation of dashboards, as discussed above. The analysis offers recommendations on what your business should do with that data. It moves in a linear fashion, like so:

You will want buy-in on your analysis and to work with your wider team to set priorities based on all the insights you gather in your dashboard and reports. You can’t build community strategy in a silo, otherwise you won’t be able to gain resources to grow over time.

One Caveat: You Can’t Do This All at Once

Even though they now have fairly sophisticated ways of measuring success, Jennifer admits that there is no way they could have done this from day one and that they still have strides to make. They are working on heading down a clear path toward mapping what community activity elicits the most engagement and lifetime value, and exactly how much community members drive referrals and revenue.

There is always work to be done. It’s important to remember that you can’t do everything at once. You just have to start small sometimes.

“As we get larger, we will have to measure how community contributes to revenue generation. For now, the team is saying ‘We already value you, so you don’t need to prove it.” For those of us in companies that deeply value community, this sentiment is familiar. But others of us have to push a little harder to be heard. That’s when we have to let results speak for themselves.

This challenge can be frustrating, but it can also allow you to grow as a professional, giving you opportunity to prove the value of your work. “Everyone goes through their own struggle,” says Megan. “It takes time.” The sooner you can get started measuring community success, the more time you’ll have to grow just like Moz’s community.