Do you know what your members want? Does your community give you its opinion, or have you ever asked? A community survey is an excellent way to gather feedback en masse to inform the business decisions that affect your community.
When planning a complete redesign of Smiley360.com, a community of consumers who experience brands for free and share their opinions, getting the input of members was essential. After all, our community lives on our site and our site lives because of our community, so members should have a say in any major changes we make to it.
We knew we needed to rebuild the site from the ground up, but how could we do that without upsetting or disregarding the 400,000 members who had joined us over four years?
We launched Smiley360 in December 2009 to meet a big need for brands looking for ways to get people talking about them in an open and honest way. People join Smiley360 for fun, to experience new products, services and content, and to share their experience with friends—and brands want the feedback Smiley members can provide.
As the community grew steadily, it became apparent that we had outgrown the original site. We decided to rebuild our application, but not before asking members what they wanted (therefore preventing community backlash down the line). To start this process the right way, we launched a Community Feedback survey for members to have their say in the future of Smiley360.
This could work for any community builder at any point in their journey: pre-launch, before a big change, even just an ongoing survey in order to let your members know that you’re listening.
Thousands of our most vocal and opinionated members took this survey. A thriving community of members who love to talk… a community manager’s dream. But collecting this feedback is no easy feat.
Here we share our process for how to create a community survey to collect useful, unbiased, and actionable feedback from members to help inform major decisions for developers, clients, or your own internal team.
Now we’ll go into each step in detail so you can collect the data you need and act right away for the benefit of your community.
Before you begin writing questions for your community survey, ask yourself and your team what your objectives are:
Make assumptions about the answers you expect to see, and write survey questions that will either confirm or reject your assumptions. For the re-launch of Smiley360.com, we focused on what features our users spent the most and least time on, what new features they wanted us to introduce next and how they felt about the design and layout, text content, and ease of use of the website.
In your community, you might run a survey to define or redefine your community guidelines, validate an idea you’re thinking of launching, or simply measure satisfaction and utility of the community.
There are a number of things to keep in mind to ensure that your community survey is easy for your members to complete, and gives you the most useful results.
Unless you have a survey professional at your disposal who knows what she’s doing, be extra careful when writing your questions so that survey questions do not lead members to answer in any particular way.
You might be tempted to write:
Instead, keep your options balanced, with a neutral opinion in the middle:
It’s important to choose your words carefully so that you do not influence the member. Consider this example of how leading questions can influence the testimony from an eyewitness of a car accident:
Which of these questions have leading language, and which one is bias-free?
There is more than one way to ask a question. There’s also a right way and a wrong way. Consider this example:
You might want to know where your members spend the most time on your site, but the problem is that this question is anything but specific. It does not account for people who visit only the FAQ pages but not the blog, or vice versa. Furthermore, analyzing the responses from this question would not give you any real insights. Instead, rephrase this question as:
We asked a question just like this to learn where our members spent the most time. The results? We were able to cut the least-used parts of our website and focus on the areas our members value the most.
When asking questions around numbers, you must make sure that everyone can fit into only one box. For example, consider this question:
What about someone who spent $20 or $40 exactly? That person doesn’t know which box to pick, but they have to choose one. This will muddle your data, and you would be surprised at how often this mistake is made. Instead, make sure that a person can fit into only one range, or better yet, ask for exact numbers—this way you can calculate means, modes and other statistically relevant information.
Maybe you really need to know your community’s ice cream flavor preferences, but a question like this will make analyzing the responses a nightmare. Why? You’re probably not managing a community of grammar-obsessed English teachers, and your members will most likely have a hundred-and-one different ways of spelling Chocolate Fudge Brownie (Choc Fudge Brownie, Chocolate Browny, Fudgy Brownie, that one with the brownie bits inside…)
The lesson? Never use an open-ended field when you can get specific. In the basic ice cream example above, give users a dropdown menu of all delicious Ben & Jerry’s flavors. The results will give you statistically accurate preferences—meaning you can treat your members to their favorite flavors.
In some cases, you will have to use an open-ended-question type. In all of our surveys, we like to end with something like:
It is always the final question, so if any thoughts popped into respondents’ minds throughout the survey, now is their chance to share.
Skip logic means designing a custom path through a survey that changes based on a participant’s responses. If a portion of your survey is about being a parent, the important question to ask is Do you have children? If the participant answers no, do not send him through 10 questions about children. Send him to the next portion of the survey that is not about having kids. That is skip logic—and you should use it at every available opportunity.
When writing a survey, your goal is to keep it as short as possible for members. If you’re asking question after question that is not relevant to the member, don’t ask it. You will waste your participant’s time, and risk not receiving any responses at all.
Now I hear you asking, what is the optimal length of a survey? The answer: as long as it needs to be—and no longer. If you need to ask three questions to accomplish your goal, ask them. If it needs to be 50, ask them. Be weary of too many non-required questions. If the question is not required, can you just save it for a follow-up interview? Just be up front about the length of time your survey takes, so the member knows what they are getting into.
Want to know the top 10 metrics you should be using to track and measure your community’s business value? Download our free report here.
Hold up! I know you want to blast this survey out and start seeing feedback right away, but don’t skip the two rounds of testing necessary to help finalize your survey. First, test internally with your team to make sure everything is in line with the goals you set in Step 1. Here’s where you also ask your team to look out for typos or awkwardly phrased questions—we are blind to our own mistakes!
The second round of testing involves piloting your survey with a small number of members. Have 50,000 community members from whom you want to collect feedback? Start with a pilot group of 500. This way, you can make sure that participants are going through the survey in the way you have intended.
You have a number of options to deploy your survey:
We deployed our Community Feedback survey on our own Smiley360 platform. We already send out an average of eight surveys each month for clients, so it made the most sense to stick to where our members are accustomed to giving feedback. Close to 10,000 members weighed in with their opinion, which was more than enough to deliver useful results.
You sent out your survey, and responses are starting to roll in. Depending on the engagement rate within your community, you might need to give members a little push to get a large enough volume of responses for statistically relevant data.
You can consider providing an incentive for your members to take action. A giveaway is a surefire way to motivate people to give up 5 to 10 minutes of their time to answer your questions. Offer up some gift cards or a prize more specific to the interests of your community.
If your survey’s deadline is approaching and you’d like to get a bump in your response rate, send out a gentle reminder email. Just be sure to filter out those who have already taken the survey as well as those who have expressed that they are not interested.
Keep in mind that the members of your community have different levels of passion and engagement. Some will give you feedback at any and every possible chance because they care deeply about the community and its future. Others might not have any opinion on your community’s next steps. It’s important to meet both of these types where they are at, as even more disengaged members may have ideas for getting more engaged.
Here’s the fun part—studying your survey data and turning that information into actionable steps for your developers, clients, or your own internal team. For a thorough and accurate analysis, follow a five-step process:
With feedback, comes action:
Some issues may come to light that you hadn’t even realized. Ask yourself, is this something that just a few particularly loud users are raising? Or is this something that affects the community at large? Be careful to filter through these ideas and suggestions and focus on the most important ones that align with your community, business, and client goals.
In our case, feedback from community members validated our roadmap for how to re-launch our application. If you’re a community manager interacting with your members every day, you have a good idea of what your members want—collecting data from an in-depth survey will give you additional learnings to drive action.