When someone says “community culture” most of us immediately think (and rightly so) about something like shared values, beliefs or social norms that define a group of people. Something that invisibly binds the group together, shaping its identity and influencing interactions among its members.
Those are good things to know but let’s get back down to earth. How can we know whether or not our own community has a good culture? Actually, it is a very tough question to answer. In this article I am going to walk you through one of the ways to do that by leveraging data.
What does the culture of a community mean?
To understand the meaning of “community culture” at the applied level, let’s imagine that you have entered a new large chain store. Most likely, your impression of the store’s “corporate culture” will not be affected by the marketing slogan on the billboard in front of the store, but by how the store’s employees interact with each other, with you, and with other customers. The reason for this is that we derive “group values, beliefs, and social norms” from our everyday interactions with other people.
For us this means that in order to assess the culture of a community, we need to look at the interactions between the users. Moreover, it is worth looking not at all users, but only at those after interacting with whom other users either leave immediately or stay in the community more often than we expected. Further in the post I will call these two dedicated groups of users as positive vibe users and negative vibe users.
Why do we even need to think about all this?
Before we continue to look into the details, let’s answer the question - Why do we need to assess the culture at all and identify users with positive and negative vibes?
1. Proactive community development
Communities with a positive inclusive culture grow, while those with a negative one die. Сulture is not something abstract, it is how users interact with each other. Knowing your “retention champions” among the users, you can set them as an example by managing the attention focus of the community. You can help the champs be even more successful and hold various social activities to keep such people in the community for as long as possible.
In addition, you can work with those who make a significant contribution to the outflow. For example, manually looking at those who post a lot and have a low success rate, you will most likely be able to see “shortcomings” in the interaction of these users. Then, contact them and offer help, or try to help implicitly. In any case, if you can make the negative vibe users change their interaction style to a more positive one, you will significantly increase retention.
2. Identify problems in the community at an early stage
As we will see later, the period of “fading” of all communities is accompanied by a decrease in activity and outflow of positive vibe users and, often, an increase in the activity of negative vibe users. By monitoring the parameters of these groups, we can learn about potential problems at an early stage and solve them effectively.
Who shapes the culture? Identifying users with positive and negative vibe
Now, let’s dive into the data. Our goal is to identify positive and negative vibe users. The first thing to do is to calculate the success rate for every user in our community. But next we will get a problem: How to divide users into those with positive vibes and those with negative vibes? Taking the average rate for all users as a threshold is not entirely correct, since your community may not have any people with negative vibes at all, but there will always be those with a higher or lower success rate.
Please take a moment and think about how you would solve this problem. (By the way, it took me a week to find a reliable solution.)
Ready? Let’s define the things.
Negative vibe users
We know that having answers in a question is one of the main predictors of the author’s continued participation in the community. We also know that some users return to the community even without receiving an answer. It seems to me, it is safe to say that if users return to the community more often without receiving an answer than after receiving an answer from some users, then there might be something wrong with such answers. In other words, if a user has a success rate lower than the percentage of users who continue participating in the community even without an answer, we will think that user has a negative vibe.
Positive vibe users
The easiest way to identify this group of users is to mark as a positive vibe user everyone who has a success rate higher than the average “continue using the site after first question” percent.
Let’s take a look at the thresholds on real data:
- Rank is (the natural logarithm of) the number of first time questions a user participated in (by giving an answer to a question).
- Success rate is the percentage of users who continue using the site among those the user has interacted with (i.e. the authors of the questions the user answered).
- Second question overall continue threshold is the percentage of users who continued using the site after asking their second question on the site.
- Overall continue threshold is the percentage of users who continued using the site after asking their first question on the site.
- Negative vibe threshold is the percentage of users who continued using the site after asking their first question on the site and not receiving even one answer.
- Mean success rate is the mean ratio for that represents the number of those who continued using the community among all first time askers whom the user has answered to.
- Number of total posts is the total number of unique questions a user needs to participate to be marked as either a positive or negative vibe user.
Users who focus on first time askers
One more note before we look at the results of the research. There is an interesting tendency in online communities that usually users contribute by focusing on a limited number of types of actions. It means that there may exist users who intentionally answer more questions of the first time askers than others. Such users may have a significant contribution to the culture of the community. So we need to look at their participation too if we want to have the full picture.
How to find such users? There are two ways to approach this problem.
- Percentage of answers. Calculate the number of answers given to the first questions on the sites and divide by the total number of answers in your community. Next, do the same for every user on the site separately. If a user has no preferences in whose questions to answer, then their coefficient will be approximately equal to the overall percentage of answers in the first questions. (Personally, I prefer this approach.)
- Percentage of questions. The second approach is to look at the percentage of the number of questions from new users to the total number of questions on the site as the threshold value.
The most interesting findings
Now that we know “what’s what,” let’s look at the main trends that are useful to know. In this post, I’ll be using data for Stack Overflow in Russian because it’s publicly available, there’s enough data to answer most questions, and I know this community well. Please note that I also looked at other communities, and these findings were true for all of them. It means that the trends that we will see should be more or less universal.
So, let’s start! For reference, here is the number of posts on the site.
1. As the community grows, the size and activity of positive and negative vibe users grow
Having both groups is a normal trend for any community. There will always be those who are better or worse at something. As a result, the more active members you have in your community, the larger the size of each group will be.
2. The community starts to fade when the number of positive vibe users decreases
Usually, the number of positive vibe users and negative vibe users are in a certain proportion. If this proportion changes in favor of negative vibe users, then the community begins to fade.
So a decrease in the number of positive vibe users leads to a decrease in the percentage of users who continue to participate in the community after their first question. At the same time, the number of people who continue participating in the community after the first question is a critical indicator of community health. In most cases, a decrease in this metric signals serious problems in the community.
3. If the total number of first time askers does not decrease, but the number of positive vibe users decreases, the negative vibe users will take the lead and close the gap
Nature abhors a vacuum. If there is a constant stream of new questions in a community where the positive vibe users aren’t participating, someone else will take their place and those may be negative vibe users which will inevitably lead to a decrease in user engagement.
4. Negative vibe users focus on first time askers more than positive vibe users
Users who focus on first time askers are distributed among all types of groups on the site but somehow you can find them more often among negative vibe users rather than positive vibe users.
I hope the insights in this research will help you develop your community more efficiently! Even though at the moment of writing we do not have this report implemented in the app, I still can run all the code for your community and help you identify the positive and negative vibe users. If you are interested, please just let me know by sending me an email at nicolas.chabanovsky@gmail.com