How to identify the independent subsystems of a community project to being able to measure them correctly?
In order for measurements to be interpretable and have practical meaning, it is necessary to isolate the object of observation. An online community is a complex system consisting of several independent subsystems. The first step to creating community metrics is to identify all independent high-level subsystems. Usually an online community consists of at least three independent parts and each part has its own set of dedicated metrics:
- Software
- Community
- Contents
Here is why those are independent subsystems. One can migrate a community (the users) and content to a completely new platform without impacting the project much. One can work on new content with an existing community or create a new community around existing content. At the same time, many different communities with different content can use the exact same software. All three parts are interconnected and form a single whole, though they are not tightly connected to one another to form a system with clear feedback loops.
If you have a unique setup for your community, you can figure out which parts of your particular setup are independent and which are dependent by trying to replace each part one by one with their analogues. If you are able to replace a part without significant changes to other parts it is independent and must have its own dedicated set of metrics.
This is a fragment of a draft of the book “Lessons Learned While Working On Stack Overflow”. Read the full book on kindle or the paperback version.