- Objective 3 [O3]: validate the indicators with results from a sample of language communities and explore their potential role in affiliate planning.
To date, there is no comprehensive study on community growth and sustainability aimed at understanding the Wikipedia communities accounting for their linguistic diversity. Visual tools have been developed for exposing different aspects of social interactions in Wikipedia, including edit histories [10], controversies [11], language gap [12], cultural diversity [13]; however, we find a lack of tools to describe the state and health of a community, beyond the basic statistics available on the Wikimedia servers (Wikimedia Statistics, https://stats.wikimedia.org/ [accessed 24 March 2022].), and tools dedicated to monitoring participation to specific events (Event Metrics, https://eventmetrics.wmflabs.org/ [accessed 24 March 2022].) or outreach programs. (Programs and Events Dashboard, https://outreachdashboard.wmflabs.org/ [accessed 24 March 2022]).
In this paper, we present a comprehensive framework to look at the evolution and current state of the active editor communities through a list of language-independent indicators we call “Vital Signs.” These indicators are computed from the digital traces left by editors on Wikipedia, and presented in user-friendly visualizations. Furthermore, we validated their usefulness, by discussing them with community members in various Wikimedia conferences, and we set target values aimed at guiding the implementation of projects that can help improve community vitality and sustainability.
Our main contributions in this paper are that:
- We study the evolution of the number of editors over time in the 50 major language editions, and identify groups of language editions following different patterns of growth, stagnation and decline;
- We propose a set of indicators (Vital signs) to assess and monitor community health, based on previous literature and community practices;
- We present the results obtained for a set of eight selected language editions, for which we received feedback from Wikimedia affiliates;
- We describe the iterative validation process based on holding focus group sessions after presentation in four Wikimedia conferences and two dedicated meetings with Wikimedia affiliates;
- We discuss implications for the Wikimedia movement and how targets based on the Vital Signs could be adopted by affiliates and integrated into their annual planning.
Figure 1 illustrates the different phases of the methodology followed in this study, in order to fulfill the three objectives introduced above. In the first phase, “Exploration”, on one hand we study growth, stagnation and decline patterns with data from the 50 largest communities, and on the other hand we revise previous literature and community practices to identify key aspects to be measured and monitored, and incorporated in our framework. In the second phase, “Design and implementation”, we define the Vital signs as a set of indicators to monitor community health, and show the results on a sample of language editions. In the third phase, “Validation”, we assess the proposed framework and results with focus groups and feedback from Wikimedia affiliates and communities. The approach is described in more detail in Section 3.
