Nevertheless, having established these goals at the Wikimedia-movement level is not effective if the communities do not embrace growth and renewal strategies as an important matter and enable the changes that effect them. In this sense, we believe the indicators we suggest will shed light on the sustainability of the current community and will become a reference baseline. The indicators will bridge the gap between the goals and narrative provided by the Strategy and the changes that may come at Wikimedia Foundation and affiliate level. In fact, Wikimedia affiliates have the capacity to influence community discussions and facilitate these changes. By means of annual plans, they are able to create initiatives and set the necessary spaces in order to raise awareness on the need to perform some actions aimed at renewing and growing the community. For this reason, our third objective is to validate the indicators and explore their potential role in affiliate planning and agenda-setting. For this reason, our Objective 3 [O3] is to validate the indicators and explore their potential role in affiliate planning.
3. Approach
In this section, we present the approach we employed to reach the three goals, and we introduce the definition of six “Vital Signs,” each of which is associated with one or more indicators that we propose for assessing and monitoring a different aspect of community sustainability.
3.1. Research Process
This research project originated as a Wikimedia project grant to provide valuable indicators to understand the state of community health and recommendations in order to improve it (Meta contributors, ’Grants:Project/Eurecat/Community Health Metrics: Understanding Editor Drop-off’, Meta, discussion about Wikimedia projects, 7 May 2021, 10:10 UTC, https://meta.wikimedia.org/w/index.php?title=Grants:Project/Eurecat/Community_Health_Metrics:_Understanding_Editor_Drop-off&oldid=21432221 [accessed 19 February 2022]). This differentiates the approach from other projects created in organizations with a specific target user. In this case, the project was approved thanks to the support from the Wikimedia affiliates and community members, who will be the users to benefit from it.
For this reason, we decided to follow an open research model approach for this project, given that it is the most convenient way to engage with Wikimedia communities. This implies sharing the results, the prototypes work-in-progress, as well as the code and data at all times, presenting preliminary work at community gatherings, and discussing the work with relevant members from communities in order to get iterative feedback [13]. This is completely in line with the Wikimedia movement ethos, which encourages being bold and improving on things incrementally.
The open research approach is iterative. We have employed three different phases: (a) Exploration, (b) Design, and (c) Validation.
3.1.1. Phase 1: Exploration
In this initial phase, we reviewed the state of the art for the Wikipedia communities’ participation and organization. Furthermore, we did not only take into account the academic literature but also the Wikimedia documentation provided in the website Meta-wiki, which is the global site for the project’s information. To understand the state of growth and renewal of the communities, we would first assess the overall trajectory in the number of monthly active editors in their entire history [O1].
To do so, we obtained the corresponding time series, and normalized the values according to the maximum number of monthly active editors for each language edition. We then grouped the time series into clusters using the k-means algorithm with dynamic time warping [40]. This algorithm allows aligning the time series and grouping those language editions with a similar temporal pattern, regardless of minor oscillations and the exact moments in which the curve changed, focusing on the general trend. Hence, by looking at
