regard to Wikipedia, this has limited significance beyond indicating the kinds of questions or hypotheses that might profitably be focused on in a subsequent study on a larger scale. Although it is perfectly legitimate to look in depth, as we have attempted to do, at the themes emerging from this small sample, in order to sharpen the focus of further research of this kind in the future, it is obviously not possible to make generalisations about Wikipedia as a whole on the basis of the comparative analysis of 22 articles. Therefore, it is important to consider what kinds of further study are most feasible.
The use of the feedback tool devised for this research produced results that were comparable across reviewers, and allowed for consistency of measurement of views, especially in terms of quantitative data. Therefore, in any future study, we consider that it would definitely be worthwhile to develop and refine this instrument. Further pilot work would be needed to test the appropriateness of specific criteria, such as conciseness, readability and enjoyment, in order that reviewers are consistent in their application of these to specific articles. Further consideration as to whether it would be appropriate to modify the tool for use in different disciplinary areas needs examination: our findings indicated that the notion of conciseness, for instance, was used differently by reviewers in science areas than in humanities and social science areas. It is also worth exploring whether an element of unstructured questioning of academics might produce more valuable qualitative data.
In terms of methods used, we do have considerable concerns about the feasibility of substantially scaling up the exact approach used in the present study. This is for a number of reasons:
- The difficulty of finding a substantially increased number of articles from other encyclopaedias against which to compare Wikipedia articles.
- The difficulty of securing and managing the participation of a sufficient number of academic experts for a large-scale representative study.
- The complexity of carrying out a sufficiently rigorous analysis of qualitative data on a large-scale without recourse to some degree of quantitative content analysis, which is not feasible with the present model of feedback tool.
In other words, we would say that the present project has produced a considerable amount of interesting and illuminating data, but we recognise that there are serious logistical problems in replicating it on a far larger and wider scale. Although it would presumably be possible to do so, given considerable investment in research staff to carry this out across multiple sites, the question arises as to whether or not this would be worthwhile.
6.3.2 The Focus for Future Research
As the end of the Section 6.2 indicated, this study has indeed raised some lines of enquiry for further research of a similar kind. As mentioned above, the qualitative results from this study indicated that academic reviewers are generally open-minded about what constitutes quality in articles in online encyclopaedias. We would therefore suggest that, on the basis of these findings, the following tentative hypothesis is worth testing in a future study: the perceived quality of
56
