5.2.1 Academics' Qualitative Judgments
In this section we shall look first of all, in section 5.2.1.1, at how the academics in this study tended to make judgments, both positive and negative, about the full range of online encyclopaedia articles in the sample. Then we shall look specifically in 5.2.1.2 at the question of whether it is possible to identify strengths and weaknesses that are characteristic of Wikipedia articles in particular. Academics were simply told that the articles given to them for review 'have been carefully chosen from popular online encyclopaedias to overlap with your area of academic expertise', and were urged not to attempt to identify the origins of articles. Therefore we aim to detect whether judgments, made blind as to the identity of different articles, revealed characteristic patterns regarding Wikipedia articles.
Whilst different reviewers sometimes expressed contradictory opinions regarding the same article (which will be discussed further in Section 6), initial analysis has indicated that there was no marked pattern of differences of opinion between student reviewers and more established academic reviewers. For that reason, we include responses from both in the following sections of this report. We do, though, in the interests of transparency, indicate throughout the qualitative data whether a comment came from a student, an established academic or a professor. It should be noted that the term 'student' covered masters students, research students reading for a variety of research degrees including doctorates, and postdoctoral students. And again, we specify these distinctions when quoting comments from reviewers.
5.2.1.1 Academics' Judgments about Online Encyclopaedia Articles in General
In this sub-section, we try to capture the aspects of articles that earned either the approval or disapproval of the academic reviewers across the full sample of articles selected. The aspects praised and criticised below are fairly evenly distributed across all sources, and are intended to illustrate the kinds of judgments about online encyclopaedia articles across the sample that were generated by the experience of comparing and reviewing one or more pairs of articles on specific topics. So although we identify the sources of the articles in the following examples, again in the interest of transparency, we are not suggesting that the characteristics discussed in this sub-section are uniquely typical of that source.
It was evident on a number of occasions that academics, having considered the criteria put before them to help them with their quality judgments, and scored each of these criteria individually, very often went on to consider the combined characteristics of an article, balancing and synthesising the individual elements to arrive at an overall judgment. This is evident for example in the responses of two different reviewers for the pair of articles on Evo Morales, in which it appears that the overall feel and coherence of the article were valued more highly than quantity or currency of information:
- "The second one [Enciclonet] is much better than the first one [Wikipedia]. It is shorter but contains all the important information up to 2006. It also has a better and more professional style." (Reviewer 3 – academic)
- Speaking as an academic, I much preferred the second piece [Enciclonet] in this case. However, this was on principally intellectual and aesthetic grounds. For the most up to date information, and for more information about the various critics of
39
