Innovation Report
assistance from WikiProject Medicine, a volunteer group of experienced Wikipedia editors who seek to ensure that the general public and health care professionals have access to free, current, accurate, and understandable medical information in their own language.
Thus far, we have run the course four times: November 2013, April 2014, November 2014, and November 2015. Each cycle has begun with a two-day orientation (Table 1) during which we introduce students to Wikipedia’s editorial tools, style, and standards. Additional didactic activities include reviewing guidelines for writing simplified English and strategies for locating and evaluating source material quality.
All students establish their own Wikipedia user accounts and choose a single healthrelated article to edit over the remainder of the course. WikiProject Medicine maintains a list of the most frequently accessed health-related Wikipedia articles, ranked by importance, and graded according to an article quality scheme that is applied to most Wikipedia Projects.5 We encourage (but do not require) students to select a “top-” or “high-importance” article that has also been tagged as needing quality improvement. At the beginning of the course, students receive custom reports for their chosen articles that contain both a list of grammar and style errors and a quantitative readability score. These Acrolinx (San Jose, California) reports use specialized natural language processing software, which considers sentence length and over 100 grammar and style rules.
Detailed course structure and materials are publicly available under free license on our course Wikipedia page.6 As we are not always certain, a priori, how much students can accomplish in a month, we encourage them to improve their selected articles “as much as feasible.” Thus far, students have completed the majority of their editing work independently. Instructors (A.A. and J.H.) and medical librarians (E.W. and L.M.) provide intermittent encouragement via e-mail, hold weekly office hours, and schedule consultation as needed. At the end of the course, faculty provide students access to custom-built tools that allow comparison of summary statistics of each article on the first versus last day of the course. While presenting the final versions of their Wikipedia pages, all students share highlights of their accomplishments and challenges, describe lessons learned, and provide feedback for future iterations of the course.
This report summarizes the outcomes of our pilot initiative at the University of California, San Francisco (UCSF). The UCSF institutional review board reviewed this research and deemed it exempt from formal review.
Effect on Wikipedia articles
We compared the state of the Wikipedia articles before and after student editing using several approaches. Two UCSFaffiliated physicians (E.W. and J.D.M.) provided independent subjective ratings of article changes and reconciled their impressions by consensus. Wikipedians with experience rating medical articles but unaffiliated with the elective received blinded unedited and edited versions of each article to evaluate using the general WikiProject article-grading rubric. We obtained changes in word, paragraph, and citation counts using Wikipedia’s analysis tools. Finally, we quantitatively assessed each article’s quality, readability, and translatability by generating postcourse Acrolinx reports.
Conveniently, all Wikipedia edits are saved in perpetuity. This archiving permits easy comparison across article versions over time. Because students edited their articles via their Wikipedia user accounts, we are able to analyze the changes (to words, paragraphs, and citations) made directly by our students. However, because anyone at any time can edit Wikipedia articles, all other metrics comparing the state of an article at the beginning and end of the course
Table 1 Instructional Content During Initial Two-Day Orientation Session of UCSF WikipediaEditing Course Day
Content
Content provider
1
Course structure • �Objectives • �Logistics • �Expectations • �Defined deliverables WikiProject Medicine • �Why Wikipedia matters and how it works • �Managing your first edits
Course faculty and guest instructors
Global reach of Wikipedia • �Principles of simple writing • �Principles of writing for translation • ��Translators Without Borders and WikiProject Medicine collaboration
Chief executive officer of Content Rules (text simplification company)
Wikipedia editing • �Core policies and the neutral point of view • ��Consensus-based collaboration and reliability through crowd-sourcing • ��Evaluating references and properly citing high-quality medical publications • �Interactive Wikipedia guided tutorial
Experienced Wikipedian and Wiki Education Foundation volunteers
Information seeking and retrieval • �Reputable sources to cite • �Finding information using UCSF resources • ��Background information (e.g., Access Medicine, other e-books) vs. foreground information (e.g., PubMed, other databases) • �Introduction to using a reference manager • ��Acknowledgment of the tension between building an open-source encyclopedia using largely closed-source information
UCSF librarians
2
Members of the WikiProject Medicine community
Abbreviations: UCSF indicates University of California, San Francisco.
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.Academic Medicine, Vol. XX, No. X / XX XXXX
