Why Some Conflicts in Wikipedia Are Not Resolved – Study Identifies Reasons and Proposes Forecasting Tools

Wikipedia has enabled wide-scale open collaboration on the world's largest general reference resource. But, as with many collaborative writing projects, content creation can be a controversial topic.

Often, many Wikipedia writers do not agree on certain changes to articles or rules. One of the main ways to formally resolve such disputes is the request for comments process. The writers in dispute will post their proceedings on a forum where other editors of Wikipedia will participate and a neutral editor will make his final decision.

Ideally, this should solve all the problems. But a new study by MIT researchers found that debilitating factors, such as excessive quarrels and poorly formulated arguments, led to about one third of unresolved RfCs.

For this study, the researchers compiled and analyzed the first comprehensive data set of RfC conversations, captured over an eight-year period, and interviewed editors who frequently close documentation centers to find out why they could not find a solution. . They have also developed a machine learning model that exploits this dataset to predict when generic resources can become obsolete. And, they recommend digital tools that could make deliberation and resolution more effective.

"It was surprising to see that one-third of the discussions were not over," says Amy X. Zhang, PhD candidate at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL). and co-author of the paper, presented at this week's ACM Conference on Computer-Assisted Cooperative Work and Social Informatics. "On Wikipedia, everyone is a volunteer, people put their work in place and they have an interest … and the editors may be waiting for someone to close in order to get back to editing. Through discussions, reading work and solving a great deliberation is difficult, especially with back and forth and litigation. [We hope to] help this person to do this work. "

The co-authors of the article are: Jane Im, first author, postgraduate student at the School of Information at the University of Michigan; Christopher J. Schilling of the Wikimedia Foundation; and David Karger, computer professor and researcher at CSAIL.

(Do not) find the closure

Wikipedia offers several ways to resolve editorial conflicts, involving two publishers who solve their problems, submit their ideas to a simple majority vote in the community or submit the debate to a group of moderators. Some previous Wikipedia research has deepened these channels and proceeded to "montage wars" between contributors. "But the resource centers are interesting because there is a lot less voting mentality," Zhang said. "With other processes, at the end of the day, you will vote and see what happens. [RfC participants] vote sometimes, but it's more about finding a consensus. What is important is what is really happening in a discussion. "

To file a change request, a publisher develops a proposal template based on a content conflict that has not been resolved in the article's "discussion" page and invites the broader community to to comment. The proposals span the gamut, from minor disagreements about a celebrity's basic information to changes to Wikipedia's policies. Any publisher can initiate a CR and any publisher – usually the most experienced – who did not participate in the discussion and who is considered neutral can close a discussion. After 30 days, a bot automatically removes the RfC model, with or without resolution. The documentation centers can formally close a summary statement of the nearest, informally because of the overwhelming agreement of participants, or be left behind, which means that they are deleted without resolution.

For their study, researchers compiled a database of about 7,000 English-language Wikipedia RfC conversations from 2011 to 2017, including closing statements, author account information, and a general response structure. They also interviewed 10 of Wikipedia's most frequent journalists to better understand their motivations and considerations in resolving a conflict.

In analyzing the dataset, the researchers found that about 57% of the control centers were officially closed. Of the remaining 43%, 78% (or about 2,300) were left to abandon without informal resolution – or about 33% of all the reference centers studied. By combining the analysis of the data sets with the interviews, the researchers then identified the main causes of the resolution's failure. Key issues include poorly articulated initial arguments, in which the initiator is unclear about the problem or writes a deliberately biased proposition; Excessive squabbles in discussions leading to more complex, longer and more difficult discussions to analyze; and the simple lack of interest of third-party publishers because topics may be too esoteric, among other factors.

Useful tools

The team then developed a machine learning model to predict the closure (formal or informal) or out of date of a given documentation center by analyzing more than 60 characteristics of the text, the Wikipedia page and information from publisher account. The model reached a 75% accuracy for predicting failure or success in the week following the start of the debate. They found that some more informative features for the prediction include the length of the discussion, the number of participants and responses, the number of revisions made to the article, the popularity and interest of the subject, the Participants experience the discussion and level of vulgarity, negativity and general aggression in the comments.

The model could one day be used by the initiators of the RC to follow an ongoing discussion. "We think it could be useful for publishers to know how to target their interventions," Zhang said. "They could post [the RfC] see you [Wikipedia forums] or invite more people, it seems that this may not be solved. "

The researchers suggest that Wikipedia could develop tools to help closed individuals organize long discussions, post persuasive arguments and changes of opinion in a thread, and encourage collaborative closure of reference centers.

In the future, the proposed model and tools could potentially be used for other community platforms involving large-scale discussions and deliberations. Zhang points to online city and community planning forums where citizens weigh on the proposals. "People are discussing [the proposals] and vote them out, so that the tools can help communities better understand the discussions … and [also] be useful for those responsible for implementing the proposals ".

Zhang, Im, and other researchers have now created an external website for editors-in-chief of all levels of expertise so that they can learn from each other and be more easily close the discussions. "The work of rapprochement is quite difficult," says Zhang, "so there is a shortage of people looking to close these discussions, especially difficult, longer, and more substantive discussions." This could help reduce barriers to # 39; entry. [for editors to become closers] and help them collaborate to close the resource centers. "

"It is surprising that a third of these discussions have never been resolved, [what’s more] Among the reasons the discussions were unsuccessful, the most interesting findings stem from qualitative analyzes, "says Robert Kraut, Emeritus Professor of Human-Computer Interactions at Carnegie Melon University. [of the study’s] The results transcend Wikipedia and can be applied to many discussions in other contexts. "Additional work, he adds, could be done to improve the accuracy of the machine learning model to provide more actionable information to Wikipedia.

The study highlights the fact that some FIU processes "are moving away from established norms, resulting in inefficiencies and biases," said Dario Taraborelli, director of research at the Wikimedia Foundation. "The results indicate that the participants' experience and the length of a discussion are highly predictive of the rapid closure of a CR. This brings new empirical evidence on the question of how to make the discussions on governance more accessible to newcomers and members of under-represented groups. "

Provided by
Massachusetts Institute of Technology

This story is republished with the permission of MIT News (web.mit.edu/newsoffice/), a popular site that covers the news of MIT's research, innovation, and teaching. .