In fact, half of the last enemies of the Jedi were not Russian trolls

If you have already spent time online, you have probably heard the news: studies show that a large majority of fans' hostility towards the public criticized The last Jedi was the work of Russian robots and trolls.

Nowadays, mixed with the "war of culture", it is tempting to take this information for cash. Screaming voiceless in the social media void.

But when you break down the numbers and look deeper, the reality is incredibly different: Star Wars does not have a Russian troll problem and the reaction of the fans is much more complex than the reports would have been believe.

If you're not aware, a pre-publication research article, written by Morten Bay of the University of Southern California, analyzed the feeling towards The Last Jedi by studying the tweets addressed to Rian Johnson enters the film's release on December 13, 2017 and July 20, 2018.

Bay is a researcher at USC's Center for Digital Future and has written about Trump's social media policy, Russian robots, misinformation, social media journalism and their ethics. He is also a fan of Star Wars.

It can be said that The Last Jedi is one of the most controversial films of the last 12 months. The speech was vitriolic – the harassment campaigns saw Kelly Marie Tran, who plays Rose Tico, leave social media and a group of fans asked redo the last Jedi.

Research like this can be important in highlighting some of the issues surrounding the politicization of social media and pop culture – and how they can be used to influence political discourse. However, although the goals are noble, the research is not definitive – a fact that seems to have been forgotten in the media storm it unleashed.

Instead, let's deepen the search.

One in 10

Bay's paper said that "50.9% of those who tweeted negatively" about The Last Jedi were likely to have political or even human motivation. It was the main course of a panoply of media that later stated that most of the negative feelings against The Last Jedi came from Russian trolls.

But this is not the truth – or at least it distorts the truth.

Bay collected 1,273 tweets using Twitter's advanced search feature, which were all tweeted to Rian Johnson's account (@rianjohnson) over a period of seven months after publication.

After "cleaning" the dataset, Bay finished with 967 tweets. He then "manually" determined if a tweet was negative, positive or neutral. In the end, the feeling of each tweet was left to Bay. To further separate the negative tweets, Bay would look in the high-activity accounts for terms such as "Trump" or "SJW" to determine their political position.

Of the 967 tweets analyzed, 206 expressed "a negative feeling" with regard to the film and its director.

Of the 206 negative comments, 61 were real people with a political agenda, 11 bots and only 33 appeared to be trolls. Of these 33 people, only 16 seemed to have features compatible with Russian troll accounts. In fact, less than one in 10 tweets came from Russian trolls – far from the 50% that were widely reported.

A less exciting story

There are some problems here. The first is that Bay's collection method is based only on tweets addressed to Rian Johnson. Other narratives related to the film, such as that of Luke Skywalker – @HamillHimself -, which has almost tripled the number of subscribers and is supposed to have a far greater reach than Johnson, have not analyzed. This greatly limits the power of the analysis.

The research has indeed caught the eye of the director of the last Jedi himself, pointing him out "What the top-line describes is my online experience." Of course, this would match his online experience, as the research paper used Johnson's tweets directly as the source of their data. He literally analyzed his online experience.

While it is true that half of the negative tweets (105 out of 206) were classified as politically motivated bots, trolls or slippers, the resulting reports diluted this message and collected all of these negative reactions. This selection of cherries helps tell a story, but it blurs the tracks.

None of this means that it was Bay's intention. The 38-page research paper explains its methodology in depth and with great clarity. It draws conclusions on the data set that it has acquired and even states that it has "limitations" and that the study is "non-exhaustive". In his conclusion, he pointed out that the claims made in his paper should be considered only in "the limited scope of the data set".

He knows it's a less exciting story.

"Having worked as a journalist for many years, I know how the game works," he says.

Bay is "moderately disappointed by some of the big media brands" who have published articles without taking the time to go a step further. He understands that some of his discoveries have been buried because they produce a less appealing light than "the army of Russian trolls invading the Star Wars galaxy".

This is just not what his research suggests.

"The so-called Russian trolls are so small that it is in principle the normal number of Russian trolls you would expect to see appear in a highly publicized online debate."

A hive of scum and malice

I'm not here to tell you that the reaction to The Last Jedi did not contain a myriad of trolls, bullies or robots. It's clear to everyone that the Star Wars fandom continues to sink into two online trading beards, almost a year after the release of The Last Jedi.

This conversation merged with right-wing politics, media diversity and the social media discourse of the Trump era. A very small minority of the group was involved in this conversation, at least on Twitter a specific instance, seems to be built artificially.

More importantly, it is impossible to make general and general statements about the state of Star Wars speech from this particular study. By the very admission of Bay, he is limited in the conclusions that he can draw. It is hampered by a relatively small sample subject to bias. It is misleading to say "half" when all the "enemies" of Star Wars have not been used in the dataset.

And to be clear, this is not a bad search. But the real story, according to Bay, shows that "US political activists have begun to use the same tactics as Russians to interfere in any kind of social media debate, where there is a gap that can be widened."

His work supports the idea that debates about pop culture on social media can be politicized and potentially used for strategic purposes. He also suggests that pop culture fandoms are another place to look to try to decipher the way political messages can be spread online.

The truth is simple: the majority of people who read sure Bay's research will never go so far as to read the study from top to bottom.

The information will be presented to them and ingested by osmosis of second hand. Via a tweet, via outlets trying to capture a snapshot of the search that most pleases their audience. There is no material analysis of the methods, no query of the dataset. And this approach only serves to ignite the speech even more.

In the end, it is ironic that at a time when false news and misinformation are so widespread, Bay's study has revealed a widespread appeal through a media narrative far removed from the truth.

iHate: CNET examines how intolerance is invading the Internet.

Infowars and Silicon Valley: all you need to know about the debate on freedom of speech in the technology sector.