Tesla CEO Elon Musk tumbled Twitter shares on Friday when he said he would “hold” his $44 billion acquisition of the social network while he scaled the stakes in fake and spam accounts of the platform examined.
Though Musk later clarified that he was still committed to the deal, he continued to pound on the subject of fake accounts. He wrote on Twitter that his team would conduct its own analysis and expressed doubts about the accuracy of the numbers Twitter reported in its most recent financial filing.
In its earnings report for the first quarter of this year, Twitter acknowledged that there are a number of “fake or spam accounts” on its platform, alongside legitimate monetizable daily active usage or users (mDAU). The company reported, “We conducted an internal review of a sample of accounts and estimate that the average false or spam accounts represented less than 5% of our quarterly mDAU in Q1 2022.”
Twitter also admitted to overstating user counts by 1.4 million to 1.9 million users over the past 3 years. The company wrote, “In March 2019, we launched a feature that allows people to link multiple separate accounts together for convenient account switching,” Twitter announced. “A mistake was made at the time so that actions taken through the primary account resulted in all linked accounts being counted as mDAU.”
While Musk may be rightly curious, pundits on social media, disinformation, and statistical analysis say his proposed approach to further analysis is woefully lacking.
Here’s what the CEO of SpaceX and Tesla said he would do to determine how many spam, fake, and duplicate accounts exist on Twitter:
“To find out, my team will conduct a random sample of 100 followers of @twitter. I invite others to repeat the same process and see what they discover.” In subsequent tweets, he explained his methodology, adding: “Choose any account with a lot of followers” and “Ignore the first 1000 followers and choose then every 10th off. I’m open to better ideas.”
Musk also said, without providing evidence, that he chose 100 as the sample size for his study because that’s the number Twitter uses to calculate numbers in their earnings reports.
“Any reasonable sampling process is fine. If many people independently get similar results for percentage of fake/spam/duplicate accounts, it will be meaningful. I chose 100 as the sample size number because Twitter uses it to calculate <5% fake/spam/duplicate."
Twitter declined comment when asked if its description of its methodology was accurate.
Facebook co-founder Dustin Moskovitz has commented on the subject via his own Twitter account, noting that Musk’s approach is not truly random, uses too small a sample, and leaves room for massive error.
He wrote, “Also, I feel like ‘not trusting the Twitter team to help pull the sample’ is its own kind of red flag.”
BotSentinel founder and CEO Christopher Bouzy said in an interview with CNBC that his company’s analysis suggests that 10% to 15% of accounts on Twitter are likely “fake,” including fakes, spammers, scammers, nefarious ones Bots, duplicates, and “individuals intentional hate accounts” that typically target and harass individuals, along with others that intentionally spread disinformation.
BotSentinel, powered primarily by crowdfunding, independently analyzes and identifies inauthentic activity on Twitter using a mix of machine learning software and teams of human reviewers. The company now monitors more than 2.5 million Twitter accounts, mostly English-speaking users.
“I don’t think Twitter realistically classifies ‘fake and spam’ accounts,” Bouzy said.
He also warns that the number of inauthentic accounts can appear higher or lower in different corners of Twitter, depending on the topic being discussed. BotSentinel has found that, for example, more inauthentic accounts tweet about politics, cryptocurrency, climate change and Covid than those discussing non-controversial topics like kittens and origami.
Carl T. Bergstrom, a University of Washington professor who co-wrote a book designed to help people make sense of data and avoid falling for false claims online, told CNBC that the sample of a hundred followers included one individual Twitter accounts should not serve as “due diligence” for a $44 billion acquisition.
He said a sample size of 100 is orders of magnitude smaller than the norm for social media researchers studying such things. The biggest problem Musk would face with this approach is something called selection bias.
Bergstrom wrote in a message to CNBC: “There is no reason to believe that the followers of the official Twitter account are a representative sample of accounts on the platform. Maybe bots follow this account less to avoid being discovered. Perhaps more likely to follow to appear legitimate. Who knows? But I just can’t believe Musk would do anything other than fool us with this silly sampling plan.