Instagram Account Suspension Error: Meta Blames Claude-Based Tool

Meta has pointed to a tool linked to Anthropic's Claude model as the cause of the suspension of the anti-Trump No Kings Coalition's Instagram account. The explanation has revived an old question in AI-powered content moderation: when an automated system makes the wrong call, who owns the mistake?

What happened?

No Kings Coalition announced that its Instagram page was taken down on Saturday, October 17. The next day, Andy Stone, Meta's vice president of communications, posted on social media. Stone said the suspension had been triggered by a tool linked to Claude. It's not yet clear exactly what role the tool plays in Meta's systems. Nor has Meta said whether the tool acted on its own or whether the decision first went through human review.

Why does it matter?

Automation in content moderation is nothing new. Meta has used its own classifiers for spam, nudity and violent content for years. According to the company's transparency reports, in some categories these systems catch most of the content that gets removed before anyone reports it. When these systems got it wrong, the company usually owned the mistake. The public also knew the error came from Meta's own algorithm.

What's different this time is that the error is being attributed to an outside, general-purpose language model. Using large language models for moderation has become widespread in recent years. In 2023, OpenAI introduced an approach in which policy documents are fed to the model, which then labels content according to those policies. The approach is fast, and the rules are easy to update. But it's much harder to explain how the model reaches a decision than it is with older classifiers. The same content can also get a different result depending on small changes in the wording of the prompt.

The AI did it explanation does little to protect Meta legally, because the company itself enforced the decision and shut down the account. From a communications standpoint, though, it spreads the blame. If a bank tried to pin an unfair loan rejection on the vendor of the software it uses, no regulator would accept that. The same logic should apply to platform moderation.

Who stands where in the chain of responsibility?

A decision like this typically passes through three layers. Most errors surface in the least visible one: the middle layer.

A language model may flag a piece of content as suspicious. But whether that flag automatically turns into an account takedown is up to whoever built the tool. If sanctions are imposed on sensitive accounts, such as political organizations, journalists and activists, without requiring human approval, the problem lies more in the design than in the model. Meta did not clarify this distinction in its statement. So for now we only know the answer to the question "which tool," not "which decision."

What does the EU Digital Services Act say about this?

No Kings is a US-based organization. But if a similar decision had affected a user in the EU, the Digital Services Act would have come into play. Article 17 of the DSA requires platforms to provide a statement of reasons for every restriction decision and to disclose whether automated tools were used. Article 20 prohibits appeals from being resolved solely by automated means and requires oversight by qualified staff. Article 21 gives users access to out-of-court dispute settlement. The statements of reasons that platforms submit are also made public through the DSA Transparency Database.

The key point here is this: under the DSA, there is no such party as a "model provider." All obligations toward the user rest with the platform. The EU AI Act imposes documentation and risk obligations on providers of general-purpose models, but it does not make them responsible for individual moderation decisions. In short, European law rules out the "AI did it" defense from the start. Whoever enforced the decision is the one who has to answer for it.

What's next?

There are several questions to watch in the coming period:

• Will Meta disclose at what stage the tool kicks in and whether human approval is involved?
• Will Anthropic make a statement about its model being used this way?
• Will Meta's Oversight Board set a principle on the use of external models in automated moderation?
• Will the European Commission begin examining language-model-based moderation as a separate item in the risk assessments of very large platforms?

Disclosing which tool a decision came from is valuable for transparency. But that disclosure doesn't erase responsibility. Meta needs to explain clearly which accounts can be shut down without human approval. Until it does, "AI did it" will sound less like an explanation and more like an excuse.