AI Recommendation Poisoning: The Silent Data Killer

In October 2025, Anthropic, the UK AI Security Institute, and the Alan Turing Institute published the study that should reset how marketers talk about model safety. Across models from 600M to 13B parameters, the researchers found that 250 poisoned documents were enough to backdoor every tested model in their setup. The full paper describes the result as a near-constant number of poison samples rather than a percentage of the training set.

The caveat matters: this was a controlled backdoor experiment, and the authors note open questions about larger models and more complex behaviors. But the training-time implication is still sharp. "We train on a lot of data" is not a poisoning defense by itself. Bigger training runs may dilute normal noise, but adversarial data is not normal noise. It is placed with intent.

Translate the finding into marketing. A brand search corpus, review archive, product Q and A library, social listening feed, CRM enrichment table, or RAG knowledge base does not need to be mostly fake to become dangerous. It only needs enough planted material in the right places to become retrievable, repeated, and eventually trusted.

Security teams already worry about model poisoning in fraud, identity, and safety systems. Marketing systems deserve the same worry with less of the theater. They are exposed because their best data is also their messiest data.

Reviews are public. Social posts are public. Forum consensus is public. Comparison pages are public. Search snippets, affiliate articles, product feeds, support tickets, call summaries, and engagement events all move through partners and platforms before a model treats them as signal. The source might look like customer truth, but the write permissions often belong to strangers, competitors, affiliates, bots, or an internal team moving too quickly to verify provenance.

The feedback loop makes the exposure worse. Recommendation systems learn from clicks, purchases, dwell, saves, returns, refunds, replies, and suppressions. Synthetic engagement is not just analytics noise. Once it is fed back into training or ranking, it becomes instruction. A click farm can teach a model that a low-quality product deserves shelf space. Agent traffic can teach a lead-scoring system that a dead segment is surging. Fake review velocity can teach a shopping assistant that consensus exists where no human market exists.

This is also where Preference Manipulation Attacks become a marketing problem. The model can be healthy and still make a poisoned recommendation if the sources or plugin descriptions it reads at query time have been shaped to promote one option and discredit another.

This is the sibling problem to attribution poisoning. Attribution poisoning corrupts measurement, so the team misreads what worked. Recommendation poisoning corrupts the system that acts on the measurement, so the model makes worse choices with confidence.

Door one is the classic training vector. A team fine-tunes on scraped content or user-generated reviews. A planted set enters the corpus. The model learns a false association or hidden trigger and later expresses it as preference.

Door two is retrieval. This is more immediate and more marketing-shaped. If the model grounds on pages, reviews, product documentation, partner feeds, or social consensus, the attacker does not need to alter weights. They need to become the context the system retrieves. That is inference-time poisoning: not changing what the model knows, but changing what the model reads before it answers.

Door three is feedback. Slow poisoning looks like normal optimization until someone asks whether the signal came from a plausible human. If bot engagement enters the learning loop, the system can optimize toward preferences nobody held.

Door four is agent memory and tools. Microsoft documented recommendation poisoning patterns in which prompts embedded in links or pages manipulate assistant memory. Its later agentic AI taxonomy update names memory poisoning, knowledge-base poisoning, cross-domain prompt injection, and agentic supply-chain compromise as operating failure modes. In plain English: a memory, tool description, MCP server, or prompt template can become a behavioral dependency.

Poisoning no longer lives only in lab papers. Gravitee's 2026 State of AI Agent Security research reports that 54 percent of organizations experienced or suspected an AI agent security or data privacy incident in the prior 12 months, and one financial-services response specifically described data poisoning that corrupted model outputs. The detail matters because it shows the shape of the failure: confident outputs, operational harm, and weak monitoring.

OWASP also treats training data poisoning as a first-class LLM application risk. The shared pattern across those sources is not that every system is doomed. It is that poisoned systems often fail quietly. A crash gets logged. A recommendation that tilts five percent toward a planted answer becomes a weekly dashboard argument.

The poisoned model rarely tells you it is poisoned. It says the product is trending. It says the audience is valuable. It says the source is relevant. It says the offer deserves another test. If the team only watches conversion rate, the poison can hide as "market movement" for months.

The attacker does not need perfect control. They need profitable influence. A competitor who nudges your recommendation model away from a flagship product steals distribution you paid to build. A seller who games review consensus gets your AI to launder their positioning with your brand's credibility attached. A fraud ring that teaches a propensity model to favor mule accounts turns your media budget into its targeting engine.

The cost side is ugly for the defender. If you cannot trace which sources fed a recommendation, you cannot tell whether a model changed because the market moved or because the diet changed. The team burns time debating channels, vendors, creative, and seasonality while the poisoned input keeps voting.

This is where poisoning connects to AI measurement drift and agent audit trails. The business needs more than a model score. It needs a source ledger, a behavioral baseline, and a rollback path. Without those, a poisoned recommendation becomes a brand problem, a revenue problem, and sometimes a liability problem, while the attacker stays invisible in the incident report.

The fix is not one magic poisoning detector. The fix is treating the model diet as production infrastructure. That means every input has an owner, a trust level, a change trail, and a detox plan.

The first control is boring on purpose: inventory the diet. List every source the recommender trains on, retrieves from, or stores in memory. Then ask the uncomfortable permission question: who can write to this source before the model sees it?

The second control is behavioral. Watch category share, top-item churn, source mix, segment recommendations, unexplained recommendation lift, and the gap between recommended demand and verified sales. If a product surges in the model before it surges in trusted revenue, you do not have proof of poison. You have enough signal to investigate.

The third control is traffic hygiene. Bot and AI-assisted traffic should not vote in the same way verified humans vote. If your attribution system already segments human, automated, and AI-assisted behavior, connect that segmentation to the learning loop. If it does not, the recommender is eating whatever the open web serves.

The fourth control is adversarial retrieval testing. Ask a blunt question: if I wanted our AI to recommend a competitor, what page would I plant, which review cluster would I seed, which comparison query would I target, and where would the retriever find it? Then search for those fingerprints.

The fifth control is memory discipline. Agent memory writes should be narrow, reviewable, reversible, and tied to a user action or trusted system event. Tool descriptions and MCP servers should be treated like supply-chain dependencies, not helpful prose.

The final control is the detox plan. Keep clean snapshots. Know which data windows can be excluded. Know how long retraining takes. A team that discovers poisoning without a clean checkpoint is not responding to an incident. It is rebuilding its map while the building is already on fire.