In August 2026, Anthropic announced that Claude now marks the text it writes with an invisible statistical watermark. That covers everything written with Claude: blog posts, landing pages, emails and product descriptions, whether they come from the Claude app, from a team’s own script or from any AI writing tool built on Claude. Each of them carries a signal that says “Claude wrote this”, and light editing afterwards does not remove it. Any content tool that writes with Claude passes that signal on to every article it produces.
We spent six weeks building a step that removes that signal while keeping everything that makes an article worth publishing: the facts, the numbers, the links, the FAQ questions and the search phrases in the headings. We call it the regeneration step, and the idea fits in two lines:
- Regenerate, don’t edit. Every section is regenerated from scratch by a model that does not watermark its output,
- Enforce it with code. A program compares each rewrite with the original and rejects it if it keeps too much of the original wording. We call this check the “novelty gate”.
Everything else in this article is about doing that without losing value. This is the full story, including:
- the parts that did not work,
- how we tested a watermark that nobody outside Anthropic can detect, and
- how our approach compares with the most popular open-source watermark remover on GitHub.
How we measure how much wording survived
One number comes up again and again in this article, so it is worth explaining first. We break the original text into every run of five consecutive words (“five-word sequences”) and count how many of them still appear, unchanged, in the new version. If 94% survive, the text was lightly edited. If 13% survive, it was genuinely rewritten.
This matters because the watermark lives in word choice. If most of the original wording survives, most of the watermark survives with it.
Two sets of test articles
We measured on two different sets of articles, because no single set can show both things we needed to know:
- 10 real production articles from our own pipeline show what the rewrite keeps or breaks: facts, numbers, links, FAQ questions and search phrases. They were written with Claude, but nobody outside Anthropic can test them for Claude’s real watermark.
- 12 watermarked test articles show how much watermark survives. We generated them ourselves with an open AI model and applied a SynthID-Text watermark with our own secret key, so we can measure it exactly (details below).
The short version:
- Editing doesn’t remove the watermark. Regenerating does. Our humanizer step, a careful polish for readers, keeps 94% of the original five-word sequences (we measure our own), which leaves the watermark untouched. Our regeneration step keeps 13%. (That 13% includes the parts we keep word for word on purpose, such as FAQ questions, product names and search phrases. In the rewritten body text alone, code caps it at 7.5%.)
- The rewritten text tested clean. On a replica of the watermark that we built and keyed ourselves, a trained detector found no watermark in the rewritten body text of any of the 12 test articles. On the same test, it still found one in 6 of 12 for the most popular open-source remover.
- Nothing of SEO value was lost. We kept 10 of 10 articles’ links intact, 23 of 23 FAQ questions and 118 of 118 search phrases in headings. The open-source tool kept links intact in 0 of 10 articles, 18 of 23 FAQ questions and 89 of 118 search phrases.
- No meaning errors. Across the same 88 article sections, our output had 0 major meaning errors. The open-source tool’s had 214.
How Claude’s watermark works
Anthropic’s watermark is a version of SynthID-Text, the method Google DeepMind published in Nature in 2024. It does not insert hidden characters. When the model chooses between several equally good next words, a secret key plus the preceding few words decide which one wins. Each single choice looks natural. Over hundreds of words the pattern becomes statistically detectable by anyone who holds the key.
Three facts from Anthropic’s own documentation shaped what we built:
- Current Claude models are marked, including the newest ones we write with, whether the text comes from the Claude apps or through the API.
- Light edits do not remove it. Anthropic says light editing “probably won’t remove the watermark completely”, while a complete rewrite will.
- There is no public detector. Detection is in private preview for regulators, researchers and similar organizations. Nobody outside Anthropic can check text for the real watermark today, us included.
Point 2 told us what to build: a complete rewrite. Point 3 told us we would have to build our own way to test it.
Step zero: why a humanizer won’t remove it
Our content process has several quality steps, and one of them is “Humanize”. Humanize was never meant to remove a watermark. Its job is to make the text better for a human reader: take out AI tells such as em dashes and stock transitions, smooth the flow, and leave everything else intact. We measured it anyway, because it shows what a humanizer does to a watermark, ours or anyone else’s.
We recovered 10 recent production articles where we had both the exact text that went into Humanize and the exact text that came out. Then we counted how many five-word sequences of the input survived in the output.
| Share of input’s 5-word sequences still present | Longest unchanged run (average) | |
|---|---|---|
| Humanize output | 94% | 617 words |
A humanizer is a careful copyeditor. Many paragraphs come out identical, which is exactly what you want when the goal is a better read without new errors. But it leaves a watermark that lives in word choice untouched, and when the humanizer is itself a watermarking model such as Claude, anything it does rewrite comes out freshly marked. The same holds for any humanizer that edits rather than rewrites. Removing the watermark needs a different kind of step.
First attempt: a careful editor (failed)
Our first prototype played it safe. It extracted the facts from each paragraph, then edited the text while preserving every one of them. We tuned it through 31 prompt versions and over 200 offline tests.
It failed in two instructive ways:
- Turning paragraphs into fact lists lost meaning. When we extracted the facts into a list and rebuilt the paragraphs from it, 0 of 8 paragraph groups passed review on our test article, in two separate runs. Qualifiers like “up to”, “optional” and “on properly equipped models” do not survive being turned into a list.
- Careful editing turned into copying. The editing approach “passed” every review, but it kept 93.8% of the original five-word sequences. Our past Humanize runs had kept 95.0%. We had built a second copy editor.
The lesson: if your only stop criteria are “the facts survived” and “it reads well”, a model will take the path of least resistance and copy. You need a hard rule that measures how much wording changed.
What we built instead: regeneration with a hard novelty gate
This is the core of our method, and the part that actually removes the watermark. Every section of the article is regenerated as a small, independent rewrite job, with strict checks around it that make sure nothing of value is lost. It runs in seven steps.
Step 1. Lock what must not change.
Before the writer sees anything, code replaces links, quotes, product labels and similar spans with placeholders, and keeps them out of reach. Numbers must come back exactly as written and appear the same number of times. FAQ questions are locked because they are written to match real searches.
Step 2. Pin the search phrases.
Code finds two-to-four-word phrases that appear in the headings and FAQ questions and also in the body, such as “tow a travel trailer”. Each phrase must appear verbatim in the paragraph that first used it. This needs no keyword list, so it also works for standalone articles that never had a brief.
Step 3. Rewrite each section from scratch, with a model that does not watermark.
This is the step that removes the watermark. The new wording has to come from a model that adds no watermark of its own; otherwise the rewrite would simply carry a fresh mark, which is also why a Claude-based humanizer cannot do this job. We use DeepSeek Flash with light reasoning: DeepSeek does not document any text watermark on its API output (as with any provider, that cannot be fully verified from outside), and it is affordable enough to rewrite every section several times. The writer gets the section and one instruction: same information, new wording. Sections run in parallel.
Step 4. Gate the result in code, not in the prompt (the novelty gate).
Asking the model to change the wording is not enough, so code rejects a rewrite automatically if:
- more than 7.5% of the original prose’s five-word sequences survive (pinned phrases excluded; headings, which are reworded only lightly, get 35%);
- any copied run is longer than 8 words;
- a number, link or required phrase changed, went missing or was added;
- it contains an em dash, or its length moved too far from the original.
Step 5. Check the meaning.
A separate AI model acts as a judge. It compares the original and the rewrite and reports only changes a careful reader would notice: a lost condition, a changed number, a dropped “only”, a claim attached to the wrong model. Synonyms and reordered sentences don’t count. The judge also flags clumsy phrasing a copy editor would fix.
Step 6. Repair, without drifting back.
Findings go back to the writer as targeted repairs. Early repairs had a habit of pasting back the original sentence. The repair prompt now tells the writer to restore the meaning in new words, and every repair goes through the same gates again.
Step 7. Check the whole article.
A final pass looks for problems that only show across sections, such as a pronoun that now points at the wrong product. One model reads the article and lists suspected problems, then a second call double-checks each one before anything is changed.
Example: one paragraph, before and after
Here is what that looks like on one paragraph from our test set:
Original: Subaru’s reputation needs no introduction; this legendary brand stands out with its keen focus on performance and quality. And when combined with their solid traction and efficient engine designs, these cars are particularly effective in slippery conditions. This means fewer chances of losing control down that steep hill.
Regenerated: Subaru’s name says enough on its own. This famed maker is known for putting performance and quality first. Pair that with dependable grip and engines that use fuel well, and the cars handle slick roads especially well. That lowers the risk of losing control while heading down a steep hill.
Same claims, same order of ideas, and almost no shared five-word sequences.
How we tested a watermark we cannot see
Since Anthropic’s detector is not public, we built a replica we could measure. This is the standard research approach: apply the real watermarking method with your own key, try to remove the watermark, then check whether your detector still finds it.
Our test setup
- The watermark: we used the official SynthID-Text implementation that ships in Hugging Face Transformers (a widely used open-source AI library), with our own secret key, on a small open model (Qwen2.5-1.5B-Instruct) running on a Mac mini.
- The test set: 12 articles generated with the watermark and 12 generated from the same prompts without it, each 500 to 1,650 tokens long and written in the same blog style as our content.
- The detectors: our main measure is the Bayesian detector described in the SynthID paper. It is trained on real watermarked and clean text and weighs each layer of the watermark separately, which makes it the strongest public way to detect SynthID-Text. We trained it on 1,000 watermarked/clean pairs generated on the Mac mini, then tested it on 150 pairs it had never seen: it caught every watermarked sample and flagged no clean ones.
- The cross-check: a simpler mean-score detector, calibrated on the clean articles. It reports a z-score, which says how far a text sits above what clean text normally produces. Clean text lands around 0, and a score of 3 or more happens by chance in only about 1 in 1,000 clean texts.
Two mistakes in this part are worth sharing:
- The Mac’s Apple GPU silently skipped the watermark. When we generated text on the Mac mini’s graphics processor (the fast route), the library produced perfectly normal text with no watermark at all, and no error message. Had we not checked, every “removal” result would have been fake. We generated everything on the slower main processor (the CPU) instead, at about 10 tokens per second, which is why generating the 2,000 training samples took many hours.
- Our first detector was three times too lenient. Our first z-score formula relied on a textbook assumption about how much clean text varies. When we measured real clean articles, they varied about nine times less, which made our z-scores about three times too low. A text we scored at 2 was really at about 6. For almost three weeks we believed the rewritten text was fully clean. After fixing it, the picture was less rosy, and we added the Bayesian detector so we would never rely on a single formula again.
The honest limit of this method
Our replica is not Anthropic’s watermark. Their key, their model and their exact settings are private, and their setup may be stronger than ours. What the replica can tell us is whether a technique destroys the kind of signal SynthID-Text creates. It cannot prove that the real watermark is gone.
The path, in data
Each round of testing changed the design. These were the turning points.
The first full article passed after 12 versions.
On our first test article, the novelty-gated pipeline went from 0 of 9 sections passing to 6 of 9, and finally 9 of 9 in version 12. Version 12 copied about 10% of the prose’s five-word sequences, against 95% for Humanize. Halfway through, our model provider retired the model we were using, so we had to recalibrate the judge on a new one. The new judge was much stricter, which led to a “materiality rule”: report only differences that change what a reader believes.
Rewritten prose looked clean, but locked parts leaked.
The first replica test showed the rewritten paragraphs scoring like clean text. But we had locked headings, tables and lists to protect them, and those parts alone still carried a strong watermark signal. On real articles, those locked parts are 8 to 26% of all tokens, enough to make a whole article detectable.
Rewriting structure created a new problem: lost search phrases.
We extended the pipeline to rewrite list items and table cells. Then an editorial read of the results showed something the metrics had missed: the title’s search phrase had been paraphrased out of the body in 3 of 3 articles where it appeared. The writer was doing exactly what we asked, which was to use new words. That is how pinned search phrases were born.
Our SEO and GEO audit found more value at risk.
We read the rewrites of 10 real production articles the way a search engine or an AI answer engine would, fixed what they lost, then rewrote and measured the same 10 articles again. This audit ran mid-project on an earlier set of 10 production articles, not the final 10 used in the comparison further down, which is why its FAQ count differs. “Before fixes” is the pipeline at that point; “after fixes” adds three changes: FAQ questions locked word for word, search phrases from the headings pinned in the body, and a phrasing check in the judge.
| Check (same 10 articles) | Before fixes | After fixes |
|---|---|---|
| FAQ questions kept word for word | 0 of 27 | 27 of 27 |
| Heading search phrases still used in the body | 80% | 89% |
| Sentences starting with a pronoun (original had 71) | 108 | 91 |
FAQ questions matter because they are written to match the way people actually search and ask AI assistants. Rewording them makes the answer harder to match to the question. So we locked them. Pronoun openers matter for AI answer engines, which often quote one passage on its own: “It tows 7,500 pounds” means little out of context, while the original named the truck.
We also turned down one fix. A rule forcing every paragraph to open with the product’s name would have cut pronoun openers further, but it risked odd, repetitive writing. We kept the article natural instead of optimizing one metric.
A judge you cannot trust is worse than no judge.
Our first whole-article reviewer flagged problems in all 11 clean articles we gave it: 36 false findings. So we built a test for the reviewers. We planted 24 real defects, such as swapped numbers and broken references, into copies of clean articles, and compared five reviewer designs on it. The winner screens fast and then verifies each finding with a second call. It caught 22 of 24 planted defects and raised no false alarms on the clean articles.
The checks caught our own bug.
When we added heading rewording, the whole-article judge flagged one test article where a list item had been replaced by a heading. The cause was a numbering collision between list blocks and heading blocks in our code. A judge that reads the full article catches things that per-section checks cannot.
The last two levers: headings, then the copy limit.
With the detector recalibrated, the remaining signal came from what we still copied word for word. We let the pipeline reword headings and table text lightly, with every search phrase pinned. Then we tightened the allowed share of copied five-word sequences in prose from 10% to 7.5%. Headings got their own, looser limit of 35%: they are mostly lists of nouns, and a light rewording cannot avoid keeping some five-word runs. Under the prose limit, heading groups took up to 15 attempts or failed outright.
The tighter prose limit cut the rewritten prose that the Bayesian detector still flags to 2 of 12 articles, and strongly flagged whole articles to 1 of 12. Whole articles still kept a flag in 7 of 12, mostly weak, because what remains there is the content we keep on purpose. With the later fixes described below, running on DeepSeek’s newest model release, the detector flags none of the rewritten prose and 4 of 12 whole articles.
Detector results, version by version
How to read the two tables below:
- Rewritten text means only the body text we rewrite. The whole article also includes the title, FAQ questions, numbers and product names that we keep word for word on purpose.
- Flagged uses a sensitive setting where the detector wrongly flags 1 in every 100 clean texts, so it errs on the side of finding a watermark. Strongly flagged means the detector is more than 50% confident the text is watermarked.
| Version (Bayesian detector) | Test articles scored | Prose flagged | Whole article flagged | Whole article, strongly flagged |
|---|---|---|---|---|
| Watermarked original | 12 | 12 of 12 | 12 of 12 | 12 of 12 |
| Ours, headings and tables locked (run 1) | 7 | 5 of 7 | 6 of 7 | 4 of 7 |
| Ours, headings and tables locked (run 2) | 8 | 3 of 8 | 6 of 8 | 3 of 8 |
| Ours, headings and tables lightly reworded | 11 | 3 of 11 | 6 of 11 | 2 of 11 |
| Ours, 7.5% copy limit | 12 | 2 of 12 | 7 of 12 | 1 of 12 |
| Ours, final pipeline (latest model) | 12 | 0 of 12 | 4 of 12 | 1 of 12 |
| Clean articles (no watermark) | 12 | 0 of 12 | 0 of 12 | 0 of 12 |
Earlier versions scored fewer than 12 articles because articles that failed our quality checks at that stage were not scored.
The z-score cross-check tells the same story. Flagged at a 1% false-positive rate; strongly flagged means a posterior above 0.5. Remember: around 0 is typical of clean text, and 3 or more happens by chance in only about 1 in 1,000 clean texts.
| Version (same articles as above) | Whole article, median z-score | Prose only, median z-score |
|---|---|---|
| Watermarked original | 21.8 | 18.2 |
| Ours, headings and tables locked | 4.2 to 5.5 (two runs) | 1.1 to 2.9 |
| Ours, headings and tables lightly reworded | 3.1 | 1.7 |
| Ours, 7.5% copy limit | 3.1 | 0.7 |
| Ours, final pipeline (latest model) | 2.2 | 1.2 |
| Clean articles (no watermark) | -1.6 to 2.3 | -1.8 to 1.8 |
Rows show medians across the test articles; the clean row shows the full range. Scores vary by about ±1 to 2 from run to run on whole articles.
How we compare with the most popular open-source remover
The best-known tool for this job on GitHub is guillaumemeyer/watermarks-remover, with about 24,000 stars. For text watermarks like SynthID, its default strategy has an AI model paraphrase the text, then a masked language model swaps about 20% of the words: it hides a word and fills the gap with its best guess. We ran its own code unmodified in two modes: the default, and paraphrase-only.
How we kept the comparison fair
To keep the comparison fair, we sent its paraphrase requests to the same model our pipeline uses, DeepSeek Flash, with the tool’s own prompt and temperature. The word swaps used its default model, roberta-large. This is not a criticism of the project. It is a general-purpose tool and was not designed for SEO articles with links, FAQs and target phrases.
We compared four outputs on the same inputs:
- our Humanize step, a readability pass that was never meant to remove watermarks, as a reference;
- our regeneration step;
- the open-source tool in its default mode;
- the open-source tool in paraphrase-only mode.
Content quality: 10 real production articles
| Humanize (readability pass) | Ours | Open-source, default | Open-source, paraphrase only | |
|---|---|---|---|---|
| Articles with every link intact | 10/10 | 10/10 | 0/10 | 5/10 |
| FAQ questions kept (of 23) | 23 | 23 | 18 | 21 |
| Heading search phrases kept (of 118) | 118 | 118 | 89 | 89 |
| Numbers missing or added | 0 | 0 | 1 added | 0 |
| Em dashes added | 0 | 0 | 21 | 21 |
| Major or critical meaning errors (88 sections) | 0 | 0 | 214 | 1 |
| Input wording still present (5-word sequences) | 94% | 13% | 29% | 35% |
| Longest unchanged run (average) | 617 words | 25 words | 61 words | 158 words |
The meaning errors were judged section by section by the same judge for every system. It is the same judge our pipeline uses internally, so our output has already passed it once, which favors us. Humanize, which barely changes anything, scored 0 on it, and the paraphrase-only mode scored 1. That puts the 214 in context: it is not judge noise. Nearly all of the open-source tool’s errors come from the word-swap step, which masks one in five content words and fills each gap with the model’s best guess, with no check on meaning.
What the errors looked like in practice
- Broken links. Links were altered or dropped in every article in the default mode and in 5 of 10 in paraphrase-only mode. For SEO, that means losing internal links and source citations.
- Swapped parts. In an installation guide, two components traded places, so the description of what sits inside the wall and what stands on the floor was reversed. Elsewhere, the part a reader should check for availability changed to a different part.
- Changed instructions. “Should be replaced” became “must be replaced”, turning advice into an obligation.
- Swapped functions. A description of filtering an inventory list became sorting it, which is a different feature.
- Rewritten FAQ questions. Questions written to match real searches were reworded, losing the exact phrasing people type.
- New AI tells. Both modes added em dashes and formal transitions, which is exactly what a humanizing step is supposed to remove.
Watermark: 12 watermarked test articles
| Flagged by the trained Bayesian detector (at 1% false-positive rate) | Ours | Open-source, default | Open-source, paraphrase only |
|---|---|---|---|
| Prose only | 0 of 12 | 6 of 12 | 5 of 12 |
| Whole article | 4 of 12 | 6 of 12 | 6 of 12 |
| Whole article, strongly flagged | 1 of 12 | 4 of 12 | 4 of 12 |
Most flags are weak: they sit just above the detector’s very low threshold (a posterior of 0.00013); “strongly flagged” means a posterior above 0.5. A 1% false-positive rate is a common research setting, but a lenient one: one clean text in a hundred gets flagged. A detector that people act on would likely use a much stricter threshold and flag fewer texts than ours did, so this is a harsher test than real use. Anthropic has not published its threshold.
On the rewritten prose, our step leaves the least detectable signal: the detector flags none of the 12 test articles’ prose. The one whole article that stays strongly flagged is one that every method left strongly flagged, both open-source modes included; it is packed with spec tables, product names and figures that a faithful rewrite has to keep.
An earlier stubborn article of the same kind, full of figures such as “197.5 in x 76 in”, exposed a bug and a gap in our pipeline. Tables written without a pipe at the start of each row, which Markdown allows, were never rewritten. And two- and three-word table labels such as “Model Year Range” were always kept as they were. We fixed both: such tables are now rewritten, and short labels get a light rewording (“Installation Complexity” became “Installation Difficulty”), while product, model, brand and venue names stay exact.
On whole articles, ours keeps a trace in 4 of 12 test articles, against 6 of 12 for both open-source modes. We know where ours comes from: the H1 title, the FAQ questions, numbers, product names and label cells. Those are the parts we keep word for word on purpose, because they carry the article’s SEO value.
Why we chose value over a perfect score
We could push the whole-article score lower by rewording the title, rewriting FAQ questions and paraphrasing search phrases. We decided not to. The article exists to rank, to be quoted by AI answer engines and to help a reader make a decision. Every one of those jobs depends on the exact things a blunt watermark remover destroys:
- Links carry internal linking and citations.
- FAQ questions match how people search and how AI assistants pull answers.
- Search phrases in headings tell search engines what the section is about.
- Exact numbers and conditions are what readers act on and what AI engines cite.
A watermark-free article with a broken link, a reversed recommendation or a missing search phrase is worth less than the article we started with. So our rule became: the watermark must go from everything we rewrite, and nothing of value may be lost to get there. Then measure both.
What it costs
- Time: about 1.5 to 3 minutes per article on DeepSeek’s latest model. Heading and table rewording and the stricter limits had pushed it to 5-12 minutes, because lists, tables and headings waited for the prose to finish. Now they run at the same time as the prose, with up to 12 model calls in parallel.
- Reliability: in the final batch, 9 of 10 real articles passed every check on the first try. The tenth exposed one last bug: a bold lead-in at the start of a list item was hidden from the writer, so the writer kept repeating it. After the fix, that article passed too. If an article fails, the step fails and the pipeline retries it rather than shipping a watermarked article.
- Cost: tightening the copy limit from 10% to 7.5% made each article about 20% more expensive, mostly from extra rewrite attempts.
- Length: rewritten articles come out around 10% longer than the input.
What is still unsolved
- We cannot prove removal of the real watermark. Our evidence comes from a replica with our own key. It shows the technique works on SynthID-style signals, not that Anthropic’s detector will agree. When the detector becomes available, we will re-test.
- Whole articles keep a weak trace. The parts we protect for SEO still carry a little signal. Each piece is weak, and whether a real detector would flag it depends on thresholds we cannot see.
- The samples are small. 12 test articles and 10 production articles, with noticeable run-to-run variation. One test article came out of the generator with a paragraph in Chinese; our pipeline handles it now, but it took several fixes and three times the normal run time.
- More small wording slips. A full rewrite draws more minor findings than a light edit: 21 across 88 sections for ours, 15 for Humanize. These are wording and style notes, not meaning errors, but they are real.
Lessons if you are building something similar
- Measure wording change with a hard number. “Keep the facts” plus “read better” produces a copy editor. A five-word-sequence limit enforced in code is what forced real rewriting.
- Rewrite small pieces and lock the rest in code. Section-sized rewrites with numbers, links and phrases enforced by code were far more reliable than asking one prompt to rewrite an article and promise to keep everything.
- Test your test. A silent GPU fallback and a miscalibrated formula would each have given us a false success. Check that your watermark is really applied, and calibrate your detector on clean text before trusting any result.
- Calibrate judges with planted errors. A reviewer that flags every article is noise. Plant known defects, measure catches and false alarms, then choose the design.
- Audit for SEO and GEO, not just the watermark score. Metrics told us the rewrite was faithful. Only a reader-and-search audit showed FAQ questions and search phrases disappearing.
- Decide what you will not trade. The last step of watermark score would have cost FAQ questions, the title and search phrases. That trade was not worth it to us.
Conclusion: removing the watermark is the easy part
Removing a watermark on its own is not hard. Rewrite the text with a model that does not watermark, and most of the signal disappears. Even the open-source tool we tested removes most of it from the prose. The hard part is doing it without breaking what the article is for: correct language, exact data, the links, FAQ questions and search phrases that search engines and AI answer engines rely on, and a text that reads at least as well as before.
That combination is what took us six weeks. A watermark-free article with broken links, reversed facts or lost search phrases is worth less than the watermarked one it replaced. Removing the watermark just for the sake of removing it, while adding errors and damaging the article, makes no sense. Our rule stays the same: the watermark has to go from everything we rewrite, and nothing of value may be lost on the way.
How we ran the tests (for the curious)
- Production comparison: 10 recent articles, each with the exact text before and after our Humanize step, recovered from our logs. Our step and the open-source tool started from the same “before” text.
- Content metrics: link, FAQ, heading-phrase, number and em-dash counts computed in code. “Input wording still present” is the share of the input’s five-word sequences found in the output.
- Meaning errors: each article split at its H2 headings, and every section judged by the same calibrated judge for every system (88 sections per system).
- Watermark replica: official SynthID-Text implementation in Hugging Face Transformers, our own key, Qwen2.5-1.5B-Instruct, CPU generation.
- Detectors: the main measure is a Bayesian detector trained on 1,000 watermarked/clean pairs (700 train, 150 validation, 150 test), with its threshold set at a 1% false-positive rate on validation data. The cross-check is a mean-score detector (z-score) calibrated on the 12 clean articles.
- Current pipeline: the 10-article comparison uses the current pipeline, including the table fix, the light rewording of short labels and two writer rules added after an earlier run (a condition on one list item stays on that item; general advice stays general). Every venue, band and product name in their tables stayed exact.
- Model versions: DeepSeek released a new version of its Flash model near the end of the project (October 2026). Our final results, both the 10 real articles and the 12 test articles, use it, and every system was judged by the same judge on it. The open-source tool’s outputs were generated a few days earlier, with its paraphrase step on the previous release.
- Open-source baseline: guillaumemeyer/watermarks-remover, run with its own code in its default and paraphrase-only modes. Paraphrasing used the same model as our pipeline; the word swaps used the tool’s default roberta-large. Its default mode is the paraphrase followed by the word swaps, so we applied its word-swap step to the paraphrase-only output. With the earlier judge, a first run with the smaller roberta-base made 270 major or critical meaning errors against 219 for roberta-large, kept 9 FAQ questions instead of 18, and removed slightly more of the watermark.
- Sources on the watermark: Anthropic, “How Claude’s text watermarking works” and “How Claude marks AI-generated content”; Dathathri et al., “Scalable watermarking for identifying large language model outputs”, Nature 634 (2024).
