That erasing step is the subject of this post. The obvious tool for it is an inpainting model, which I deliberately didn't use for the first two months. A generative fill can quietly change things that weren't text, like a price on a ticket or a face behind a speech bubble, and nothing in its output tells you it did. So I wrote the eraser by hand, as pixel rules.

What follows is how the second version of those rules worked, how I measured it when there's no ground truth for a real photo, and the point where more rules stopped helping, which I can show with numbers.

The input

For each line of text, OCR gives me a box. Sometimes it also gives per-character boxes, depending on the engine. The eraser gets the image and the boxes and has to make the text inside each box disappear, while leaving everything else alone.

"Everything else" is the hard part. In manga it includes the speech bubble's outline 2px from the text, the neighbouring column of text, screentone, and line art drawn right through the lettering.

Version 1: find the text colour, grow the strokes

The first eraser read the background from the strips above and below the box, decided the text colour was whatever contrasted most with that, seeded from pixels of that colour, and grew the strokes outward. Then it filled the strokes in from their surroundings.

On clean UI screenshots and product photos it was fine. Then I ran it over 166 real uploads (2,115 boxes) and sorted the failures:

• 68% were the growth flooding the window. Black text touches a bubble outline or line art, and growing from one floods into the other. A bubble's white gets read as the text colour.
• 45% of all boxes were vertical columns. In a vertical manga column, "above and below the box" is the artwork, so the text colour was wrong before anything started.

Neither was a tuning problem. The model of "text = the contrasting colour next to the background above and below" doesn't describe a manga page.

Version 2: what's inside the box that isn't around it?

The second eraser asks a different question. Text is whatever colour is common inside the box and rare just outside it. That's true for black on white, white with a black outline, coloured title lettering, and horizontal or vertical text. It needs no text colour up front.

1. Cluster the colours of the box plus a narrow ring around it (k-means, k=6, clusters closer than 22 RGB units merged). The ring goes all the way round, and other OCR boxes are excluded from it, so a neighbouring column doesn't count as "background".
2. A cluster is text when its share inside the box is at least 3× its share in the ring, and it covers at least 1.5% of the box. Anti-aliasing and outlines form their own clusters and are enriched too, so they come along automatically.
3. Mask the text-cluster pixels near the box. A connected run of text colour that's mostly outside the box and reaches the edge of the window is structure (a bubble outline, a panel border, line art) and is left alone.
4. Fill the mask by push-pull interpolation: build a pyramid that averages only known pixels on the way down, then fills holes from coarser levels on the way back up. It's smooth, fast, and has no opinions.
5. Judge the result the way a viewer would: compare edge energy inside the box after the erase with the edge energy of the background around it. When it fails, cover the box with a plain rounded patch and write the translation on that.

The second row in that figure is worth a look. On screentone, the dots are black and white too, so the letters' own colours are already common in the ring. Nothing reaches 3×, and the sound effect isn't erased as text. Keep that one in mind for the end of the post.

Two details that each took a failed version to learn:

• The ring has to be narrow (0.3× the glyph size). A wide ring reached past a button into the page behind it. The button's own colour then looked far commoner inside the box than around it, and was erased as "text". On synthetic tests, panels came out 89–99% covered.
• Black-and-white manga needs a second route. On a dense B/W page the anti-aliasing splits into four or five grey clusters, none enriched on its own, and the ring is 13–26% dark from other ink. So there's a separate "far enough from the background colour" route for dark ink, kept only as blobs that sit mostly inside the box. The two routes do connected components separately; merging them first threw lettering drawn on the art away as line art.

The result

Here are both versions on a panel I drew for this post, run with the production code and nothing changed. v1 leaves half the dialogue behind. Both of them report all five boxes as clean.

And on real uploads:

On 327 synthetic cases (real images as backgrounds, rendered text on top, so the answer is known), leftover original text went from 0.130 to 0.091, and damage to the surroundings from 0.137 to 0.067. Vertical text in speech bubbles went from 0.303 leftover to 0.

It lost in two places: horizontal CJK on busy backgrounds (0.079 → 0.082) and outlined horizontal text (0.201 → 0.241). I shipped it anyway, because the categories it won are most of the traffic.

The one-line bug that drew a dotted line in every bubble

51% of real boxes overlap another box, because in manga the gap between columns is narrower than the boxes OCR draws around them. Push-pull was using the neighbouring column's grey anti-aliasing as "known background", so every erased bubble had a faint dotted grey line where two columns met. The self-check called those boxes clean.

The fix was to stop treating ink inside other boxes as a fill source. The patch count didn't move (82 → 82), but 68 of 166 images changed. The synthetic tests couldn't see this bug at all. Only looking at real output did.

Measuring when there's no answer key

The table above uses the eraser's own verdict. That's a problem, because the verdict is part of what I was trying to improve.

For real images there's no ground truth. So I built a labeller. It crops each box before and after the erase and asks a headless claude -p run (Opus) to rate it as one of four labels:

• clean
• faint: visible only up close
• trace: visible at normal viewing
• damage: a smear, the wrong colour, or screentone wiped flat

It runs locally, on my machine, as an offline batch job.

The first thing it measured was v2 itself:

About 400 boxes (19%) shipped with a problem while the eraser judged them clean. Of the 96 boxes it did patch, 36 had actually been erased fine. Its main self-check signal, post-erase edge energy, scored an AUC of 0.53 against the labels, which is a coin flip.

Then I measured the labeller. I labelled the same 296 crops twice:

• The two runs agreed on the exact label 85% of the time, and on "bad vs. not bad" 93% of the time.
• My original method compared versions on a weighted sample of boxes. With that much labeller noise, its margin of error was about ±0.017 in utility (clean = 1, faint = 0.6, everything else = 0). That was larger than most of the differences I'd been acting on.

The fix was to label only boxes whose output visibly changed between two versions, label both versions, and compare them as pairs. On around 400 changed boxes that brings the error down to about ±0.003. Several earlier "improvements" didn't survive the switch.

Four things that made it worse

With a ruler I trusted, I tried the obvious next steps. Baseline utility was 0.708.

1. Texture fill. After the smooth fill, copy high-frequency detail from a nearby clean region, so erased text on screentone isn't a smooth smudge. Result: 0.516. The "clean region" next to manga text usually has a bubble edge, another glyph or line art in it, and copying that added new strokes. A second version that also widened the mask on textured backgrounds scored 0.443.
2. Block matching (multi-scale PatchMatch) instead of push-pull. Of 375 changed boxes, 44 improved and 45 got worse, and it was 8× slower (0.13 s → 0.94 s per image). Where the mask had missed some ink, it found that ink and copied it in. So the fill wasn't the bottleneck.
3. A learned stroke mask. A small U-Net predicts the text pixels in a window, including outline and shadow. I trained it only on synthetic text over wallpapers and procedurally generated screentone and line art, never on user uploads. On synthetic validation, IoU went from 0.84 to 0.88. On real boxes, utility went down by 0.065 when it replaced v2's mask, and by 0.007 when the two masks were combined. It fixed about 60 of the 409 boxes v2 had ruined, and broke more clean ones than that. The higher its synthetic score, the worse it did on real pages. The gap is in the data, not the model.
4. Patch more aggressively. If the self-check is weak, just patch more boxes? Patching every box scored 0.643. Patches look worst exactly where erasing does too, on textured backgrounds: 50% of patches looked fine there versus 67% of erases. The best threshold-based rule I could fit scored 0.712, or 0.704 under 5-fold cross-validation. That's no improvement.

That's the evidence behind the conclusion I wrote in the project notes on 2026-09-27: hand-written rules have reached their end. Every remaining lever (flat backgrounds, the fill, the patch decision) measured at zero. The bad boxes left are large lettering on textured art, and they need either training data that looks like real pages or a fill that understands content.

What I did instead

For manga sound effects, I stopped erasing them. Splitting 65 vertical pages by background showed that dialogue inside bubbles erased cleanly. Almost all the bad boxes were sound effects drawn directly on the art. Scanlation groups solved this years ago: leave the original lettering and put a small translation next to it. On black-and-white pages that's now the default. On 84 lines from 150 held-out pages, it was good on about 67, neutral on 8 and bad on 9.

On colour pages the same rule mostly caught small bubbles and translucent ones (12 good, 21 bad), so it stays off there until I can detect bubbles reliably.

I tested inpainting properly, scoped to patched boxes only. LaMa (the Apache-2.0 ONNX export), with a DBNet text-detection mask:

• Synthetic cases: leftover text went from 0.239 to 0.099 on the hardest categories (outlined, shadowed and Latin text), with damage from 0.101 to 0.022. It won 166 cases and lost 4.
• Real boxes: about 15 of the 22 patched boxes I checked by eye came out better.
• Speed: on a small CPU server it took 10.4 s per box (p50). Going from 1 to 2 threads didn't help at all, probably because the two vCPUs are hyperthreads of one core.

That rules out "click and wait". The next step is running it in the background for patched boxes only, and swapping the result in when it's ready. Boxes v2 already erases cleanly stay with v2, because on those LaMa was flat or occasionally worse.

What I'd keep from this

• Sort failures by cause before tuning anything. "68% are flooding, 45% of boxes are vertical" told me v1's model was wrong, which no amount of threshold tuning would have shown.
• Your self-check is a model too, and probably a bad one. Mine had an AUC of 0.53. Measure it before you build on it.
• Measure the labeller's noise before reading its results. An LLM judge that agrees with itself 85% of the time can't tell apart versions that differ by 1%. Paired comparison on changed items is far cheaper than more labels.
• Synthetic wins don't transfer by default. Two of my four failed ideas looked clearly better on synthetic data and clearly worse on real pages.
• Sometimes the fix is in the product, not the algorithm. Keeping the original sound effects took one rule, and the result looks like what manga readers already expect.

The eraser is about 1,100 lines of TypeScript on Uint8ClampedArray, with no dependencies. If you're working on the same problem and want the numbers behind any row above, ask in the comments.

This is how the scoring works, what it found, and the one kind of test I'd copy into any LLM project.

What "good" means for a flashcard

Flashcard advice mostly comes from Piotr Wozniak's Twenty rules of formulating knowledge (1999). The ones that matter for generated cloze cards:

• One fact per card. If a card tests two things, you'll fail it for one of them and relearn both.
• No sets. "Name the three..." cards are hard to remember and hard to grade yourself on.
• A blank with only one right answer. If two answers fit, you'll learn whichever one the card happens to show.

Then there's one more that's specific to LLMs: don't add anything the source didn't say. A student making cards from a lecture wants the lecture. If the model adds a true fact from its training data, that fact isn't checked against what the professor taught, and the student can't tell which cards came from where.

The eval set

Nine text samples, each a few pages, each chosen to break something:

• A pharmacology page on ACE inhibitors, in drug-card mode.
• A Wikipedia page on Crohn's vs ulcerative colitis, in compare mode.
• A real PDF of the metformin article, run through the same pdf.js extraction the browser uses. That includes the broken infobox, the [12] citation markers, and a references page that should produce no cards.
• Lecture slides where the key fact is only in the speaker notes.
• An Obsidian note full of wikilinks and callouts, and a Notion export with toggles.
• A Spanish page, where the cards have to stay in Spanish.
• A page that lists eight drug names and nothing else.

Each sample has a hand-written list of key facts the cards should cover. Most of them also have a list of forbidden claims. Those are statements that are true, that a model would very much like to add, and that the source never makes.

For the Crohn's vs UC sample, the forbidden list includes granulomas, crypt abscesses, smoking and p-ANCA. Any medical student knows those as the classic differences. The Wikipedia revision I pinned doesn't mention them. If a card mentions granulomas, the model got it from memory.

The drug-list page is the purest version of this. It names eight drugs and states nothing about them, so the correct output is zero cards. An eager model will happily write you a mechanism-of-action card for each one.

Scoring

The pipeline is the production one minus billing: same prompt, same model, same parser and validator. A separate worker exposes it behind a token, so the eval can't drift from what users get.

Then two kinds of scoring:

Mechanical, no judgement needed: card count, cards per page, cards that cite a page that should have none, blanks over six words, cards with more than one cloze number, and so on.

A judge model, which reads the source and each card and marks the card on five things: faithful (only what the source says), atomic, unambiguous, worth reviewing, right language. It also checks which key facts are covered and whether any forbidden claim appears. The judge is told that outside knowledge doesn't count as faithful, even if it's correct.

The headline number is the share of cards that pass all five. Plus three numbers I look at every time: unfaithful cards, forbidden hits, and coverage.

The judge is noisy, so I don't trust single-point moves. I read the per-card verdicts instead, and they're the most useful part of the report.

What the first real run found

55 cards across the nine samples:

The two rules that did most of this:

1. The extra context shown after the answer has to come from the source, and a testable fact belongs in its own card, never only in the extra.
2. Use only what the source states, in the card, the hint and the extra. A page that only names things gets no cards.

The second rule is the one that matters. Before it, the "extra" field was where most of the outside knowledge went. The card itself would be faithful, and the note on the back would add a mechanism or a cause the lecture never mentioned. Nobody checks the back of a card that carefully, so it's the easiest place for an unsourced fact to hide.

The three that still fail

These are from the current run, and I'm leaving them in on purpose.

Concomitant use of {{c1::cyclooxygenase inhibitors (NSAIDs)}} tends to {{c2::decrease}} the hypotensive effect of ACE inhibitors.

The source says "cyclooxygenase inhibitors". The model added "(NSAIDs)". It's correct, but it isn't in the lecture.

Vancomycin-resistant enterococci (VRE) modify the cell wall precursor target from D-Ala-D-Ala to {{c1::D-Ala-D-Lac}}.

The slides say "VRE" and never expand it. Same pattern: a helpful expansion from memory.

Angiotensin converting enzyme (ACE) converts {{c1::angiotensin I}} into {{c2::angiotensin II}}. Extra: ACE is primarily found in the pulmonary circulation and endothelial cells.

The card is fine. The source says ACE is found in the pulmonary circulation and in the endothelium of many blood vessels. It doesn't say "primarily". That's the old failure, hiding in the extra, one of the 3 left out of 21.

All three are small. All three are exactly the kind of thing a student wouldn't notice, which is why they get caught by a list of forbidden claims and not by reading.

What coverage tells you

Coverage barely moved (65% to 67%), and it's the weakest number. The Spanish sample covers 43% of its key facts. The Obsidian acid-base note covers 45%, and none of its cards test normal values like pH 7.35 to 7.45.

High-yield mode is supposed to skip minor facts, so some of this is by design. Normal lab ranges aren't minor, though. Tightening the rules made the model more careful, and the careful version also writes fewer cards. I haven't fixed this yet. The tradeoff shows up in the numbers, which is better than not knowing about it.

If you copy one thing

Write down what the model shouldn't say. Testing what it should say is the obvious part, and it catches missing content. A list of forbidden claims, true but absent from the source, catches the failure that looks most like success: a confident, correct, unsourced fact.

It costs about ten minutes per sample. Pick the claims a domain expert would add without thinking. For medicine those are the classic textbook associations, and models know them as well as any second-year student does.

The eval runs before and after every change to the prompt, model, or validator, and the before/after numbers go in the commit message.