HOW CHESSDIARY READS MESSY HANDWRITING (AND WHEN IT CAN'T)

By Venugopal Ramamoorthy — tournament chess player and creator of ChessDiary

Every scoresheet OCR system eventually meets a page it can't fully read — a rushed time-scramble scrawl, a smudged pencil line, or a "1" that could just as easily be a "7". ChessDiary's parser, built on Google's Gemini model, doesn't just guess silently in these cases. It flags exactly how confident it is in what it read, and where.

Understanding what that confidence signal means — and what to do when it's low — turns a frustrating half-parsed game into a two-minute fix instead of a scoresheet you give up on.

1. WHAT THE PARSER ACTUALLY DOES WITH UNCLEAR WRITING

When Gemini reads a scoresheet image, it isn't matching letterforms against a fixed template the way older OCR engines do — it's reasoning about the page the way a person would: this looks like a knight move because it's followed by a file letter, this squiggle is probably a "5" because the column above and below it are also numbers. That context-based reading is what lets it get through genuinely messy handwriting that a strict character-recognition system would reject outright. It's also why an isolated unclear character rarely breaks the whole game — the parser can often infer it from the notation pattern around it.

2. WHY AI READING BEATS TRADITIONAL OCR HERE

Traditional OCR is built for printed text — clean, consistent letterforms in a known font. Handwritten chess notation breaks nearly every assumption that relies on: everyone's handwriting is different, players write faster as the clock runs down, and a symbol like x carries specific chess meaning a generic text reader has no way to know. Gemini's model was chosen specifically because it reasons about the page in context rather than matching shapes, which is what makes it usable on genuinely rough club-level scoresheets.

3. HOW CONFIDENCE GETS FLAGGED

Every parsed scoresheet comes back with a confidence level — high, medium, or low — along with any specific notes about what was hard to read. A high-confidence parse usually means the handwriting was clean enough that there's little to double-check. A low-confidence result doesn't mean the parse failed; it means the model made its best guess on one or more moves and is telling you exactly where to look before you trust the result.

4. THE MOST COMMON MISREADS (AND WHY)

• e vs. a — a hastily written lowercase e can close up into something that looks like an a, especially under time pressure. Both are common file letters, so a misread here often looks plausible at a glance, which is exactly why it's worth checking.
• 1 vs. 7 — a "1" with a small serif at the top and a fast "7" can look nearly identical in cursive scoresheet writing. This is the single most common numeric misread across handwritten sheets.
• Missing N — knight moves use N, and a rushed capital N can look like a stray mark that gets dropped entirely, turning "Nf3" into just "f3" — a pawn move that isn't legal on that square yet.
• Capture and check marks — the x for a capture and + for check are small and easy to lose in fast handwriting; when they're missing, the parser infers them from the position where it can, but a genuinely ambiguous case gets flagged.

5. WHAT TO DO WITH A LOW-CONFIDENCE PARSE

Every parsed game — regardless of confidence — is shown to you as a move list before it's saved, not saved automatically in the background. Step through it against the actual scoresheet (or your memory of the game) and correct anything that looks wrong before confirming. This review step exists specifically because no OCR system, however good, should be trusted blindly with a record you might want to study seriously later.

6. WHEN IT'S WORTH RE-PHOTOGRAPHING INSTEAD OF CORRECTING

If more than a handful of moves come back uncertain, it's usually faster to retake the photo with better lighting than to manually fix each one — see our guide on photographing a scoresheet for what actually improves parse accuracy. A five-second reshoot with even lighting often turns a low-confidence parse into a clean one.

7. A WORKED EXAMPLE: FIXING A LOW-CONFIDENCE GAME

Say a 40-move tournament game comes back flagged as low confidence, with three specific moves called out: move 14 Black, move 27 White, and move 33 Black. Here's the actual review process, not just the general advice:

1. Open the three flagged moves first, not the whole game top to bottom. The model already told you exactly where it's uncertain — reviewing 37 moves it was confident about wastes time the flags were designed to save.
2. Check each flagged move against the position, not just the handwriting. If move 14 Black is unclear between Nd7 and Nbd7, look at whether a second knight could actually reach d7 at that point in the game — if not, the disambiguation letter is a misread, not a real option, and the simpler reading is correct.
3. For a genuinely illegible move (not just ambiguous between two legal options), check your own memory of the game or, if you played with an opponent, ask them to confirm — a tournament scoresheet is a shared record, and your opponent's copy of the same game is often the fastest way to resolve a true illegible gap.
4. Confirm the whole game only after the three flags are resolved, not before. ChessDiary won't silently save a flagged game without your review, but it also won't stop you from confirming it with the uncertain moves still wrong if you skip past them.

In practice, the vast majority of low-confidence flags resolve in favor of the simpler, more common reading once you check it against the actual position — the model is usually right even when it says it isn't sure, which is exactly why it's worth checking rather than re-photographing by default.