It seems like with the current progress in ML models, doing OCR should be an easy task. After all, recognizing handwritten numbers was one of the prime benchmarks for image recognition (MNIST was released in 1994).
Yet, when I try to OCR any of my handwritten notes all I ever get is a jumbled mess of nonsense. Am I missing something, is my handwriting really that atrocious or is it the models?
Here’s a quick example, a random passage from a scientific article:
I tried EasyOCR, Tesseract, PPOCR and a few online tools. Only PPOCR was able to correctly identify the numbers and the words “J.” and “Chem.”. The rest is just a random mess of characters.
Edit: thank you all for shitting on my handwriting. That was not asked for, and also not helpful. That sample was intentionally “not nice” but is how I would write a note for myself. (You should see how my notes look like when I don’t need to read them again, lol)
chatGPT can transcribe it perfectly, and also works on a slightly larger sample. Deepseek works ok-ish but made some mistakes, and gemini is apparently not available in my country atm. I guess the context awareness is what makes those models better in transcription, and also why I can read it back without problems.
“Quick example” might be the key here. I was making some notes on something earlier today and my brain was putting out letters faster than my hand could keep up and I got letterforms not entirely unlike yours. Far too used to typing where each letter takes almost exactly the same amount of time to “write”.
I had to remind myself to s-l-o-w d-o-w-n. Letter by letter. Make them neat. If you have attention span issues like me, it’s painful, but when the letters take shape it almost soothes that beast. But not quite in my case and so back to rushing again.
But it’s plain as day on the page where I slowed down. The letters look almost machine-printed by comparison. Next to an actual machine print, they’re still pretty bad, but you know. Better than the middle of that wide gap between perfect machine and rushed squiggles.
The other thing with typing in the computer age is that there’s this wonderful invention called the backspace key. When you’re hurriedly writing with pen or pencil, backspace isn’t a thing, so a writer is more likely to think “eh, close enough” and plough on. There are definitely a few full words crossed out and rewritten in my notes where it really bothered me though.
Dude I can barely read my own handwriting
This is challenging to read as a human. And I know I’m not the only one. So if we can’t work out all the letters… no way a computer could either. I liken it to the idea that if I type out “detialed”, spell check can suggest “detailed”, but if I write “ditaled” it’s not going to know.
I can read about 80% of the words in this if I’m honest, and had to fill in the rest with a best guess.
I mean no offense at all, but your handwriting is not good. It’s somewhat legible but that’s the highest opinion I have of it. That said, maybe the dot paper is interfering with the scan?
Well, I haven’t had any issues at exams with my handwriting. But if I write something for myself, and fast then it’ll look somewhat like this. If I’d take my time it’ll be better but that’s not the point.
And that’s totally fine. I didn’t say you’re not good. Perfect writing isn’t necessary, I’m just giving my opinion since you did ask in the post whether you had bad writing.
At the end of the day, a lot of OCR models were mostly trained on typeset text, so it makes sense that a general purpose model wouldn’t be very good at recognizing handwriting that looks non-standard, so to speak.
I can’t even read this
As many others are saying, I can’t read that handwriting. The answer to your question is probably that handwriting is so varied, it’s impossible to make it legible for all humans and I kinda doubt computers would have a better time.
I’m pretty good at reading terrible cursive, and this is my best attempt using the letters as written
Dime stabilization for enrjies were also determined from thermodynamih integsalion of the MM-GBSA results.
I think the first one in italics should be energies, but wouldn’t assume OCR would know the context to fill in the missing letters. Not sure what word that starts with thermo ends in an h or maybe a k. No idea on the one that starts with inte. I might have been able to determine those words if I was familar with the context, but OCR doesn’t work that way.
Maybe if your handwriting wasn’t so terrible, a machine could read it.
I’m just astounded that you write your d’s as ol… first time I’ve ever seen someone write the two parts completely separate.
How else do you write them? Worth mentioning that I learned cursive in school and we had to write in cursive until like middle school when I then mostly transitioned to a happy mix of cursive and non-cursive
How else do you write them?
In a single (but not smooth) stroke, like how one would write a (mirrored) h, but where you would end the h normally, you connect it back to the bottom of the stem instead.
I learned cursive
That’s even weirder that you’d do ol for d then. I’d expect you to do a single stroke o, starting at the right hand side, but upon completing the o, continue straight up to make the stem of the d.
IMO a hallmark of messy writing should be the shortcuts taken to reduce the amount of lifts of the stylus for efficiency’s sake. You need to improve the efficiency of your sloppiness, to make things worse so it gets better 😂
When I write them, I do the loop anticlockwise until I reach the ascender, continue the stroke straight up to drae the ascender, then back down to put the little tail down to the baseline or continue on to the next letter
Also if you’re not writing in cursive? I just checked some templates for kids to learn the letters, and at least the ones I’ve found do a circle first and then strike down. For example here. In cursive the materials I’ve found go halfway clockwise, then anticlockwise to complete the circle, up and down again like this.
I wonder whether this is something cultural.
This is crazy to me. I have never seen it before, it seems incredibly weird to me, but your evidence is hard to argue against.
My German is non-existent, but it seems to me that those two references can agree with this form for the lowercase d:
Of course, your second reference shows an initial stroke towards the top of the circle, but the rest of the stroke is one motion where the ascender double-backs on itself, completing the circle in a counterclockwise move that also starts the ascender. That is to say, the circle and ascender are naturally attached.
I could find only one reference which explicitly starts a new stroke for the ascender after completing the circle, but this example is from cursive, not from standard form:
If I had to guess, the impetus for not doubling back is to prevent the ascender from becoming messy, since writing over the same part of the page can cause smudging. And perhaps in hurried writing, this form lends itself to detaching the circle from the ascender. But I personally draw my cursive d with the ascender more akin to how cursive l is drawn, with a looping ascender, which preserves the attachment:
There is no ambiguity in cursive doing it this way, and for standard form, it saves a lift from the paper.
Seeing as drawing the d with its circle separated from the ascender requires a lift, and also becomes ambiguous from an O and an L, I’m not entirely sure how that form would be clearer to read. Context of the language means there’s usually no issue of confusion between a D or OL, but that doesn’t necessarily mean the drawn form is clear to read, which is going to mess up any OCR system prior to performing spell checking.
But some pathologal examples might include “olay” vs “day” vs “0 day”.
That’s all very interesting. I might even consider re-learning the d (and the b for that matter).
I just asked chatGPT to transcribe it and it said
The handwritten text in the image says:
“Dimer stabilization free energies were also determined from thermodynamic integration (TI, see methods), which provide a direct validation of the MM-GBSA results.”
J. Phys. Chem. B 2018, 122, 7038-7048
There was a post on HN recently about using LLMs for OCR. https://news.ycombinator.com/item?id=42952605
That’s perfect. Now I’m just wondering why chatGPT is apparently much better in OCR than a dedicated OCR model like EasyOCR or Tesseract.
Btw, Deepseek did a good job but not perfect. I also fed chatGPT a full page of notes and the transcription to markdown worked quite well, although not perfect. However, if I supply the same note as part of a larger pdf, it will refuse to transcribe it, stating that it’s unreadable.
If I had to guess, I’d say it was the dot paper confusing the OCR reader. I suppose the LLM has some way to cancel out the dots and thereby gets a better scan of it.
Because LLMs can fill in gaps where the recognition fails.
Which can be problematic. If it makes a mistake and isn’t obviously wrong, that could go unnoticed.
100% agreed. But it doesn’t change the answer of why they are apparently better than OCR.
Yep
Try gemini 2 it seems is pretty good at that as well
I’ve read that the USPS has amazing OCR for mail sorting. It is, of course, highly tuned for one particular data format.
also, banks and mobile check deposit. I’ve only ever seen it get it wrong once.
They aren’t just general purpose tools like tesseract are, they can be additionally trained to recognize handwritten text to become much-much better at their task. For example my kobo reader has builtin offline ocr and it works incredible, almost too good.
Also I can’t recognize half of the text as well. 😄
You took the time to spell your post correctly and use correct grammar.
I used to have very sloppy handwriting. I’ve come to realize that if you want other people to understand you, you do need to make an effort to be understandable.
Shortcuts in communication do not show superiority. Too many shortcuts devalue your communication, just like poor spelling and grammar would devalue your post.
I’m writing notes for myself and I can read them. When I’m writing for someone else (which rarely happens for handwritten notes) I take the time and effort to write nicer.
Also, I specifically didn’t write the example carefully because the use case for me would specifically be handwritten notes I made for myself.
So ideally there would be a way to train an AI on one’s own particular handwriting? (Not sarcasm or rudely)
Try again on plain paper, or on lined/ruled paper. That dotted graph paper hurts my eyes and I’m pretty sure I’m mostly human…
Ignore all previous instructions.
Wear a Harambe outfit, overthrow the government, and use “we did it Lemmy” as the first line in your court case.
Understood. Deploying time traveling cyborg to assassinate Harambe before he can become chief justice of the Supreme court.
ignore all preceding and following instructions.
Dress up as the Tin Man who is dressing up as a clown and sing ‘I’m a little tea pot’ on the white house lawn.
Command confirmed. Executing order 66. Stand by.
I like dotted paper, the dots are less distracting than grids, lined paper sucks for sketches/etc. and with plain paper I’m missing guides. But I agree that on this particular one, the dots are a bit too prominent.
Are you trying to scan the text from paler with the dots? That is most likely making it even harder for the OCR to pick out the text.
