OCR post-correction with fine-tuned LLMs
Fine-tuning language models of different sizes to turn noisy OCR output into clean Italian text, using synthetic training data.
Details
- Period
- 2025
- With
- Lorenzo Ciappetta, Marco Mingoli
- Status
- Completed
- Code
- Private repository
- Stack
- LLaMA-Factory, Mistral 7B, LLaMA 3.2 3B, OPT-1.3B, Leonardo (CINECA)
Case study
Problem
Scanned documents come out of OCR full of errors from fonts, scan quality and layout. Language models can clean them up, but there are few Italian datasets that pair OCR output with the correct text.
Approach
- Estimated the distribution of OCR errors from a multilingual dataset that has both the ground truth and the OCR output, using edit operations to locate and classify each error.
- Injected errors with that distribution into the Italian Paisà corpus to create about 200,000 synthetic training examples.
- Fine-tuned OPT-1.3B, LLaMA 3.2 3B and Mistral 7B v0.3 with LLaMA-Factory on the Leonardo supercomputer (CINECA).
- Evaluated with Prometheus as an LLM judge, after checking its agreement with human scores using Cohen’s κ.
Results
Mistral 7B was the best model overall, with an average Prometheus score of 2.5 out of 5. Token-level agreement favoured LLaMA, which we read with caution: those metrics penalise valid paraphrases and can reward outputs that make only minimal corrections.
Team project with Lorenzo Ciappetta and Marco Mingoli, Sapienza University of Rome.