← ProjectsComputer Vision2026Deployed

PP-OCR Autopsy

PaddleOCR's document pipeline taken apart piece by piece: every step that is not a neural network rewritten by hand and checked identical to the original, on a CV and a heritage catalogue form.

PaddleOCRPP-StructureV3OpenCVNumPyPythonTypeScriptReactDocker
217
Text lines traced end to end
217/217
Lines identical to PaddleOCR
6.6 MB
Networks behind detection + recognition
01

Introduction

SummaryDocument intelligence

PaddleOCR turns a PDF into text, and with PP-StructureV3 into Markdown with headings, images and tables. One call does it all, and that is the problem: when the output is wrong there is no way to tell which part failed.

So I took it apart. Each component runs on its own, its inputs and outputs are recorded, and every step that is not a neural network is rewritten from scratch and checked against the library until both give the same result, number for number.

The page follows the data through the stages in order, from the PDF to the table HTML. Each stage runs the rewritten code live in your browser on the real outputs of the networks, so you can move a threshold and watch what changes.

The two documentsReal inputs
CV
My own CV: two columns, a photo, headings, icons and bullet lists. It is the case where reading order and images matter.
ICCD form
A catalogue record from Italy's heritage census (Santa Maria del Monte Serra, Cetraro): a two-page grid of field codes and long Italian text, a real table.

Pick a document at the top of any stage and every stage switches to it.

02

Methodology

The route through the autopsy
PDF → pixelsDetection (DB)Sort + cropRecognition (CTC)Pipeline by handLayout + regionsImages & tables

The stages follow the data. First the plain OCR (detection, glue, recognition), rebuilt by hand and compared with PaddleOCR. Then the document layer PP-StructureV3 adds on top: block types, regions, reading order, images and tables.

Where the learned parts are
PDF ─► render ─► DET ─► sort + crop ─► REC ─► text            PaddleOCR(...).predict()
         │      [net]     (no model)    [net]
         │       └ pre-process, DB post-process      └ resize to 48 px, CTC decoding
         │
         └─► LAYOUT [net] + REGIONS [net] ─► rules ─► reading order ─► Markdown   PP-StructureV3
                  └ thresholds, NMS, merges     └ blocks → regions, lines → blocks
                  └─► table: classifier [net] ─► structure [net] ─► match lines ↔ cells ─► HTML

[net] = a trained network, the only borrowed part. Everything else is rewritten here.
How each piece was isolatedSame recipe every stage
  • Call the component alone (TextDetection, TextRecognition, LayoutDetection…) with the parameters the full pipeline gives it. They are not the module defaults.
  • Open its predictor: the pre-process steps, the network runner and the post-process object, run one at a time.
  • Rewrite every step that is not the network with NumPy and OpenCV, and assert that the result equals PaddleX's.
  • Export the network's raw output, so this page can run the same code again in TypeScript on top of it.
What is actually learnedModel files
2.0 MB
PP-OCRv6_tiny_det
4.6 MB
PP-OCRv6_tiny_rec
130.4 MB
PP-DocLayout_plus-L
129.9 MB
PP-DocBlockLayout

The OCR networks are tiny. The two layout detectors are RT-DETR-L models, 130 MB each. Anything not listed here is code you can read.

03

00 · PDF → pixels

Document
The page as pixelszoom 2 · hover to read pixels
hover: [row, column] → B G R
The same line at three zooms1 px = 1 square
zoom=1 · 72 DPI(843, 596, 3) · strip 264×19
A line rendered at zoom 1
zoom=2 · 144 DPI(1685, 1191, 3) · strip 528×39
A line rendered at zoom 2
zoom=3 · 216 DPI(2527, 1787, 3) · strip 792×58
A line rendered at zoom 3

A PDF stores text and vectors, not pixels. The models only see pixels, so the first step is to photograph each page. The zoom decides how many pixels a letter gets: at 72 DPI small type is a few pixels tall, at 216 DPI everything gets slower. PaddleX renders at zoom 2 (144 DPI).

In → out
in
CV_2026_10.pdf · 1 pages
page
595.5 × 842.2 pt (1 pt = 1/72")
render
pypdfium2 · zoom = 2 · 144 DPI
out
ndarray (1685, 1191, 3) uint8 · BGR
a pixel
three integers 0–255, blue first (OpenCV order)
04

01a · Detection

Document
83 text boxes

The detector does not read. It marks blobs, and they come out in the order the contour search finds them, from the bottom of the page up. Putting them in reading order is stage 02's job.

Detector settings
Network input size (limit_side_len)
Probability map1696 × 1184
Network time (build CPU)646 ms
Boxes83
Discarded36 too small · 8 low score
Checkmatches PaddleX· 83/83 boxes within 3 px (max 2.2 px)
Same model, different defaults
parameterTextDetectionPaddleOCR(…)
limit_side_len / type960 · max64 · min
thresh0.20.3
box_thresh0.40.6
unclip_ratio1.41.5

The standalone module and the full PaddleOCR pipeline do not use the same defaults. To reproduce the pipeline, the module has to be called with the pipeline's values. The live boxes are computed in your browser from the network's probability map. They land within a couple of pixels of PaddleX's: the rest is pyclipper rounding to integers and OpenCV working in float32.

05

01b · Detection, inside

Document
hover: probability that this pixel is text
1 · Pre-process (rewritten, identical tensor)
page (H, W, 3) uint8 BGR
  Read       BGR → RGB
  Resize     H, W → nearest multiple of 32      # the net halves the image 5×: 2⁵ = 32
  Normalize  (x / 255 − mean) / std              # mean (0.485, 0.456, 0.406), std (0.229, 0.224, 0.225)
  ToCHW      (H, W, 3) → (3, H, W)
  ToBatch    (3, H, W) → (1, 3, H, W) float32
Shapes at each step
page
(1685, 1191, 3) uint8
resized
(1696, 1184, 3)
tensor
(1, 3, 1696, 1184) float32 · [-2.015, 2.64]
2 · network
PP-OCRv6_tiny_det, 1.97 MB, the only learned step
output
(1, 1, 1696, 1184) · mean 0.1
3 · DB post-process (Differentiable Binarization)
  1. Threshold: a pixel is text if p > thresh.
  2. Blobs: group touching pixels (8-connectivity).
  3. Fit the minimum-area rectangle around each blob; drop it if its short side is under 3 px.
  4. Score it by the mean probability inside; drop it under box_thresh.
  5. Unclip: push every side out by area × ratio / perimeter, then scale back to the page.
Blobs127
Too small36
Low score8
Boxes83
PaddleX (OpenCV contours)137 contours → 83 boxes

Why unclip? DB is trained to mark a shrunk core of each line, so two lines close together never melt into one blob. The box only gets its real size back at the end. OpenCV also counts holes inside blobs as contours, which is why its contour count can be higher than the blob count. Those holes never survive the filters.

Module defaults in the model's inference.yml: thresh 0.2 · box_thresh 0.4 · unclip 1.4 · max_candidates 3000

06

02 · Sort + crop

Document
Boxes, numbered

SortQuadBoxes: by the y of the top-left corner, then neighbours less than 10 px apart are swapped left to right. It is a simple rule with no idea of columns. On the CV, the left sidebar and the main column interleave line by line, which is exactly why PP-StructureV3 adds a layout model (stage 05).

Strip 12

Minimum-area rectangle → corners in a fixed order → perspective warp onto a straight rectangle. Strips at least 1.5× taller than wide are turned 90°. The recogniser never sees the page, only these strips. Here the canvas does the warp.

Ordermatches PaddleX
Strip sizesmatches PaddleX· 83 strips, ±1 px
83 stripsclick to select
07

03a · Recognition

Document
83 strips → text (1901 ms on the build CPU)
One line in, one string out

PP-OCRv6_tiny_rec answers one question: what does this strip say? It can only read a single line that has already been cut out, and it returns the text plus a confidence, the mean over the characters it wrote.

Sort by lowest score: the worst lines on the CV are its icons. A phone or a GitHub logo is not text, but the model has to write something, so it picks the closest symbol in its dictionary. That can be a Chinese character, or nothing at all.

Breaking it on purposeinput changed → output
  • original0.98original« interpretazione dei dati, con un focus sulla modellazione economica. »
  • left half only0.99left half only« interpretazione dei dati, con un f »
  • top half only (letters cut)0.57top half only (letters cut)« intoMnmoteziep.edeidotieenunfoeueoulem2.e dolezioneeeene3oiee »
  • rotated 180°0.51rotated 180°« 'ojouoea euolzeepo eins snoo} un uoo ‘nep ep euolzreudeu! »
  • blurred0.73blurred« inßergretaaigve deiedafi. con un focus sullamoielliaaioneegonomica »
  • ⅓ resolution0.70⅓ resolution« inlorμrclaliurrc ciculi Lu11 un1 Fucurasulliu muckelluiurc cu1u=icni. »
  • half of one line + half of the next0.77half of one line + half of the next« Anerprelazitne: deh cdaanincoo automociose sutidizzoodectzlonemnetonoo ca. »

The last case is what a badly cut box looks like: half of two lines. The tiny model then produces confident-looking gibberish. When an OCR result is garbage, the cause is often the box, not the recogniser.

08

03b · Recognition, inside

Document
line 12 / 83
What the network sees: 48 × 1460 px, cut into T = 182 columns≈ 8 px per column · hover a bar
·i·n··t·e·r··p·r··e·t·a·z·i·o··n··e··␣␣···d·e·i··␣␣··dd·a·t·i,␣␣␣··c··o··n··␣␣···u·n··␣␣··f··o·c··u··s·␣␣··s··ull·la··␣␣····m··o··d··el·l·a·z·i·o··n··e··␣␣··e··c··o··n·o····m·i·c·a·.
writes a characterrepeat, collapsedblankt=62: '␣' 0.501 '·' 0.449 ',' 0.050
CTC decoding, by handmatches PaddleX· text and score
1. best symbol in each of the 182 columns·i·n··t·e·r··p·r··e·t·a·z·i·o··n··e··␣␣···d·e·i··␣␣··dd·a·t·i,␣␣␣··c··o··n··␣␣···u·n··␣␣··f··o·c··u··s·␣␣··s··ull·la··␣␣····m··o··d··el·l·a·z·i·o··n··e··␣␣··e··c··o··n·o····m·i·c·a·.
2. collapse repeats·i·n·t·e·r·p·r·e·t·a·z·i·o·n·e·␣·d·e·i·␣·d·a·t·i,␣·c·o·n·␣·u·n·␣·f·o·c·u·s·␣·s·ul·la·␣·m·o·d·el·l·a·z·i·o·n·e·␣·e·c·o·n·o·m·i·c·a·.
3. drop the blanksinterpretazione dei dati, con un focus sulla modellazione economica.
Score (mean of emitting columns)0.9827 (PaddleX 0.9827)
Doubtful columns (< 0.80)t=62

The network does not read letter by letter. It sweeps the strip in T vertical slices and gives each one a probability for all 6,906 symbols. The blank (·) means "nothing new here". It is also what lets a word contain double letters: l·l gives ll, while ll collapses to l.

The dictionary: 6,906 symbols
[0]
blank: "no new character"
[1…39]
! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~ 0 1 2 3 4 5 6 …
[6905]
the space
ñ ✓Ñ ✓à ✓è ✓ì ✓ò ✓ù ✓€ ✓§ ✓« ✓» ✓ß ✓四 ✓

The model can only ever write what is in this list. Over 6,100 of its symbols are Chinese characters; the rest are Latin, Greek, full-width and circled forms. That is why an icon can come out as 四.

09

04 · Pipeline by hand

Document
The OCR pipeline, chained by hand
def ocr_by_hand(page):
    polys  = det.predict(page, **PIPELINE_PARAMS)   # 01  network + DB post-process
    polys  = sort_quad_boxes(polys)                 # 02  plain code
    strips = [crop(page, p) for p in polys]         # 02  plain code
    order  = argsort([w / h for h, w in strips])    # what PaddleOCR does before batching
    texts  = rec.predict([strips[i] for i in order], batch_size=6)
    return unsort(texts, order), polys              # 03  network + CTC
By hand vs PaddleOCR(…).predict()
Lines83
Same boxesmatches PaddleX
Same textdiffers from PaddleX· 77/83
Detection (build CPU)644 ms
Sort + crop13 ms

First attempt: same boxes, same strips, same models, but a few lines differ. Recognition runs in batches of six, and every strip in a batch is padded to the widest one. The padding changes what the network sees for short strips.

Lines that differ: 6
#by handPaddleOCR
14«+39 351739 5315»«+39 351 739 5315»
17«V»«»
28«9»«四»
31«05/2023-PRESENTE»«05/2023 - PRESENTE»
38«2022- 2024»«2022 -2024»
48«06/2025-08/2025»«06/2025 -08/2025»
The page as text (PaddleOCR)
  0  HÉCTOR ARÍSTIDES
  1  05/10/2026
  2  LICEA BEATÓN
  3  ECONOMISTA
  4  DESCRIZIONE
  5  Economista
  6  con
  7  solida
  8  esperienza
  9  nell'estrazione,
 10  modellazione
 11  CONTATTO
 12  interpretazione dei dati, con un focus sulla modellazione economica.
 13  Appassionato di web scraping e automazione, utilizzo queste competenze per
 14  +39 351 739 5315
 15  trasformare i dati in conoscenza applicata alle scienze sociali. Negli ultimi anni
 16  ho sviluppato (Python) e seguito progetti innovativi basati sull’intelligenza
 17  
 18  [email protected]
 19  artificiale. Dispongo di competenze in machine learning, analisi di embeddings,
 20  clustering e visualizzazione dei dati, con particolare attenzione allintegrazione
 21  Rende, Cosenza
 22  tra approcci classici/statistici ed Al generativa. Per approfondire le mie
 23  competenze pratiche, puoi consultare il mio Portfolio Interattivo:
 24  
 25  Interactive Portfolio
 26  https://data-mia.org
 27  ESPERIENZA LAVORATIVA
 28  四
 29  Github
 30  Wish S.R.L
 31  05/2023 - PRESENTE
 32  Steam Account
 33  Data Scientist
 34  • Sviluppo e implementazione di modelli di machine learning
 35  ISTRUZIONE
 36  Implementazione di modelli di IA
 37  • Pulizia e preparazione dei dati per analisi approfondite
 38  2022 -2024
 39  Utilizzo di linguaggi di programmazione come Python e R per analisi
 40  UNIVERSITA DELLA CALABRIA
 41  statistiche
 42  ITALIA
 43  • Analisi e interpretazione dei risultati della applicazione di modelli
 44  statistici
 45  Economia e Commercio
 46  
 47  UNICAL (DISCAG)
 48  06/2025 -08/2025
 49  2013 - 2018
 50  UNIVERSIDAD DE ORIENTE
 51  Master Gestione della transizione verde e dei rischi climatici
 52  CUBA
 53  • Docente di Python per Analisi di Dati nel Master Gestione della
 54  Economia
 55  transizione verde e dei rischi climatici
 56  COMPETENZE
 57  
 58  Universidad de Oriente
 59  08/2018-09/2021
 60  Professore
 61  • Analisi dei dati
 62  • Professore Istruttore a tempo pieno, Economia Cubana.
 63  Conoscenza delle teorie economiche
 64  Competenze in statistica
 65  CERTIFICATI E CORSI
 66  Pensiero critico
 67  Programmazione Python
 68  Programmazione R
 69  Curso completo de Python: Programación en Python desde cero
 70  Deploy di Workflow IA
 71  Udemy
 72  Web Scraping
 73  C# Total - Programador Experto en 28 días
 74  • SPARQL
 75  Udemy
 76  LINGUE
 77  PUBBLICAZIONI
 78  Spagnolo (Madrelingua)
 79  Italiano (Fluent)
 80  Inglese (Fluent)
 81  Employment and Self-Employment in the Santiago de Cuba Province.
 82  Anuario Facultad de Ciencias Económicas y Empresariales
10

05a · Layout

Document
54 blockstext ×41paragraph_title ×10header ×1doc_title ×1image ×1

PP-DocLayout_plus-L does not read either. It decides that something is a title, a paragraph or a picture purely from how it looks: position, size, weight and white space around it.

Post-process over 300 candidates

PP-StructureV3 reads a YAML with a threshold per class (0.3 for titles, 0.4 for text, 0.5 for most others), turns NMS on, and merges boxes class by class.

67117211212442343456367611300.250.50.751
Raw candidates300
After per-class threshold62
After NMS54
After the giant-image filter54
After merging54
Checkmatches PaddleX· 54
What the network sees
The page squashed to 800 by 800 pixels
ReadImage
(1685, 1191, 3)
Resize
(800, 800, 3)
Normalize
(800, 800, 3)
ToCHWImage
(3, 800, 800)
network
RT-DETR-L · 130 MB · 6336 ms
out
300 × [class, score, x1, y1, x2, y2]

Unlike the text detector, this one does not keep the aspect ratio: the page is squashed to 800×800, and a scale factor maps the boxes back. It always returns 300 candidates, most of them noise with a tiny score. All of the filtering above is code.

11

05b · Regions & order

Document

PP-DocBlockLayout is the same kind of detector with a single class: Region. It shows what belongs together (dashed). Then plain rules pour blocks into regions, smallest region first: a block joins at ≥ 60 % overlap and the region snaps to its blocks. Leftover blocks share a SupplementaryRegion. Out-of-flow blocks such as page numbers and headers each get their own, which is why that region always scores exactly 1.00.

Regions by handmatches PaddleX· 2 regions
PP-StructureV3 output

HÉCTOR ARÍSTIDES LICEA BEATÓN

CONTATTO

+39 351 739 5315

[email protected]

Rende, Cosenza

Interactive Portfolio

四Github

Steam Account

ISTRUZIONE

2022 -2024

UNIVERSITA DELLA CALABRIA

ITALIA

Economia e Commercio

2013 - 2018

UNIVERSIDAD DE ORIENTE

CUBA

Economia

COMPETENZE

• Analisi dei dati

Conoscenza delle teorie economiche

Competenze in statistica

Pensiero critico

Programmazione Python

Programmazione R

Deploy di Workflow IA

Web Scraping

• SPARQL

LINGUE

Spagnolo (Madrelingua)

Italiano (Fluent)

Inglese (Fluent)

ECONOMISTA

DESCRIZIONE

Economista con solida esperienza nell'estrazione,modellazione interpretazione dei dati, con un focus sulla modellazione economica.Appassionato di web scraping e automazione, utilizzo queste competenze per trasformare i dati in conoscenza applicata alle scienze sociali. Negli ultimi anni ho sviluppato (Python) e seguito progetti innovativi basati sull’intelligenza artificiale. Dispongo di competenze in machine learning, analisi di embeddings,clustering e visualizzazione dei dati, con particolare attenzione allintegrazione tra approcci classici/statistici ed Al generativa. Per approfondire le mie competenze pratiche, puoi consultare il mio Portfolio Interattivo:https://data-mia.org

ESPERIENZA LAVORATIVA

Wish S.R.L

05/2023 - PRESENTE

Data Scientist

• Sviluppo e implementazione di modelli di machine learning

Implementazione di modelli di IA

• Pulizia e preparazione dei dati per analisi approfondite

Utilizzo di linguaggi di programmazione come Python e R per analisi statistiche

• Analisi e interpretazione dei risultati della applicazione di modelli statistici

UNICAL (DISCAG)

06/2025 -08/2025

Master Gestione della transizione verde e dei rischi climatici

• Docente di Python per Analisi di Dati nel Master Gestione della transizione verde e dei rischi climatici

Universidad de Oriente

08/2018-09/2021

Professore

• Professore Istruttore a tempo pieno, Economia Cubana.

CERTIFICATI E CORSI

Curso completo de Python: Programación en Python desde cero Udemy

C# Total - Programador Experto en 28 días Udemy

PUBBLICAZIONI

Employment and Self-Employment in the Santiago de Cuba Province.Anuario Facultad de Ciencias Económicas y Empresariales

Each block becomes Markdown by label: doc_title → #, paragraph_title → ##, text → a paragraph, image → <img>, table → HTML. Blocks labelled number, header, header_image, footer, footer_image, aside_text are dropped.

12

06a · Image or table?

Document
1 image / table zones
Zone labelled imagecandidates competing for it
  • image0.848
  • header0.272
  • paragraph_title0.059
  • text0.049
  • image0.047

An image block is cropped and saved next to the Markdown. Any text the OCR found inside it is discarded.

<img src="imgs/img_in_image_box_65_63_328_322.jpg">
Same detector, different routes
  • text · titlesCollect the OCR lines inside the block → paragraph or heading.
  • imageCrop to imgs/img_in_image_box_x1_y1_x2_y2.jpg → <img>.
  • tableCrop → classifier → structure model → match OCR lines to cells → HTML.
  • chart · formula · sealOnly with their own recognisers switched on, which this run did not.
  • number · header · footerKept for the order, dropped from the Markdown.
Experiment: the page at 35 % brightness
text 0.98text 0.95paragraph_title 0.94paragraph_title 0.93paragraph_title 0.93text 0.92text 0.92paragraph_title 0.89text 0.88text 0.88paragraph_title 0.88text 0.88header 0.88text 0.87doc_title 0.86paragraph_title 0.85text 0.85text 0.82text 0.82text 0.81text 0.79text 0.79text 0.79paragraph_title 0.79text 0.78text 0.77text 0.76text 0.75paragraph_title 0.74text 0.74text 0.74text 0.72text 0.72text 0.71paragraph_title 0.67text 0.64paragraph_title 0.64text 0.63image 0.63text 0.62text 0.61text 0.60paragraph_title 0.59text 0.59text 0.58text 0.58text 0.58text 0.57text 0.55text 0.53text 0.50

There is no separate "is this a picture?" model, so a dark scan could in principle look like one big photo. That is what the giant-image filter in 05a is for: an image box covering more than 93 % of a portrait page is dropped. Here the labels survive the darkening.

13

06b · Table parser

Document
The CV has no table. Pick a page of the ICCD form.
14

Results

ResultsThree pages, two documents
0
Text lines traced end to end
217/217
Lines identical to PaddleOCR
6.6 MB
Networks behind detection + recognition
  • Every plain-code step (DB post-process, sorting, cropping, CTC decoding, layout post-process, region rules, both table parsers) was rewritten and asserted identical to PaddleX on all three pages.
  • Chained by hand, the pipeline first agreed with PaddleOCR on 206 of 217 lines. The 11 differences came from batching: PaddleOCR sorts strips by aspect ratio and pads each batch of six to its widest strip. Doing the same gives 217/217.
  • Only the networks are learned: 2.0 MB for the text detector and 4.6 MB for the recogniser, 130 MB each for the layout and region detectors. Everything around them is geometry and bookkeeping.
  • The two pages of the same ICCD form take different routes: page 1 is classified as a wired table (69 %) and goes through a cell detector, page 2 as wireless (69 %) and is parsed end to end by the structure model.
  • The page re-runs the same post-processing live in the browser: DB on the shipped probability maps, CTC on the recogniser's columns, the layout filters on all 300 raw candidates, the region rules and the table assembly.
15

Technical explanation

How the data was producedOffline, once

One script runs every stage in a throw-away Docker image with PaddlePaddle 3.3 and PaddleOCR 3.7. Each hand-written step is asserted equal to PaddleX's own, and the build stops if one differs.

Asserted
Detection tensor, DB boxes, sort, crops, CTC text and score, layout post-process, regions, table cell boxes and HTML
Shipped
Page images, the detector's probability maps (16-bit PNG), the top-3 symbols per CTC column, all 300 layout candidates, region and table outputs
Size
About 4 MB for three pages, loaded one stage at a time
Why nothing runs on the serverRaspberry Pi

This portfolio is served from a Raspberry Pi. The full PP-StructureV3 stack needs gigabytes of RAM, has poor ARM support, and takes seconds to minutes per page on a CPU. It does not belong on the Pi.

It does not need to be there either. The networks ran once at build time. Everything this page lets you change happens after them, and that is exactly the code that was rewritten, so the browser can run it.

16

Conclusion

RetrospectiveWhat the autopsy showed
  • Most of an OCR pipeline is not AI. Out of a dozen stages only the networks are learned. Thresholds, geometry, sorting, decoding and matching rules are plain code, and that is where most of the behaviour you can tune lives.
  • The surprises were in the glue: different defaults for the same model in two entry points, batch padding changing the text, a rule that lets one line feed two blocks, and two pages of the same form sent to two different table parsers.
  • Knowing each piece makes the system repairable. A bad line can now be traced to its box, its strip, its columns or its block, and fixed at that step, instead of swapping the whole library.
  • The same method works for any packaged model: isolate, record inputs and outputs, rewrite the code around the network, and prove it identical before changing anything.
© 2026 Héctor LiceaDesign concept · placeholder contentBuild 2026.08Rendered client-side · no tracking