← ProjectsComputer Vision2024Deployed

UNICAL GAMES Dataset Builder

How the UNICAL GAMES training set was built from Google's Quick, Draw! strokes — replayed, auto-labelled and exported as COCO and YOLO annotations in the browser.

PythonOpenCVmatplotlibQuick, Draw!YOLOCOCOReactCanvas
10,000
Single-object images
1,500
Composite scenes
0 px
Box error vs notebook (gallery)
01

Introduction

ObjectiveWhy this exists

UNICAL GAMES is a drawing game scored by a YOLO object detector. Before the detector could score anything it needed a dataset: thousands of drawings, each with a box around the object and a class label.

From notebook to browser

Google's Quick, Draw! dataset contains millions of doodles, but no images ready to be used by an object detection model. Each drawing is stored as a sequence of pen strokes: a list of (x, y) points recorded in the order they were drawn.

And that's exactly where the work begins.

In my Jupyter Notebook I built a pipeline that takes this raw data and turns it, step by step, into something a model like YOLO can use: the strokes are rendered as images, the drawing's bounding box is derived from its points, and finally the annotations are written in the formats needed for training.

But reading these operations inside a notebook isn't always the most engaging way to understand what's going on.

So... why not show it?

This section is an interactive version of what happens in the notebook.

Instead of just showing you the code and telling you “this is where it happens”, you can follow the process yourself: start from the original data, watch how the pen strokes are interpreted, see how the bounding box is computed, and get all the way to the final annotation.

The following sections reproduce the same pipeline logic, right in your browser. You can use real records from the dataset or try a drawing of your own and see how it gets transformed.

The goal isn't just to show you the final result, but to let you see every step that would normally stay hidden inside a cell of my Jupyter Notebook.

In other words: I'm not asking you to read this notebook. I'm letting you explore it.

house_0 · stroke by stroke
stroke 1/5 · 12/36 pts
The notebook's datasetdownload.ipynb
Source
Quick, Draw! · simplified ndjson
Classes
circle, triangle, house, car, bicycle
Sample
First 2,000 records of each file
Images
200 × 200 px PNG, black on white
Labels
YOLO .txt, one row per object
Scenes
1,500 × 640 px, 2–5 objects each
Split
train only
02

Methodology

The pipeline, as implementeddownload.ipynb
  1. 01Cell 1

    Downloading the data: starting from the pen stroke

    The first step is getting the data to work with.

    For each of the five classes, the notebook downloads the whole simplified file from Google's public bucket. Each file is in NDJSON format: one line corresponds to a single drawing.

  2. 02Cells 2–3

    From record to drawing: rebuilding the strokes

    Once the files are downloaded, we need to understand how a single drawing is represented. Each record holds several pieces of information — word, countrycode, timestamp, recognized, key_id — but for our pipeline the most important one is drawing.

    drawing contains a list of strokes, where each stroke is represented by two arrays: one for the x coordinates and one for the y coordinates.

  3. 03Cell 4

    From vector to image: making the drawing usable

    At this point we have the drawing back, but YOLO works with images. So a conversion step is needed.

    The drawing_to_image function takes every stroke and renders it as a black line 3 pt thick on a 2 × 2 inch figure, with no margins. The axes are fixed to the range −60…315, leaving a padding of 60 units around the original 0–255 area.

  4. 04Cell 6

    From ink to bounding box: creating the annotations

    Now we finally have an image. But to train YOLO it is not enough to know that the image contains, say, a bicycle: we also have to tell it where it is.

    The get_bbox function converts the image to greyscale and treats as "ink" every pixel with an intensity below 250. From these pixels it takes the minimum and maximum coordinates, which gives the drawing's bounding box.

    The box is then converted to YOLO format by yolo_bbox, which describes each object through x_center y_center width height, all expressed as values normalised to the image size.

  5. 05Cells 5–6

    From one object to a scene: building multi-object images

    Training a detector only on images containing a single object would be rather limiting. So the next step is to create real artificial scenes with several objects.

    generate_combined_images builds 1,500 images of 640 × 640 pixels. For each scene, 2 to 5 already-labelled drawings are picked at random. Each drawing is cropped using its own bounding box, tightened further to the actual ink area, and rescaled by a random factor between 0.4 and 1.2.

    The system then tries up to 80 random positions, looking for one that keeps at least 4 px of distance from the objects already in the scene.

  6. 06Cell 7

    The final check: can we trust our annotations?

    Automatically creating thousands of images and bounding boxes is useful, but there is one fundamental question: did we generate the annotations correctly?

    To check, show_labeled_examples picks 9 images from the dataset, reads the YOLO coordinates back, converts them into pixels again and draws the bounding box and class name over each image.

    It is a simple check, but an important one: it lets you verify visually that image, object position and label are still perfectly aligned. If something went wrong in the previous steps, this is where it should become immediately obvious.

03

Explore

bicycle_0 · replayOriginal stroke order
stroke 2/8 · 17/53 pts

1.2 s / 5.2 s · constant pen speed — the file stores order, not timing

Reference gallery10 drawings · bundled
Recordbicycle.ndjson · line 1
word
bicycle
countrycode
DE
timestamp
2017-03-23 12:59:36.68729 UTC
recognized
true
key_id
6426650621247488
drawing
8 strokes · 53 points
notebook file
dataset/images/train/bicycle_0.png
{
  "word":"bicycle",
  "countrycode":"DE",
  "timestamp":"2017-03-23 12:59:36.68729 UTC",
  "recognized":true,
  "key_id":"6426650621247488",
  "drawing":[[[1,0,5,26,52,77,86,84,79,70,46,27,15,5,4,8],[145,153,159,163,152,134,101,77,71,66,67,79,95,132,142,143]],[[51,51,43],[66,94,154]],[[10,47,84],[118,108,107]],[[191,185,184,189,200,215,235,245,251,251,245,235,224,207,187],[91,107,133,147,157,157,144,134,120,78,69,66,66,75,119]],[[192,255,255],[111,110,108]],[[226,225],[59,170]],[[54,63,76,107,129,207,226,229,229],[61,47,35,28,35,37,42,46,68]],[[188,186],[37,0]]]
}

Send this drawing through AutoLabel, or draw your own below.

AutoLabel it →
04

Create

New drawing0 strokes
Nothing drawn yet

Draw on the pad. The first stroke creates Custom_n; you can then rename it, give it a class and replay it.

05

AutoLabel

Quick, Draw!
COCO category 5 · YOLO class 4
  1. Strokesdone
  2. Boxpending
  3. Annotationpending
Notebook image · 200 × 200
not boxed yet
ink boxextreme pixelsdashed square = 0–255 data area
How the box is foundWaiting
  1. 01
    Render the imageEvery stroke drawn in black on a white 200 × 200 image, data window −60…255+60, line 3 pt = 4.17 px (+0.25 px to match matplotlib's anti-aliasing).drawing_to_image(drawing)
  2. 02
    Find the inkConvert to grey and keep every pixel darker than 250.mask = gray < 250
  3. 03
    Take the extremesThe smallest and largest column and row that contain ink. The four pixels that set them are marked on the image.coords.min(axis=0), coords.max(axis=0)
  4. 04
    Write the boxCOCO stores [x, y, width, height]. Width is x_max − x_min, as in the notebook, so the right and bottom edges sit on the last ink pixel rather than past it.[x_min, y_min, x_max − x_min, y_max − y_min]
  5. 05
    Attach the classLabel "bicycle" → COCO category 5 → YOLO class 4.
Checked against the notebook
notebook [29, 29, 170, 124]Ground truth comes from running the notebook's own drawing_to_image and get_bbox on this drawing. The browser port matches all 10 gallery drawings exactly, and 100 held-out drawings to within 1 px (347 of 400 box edges exact); what remains is the difference between matplotlib's anti-aliasing and the browser's.
06

Inspect

Quick, Draw!
COCO category 5 · YOLO class 4
  1. Strokesdone
  2. Boxpending
  3. Annotationpending
This drawing has no box yet. Run AutoLabel to add it to the annotation.
{
  "info": {
    "description": "UNICAL GAMES Dataset Builder — session export",
    "version": "1.0",
    "year": 2026,
    "date_created": "2026-11-16T20:08:49.993Z",
    "url": "https://quickdraw.withgoogle.com/data"
  },
  "licenses": [
    {
      "id": 1,
      "name": "CC BY 4.0",
      "url": "https://creativecommons.org/licenses/by/4.0/"
    }
  ],
  "images": [],
  "annotations": [],
  "categories": [
    {
      "id": 1,
      "name": "circle",
      "supercategory": "quickdraw"
    },
    {
      "id": 2,
      "name": "triangle",
      "supercategory": "quickdraw"
    },
    {
      "id": 3,
      "name": "house",
      "supercategory": "quickdraw"
    },
    {
      "id": 4,
      "name": "car",
      "supercategory": "quickdraw"
    },
    {
      "id": 5,
      "name": "bicycle",
      "supercategory": "quickdraw"
    }
  ]
}
Fields for this drawingLive values
images[].id
—
width × height
—
annotations[].id
—
image_id
—
category_id
—
bbox
—
area
—
segmentation
[]
What is in it — and what is not
  • bbox[x, y, width, height] in image pixels, origin top-left. Width = x_max − x_min, the notebook's convention, so COCO and YOLO here carry the same numbers.
  • areaThe box area. COCO normally takes area from the segmentation mask; without one, the box area is the usual stand-in.
  • segmentationEmpty on purpose. The notebook keeps no masks or polygons. The strokes could be turned into one, but that would be a new annotation the notebook never produced.
  • iscrowd0 — every annotation is a single object.
  • category_idStarts at 1. The YOLO class index is category_id − 1.
  • licenseGallery images are Quick, Draw! data (CC BY 4.0). Custom drawings carry none.

Turn the box into a YOLO label.

To YOLO →
07

YOLO

Quick, Draw!
COCO category 5 · YOLO class 4
  1. Strokesdone
  2. Boxpending
  3. Annotationpending
  1. 01Strokes

    What Quick, Draw! stores: ordered strokes of integer points. No pixels, no box.

    [[1,0,5,26,…],
     [145,153,159,163,…]] × 8
  2. 02Image

    What the detector sees: the strokes rendered by drawing_to_image. Only this goes into YOLO.

    images/train/bicycle_0.png
    200 × 200 px
  3. 03COCO

    Absolute pixels — top-left corner and size — with the class as a category id.

    —
  4. 04YOLO

    One text row per object: class index, then centre and size divided by the image size.

    —
COCO → YOLOW = H = 200

COCO stores the top-left corner and the size in pixels; YOLO stores the centre and the size as fractions of the image. Move to the centre, then divide by the image width W and height H:

Run AutoLabel on this drawing to see the conversion with its numbers.

The notebook computes YOLO directly from the corner pixels — ((x_min + x_max) / 2) / W — which is the same arithmetic. For the gallery drawings both routes give, digit for digit, the rows the notebook wrote.

Class mappingnc = 5
labelCOCO idYOLOorigin
circle10notebook
triangle21notebook
house32notebook
car43notebook
bicycle54notebook

The notebook's class_map fixes circle → 0 … bicycle → 4. A label it does not know (say "dog") would raise a KeyError there; here it is appended as the next index, which also changes nc in data.yaml. COCO ids start at 1; YOLO index = category_id − 1, the mapping Ultralytics' COCO converter uses.

data.yaml
path: dataset
train: images/train
val: images/val    # the notebook writes train only
test: images/test  # optional

nc: 5
names:
  0: circle
  1: triangle
  2: house
  3: car
  4: bicycle
This is one example, not a datasetWhat training needs

One box shows the format. Training a detector that generalises needs, at least:

  • Thousands of images per class — the notebook used 2,000 drawings each, plus composites.
  • Separate train, validation and test splits, with no drawing (key_id) in more than one. The notebook writes train only.
  • Mirrored images/ and labels/ folders: one .txt per image with the same name, one row per object.
  • A data.yaml whose names match the indices in the label files, with nc updated whenever a class is added.
  • Scenes with several objects at varied positions and sizes — single-object images always sit in the same region, which the composite step corrects.
  • Background images with no objects (an empty label file), so the model learns when to predict nothing.
  • Balanced classes, and a check that no box was clipped by the image border.
© 2026 Héctor LiceaDesign concept · placeholder contentBuild 2026.08Rendered client-side · no tracking