UNICAL GAMES Dataset Builder
How the UNICAL GAMES training set was built from Google's Quick, Draw! strokes — replayed, auto-labelled and exported as COCO and YOLO annotations in the browser.
Introduction
UNICAL GAMES is a drawing game scored by a YOLO object detector. Before the detector could score anything it needed a dataset: thousands of drawings, each with a box around the object and a class label.
From notebook to browser
Google's Quick, Draw! dataset contains millions of doodles, but no images ready to be used by an object detection model. Each drawing is stored as a sequence of pen strokes: a list of (x, y) points recorded in the order they were drawn.
And that's exactly where the work begins.
In my Jupyter Notebook I built a pipeline that takes this raw data and turns it, step by step, into something a model like YOLO can use: the strokes are rendered as images, the drawing's bounding box is derived from its points, and finally the annotations are written in the formats needed for training.
But reading these operations inside a notebook isn't always the most engaging way to understand what's going on.
So... why not show it?
This section is an interactive version of what happens in the notebook.
Instead of just showing you the code and telling you “this is where it happens”, you can follow the process yourself: start from the original data, watch how the pen strokes are interpreted, see how the bounding box is computed, and get all the way to the final annotation.
The following sections reproduce the same pipeline logic, right in your browser. You can use real records from the dataset or try a drawing of your own and see how it gets transformed.
The goal isn't just to show you the final result, but to let you see every step that would normally stay hidden inside a cell of my Jupyter Notebook.
In other words: I'm not asking you to read this notebook. I'm letting you explore it.
- Source
- Quick, Draw! · simplified ndjson
- Classes
- circle, triangle, house, car, bicycle
- Sample
- First 2,000 records of each file
- Images
- 200 × 200 px PNG, black on white
- Labels
- YOLO .txt, one row per object
- Scenes
- 1,500 × 640 px, 2–5 objects each
- Split
- train only
- 02MethodologyWhat each notebook cell does, and what it assumes.
- 03ExploreReplay real dataset records from their original strokes.
- 04CreateDraw an object; it is converted to the same format.
- 05AutoLabelBox the drawing with the notebook's own method.
- 06InspectThe result as a COCO annotation you can copy.
- 07YOLOFrom COCO to a YOLO label row, and what training still needs.
Methodology
- 01Cell 1
Downloading the data: starting from the pen stroke
The first step is getting the data to work with.
For each of the five classes, the notebook downloads the whole
simplifiedfile from Google's public bucket. Each file is inNDJSONformat: one line corresponds to a single drawing. - 02Cells 2–3
From record to drawing: rebuilding the strokes
Once the files are downloaded, we need to understand how a single drawing is represented. Each record holds several pieces of information —
word,countrycode,timestamp,recognized,key_id— but for our pipeline the most important one isdrawing.drawingcontains a list of strokes, where each stroke is represented by two arrays: one for thexcoordinates and one for theycoordinates. - 03Cell 4
From vector to image: making the drawing usable
At this point we have the drawing back, but YOLO works with images. So a conversion step is needed.
The
drawing_to_imagefunction takes every stroke and renders it as a black line3 ptthick on a2 × 2inch figure, with no margins. The axes are fixed to the range−60…315, leaving a padding of 60 units around the original0–255area. - 04Cell 6
From ink to bounding box: creating the annotations
Now we finally have an image. But to train YOLO it is not enough to know that the image contains, say, a bicycle: we also have to tell it where it is.
The
get_bboxfunction converts the image to greyscale and treats as "ink" every pixel with an intensity below250. From these pixels it takes the minimum and maximum coordinates, which gives the drawing's bounding box.The box is then converted to YOLO format by
yolo_bbox, which describes each object throughx_center y_center width height, all expressed as values normalised to the image size. - 05Cells 5–6
From one object to a scene: building multi-object images
Training a detector only on images containing a single object would be rather limiting. So the next step is to create real artificial scenes with several objects.
generate_combined_imagesbuilds 1,500 images of640 × 640pixels. For each scene, 2 to 5 already-labelled drawings are picked at random. Each drawing is cropped using its own bounding box, tightened further to the actual ink area, and rescaled by a random factor between0.4and1.2.The system then tries up to 80 random positions, looking for one that keeps at least
4 pxof distance from the objects already in the scene. - 06Cell 7
The final check: can we trust our annotations?
Automatically creating thousands of images and bounding boxes is useful, but there is one fundamental question: did we generate the annotations correctly?
To check,
show_labeled_examplespicks 9 images from the dataset, reads the YOLO coordinates back, converts them into pixels again and draws the bounding box and class name over each image.It is a simple check, but an important one: it lets you verify visually that image, object position and label are still perfectly aligned. If something went wrong in the previous steps, this is where it should become immediately obvious.
Explore
1.2 s / 5.2 s · constant pen speed — the file stores order, not timing
- word
- bicycle
- countrycode
- DE
- timestamp
- 2017-03-23 12:59:36.68729 UTC
- recognized
- true
- key_id
- 6426650621247488
- drawing
- 8 strokes · 53 points
- notebook file
- dataset/images/train/bicycle_0.png
{
"word":"bicycle",
"countrycode":"DE",
"timestamp":"2017-03-23 12:59:36.68729 UTC",
"recognized":true,
"key_id":"6426650621247488",
"drawing":[[[1,0,5,26,52,77,86,84,79,70,46,27,15,5,4,8],[145,153,159,163,152,134,101,77,71,66,67,79,95,132,142,143]],[[51,51,43],[66,94,154]],[[10,47,84],[118,108,107]],[[191,185,184,189,200,215,235,245,251,251,245,235,224,207,187],[91,107,133,147,157,157,144,134,120,78,69,66,66,75,119]],[[192,255,255],[111,110,108]],[[226,225],[59,170]],[[54,63,76,107,129,207,226,229,229],[61,47,35,28,35,37,42,46,68]],[[188,186],[37,0]]]
}Send this drawing through AutoLabel, or draw your own below.
AutoLabel it →Create
Draw on the pad. The first stroke creates Custom_n; you can then rename it, give it a class and replay it.
AutoLabel
- Strokesdone
- Boxpending
- Annotationpending
- 01Render the imageEvery stroke drawn in black on a white 200 × 200 image, data window −60…255+60, line 3 pt = 4.17 px (+0.25 px to match matplotlib's anti-aliasing).
drawing_to_image(drawing) - 02Find the inkConvert to grey and keep every pixel darker than 250.
mask = gray < 250 - 03Take the extremesThe smallest and largest column and row that contain ink. The four pixels that set them are marked on the image.
coords.min(axis=0), coords.max(axis=0) - 04Write the boxCOCO stores [x, y, width, height]. Width is x_max − x_min, as in the notebook, so the right and bottom edges sit on the last ink pixel rather than past it.
[x_min, y_min, x_max − x_min, y_max − y_min] - 05Attach the classLabel "bicycle" → COCO category 5 → YOLO class 4.
notebook [29, 29, 170, 124]Ground truth comes from running the notebook's own drawing_to_image and get_bbox on this drawing. The browser port matches all 10 gallery drawings exactly, and 100 held-out drawings to within 1 px (347 of 400 box edges exact); what remains is the difference between matplotlib's anti-aliasing and the browser's.Inspect
- Strokesdone
- Boxpending
- Annotationpending
{
"info": {
"description": "UNICAL GAMES Dataset Builder — session export",
"version": "1.0",
"year": 2026,
"date_created": "2026-11-16T20:08:49.993Z",
"url": "https://quickdraw.withgoogle.com/data"
},
"licenses": [
{
"id": 1,
"name": "CC BY 4.0",
"url": "https://creativecommons.org/licenses/by/4.0/"
}
],
"images": [],
"annotations": [],
"categories": [
{
"id": 1,
"name": "circle",
"supercategory": "quickdraw"
},
{
"id": 2,
"name": "triangle",
"supercategory": "quickdraw"
},
{
"id": 3,
"name": "house",
"supercategory": "quickdraw"
},
{
"id": 4,
"name": "car",
"supercategory": "quickdraw"
},
{
"id": 5,
"name": "bicycle",
"supercategory": "quickdraw"
}
]
}- images[].id
- —
- width × height
- —
- annotations[].id
- —
- image_id
- —
- category_id
- —
- bbox
- —
- area
- —
- segmentation
- []
bbox[x, y, width, height] in image pixels, origin top-left. Width = x_max − x_min, the notebook's convention, so COCO and YOLO here carry the same numbers.areaThe box area. COCO normally takes area from the segmentation mask; without one, the box area is the usual stand-in.segmentationEmpty on purpose. The notebook keeps no masks or polygons. The strokes could be turned into one, but that would be a new annotation the notebook never produced.iscrowd0 — every annotation is a single object.category_idStarts at 1. The YOLO class index is category_id − 1.licenseGallery images are Quick, Draw! data (CC BY 4.0). Custom drawings carry none.
Turn the box into a YOLO label.
To YOLO →YOLO
- Strokesdone
- Boxpending
- Annotationpending
- 01Strokes
What Quick, Draw! stores: ordered strokes of integer points. No pixels, no box.
[[1,0,5,26,…], [145,153,159,163,…]] × 8
- 02Image
What the detector sees: the strokes rendered by drawing_to_image. Only this goes into YOLO.
images/train/bicycle_0.png 200 × 200 px
- 03COCO
Absolute pixels — top-left corner and size — with the class as a category id.
—
- 04YOLO
One text row per object: class index, then centre and size divided by the image size.
—
COCO stores the top-left corner and the size in pixels; YOLO stores the centre and the size as fractions of the image. Move to the centre, then divide by the image width W and height H:
Run AutoLabel on this drawing to see the conversion with its numbers.
The notebook computes YOLO directly from the corner pixels — ((x_min + x_max) / 2) / W — which is the same arithmetic. For the gallery drawings both routes give, digit for digit, the rows the notebook wrote.
| label | COCO id | YOLO | origin |
|---|---|---|---|
| circle | 1 | 0 | notebook |
| triangle | 2 | 1 | notebook |
| house | 3 | 2 | notebook |
| car | 4 | 3 | notebook |
| bicycle | 5 | 4 | notebook |
The notebook's class_map fixes circle → 0 … bicycle → 4. A label it does not know (say "dog") would raise a KeyError there; here it is appended as the next index, which also changes nc in data.yaml. COCO ids start at 1; YOLO index = category_id − 1, the mapping Ultralytics' COCO converter uses.
path: dataset
train: images/train
val: images/val # the notebook writes train only
test: images/test # optional
nc: 5
names:
0: circle
1: triangle
2: house
3: car
4: bicycleOne box shows the format. Training a detector that generalises needs, at least:
- Thousands of images per class — the notebook used 2,000 drawings each, plus composites.
- Separate train, validation and test splits, with no drawing (key_id) in more than one. The notebook writes train only.
- Mirrored images/ and labels/ folders: one .txt per image with the same name, one row per object.
- A data.yaml whose names match the indices in the label files, with nc updated whenever a class is added.
- Scenes with several objects at varied positions and sizes — single-object images always sit in the same region, which the composite step corrects.
- Background images with no objects (an empty label file), so the model learns when to predict nothing.
- Balanced classes, and a check that no box was clipped by the image border.