← ProjectsData Engineering2026In progress

Corso Mazzini Dataset Constructor

An Apache Airflow DAG that uses Selenium to open a public webcam over Corso Mazzini, records the live stream in fixed chunks, cuts them into frames and feeds a Label Studio loop where a YOLO model retrained every 100 labels pre-annotates the next batch.

Apache AirflowSeleniumFFmpegLabel StudioYOLOPython
60s
Recording chunk
1,440
Chunks per 24 h run
100
Labels per retrain
01

Introduction

SummaryTwo minutes

Before a model can learn what Corso Mazzini looks like, someone has to collect Corso Mazzini. This project is that collector: an Apache Airflow DAG that drives a real browser with Selenium to a public webcam pointed at the street, records what it streams, turns the recordings into images, and hands them to a labelling loop in which the model being trained helps label its own training data.

What comes out the other end is a labelled dataset in YOLO format — the data behind the Corso Mazzini pedestrian tracker.

02

Context

ProblemBefore

A detector for one specific street needs pictures of that street — its angle, its light, its crowds at every hour — and no public dataset has them. The only source is a public webcam that streams live and keeps nothing.

ApproachAfter

Schedule the capture as an Airflow DAG: Selenium brings the webcam's player to a live state, the stream is recorded in one-minute chunks, the chunks become frames, and the frames are labelled in Label Studio with a YOLO model that is retrained every 100 labels to pre-annotate the rest.

03

Data

The sourceOpen webcam

The footage comes from an open, public webcam on SkylineWebcams that films Corso Mazzini in Cosenza from a fixed position. It is a live stream with no archive: whatever is not recorded while it airs is gone. The page sits behind a cookie-consent banner and a player that has to be clicked before it starts, and that player does not always load on the first try — which is why a plain download script is not enough and a real browser is.

Source
Public SkylineWebcams page for Corso Mazzini, Cosenza
Camera
Fixed position, so every frame shares one viewpoint
Stream
Live HLS video, no history kept by the site
Unit
One MP4 file per minute of footage
Window
24 hours per run — day, night, rush hour and empty street
04

Methodology

Pipeline
Airflow scheduleSelenium opens the webcamStream recorded in chunksChunks cut into framesLabel StudioYOLO retrain every 100 labelsExported dataset

It started as a notebook run by hand: open the page, click through, record. Moving it into Apache Airflow turned each of those steps into a task in a DAG, with a schedule, retries and a log per run, so collecting footage no longer depends on someone being at the computer when the street is busy.

The DAG, task by taskAirflow

Each stage is its own task, so a failure is retried where it happened instead of restarting the whole capture.

Open the page
Selenium starts Chrome and loads the webcam page for Corso Mazzini
Accept consent
Waits for the cookie banner and clicks it away, so the player is reachable
Start the player
Clicks the player poster and waits for the loading spinner to disappear; if the stream stalls, the page is refreshed and the task tries again
Record in chunks
Once the stream is live, FFmpeg copies it to disk one minute at a time, without re-encoding, each file named by the time it was captured
Stop and clean up
When the 24-hour window is over the loop ends and the browser is closed

Recording in one-minute pieces is deliberate. A network hiccup or an expired stream costs a single minute instead of a corrupted day-long file, every chunk is a complete, playable video on its own, and later stages can start on the first chunks while the capture is still running.

05

Implementation

From clips to imagesFrames

A detector is trained on pictures, not videos. The recorded clips are cut into still frames, and those frames are the unit everything downstream works on: they are what gets imported into Label Studio, what the model is trained on and what the final dataset is made of.

Labelling with the model in the loopEvery 100 images

The frames are labelled in Label Studio and exported as a YOLO dataset. Drawing every box by hand does not scale to a day of a busy street, so the labelling is done in rounds, and the model learns along the way:

Label
A batch of frames is labelled in Label Studio
Train
Every 100 labelled images, the dataset is exported and the same YOLO model is retrained on everything labelled so far
Pre-label
The new checkpoint predicts boxes on the next unlabelled frames, which show up in Label Studio as suggestions
Correct
The suggestions are reviewed: wrong boxes fixed, missing ones added, false ones deleted — and the round starts again

Each round the model is a little better, so each round needs a little less correcting. Nothing it suggests goes into the dataset unchecked: the model speeds the labelling up, a person still decides what is true.

06

Results

ResultsCapture and labelling
60s
Recording chunk
1,440
Chunks per 24 h run
100
Labels per retrain
  • Capture runs unattended: one DAG run records a full day as 1,440 one-minute clips, so a failed chunk costs a minute of footage, not a day.
  • The browser steps that used to be done by hand — consent banner, play button, waiting out a stalled player — are tasks with retries instead of a person at a keyboard.
  • Labelling is a loop rather than a slog: every 100 human-checked images produce a new YOLO checkpoint that pre-labels the next batch, so the work shifts from drawing boxes to correcting them.
  • The exported dataset is the training material behind the Corso Mazzini pedestrian tracker.
From a live webcam to a labelled datasetAirflow · Selenium · Label Studio · YOLO
retrain every 100 → pre-labelWebcam1-min clipsFramesLabel StudioYOLO
07

Conclusion

What held upRetrospective
  • The hard part of scraping a live webcam is not the video, it is the page in front of it: consent banners, click-to-play and a player that sometimes never loads. Treating those as waits and retries inside a task is what made the capture reliable.
  • Airflow earned its place by making failures visible and local. A run that breaks at minute 700 shows exactly which task failed, and the 699 minutes before it are already on disk.
  • One fixed camera filmed for a full day covers more variation than it sounds — light, shadows, night, crowds and empty pavement — while keeping the viewpoint the final model will actually see.
  • Retraining every 100 labels is a trade-off: often enough that the suggestions improve quickly, rarely enough that training time does not dominate labelling time.
  • Model-assisted labelling inherits the model's blind spots. If it never proposes a box for something, that something is easy to miss, which is why every pre-labelled frame is still reviewed by hand.
© 2026 Héctor LiceaDesign concept · placeholder contentBuild 2026.08Rendered client-side · no tracking