Corso Mazzini Dataset Constructor
An Apache Airflow DAG that uses Selenium to open a public webcam over Corso Mazzini, records the live stream in fixed chunks, cuts them into frames and feeds a Label Studio loop where a YOLO model retrained every 100 labels pre-annotates the next batch.
Introduction
Before a model can learn what Corso Mazzini looks like, someone has to collect Corso Mazzini. This project is that collector: an Apache Airflow DAG that drives a real browser with Selenium to a public webcam pointed at the street, records what it streams, turns the recordings into images, and hands them to a labelling loop in which the model being trained helps label its own training data.
What comes out the other end is a labelled dataset in YOLO format — the data behind the Corso Mazzini pedestrian tracker.
Context
A detector for one specific street needs pictures of that street — its angle, its light, its crowds at every hour — and no public dataset has them. The only source is a public webcam that streams live and keeps nothing.
Schedule the capture as an Airflow DAG: Selenium brings the webcam's player to a live state, the stream is recorded in one-minute chunks, the chunks become frames, and the frames are labelled in Label Studio with a YOLO model that is retrained every 100 labels to pre-annotate the rest.
Data
The footage comes from an open, public webcam on SkylineWebcams that films Corso Mazzini in Cosenza from a fixed position. It is a live stream with no archive: whatever is not recorded while it airs is gone. The page sits behind a cookie-consent banner and a player that has to be clicked before it starts, and that player does not always load on the first try — which is why a plain download script is not enough and a real browser is.
- Source
- Public SkylineWebcams page for Corso Mazzini, Cosenza
- Camera
- Fixed position, so every frame shares one viewpoint
- Stream
- Live HLS video, no history kept by the site
- Unit
- One MP4 file per minute of footage
- Window
- 24 hours per run — day, night, rush hour and empty street
Methodology
It started as a notebook run by hand: open the page, click through, record. Moving it into Apache Airflow turned each of those steps into a task in a DAG, with a schedule, retries and a log per run, so collecting footage no longer depends on someone being at the computer when the street is busy.
Each stage is its own task, so a failure is retried where it happened instead of restarting the whole capture.
- Open the page
- Selenium starts Chrome and loads the webcam page for Corso Mazzini
- Accept consent
- Waits for the cookie banner and clicks it away, so the player is reachable
- Start the player
- Clicks the player poster and waits for the loading spinner to disappear; if the stream stalls, the page is refreshed and the task tries again
- Record in chunks
- Once the stream is live, FFmpeg copies it to disk one minute at a time, without re-encoding, each file named by the time it was captured
- Stop and clean up
- When the 24-hour window is over the loop ends and the browser is closed
Recording in one-minute pieces is deliberate. A network hiccup or an expired stream costs a single minute instead of a corrupted day-long file, every chunk is a complete, playable video on its own, and later stages can start on the first chunks while the capture is still running.
Implementation
A detector is trained on pictures, not videos. The recorded clips are cut into still frames, and those frames are the unit everything downstream works on: they are what gets imported into Label Studio, what the model is trained on and what the final dataset is made of.
The frames are labelled in Label Studio and exported as a YOLO dataset. Drawing every box by hand does not scale to a day of a busy street, so the labelling is done in rounds, and the model learns along the way:
- Label
- A batch of frames is labelled in Label Studio
- Train
- Every 100 labelled images, the dataset is exported and the same YOLO model is retrained on everything labelled so far
- Pre-label
- The new checkpoint predicts boxes on the next unlabelled frames, which show up in Label Studio as suggestions
- Correct
- The suggestions are reviewed: wrong boxes fixed, missing ones added, false ones deleted — and the round starts again
Each round the model is a little better, so each round needs a little less correcting. Nothing it suggests goes into the dataset unchecked: the model speeds the labelling up, a person still decides what is true.
Results
- Capture runs unattended: one DAG run records a full day as 1,440 one-minute clips, so a failed chunk costs a minute of footage, not a day.
- The browser steps that used to be done by hand — consent banner, play button, waiting out a stalled player — are tasks with retries instead of a person at a keyboard.
- Labelling is a loop rather than a slog: every 100 human-checked images produce a new YOLO checkpoint that pre-labels the next batch, so the work shifts from drawing boxes to correcting them.
- The exported dataset is the training material behind the Corso Mazzini pedestrian tracker.
Conclusion
- The hard part of scraping a live webcam is not the video, it is the page in front of it: consent banners, click-to-play and a player that sometimes never loads. Treating those as waits and retries inside a task is what made the capture reliable.
- Airflow earned its place by making failures visible and local. A run that breaks at minute 700 shows exactly which task failed, and the 699 minutes before it are already on disk.
- One fixed camera filmed for a full day covers more variation than it sounds — light, shadows, night, crowds and empty pavement — while keeping the viewpoint the final model will actually see.
- Retraining every 100 labels is a trade-off: often enough that the suggestions improve quickly, rarely enough that training time does not dominate labelling time.
- Model-assisted labelling inherits the model's blind spots. If it never proposes a box for something, that something is easy to miss, which is why every pre-labelled frame is still reviewed by hand.