Sprint auto-split experiments — morning report (2026-10-09 overnight)

Result label everywhere: App-Measured. Tape owns the metre. No auto-DQ. Every auto event below is auto_detected_stub, humanReviewed=false. Nothing was sent, posted or hosted. No app code was changed (branch sprint-direction untouched; 323 tests not affected).

1. Footage found (details + URLs in the footage page)

No free clip matches the ideal (fixed tripod, cones at known distances, behind/front). Best available: | clip | licence | use | |---|---|---| | yt_wKP0iWaAlZI_30m_sprinting.mp4 (YouTube, 1280x720 30 fps, 6 reps) | CC BY | main test set: static ground-level camera, athlete runs straight at lens (true down-the-line). No cones; real distance unknown | | yt_3hREO5HBdw0_block_start.mp4 (25 fps) | CC BY | start-detection test (side view, block start) | | yt_A4hq-rH9z_I_30m_dash.mp4 | CC BY | handheld/panning failure case | | enugu_100m_boys.webm, athletissima_100m_f.ogv (+3 spares) | CC BY-SA 4.0 / CC BY 3.0 (Wikimedia Commons) | multi-runner, moving camera failure cases | Local assets (Gautam beep-test clips, shuttle videos) are not sprints and were not used. Pexels blocks scripted download (not circumvented); non-CC YouTube clips not downloaded.

2. What was run

• Pose: existing /workspace/yolov8s-pose.pt (ultralytics, CPU) with ByteTrack at imgsz 960 (pose_track.py); installed only lapx, scipy, matplotlib (user site). Because the runner is tiny when far, a x3 upscaled crop pass (pose_crop.py) was added for the 6 reps.
• Runner isolation: static false detections (a bush detected as a person) removed by position clustering; runner chosen by track continuity (sprintdet.py, wk.py).
• Ground-progress proxy: with the camera at ground level the ground plane collapses onto the horizon (feet-row is useless — measured: feet y stays ~390 px for the whole run). Only apparent size works. Bounding-box height is posture-confounded (crouch h≈93 px vs standing 155 px); shoulder-width (px) is posture-invariant (flat through crouch) and was used: distance ≈ K/s, K = f·0.40 m, f=931 px assumed. Assumed Z0 ≈ 12.8–14.4 m per rep. Gates at 3/5/7 "proxy metres" = shoulder-width ratios ≈1.29/1.59/2.08 vs the set-position width.
• Start = first-movement: keypoint-activity burst (normalised keypoint speed) preceding a sustained scale rise.
• Events emitted in app shape via sprintmath.py (faithful Python port of SprintMath/SprintRun JSON, beep-sprint-run/1). Kotlin tests not run (port only).

3. Results

Start (first movement) vs manual frame inspection (my reading of 12-tile close-ups, itself ±1–2 frames)

Split-gate crossings (shoulder-scale gates)

• Found 17/18 (94%); the missing one is the 7 m gate in run 5 (pose lost as the runner fills the frame near the lens).
• No ground-truth gate error could be established on this footage: there are no visible markers, and a human check of "shoulder width equals a drawn bar" (out/gate_bar_check_run1.png) was only reliable to ~±2–3 frames (±70–100 ms) because arms hide the shoulders and width changes 15–25 %/frame near the lens. Honest statement: gate timing accuracy vs truth = unknown.
• Noise-based timing uncertainty (raw-vs-smoothed scale residual ÷ scale slope): median ~130 ms (range 10–780 ms, n=17); worst at the early 3–5 m where scale changes slowly (perspective compression), best near the lens (10–25 ms). This is a conservative upper-ish bound (residual includes real stride motion).
• Metre scale: distance reached at a fixed scale ratio scales linearly with K (f × shoulder width). ±20 % K → the "5 m" gate really 4–6 m; at 4.5–7 m/s that is ±150–200 ms. So person-size alone cannot give the metre; a tape/cone reference is required (matches product law).

Pose detection coverage vs runner size (6 reps; reference = crop pass)

Frame-rate sensitivity (30 fps gate times subsampled; error vs 30 fps result)

Moving-camera clips (failure modes, measured)

Median global camera motion px/frame / % frames moving >2 px: wKP0 static 0.00 / 4 %; A4hq (handheld) 29 % moving, peaks 334 px/frame; Enugu 42 %; 3hRE 56 % (camera drifts 1–3 px/f even before the start); Athletissima broadcast 78 %. Fixed image gates are meaningless there; I did not run gate detection on them. Other failure modes seen: multi-person (8 lanes/crowd — tracker ID swaps, bystanders), tiny far athlete, bush false positives, pose loss when athlete fills frame (<~2–3 m), arms occluding shoulders, motion blur at the lens.

4. Simulated app output

out/sim_tables.md, out/sim_runs.json (full beep-sprint-run/1 JSON for each of 6 reps, auto start vs manual start). Example run 1 (auto start): 3 m 0.873 s, 5 m 1.287 s, 7 m 1.600 s; intervals 0.873/0.414/0.313 s (3.44/4.83/6.39 m/s); profile "Still speeding up at the finish (fastest 5–7 m, 6.39 m/s)…"; flags "not yet confirmed by a person" ×3. Insight: start error shifts all cumulative splits equally but leaves later intervals unchanged. Distances here are proxy metres — illustrative only, not a result. Note StartSource has no auto value; an auto start currently has to be tagged video_scrub/go_cue — suggest adding AUTO_DETECTED.

5. Visuals (out/)

• run1_annotated.mp4 (best, 0.27 MB) and run0_annotated.mp4: skeleton, START marker, running time, split flags, proxy-distance bar, App-Measured/unreviewed banner. Keyframes: run1_keyframes.jpg, run0_keyframes.jpg.
• summary_scale_gates.png: shoulder-width curve, auto vs manual start, gates, all 6 reps.
• start_check_3hRE_manual_frames.png, start_check_run0_manual_frames.png: the manual-reference frames.

6. Verdict

• Start (first movement) auto-detection from pose: viable as a proposal (≈±70 ms typical, one −167 ms miss; human can nudge on scrub).
• Automatic split crossings from a single down-the-line phone view: not viable as an unreviewed measurement today. Reasons: (1) metre scale cannot come from the video alone (±20 %); (2) ground-level/low cameras lose ground-plane information, high ones need a calibrated homography; (3) scale changes slowly at far range so timing noise is ~100+ ms early on; (4) small far athlete needs crop/upscale; (5) any camera motion breaks fixed gates. Viable as "suggested marks for a person to confirm on scrub", provided the calibration comes from tapped cones/tape. Absolute ms accuracy of gates remains unmeasured — needs a marked-course test.

7. Recommended capture setup

Tripod/fixed phone (no pan, no handheld, no auto-zoom/stabilisation), 60 fps (30 minimum), locked exposure/short shutter. Place the phone 8–10 m behind (or in front of) the start line on the line of the lane, 1.0–1.5 m high (not on the ground), framing the whole course incl. start; or a fixed side/oblique view 15–20 m away with the full course visible (best for per-metre accuracy; down-the-line mainly compresses distance). Cones/flat markers at 0, 5, 10, 20 m placed with tape on both sides of the lane (wide enough to be visible from behind), high-contrast colour. One athlete in frame; others out of shot. Athlete in plain contrasting clothing.

8. Next build steps (proposed, no code changed)

1. Calibration on frame 1: user taps the cone/line points (or 4 lane corners) → homography to ground metres; extend SprintCrossingDetector.detect to take calibration (image points + tape metres) and return confidence; add StartSource.AUTO_DETECTED.
2. Detector: pose (ML Kit/MediaPipe on device) on a crop around the athlete with tracking; start = activity burst; crossing = pelvis ground-projection through gate line, sub-frame interpolation, monotone envelope.
3. Review UI: proposals appear as unconfirmed ticks on the scrub bar (existing flags text); never auto-judge.
4. Collect a small marked-course dataset (Gautam/Luke, 10 m + 5 m cones, 60 fps, ≥20 runs) and measure real gate error vs manual frames before shipping any detector.

9. Risks / limits

Manual references are mine and ±1–2 frames; only 6 reps of one athlete/one camera; scale assumptions (f, shoulder width) unverified; CPU model, not a phone model; far-pose crop positions were hand-chosen per clip; clips from YouTube CC BY (attribution needed if shown). Detection numbers are on a clean static clip — real-world will be worse.

Files: scripts pose_track.py pose_crop.py sprintdet.py wk.py analyse_wk.py exp_wk.py exp_final.py render.py sprintmath.py camshake.py; tracks work/tracks/*.json; metrics out/results.json.

10. Luke's own clip — footage/luke_run_20261011.mp4 (2026-10-11)

All numbers: App-Measured, proxy / illustrative, unreviewed (auto_detected_stub). No marker distance exists, so metres are assumed-scale. Nothing sent/deployed. Code: luke_analysis.py luke_sim.py luke_render.py; tracks work/tracks/luke_crop2.json; output out/luke_sim.json, out/luke_summary.md, out/luke_run{1,2}_annotated.mp4 (+_keyframes.jpg).

Video facts. H.264, 368x496 portrait (low-res, ~3:4 downscale), 30 fps, 1032 frames, 34.40 s. 0–5.9 s handheld/fingers over lens/camera being set down (settles ~frame 172); static ground-level camera, tilted up at trees, sun behind runner (backlit). Runner (white tee, grey shorts, sandals) ≈ 25–40 px tall at the far end.

Timeline verified by frames (30 fps; ±~6 f pass-by, ±~8 f turn). The watcher's times are roughly right but the start is not a standing start: the runner starts beside/behind the camera and is first seen as a foot at frame 176 (5.87 s), passing the lens ~f188 (6.3 s). Run 1: pass f188 → turn ~f408 (13.6 s; side-on f396–420) → returns, passes camera ~f580–600. Run 2 is a flying/rolling start: the returning runner passes the lens ~f603 (20.1 s) and carries straight on away from camera; turn ~f795 (26.5 s; side-on f783–805); back at lens ~f940 (31.3 s). So "first movement" is not observable in either run (run 1 starts out of shot, run 2 has no stop). Manual reference START = camera pass-by (not first movement).

Pose detection / tracking. Full-frame never tested at far range (earlier work: 0 % below ~150 px); used the x3 upscaled crop (pose_crop.py, crop x40–340,y0–340). Runner found in 323 frames of run 1 window (f176–560; 84 %) and 299/346 of run 2 (f600–945; 86 %); misses are at the lens (body mostly out of frame) and the dim turn. Pose at 25–40 px tall is coarse (box heights still stable). Runner reaches the assumed-scale ~20 m (plausible range 16–25 m) at the turn.

Start detector vs manual. Activity/first-confident-pose trigger: run 1 first pose f178 (−10 f, −333 ms vs pass-by ref), activity burst f214 (+26 f, +867 ms — fires only once the body is in view and swinging); run 2 f608 / f606 (+5 / +3 f, +167 / +100 ms). I.e. it detects "runner enters / starts running in view", within ~±100–330 ms of my reference when the runner is in shot, but it cannot detect a true start that happens off-frame. Not comparable to the 6/7 ±67 ms result on the CC BY footage.

Distance proxy. Pinhole: Z = f·1.75 m / bbox height (px), f=413 px (assumed ≈48° horizontal FOV; real f unknown), h smoothed. Valid only when the runner is >~3.8 m (h<190 px); the first ~4 m is extrapolated. Uncertainty: FOV/f ±25 % dominant, height ±3 %, upward camera pitch/posture/box noise → metres ±25–30 %. Monte-Carlo (scale 0.8–1.25×, start ±6 f) gives 10–90 % ranges below.

Relative (scale-free) timing, time from pass-by to reach a fraction of the turnaround distance: run 1: 25 % 2.30 s, 50 % 4.20 s, 75 % 5.30 s, 90 % 6.47 s (turnaround reached ~7.5–8 s). Run 2: 25 % 1.80 s, 50 % 3.30 s, 75 % 4.87 s, 90 % 5.63 s. Run 2 is ~25 % faster early (rolling start) — this part is robust to the metre-scale problem.

Simulated app output (same sprintmath.py; start source tagged video_scrub, see §4 note): | config | run | split 5 m | split 10 m | split 20 m | intervals (s, m/s) | |---|---|---|---|---|---| | 10 m (marks 5,10) | 1 | 2.30 s (1.9–2.7) | 4.18 s (3.4–4.7) | – | 0–5 2.30 s 2.17; 5–10 1.87 s 2.67 (2.5–3.2) | | 10 m | 2 | 1.80 s (1.5–2.2) | 3.31 s (2.7–4.0) | – | 0–5 1.80 s 2.78; 5–10 1.51 s 3.31 (2.9–4.0) | | 20 m (turn≈20 m; 20 m = turnaround plateau onset) | 1 | 2.31 | 4.21 | 7.59 | 10–20 3.38 s 2.96 m/s | | 20 m | 2 | 1.80 | 3.31 | 5.73 | 10–20 2.42 s 4.13 m/s | Profile line for 10 m (both): "Still speeding up at the finish (fastest 5–10 m …). Top speed not shown on this distance." Both runs are jogs/rolling runs (~2–4 m/s), not sprints (nowhere near 7–9 m/s). The 20 m column depends on the assumption turnaround = 20 m and the runner decelerating into the turn contaminates 10–20 m; treat as illustration of the UI only. Review flags: every mark "not yet confirmed by a person".

Verdict / comparison with CC BY footage. Earlier clip (wKP0): static camera 1.0 m-ish, runner comes toward the lens from a visible crouched start, 1280x720, 6 reps, start ±67 ms typical. Here: 368x496, runner already moving at first sight, fully backlit, no start posture, runner far (25–40 px), scale unknown, run 2 has no start at all. Start detection that worked there is not applicable; gate timing is dominated by ±25 % scale uncertainty (≈±0.3–0.5 s on the 10 m split) plus ±6 f start ambiguity. Useful as a demonstration that the pipeline runs end-to-end on real phone footage; not as a measurement.

What would make it trustworthy: (1) phone on a tripod/stable object 1–1.5 m high, 8–10 m behind the start line (or side-on 15–20 m away) so the start line, the athlete in the set position and the whole 20 m are in frame; (2) cones/tape at 0, 5, 10, 20 m (bright, both sides of lane) visible in frame — then the app taps them for calibration instead of assuming height/FOV; (3) a real standing start (static set position, a visible first movement), runner starting in frame; (4) 60 fps (30 minimum), 1080p, locked exposure/no auto-zoom, sun not directly behind the athlete; (5) a hard sprint, not a jog, ideally ≥3 reps with the same setup; (6) turnaround well beyond the last mark (e.g. 30 m) so the 20 m mark isn't confused with deceleration.

11. Iterations — app v0.2.2 → v0.2.6 (2026-10-11, branch sprint-direction)

Full per-iteration detail with tables: out/iterations.md (+ out/iter2_results.txt, iter3_results.json, iter4_results.txt, iter4_quality.json, baseline_v0.2.1.json). Code: motion_signal.py (frame-difference sampler), startsuggest.py (reference of Kotlin StartSuggester), calib.py (reference of Kotlin CourseCalibration), iter*_eval.py, iter_metrics.py, build_iter.sh. All numbers App-Measured / proxy / unreviewed; every auto event is an unconfirmed suggestion; nothing sent or deployed. No emulator or phone was used: "unit-tested" = JVM tests; "compile-checked" = builds into the APK, never run.

Honest bottom line. What is solid: the start suggestion on a static, ≥720p camera (≈ ±0.1 s vs my manual frames), the calibration maths, the uncertainty wording, the 0.2.1 fixes. What is not: Luke's low-res handheld rolling runs (suggestions are late/wrong and the app now says "poor footage"), anything on a moving camera, and every piece of UI (never run). Calibration from behind the start line is geometrically poor at 10–20 m (3–4 px tap error = 1–3 m) — the app warns; a side-on view is ~10x better in the sim.

Film next (a marked-course test): phone on a tripod/stable object, 1.0–1.5 m high, 60 fps, 1080p, locked exposure, no hand on the phone; cones/tape at 0, 5, 10, 20 m placed ON the lane line; start with the camera SIDE-ON ~15 m away looking at the 10 m cone (best for calibration) and again from 8 m behind the start; athlete in the set position for ≥2 s before going, then runs; ≥10 runs each; keep one clip with a rolling start. Note the tape-measured cone distances and the phone model. Then open them in the app via "Open a video from this phone".

← Experiments overview