A commentator starts raising their voice before the shot is taken.
They aren't reacting. They're anticipating — reading the shape of the play, the striker's body angle, the defender who is a half-step out of position, and concluding that something is about to happen. By the time the ball is actually struck they are already at full volume.
Almost every piece of football video AI I'd worked with does the opposite. It watches a clip and reports what occurred: that was a pass, that was a shot, that was a throw-in at the 47th minute. Useful, and fundamentally retrospective. The model is the person who says "oh, a goal" after the net moves.
The SoccerNet 2026 challenge has a track that asks for the commentator's version instead. Given broadcast footage up to now, name the ball action that is about to happen, and say when. I spent March on it. The final numbers say something I didn't expect and find more interesting than the score itself: the model is much better at knowing what is coming than at knowing when.
01Why anticipation is a different problem, not a harder version of the same one
The instinct is that anticipation is detection with the labels shifted a few seconds earlier. Train on "the two seconds before a pass" instead of "the pass" and you're done. That instinct is wrong in a way that matters, and it took me a while to see it.
Detection has a ground truth that is visible in the frame. Somewhere in the clip, the boot makes contact with the ball. The evidence for "this is a pass" is in the pixels. A model that can't find it is a model that isn't looking properly.
Anticipation has no such thing. In the frames before a pass there is no pass. There is a player in possession, some teammates moving, some opponents closing. That configuration is genuinely consistent with a pass, a dribble, a shot, or losing the ball. The evidence for the label is not present in the input, because the thing being labelled has not happened yet.
Which means the ceiling isn't set by the model. It's set by the sport. Football is played by people specifically trying to be unpredictable — the whole point of a feint is to make the next action unguessable. Some fraction of these predictions are unknowable, not merely hard, and no architecture fixes that. That reframes what a good score means. You are not measuring how well the model sees. You are measuring how much of football is legible in advance.
02The setup
I built on FAANTRA, the anticipation framework released with a CVPRW 2025 paper by Dalal and colleagues, which is the reference implementation for this task. The model I trained ended up as faantra_decoupled_rny008_BAS_448p_SoccerNetBall_transformer. Reading that filename is most of the architecture, and each piece is a decision:
| Component | Choice | Why |
|---|---|---|
| Backbone | rny008 (RegNetY-008) | A deliberately small image encoder. Every frame passes through it, so its cost is multiplied by the frame count — this is where a free-tier budget actually binds |
| Resolution | 448p | Enough to resolve a player's body orientation, the signal that matters. Higher costs quadratically and adds detail about grass |
| Temporal head | transformer | Attention over the frame sequence, so the model can weight the moment the defender committed rather than treating all frames equally |
| Structure | decoupled | Predicting what and when through separate paths rather than one joint output |
| Data | SoccerNetBall + BAS | The ball-action set, plus the broader action-spotting set for extra supervision |
That decoupled choice is the one that turned out to be load-bearing, and I'll come back to it — the results make an argument about it that I didn't anticipate when I picked it.
03Reading a metric that has six numbers
Here is the official challenge server's scoring of my first submission — not my own validation, the organisers' evaluation of a held-out split I never see.
scoring_result (1)/(2).zip → scores.json.| Metric | Submission 1 | Submission 2 | Δ |
|---|---|---|---|
| mAP@1 | 6.78 | 3.14 | −3.64 |
| mAP@2 | 12.74 | 8.10 | −4.64 |
| mAP@3 | 17.13 | 12.96 | −4.17 |
| mAP@4 | 20.90 | 17.70 | −3.20 |
| mAP@5 | 22.47 | 20.17 | −2.30 |
| mAP@∞ | 23.59 | 21.04 | −2.55 |
| mAP_Average | 17.69 | 14.20 | −3.49 |
The δ is a tolerance window, in seconds. A prediction counts as correct if it names the right action and lands within δ seconds of when that action actually occurred. So mAP@1 demands the model be right about what, and within one second about when. mAP@5 allows five seconds of slack. And mAP@∞ removes the timing requirement entirely — get the action right and you score, whenever you said it would happen. That last one is the diagnostic, because it separates the two halves of the task:
- mAP@∞ = 23.59 — how good the model is at what.
- mAP@1 = 6.78 — how good it is at what and when together.
The model retains only 29% of its performance when you demand one-second precision. It identifies the right upcoming action roughly three and a half times more often than it can place that action within a second of its actual moment. Almost all of the difficulty is in the timing.
In hindsight this is obvious. "A shot is coming" is legible from the shape of the play — that's exactly what the commentator reads. "A shot is coming in 1.4 seconds" depends on whether the striker takes one more touch, whether the defender lunges, whether the pass is slightly behind them. Those are decisions made in the moment by people, not properties of the configuration on screen. Which is why I'd argue mAP@∞ is the more honest number for this task, and mAP@1 is close to measuring the sport's inherent randomness rather than the model. The average of all six — 17.69 — hides the interesting structure entirely.
That's also the argument for the decoupled architecture. When what and when have error profiles this different, forcing them through a shared output head means one signal's difficulty degrades the other's. The results suggest the separation is worth keeping and, if anything, should go further.
04The second submission was worse
Five days later I submitted again, and it was worse across every tolerance window — and worst at the tight ones. Whatever changed hurt timing precision roughly twice as much as it hurt classification: mAP@1 lost 54% of its value, mAP@∞ only 11%.
That shape is informative on its own. A change that damaged the model's general understanding would have pulled mAP@∞ down hard. This one mostly left what intact and degraded when — the signature of something that blurred the temporal resolution rather than the semantics. Averaging predictions across models with slightly different timing biases would do exactly that: the classifications reinforce, the timings smear.
I'm being careful with the word "would." I have both scored submissions and I have ensemble outputs sitting in the same directory, but the ensemble files are timestamped two hours after the second submission was scored — so I can't honestly claim from the artifacts alone that submission 2 was the ensemble. Reconstructing exactly what changed, and testing the smearing hypothesis properly, is Part 2. The general lesson is already clear enough to state: an aggregate metric that moves 3.5 points tells you almost nothing; the same metric decomposed across six tolerance windows tells you which half of the model you broke.
05What broke, and what I'd do differently
I optimised for the average. The headline mAP_Average weights all six windows equally, so improving mAP@5 and improving mAP@1 look identical on the scoreboard while being completely different engineering problems. If I'd been tracking the ∞-to-1 ratio from the start, I'd have known within the first week that timing was the whole game and spent March on the temporal head instead of spreading effort evenly.
I treated the submission budget as free. It isn't. Each scored submission is a measurement of a held-out set you otherwise never observe, and I spent one of them on a change I hadn't validated well enough locally to predict the direction of. That's a wasted observation of the only ground truth that counts.
Next time I'd predict the score before submitting. Writing down "I expect mAP@1 to go up about a point, mAP@∞ to stay flat" costs thirty seconds and converts every submission into a test of your understanding rather than a lottery ticket. When submission 2 came back down 3.5 points, I had no prior to be surprised against — which means I learned much less from it than I should have.
The next part reconstructs what actually changed between those two runs. The third looks at whether mAP@∞ deserves to be the headline number for anticipation tasks generally, because I now suspect the averaged metric is quietly steering the whole field toward optimising the half of the problem that isn't the hard one.
The transferable lesson
- Anticipation isn't detection with shifted labels — the label isn't in the pixels, so the ceiling is set by the sport, not the model.
- A metric decomposed across tolerance windows tells you which half of the model you broke; the aggregate tells you nothing.
- When two sub-tasks have different error profiles, a decoupled head stops one from dragging the other down.
- A scored submission is a scarce measurement of ground truth. Predict the result before you spend one.
