Accuracy & methodology

The headline number

Across the tested fleet, VPM predicts a vessel's next port of call correctly 77.9% of the time, weighted by event. On a strict chronological hold-out — training only on the earlier part of history and scoring only the later part — the same measure is 76.2%.

All three come from the same back-test. They differ only in how it is counted:

• 77.9% — every scored call across the full history, weighted by call. A vessel making a thousand calls counts for more than one making a hundred, so this figure tracks the fleet as it actually moves.
• 76.2% — the same measure, but scored only on the held-out later stretch of history, after the model was built on the earlier stretch alone. This is the number that shows the result is a genuine forecast, not a fit to the past.
• 71% — each vessel's own hit rate, then averaged with every ship weighted equally. It sits lower because the hardest vessels to predict are also the least active, so equal weighting gives them more pull.

All three are shown; none is hidden.

What "Top-1" and "Top-3" mean

For each move, the model produces a ranked list of candidate next ports with probabilities.

• Top-1 is a hit when the single most likely port is the one the vessel actually called at.
• Top-3 is a hit when the actual port is anywhere in the top three candidates.

Top-1 is the honest headline because it is what a user acts on. Top-3 matters for planning — three candidates that between them cover a call are still useful to a supplier deciding where to position stock.

The back-test protocol

Accuracy is measured the way it would have to hold up in production:

• Chronological, expanding window. To score the call on a given date, the model is rebuilt from that vessel's history up to that date only. It never sees the future it is being tested against.
• Held-out tuning. Any parameter that could be tuned is chosen on the first 60% of history and evaluated on the last 40%. The hold-out number is the one that survives that separation.
• Scored against real calls. Every prediction is checked against a port the vessel physically visited — never a simulation, never a synthetic route.
• Validated against production. The measurement engine was run side by side with the live prediction engine across all 36 vessels, with zero divergence, so the number describes the model that actually runs.

Where accuracy comes from

The fleet number is an average of two very different regimes. VPM predicts from two signals — the vessel's movement pattern and the destination it declares over AIS — and which one carries a given call decides how good the call is.

The vessels that resist prediction

The weakest five vessels in the fleet sit between 15% and 41% Top-1. Every one of their errors is in the pattern-only branch, and every one of those errors is a call where no destination was declared. Their problem is not the model — it is the absence of a signal to model. These are tramp vessels working the spot market, which by definition run no fixed schedule; a missing pattern there is the economics of the trade, not a gap in the data.

VPM reports these vessels honestly as low-confidence rather than presenting a weak guess as a firm one. A route's concentration — how repetitive its history is — is shown alongside every vessel, so a user can see in advance which ships predict well.

Predicting further ahead

Beyond the next call, VPM predicts the second and third ports ahead. Measured on a 6,359-point chronological hold-out:

These are lower by design — uncertainty compounds with each hop. The useful output here is Top-3: three candidates that cover roughly half of second-port calls is enough to plan around, which a single best guess at 31% is not.

Two definitions of a hit

Two scoring rules need stating plainly, because a sceptical reader should know exactly what counts.

Same port area

Two different port names within 25 km of each other are treated as the same call — Muuga and Tallinn, Immingham and Hull. A prediction that lands on the neighbouring berth of the same port complex is a hit, not a miss. A stricter figure without this rule is tracked alongside and runs a few tenths of a percentage point lower.

Canal transit

VPM's own history occasionally misses a stop at a canal waypoint, because a vessel slowing through a lock does not always drop below the speed threshold that registers a call. When the model correctly predicted a canal transit and the vessel demonstrably used that canal to reach the port it actually called at next, on the far side, the prediction is scored as correct. This reclassified roughly 250 of the 16,100 events, across 17 vessels — about 1.6 percentage points of the headline. It is disclosed here rather than buried.

Back-tested vs live accuracy

The figure shown next to a vessel inside the product is not this back-test number. It is scored from live predictions as they resolve — the vessel has to actually arrive before that prediction can be marked right or wrong — so it lags a model change by weeks. A separate historical figure, computed the back-test way, is shown alongside so an improvement is visible immediately. When the two disagree, the live figure is catching up, not contradicting.

What moves the number now

Seven families of algorithmic refinement have been tested against this back-test. The ones that worked are in production; the rest — every smoothing- or prior-based idea — measured negative and were dropped. The remaining lever is not the algorithm.

27.5% of calls fall into the weak pattern-only branch only because no destination was declared. Give half of those a resolved destination and the headline moves by about 6.3 percentage points — more than the entire algorithmic package combined. That is a data-coverage problem, and it is where the work goes next.