calibration

`sleap_nn.tracking.mot.calibration` ¶

Per-video self-calibration for the MOT trackers.

:func:calibrate looks at every detection of a video BEFORE tracking it (all vectors already exist in sleap-nn's offline tracking flow) and measures:

Geometry: body size, nearest-neighbor spacing, the processing stride and per-step motion, from anchor tracklets -- detections chained frame-to-frame only where the association is unambiguous (mutual nearest neighbors, well separated from any competitor), cut at any doubt.
Time base: the stride and how time advances between processed frames, through the shared step helpers (:func:stride_from_counts, :func:resolve_time_base, :func:steps_between, :func:frame_steps) that the trackers and the identity linker use too, so all of them count the same steps.
Scores: detection-confidence quantiles (:func:score_normalizer turns them into the per-video score normalizer of MOTParams.normalize_scores).
Appearance (when detections carry re-ID vectors): the distributions of the exact DECISION STATISTICS the trackers and the identity linker use, measured on synthetic trials over anchor tracklets. Two anchors that overlap in time are guaranteed to be different animals, so co-occurring anchors provide true negatives; later vectors of the same anchor provide positives. Calibrating the statistic that is thresholded (EMA cosine, gallery max, tracklet-to-gallery top-k mean) -- not single-pair cosines -- keeps the thresholds' error rates meaningful.

When there is not enough evidence (no vectors, a single animal, too few trials) the appearance part reports status != "ok" and every "auto" threshold disables appearance rather than falling back to a constant tuned on person re-ID.

Classes:

Name	Description
`VideoCalibration`	Calibration of one video. Every field is a plain Python type.

Functions:

Name	Description
`calibrate`	Calibrate a video from all of its frames' detections.
`frame_steps`	Step position of each processed frame (the first at step 0).
`frame_stride`	`(frame_step, regular)` of a sequence of frame indices.
`link_rule_tables`	The calibrated linking rule for each query length.
`pair_geometry`	Same-frame spacing of `cur` and its unambiguous links to `prev`.
`resolve_body_size`	The body size (px) and its source (the shared fallback chain).
`resolve_time_base`	Resolve `MOTParams.time_base` to `"frames"` or `"steps"`.
`score_normalizer`	Per-video score normalizer of `MOTParams.normalize_scores`.
`steps_between`	Time steps between two processed frames `delta` frame indices apart.
`stride_from_counts`	Processing stride from the counts of the positive frame-index deltas.

`VideoCalibration` ¶

Calibration of one video. Every field is a plain Python type.

Attributes:

Name	Type	Description
`status`	`str`	Appearance-calibration status: `"ok"`, `"no_vectors"`, `"insufficient"` or `"skipped"` (`calibrate(appearance=False)`: nobody reads the appearance part, so it was not computed).
`n_frames`	`int`	Frames seen.
`n_detections`	`int`	Valid detections seen.
`frame_step`	`int`	Most common positive frame-index delta (1 if unknown).
`regular_stride`	`bool`	Whether at least half of the deltas equal `frame_step`.
`time_base`	`str`	The RESOLVED time base, `"frames"` or `"steps"` (see :func:`resolve_time_base`): with :func:`frame_steps` (`frame_step` in the `"frames"` time base) it gives every processed frame's step position, exactly as the trackers count steps.
`body_size`	`float`	Body size (px), from the fallback chain of :func:`resolve_body_size`: the median raw-box diagonal of valid masks and multi-node poses; else (single-node poses) a motion-derived size; else `nn_distance`; else 50. A `body_size` given to :func:`calibrate` replaces all of them.
`body_size_source`	`str`	Where `body_size` came from: `"measured"`, `"motion"`, `"nn_distance"`, `"default"` or `"override"`.
`nn_distance`	`float`	Median same-frame nearest-neighbor centroid distance (px), ignoring duplicate pairs (IoMin > 0.5); falls back to `body_size`.
`step_p95`	`float`	95^th percentile per-step centroid displacement along anchors (px).
`step_p99`	`float`	99^th percentile per-step centroid displacement along anchors (px).
`score_quantiles`	`tuple`	Score quantiles at 0.05, 0.25, 0.5, 0.75 and 0.95, of the RAW detection scores (before any normalization).
`n_anchors`	`int`	Anchor tracklets found.
`n_trials`	`int`	Appearance trials run at `g = 1`.
`assoc_rank1`	`Optional[float]`	Fraction of `g = 1` trials where the true track's EMA cosine beats every co-occurring competitor's.
`m_assoc`	`Optional[float]`	Association margin: `max(0.02, q10)` of (true - best competitor) EMA cosine at `g = 1`.
`ema_cos_gate`	`Optional[float]`	q95 of competitor EMA cosines at `g = 1`.
`gallery_cos_gate`	`Optional[float]`	q95 of competitor gallery-max cosines at `g = 1`.
`mu_pos`	`Optional[float]`	Median true EMA cosine at `g = 1`.
`mu_neg`	`Optional[float]`	Median competitor EMA cosine at `g = 1`.
`link_thresh`	`Optional[float]`	q99 of competitor linking statistics at the long gap, for a full query (:data:`LINK_QUERY_LEN` vectors, gallery :data:`LINK_GALLERY_LEN`).
`link_margin`	`Optional[float]`	Smallest margin in :data:`LINK_MARGIN_GRID` giving linking precision `>= LINK_PRECISION` on the trials, or `None`.
`link_recall`	`Optional[float]`	Recall of the calibrated linking rule on the trials.
`link_thresh_by_len`	`tuple`	`link_thresh` for a query of `n` vectors at index `n - 1` (`n = 1 .. LINK_QUERY_LEN`; the last equals `link_thresh`): a short query is noisier, so it gets its own, stricter threshold. Empty when not calibrated.
`link_margin_by_len`	`tuple`	`link_margin` per query length, likewise (`None` where no grid margin reaches the precision at that length: such a query is too short for appearance to decide).
`neg_collision_rate`	`Optional[float]`	Fraction of same-frame negative pairs whose cosine is at or above the q05 of `g = 1` positives.
`appearance_for_association`	`bool`	`status == "ok"` and `assoc_rank1 >= MIN_ASSOC_RANK1`.
`appearance_for_linking`	`bool`	`status == "ok"`, `link_margin is not None` and `link_recall >= MIN_LINK_RECALL`.

Methods:

Name	Description
`as_dict`	JSON-serializable dict (NaN becomes `None`).
`summary`	One-line human-readable summary for logs.

Source code in sleap_nn/tracking/mot/calibration.py

@attrs.frozen
class VideoCalibration:
    """Calibration of one video. Every field is a plain Python type.

    Attributes:
        status: Appearance-calibration status: ``"ok"``, ``"no_vectors"``,
            ``"insufficient"`` or ``"skipped"`` (``calibrate(appearance=False)``:
            nobody reads the appearance part, so it was not computed).
        n_frames: Frames seen.
        n_detections: Valid detections seen.
        frame_step: Most common positive frame-index delta (1 if unknown).
        regular_stride: Whether at least half of the deltas equal ``frame_step``.
        time_base: The RESOLVED time base, ``"frames"`` or ``"steps"`` (see
            :func:`resolve_time_base`): with :func:`frame_steps` (``frame_step``
            in the ``"frames"`` time base) it gives every processed frame's step
            position, exactly as the trackers count steps.
        body_size: Body size (px), from the fallback chain of
            :func:`resolve_body_size`: the median raw-box diagonal of valid
            masks and multi-node poses; else (single-node poses) a
            motion-derived size; else ``nn_distance``; else 50. A
            ``body_size`` given to :func:`calibrate` replaces all of them.
        body_size_source: Where ``body_size`` came from: ``"measured"``,
            ``"motion"``, ``"nn_distance"``, ``"default"`` or ``"override"``.
        nn_distance: Median same-frame nearest-neighbor centroid distance (px),
            ignoring duplicate pairs (IoMin > 0.5); falls back to ``body_size``.
        step_p95: 95th percentile per-step centroid displacement along anchors (px).
        step_p99: 99th percentile per-step centroid displacement along anchors (px).
        score_quantiles: Score quantiles at 0.05, 0.25, 0.5, 0.75 and 0.95, of
            the RAW detection scores (before any normalization).
        n_anchors: Anchor tracklets found.
        n_trials: Appearance trials run at ``g = 1``.
        assoc_rank1: Fraction of ``g = 1`` trials where the true track's EMA
            cosine beats every co-occurring competitor's.
        m_assoc: Association margin: ``max(0.02, q10)`` of (true - best
            competitor) EMA cosine at ``g = 1``.
        ema_cos_gate: q95 of competitor EMA cosines at ``g = 1``.
        gallery_cos_gate: q95 of competitor gallery-max cosines at ``g = 1``.
        mu_pos: Median true EMA cosine at ``g = 1``.
        mu_neg: Median competitor EMA cosine at ``g = 1``.
        link_thresh: q99 of competitor linking statistics at the long gap, for
            a full query (:data:`LINK_QUERY_LEN` vectors, gallery
            :data:`LINK_GALLERY_LEN`).
        link_margin: Smallest margin in :data:`LINK_MARGIN_GRID` giving linking
            precision ``>= LINK_PRECISION`` on the trials, or ``None``.
        link_recall: Recall of the calibrated linking rule on the trials.
        link_thresh_by_len: ``link_thresh`` for a query of ``n`` vectors at index
            ``n - 1`` (``n = 1 .. LINK_QUERY_LEN``; the last equals
            ``link_thresh``): a short query is noisier, so it gets its own,
            stricter threshold. Empty when not calibrated.
        link_margin_by_len: ``link_margin`` per query length, likewise (``None``
            where no grid margin reaches the precision at that length: such a
            query is too short for appearance to decide).
        neg_collision_rate: Fraction of same-frame negative pairs whose cosine is
            at or above the q05 of ``g = 1`` positives.
        appearance_for_association: ``status == "ok"`` and ``assoc_rank1 >=
            MIN_ASSOC_RANK1``.
        appearance_for_linking: ``status == "ok"``, ``link_margin is not None``
            and ``link_recall >= MIN_LINK_RECALL``.
    """

    status: str = "no_vectors"
    n_frames: int = 0
    n_detections: int = 0
    frame_step: int = 1
    regular_stride: bool = True
    time_base: str = "frames"
    body_size: float = 50.0
    body_size_source: str = "default"
    nn_distance: float = 50.0
    step_p95: float = float("nan")
    step_p99: float = float("nan")
    score_quantiles: tuple = ()
    n_anchors: int = 0
    n_trials: int = 0
    assoc_rank1: Optional[float] = None
    m_assoc: Optional[float] = None
    ema_cos_gate: Optional[float] = None
    gallery_cos_gate: Optional[float] = None
    mu_pos: Optional[float] = None
    mu_neg: Optional[float] = None
    link_thresh: Optional[float] = None
    link_margin: Optional[float] = None
    link_recall: Optional[float] = None
    link_thresh_by_len: tuple = ()
    link_margin_by_len: tuple = ()
    neg_collision_rate: Optional[float] = None
    appearance_for_association: bool = False
    appearance_for_linking: bool = False

    def as_dict(self) -> dict:
        """JSON-serializable dict (NaN becomes ``None``)."""
        return {k: _json_safe(v) for k, v in attrs.asdict(self, recurse=False).items()}

    def summary(self) -> str:
        """One-line human-readable summary for logs."""
        stride = "regular" if self.regular_stride else "irregular"
        geometry = (
            f"{self.n_frames} frames, {self.n_detections} detections, frame step "
            f"{self.frame_step} ({stride}, time base {self.time_base}), body "
            f"{_fmt(self.body_size, 1)} px ({self.body_size_source}), "
            f"NN {_fmt(self.nn_distance, 1)} px, step p95/p99 "
            f"{_fmt(self.step_p95, 1)}/{_fmt(self.step_p99, 1)} px, "
            f"{self.n_anchors} anchors"
        )
        if self.status == "skipped":
            appearance = "appearance not calibrated (not used)"
        elif self.status != "ok":
            appearance = (
                f"appearance {self.status} ({self.n_trials} trials): association "
                "and linking by geometry only"
            )
        else:
            appearance = (
                f"appearance ok ({self.n_trials} trials): rank-1 "
                f"{_fmt(self.assoc_rank1)}, m_assoc {_fmt(self.m_assoc)}, EMA gate "
                f"{_fmt(self.ema_cos_gate)}, gallery gate "
                f"{_fmt(self.gallery_cos_gate)}, link thresh "
                f"{_fmt(self.link_thresh)} margin {_fmt(self.link_margin, 2)} "
                f"recall {_fmt(self.link_recall)}, negative collisions "
                f"{_fmt(self.neg_collision_rate)}; association "
                f"{'on' if self.appearance_for_association else 'off'}, linking "
                f"{'on' if self.appearance_for_linking else 'off'}"
            )
        return f"MOT calibration: {geometry}; {appearance}"

`as_dict()` ¶

JSON-serializable dict (NaN becomes None).

Source code in sleap_nn/tracking/mot/calibration.py

def as_dict(self) -> dict:
    """JSON-serializable dict (NaN becomes ``None``)."""
    return {k: _json_safe(v) for k, v in attrs.asdict(self, recurse=False).items()}

`summary()` ¶

One-line human-readable summary for logs.

Source code in sleap_nn/tracking/mot/calibration.py

def summary(self) -> str:
    """One-line human-readable summary for logs."""
    stride = "regular" if self.regular_stride else "irregular"
    geometry = (
        f"{self.n_frames} frames, {self.n_detections} detections, frame step "
        f"{self.frame_step} ({stride}, time base {self.time_base}), body "
        f"{_fmt(self.body_size, 1)} px ({self.body_size_source}), "
        f"NN {_fmt(self.nn_distance, 1)} px, step p95/p99 "
        f"{_fmt(self.step_p95, 1)}/{_fmt(self.step_p99, 1)} px, "
        f"{self.n_anchors} anchors"
    )
    if self.status == "skipped":
        appearance = "appearance not calibrated (not used)"
    elif self.status != "ok":
        appearance = (
            f"appearance {self.status} ({self.n_trials} trials): association "
            "and linking by geometry only"
        )
    else:
        appearance = (
            f"appearance ok ({self.n_trials} trials): rank-1 "
            f"{_fmt(self.assoc_rank1)}, m_assoc {_fmt(self.m_assoc)}, EMA gate "
            f"{_fmt(self.ema_cos_gate)}, gallery gate "
            f"{_fmt(self.gallery_cos_gate)}, link thresh "
            f"{_fmt(self.link_thresh)} margin {_fmt(self.link_margin, 2)} "
            f"recall {_fmt(self.link_recall)}, negative collisions "
            f"{_fmt(self.neg_collision_rate)}; association "
            f"{'on' if self.appearance_for_association else 'off'}, linking "
            f"{'on' if self.appearance_for_linking else 'off'}"
        )
    return f"MOT calibration: {geometry}; {appearance}"

`calibrate(frames, *, track_buffer=30, gallery_size=30, query_len=LINK_QUERY_LEN, ema_alpha=0.8, max_trials=2000, seed=0, time_base='auto', body_size=None, appearance=True)` ¶

Calibrate a video from all of its frames' detections.

Procedure (deterministic given seed):

Geometry: frame_step / regular_stride from the frame indices and the resolved time_base (:func:frame_stride, :func:resolve_time_base; step positions from :func:frame_steps); nn_distance (median same-frame nearest-neighbor centroid distance, duplicate pairs ignored) and the score quantiles of the valid detections.
Anchor tracklets: detections of consecutive processed frames are chained when they are mutual nearest neighbors closer than 0.5 * nn_distance (no distance gate when no frame holds two detections: a lone detection has no competitor), the runner-up is >= 2x farther in both directions, and neither is a same-frame duplicate (IoMin > 0.5); chains are cut at any violation and kept when >= :data:MIN_ANCHOR_LEN long. step_p95 / step_p99 are per-STEP displacements along the anchor links.
Body size (:func:resolve_body_size): the median raw-box diagonal of valid masks and poses with >= 2 visible nodes; else (single-node poses) MOTION_BODY_FACTOR * step_p99 clipped to [MIN_MOTION_BODY, nn_distance]; else nn_distance; else 50 px. body_size_source records which; a WARNING is logged when only the spacing or the default was left.
Appearance trials at g = 1 and g = track_buffer steps: for anchor a split every stride vectors (clip(candidates // max_trials, 1, MAX_TRIAL_STRIDE)) at step t, the true memory is a's last gallery_size vectors at or before t - g plus its running EMA there (seeded and renormalized like :class:~sleap_nn.tracking.mot.appearance.AppearanceMemory), the query is a's next query_len vectors from t, and the competitors are the anchors with a vector in the true gallery's time window (they share a frame with a, so they are different animals), with memories built the same way. Statistics: s_ema = cos(q, ema), s_gal = max cos(q, gallery) (q the first query vector) and s_link = mean over the query of the top-5 mean gallery cosine. See :class:VideoCalibration for the thresholds. At most max_trials per gap (subsampled with seed). status is "no_vectors" without vectors and "insufficient" with fewer than :data:MIN_TRIALS g = 1 trials having a co-occurring competitor (e.g. a single animal); linking thresholds also need :data:MIN_TRIALS trials at the long gap. The long-gap (linking) trials always use the linker's gallery length :data:LINK_GALLERY_LEN and calibrate the rule for every query length up to query_len (link_thresh_by_len).

Logs a one-line INFO summary, and a WARNING when more than :data:NEG_COLLISION_WARN of the same-frame pairs of different animals look like the same animal (the embedding may encode class/sex, not identity).

Parameters:

Name	Type	Description	Default
`frames`	`Sequence[FrameDetections]`	The video's :class:`FrameDetections`, in temporal order.	required
`track_buffer`	`int`	Long-gap length (steps) for the linking statistic.	`30`
`gallery_size`	`int`	Gallery length of the `g = 1` trials: the length of the tracker's appearance memory, which `gallery_cos_gate` describes (DeepSORT: `gallery_budget`). The linking trials use :data:`LINK_GALLERY_LEN` regardless.	`30`
`query_len`	`int`	Query tracklet length used for the linking statistic.	`LINK_QUERY_LEN`
`ema_alpha`	`float`	EMA momentum used for the EMA statistic.	`0.8`
`max_trials`	`int`	Cap on trials per gap (subsampled deterministically).	`2000`
`seed`	`int`	Seed for subsampling.	`0`
`time_base`	`str`	How time advances between processed frames, as :class:`~sleap_nn.tracking.mot.params.MOTParams` `time_base` (`"auto"`, `"frames"` or `"steps"`); sets the step unit of the displacement statistics and the gaps. The resolved value is `VideoCalibration.time_base`.	`'auto'`
`body_size`	`Optional[float]`	A known body size (px, e.g. `MOTParams.body_size`) that replaces the measured one (`body_size_source="override"`).	`None`
`appearance`	`bool`	Calibrate the appearance part. `False` (a geometry-only tracker without identity linking: nothing reads it) skips the vector work entirely and reports `status="skipped"`; the geometry is identical either way.	`True`

Returns:

Name	Type	Description
`The`	`VideoCalibration`	class:`VideoCalibration`. Never raises on degenerate input (empty video, single animal, no vectors).

Raises:

Type	Description
`ValueError`	On an unknown `time_base`.

Source code in sleap_nn/tracking/mot/calibration.py

def calibrate(
    frames: Sequence[FrameDetections],
    *,
    track_buffer: int = 30,
    gallery_size: int = 30,
    query_len: int = LINK_QUERY_LEN,
    ema_alpha: float = 0.8,
    max_trials: int = 2000,
    seed: int = 0,
    time_base: str = "auto",
    body_size: Optional[float] = None,
    appearance: bool = True,
) -> VideoCalibration:
    """Calibrate a video from all of its frames' detections.

    Procedure (deterministic given ``seed``):

    1. **Geometry:** ``frame_step`` / ``regular_stride`` from the frame indices
       and the resolved ``time_base`` (:func:`frame_stride`,
       :func:`resolve_time_base`; step positions from :func:`frame_steps`);
       ``nn_distance`` (median same-frame nearest-neighbor centroid distance,
       duplicate pairs ignored) and the score quantiles of the valid detections.
    2. **Anchor tracklets:** detections of consecutive processed frames are
       chained when they are mutual nearest neighbors closer than ``0.5 *
       nn_distance`` (no distance gate when no frame holds two detections: a
       lone detection has no competitor), the runner-up is >= 2x farther in
       both directions, and neither is a same-frame duplicate (IoMin > 0.5);
       chains are cut at any violation and kept when >= :data:`MIN_ANCHOR_LEN`
       long. ``step_p95`` / ``step_p99`` are per-STEP displacements along the
       anchor links.
    3. **Body size** (:func:`resolve_body_size`): the median raw-box diagonal of
       valid masks and poses with >= 2 visible nodes; else (single-node poses)
       ``MOTION_BODY_FACTOR * step_p99`` clipped to ``[MIN_MOTION_BODY,
       nn_distance]``; else ``nn_distance``; else 50 px. ``body_size_source``
       records which; a WARNING is logged when only the spacing or the default
       was left.
    4. **Appearance trials** at ``g = 1`` and ``g = track_buffer`` steps: for
       anchor ``a`` split every ``stride`` vectors (``clip(candidates //
       max_trials, 1, MAX_TRIAL_STRIDE)``) at step ``t``, the
       true memory is ``a``'s last ``gallery_size`` vectors at or before ``t - g``
       plus its running EMA there (seeded and renormalized like
       :class:`~sleap_nn.tracking.mot.appearance.AppearanceMemory`), the query
       is ``a``'s next ``query_len`` vectors from ``t``, and the competitors are
       the anchors with a vector in the true gallery's time window (they share
       a frame with ``a``, so they are different animals), with memories built
       the same way. Statistics: ``s_ema = cos(q, ema)``, ``s_gal = max cos(q,
       gallery)`` (``q`` the first query vector) and ``s_link`` = mean over the
       query of the top-5 mean gallery cosine. See :class:`VideoCalibration` for
       the thresholds. At most ``max_trials`` per gap (subsampled with
       ``seed``). ``status`` is
       ``"no_vectors"`` without vectors and ``"insufficient"`` with fewer than
       :data:`MIN_TRIALS` ``g = 1`` trials having a co-occurring competitor (e.g.
       a single animal); linking thresholds also need :data:`MIN_TRIALS` trials
       at the long gap. The long-gap (linking) trials always use the linker's
       gallery length :data:`LINK_GALLERY_LEN` and calibrate the rule for every
       query length up to ``query_len`` (``link_thresh_by_len``).

    Logs a one-line INFO summary, and a WARNING when more than
    :data:`NEG_COLLISION_WARN` of the same-frame pairs of different animals look
    like the same animal (the embedding may encode class/sex, not identity).

    Args:
        frames: The video's :class:`FrameDetections`, in temporal order.
        track_buffer: Long-gap length (steps) for the linking statistic.
        gallery_size: Gallery length of the ``g = 1`` trials: the length of the
            tracker's appearance memory, which ``gallery_cos_gate`` describes
            (DeepSORT: ``gallery_budget``). The linking trials use
            :data:`LINK_GALLERY_LEN` regardless.
        query_len: Query tracklet length used for the linking statistic.
        ema_alpha: EMA momentum used for the EMA statistic.
        max_trials: Cap on trials per gap (subsampled deterministically).
        seed: Seed for subsampling.
        time_base: How time advances between processed frames, as
            :class:`~sleap_nn.tracking.mot.params.MOTParams` ``time_base``
            (``"auto"``, ``"frames"`` or ``"steps"``); sets the step unit of the
            displacement statistics and the gaps. The resolved value is
            ``VideoCalibration.time_base``.
        body_size: A known body size (px, e.g. ``MOTParams.body_size``) that
            replaces the measured one (``body_size_source="override"``).
        appearance: Calibrate the appearance part. ``False`` (a geometry-only
            tracker without identity linking: nothing reads it) skips the
            vector work entirely and reports ``status="skipped"``; the geometry
            is identical either way.

    Returns:
        The :class:`VideoCalibration`. Never raises on degenerate input (empty
        video, single animal, no vectors).

    Raises:
        ValueError: On an unknown ``time_base``.
    """
    if time_base not in ("auto",) + RESOLVED_TIME_BASES:
        raise ValueError(
            f"time_base must be 'auto', 'frames' or 'steps', got {time_base!r}."
        )
    frames = list(frames)
    gallery_size = max(1, int(gallery_size))
    query_len = max(1, int(query_len))
    max_trials = max(0, int(max_trials))

    frame_idx = np.array([int(fd.frame_idx) for fd in frames], dtype=np.int64)
    frame_step, regular = frame_stride(frame_idx)
    resolved_time_base = resolve_time_base(time_base, regular)
    step_idx = frame_steps(frame_idx, resolved_time_base, frame_step)
    steps = np.diff(step_idx)
    flat = _flatten(frames, vectors=bool(appearance))
    n_total = len(flat.frame_pos)

    # Same-frame geometry: spacing, duplicates, negative cosines.
    nn_dist, duplicate, neg_cos = _same_frame_pairs(flat)
    finite_nn = nn_dist[np.isfinite(nn_dist)]
    nn_measured = float(np.median(finite_nn)) if finite_nn.size else float("nan")
    diag = np.hypot(*(flat.raw_boxes[:, 2:] - flat.raw_boxes[:, :2]).T)
    diag = diag[flat.sized & np.isfinite(diag) & (diag > 0)]
    measured_body = float(np.median(diag)) if diag.size else float("nan")
    finite_scores = flat.scores[np.isfinite(flat.scores)]
    score_quantiles = (
        tuple(float(q) for q in np.quantile(finite_scores, SCORE_QUANTILE_LEVELS))
        if finite_scores.size
        else ()
    )

    # Anchor tracklets. Without any same-frame pair (one animal), a link has no
    # competitor: no distance gate.
    max_link = (
        _ANCHOR_MAX_NN_FRACTION * nn_measured if np.isfinite(nn_measured) else np.inf
    )
    nxt = _anchor_links(flat, duplicate, max_link)
    roots = _chain_roots(nxt)
    lengths = np.bincount(roots, minlength=n_total)
    kept = lengths[roots] >= MIN_ANCHOR_LEN
    anchor_roots = np.unique(roots[kept])
    anchor_of = np.full(n_total, -1, dtype=np.int64)
    anchor_of[kept] = np.searchsorted(anchor_roots, roots[kept])
    n_anchors = len(anchor_roots)

    step_p95 = step_p99 = float("nan")
    src = np.flatnonzero((nxt >= 0) & kept)
    if src.size:
        disp = np.linalg.norm(flat.centroids[nxt[src]] - flat.centroids[src], axis=1)
        disp = disp / steps[flat.frame_pos[src]]
        step_p95, step_p99 = (float(v) for v in np.percentile(disp, [95, 99]))

    # Body size (fallback chain), then the spacing's own fallback.
    if body_size is not None and np.isfinite(body_size) and body_size > 0:
        body, body_source = float(body_size), "override"
    else:
        body, body_source = resolve_body_size(measured_body, step_p99, nn_measured)
    nn_distance = nn_measured if np.isfinite(nn_measured) else body

    if appearance:
        appearance = _appearance(
            flat,
            anchor_of,
            n_anchors,
            step_idx,
            neg_cos,
            track_buffer=track_buffer,
            gallery_size=gallery_size,
            query_len=query_len,
            ema_alpha=ema_alpha,
            max_trials=max_trials,
            seed=int(seed),
        )
    else:
        appearance = {"status": "skipped"}

    cal = VideoCalibration(
        status=str(appearance.get("status", "no_vectors")),
        n_frames=len(frames),
        n_detections=int(n_total),
        frame_step=int(frame_step),
        regular_stride=bool(regular),
        time_base=resolved_time_base,
        body_size=float(body),
        body_size_source=body_source,
        nn_distance=float(nn_distance),
        step_p95=step_p95,
        step_p99=step_p99,
        score_quantiles=score_quantiles,
        n_anchors=int(n_anchors),
        n_trials=int(appearance.get("n_trials", 0)),
        assoc_rank1=_opt_float(appearance.get("assoc_rank1")),
        m_assoc=_opt_float(appearance.get("m_assoc")),
        ema_cos_gate=_opt_float(appearance.get("ema_cos_gate")),
        gallery_cos_gate=_opt_float(appearance.get("gallery_cos_gate")),
        mu_pos=_opt_float(appearance.get("mu_pos")),
        mu_neg=_opt_float(appearance.get("mu_neg")),
        link_thresh=_opt_float(appearance.get("link_thresh")),
        link_margin=_opt_float(appearance.get("link_margin")),
        link_recall=_opt_float(appearance.get("link_recall")),
        link_thresh_by_len=tuple(
            _opt_float(v) for v in appearance.get("link_thresh_by_len", ())
        ),
        link_margin_by_len=tuple(
            _opt_float(v) for v in appearance.get("link_margin_by_len", ())
        ),
        neg_collision_rate=_opt_float(appearance.get("neg_collision_rate")),
        appearance_for_association=bool(
            appearance.get("appearance_for_association", False)
        ),
        appearance_for_linking=bool(appearance.get("appearance_for_linking", False)),
    )
    logger.info(cal.summary())
    if n_total and body_source in ("nn_distance", "default"):
        logger.warning(
            "MOT calibration: no body size could be measured (no detection with "
            "an extent, e.g. single-node poses) and too little unambiguous motion "
            f"to size it from; using the {body_source.replace('_', ' ')} "
            f"({body:.1f} px) as the body size, which sets the centroid-similarity "
            "gate and pseudo-box size. Pass the animal length in px as "
            "MOTParams.body_size (--mot_param body_size=<px>) if it is far off."
        )
    if (
        cal.neg_collision_rate is not None
        and cal.neg_collision_rate > NEG_COLLISION_WARN
    ):
        logger.warning(
            f"MOT calibration: {cal.neg_collision_rate:.1%} of same-frame pairs of "
            "different animals are as similar as the same animal one step apart "
            f"(more than {NEG_COLLISION_WARN:.0%}): the embedding may encode "
            "class/sex rather than identity."
        )
    return cal

`frame_steps(frame_indices, time_base, frame_step=1)` ¶

Step position of each processed frame (the first at step 0).

"frames": consecutive frames are :func:steps_between steps apart, so a strided video behaves like a dense one and genuine gaps still count; "steps": one step per processed frame. With a calibration, call it as frame_steps(idx, cal.time_base, cal.frame_step).

Parameters:

Name	Type	Description	Default
`frame_indices`	`Sequence[int]`	Frame indices of the processed frames, in processing order.	required
`time_base`	`str`	The RESOLVED time base, `"frames"` or `"steps"`.	required
`frame_step`	`int`	The stride (used in the `"frames"` time base).	`1`

Returns:

Type	Description
`ndarray`	`(F,)` int64 step positions.

Raises:

Type	Description
`ValueError`	On a time base other than `"frames"` / `"steps"`.

Source code in sleap_nn/tracking/mot/calibration.py

def frame_steps(
    frame_indices: Sequence[int], time_base: str, frame_step: int = 1
) -> np.ndarray:
    """Step position of each processed frame (the first at step 0).

    ``"frames"``: consecutive frames are :func:`steps_between` steps apart, so
    a strided video behaves like a dense one and genuine gaps still count;
    ``"steps"``: one step per processed frame. With a calibration, call it as
    ``frame_steps(idx, cal.time_base, cal.frame_step)``.

    Args:
        frame_indices: Frame indices of the processed frames, in processing order.
        time_base: The RESOLVED time base, ``"frames"`` or ``"steps"``.
        frame_step: The stride (used in the ``"frames"`` time base).

    Returns:
        ``(F,)`` int64 step positions.

    Raises:
        ValueError: On a time base other than ``"frames"`` / ``"steps"``.
    """
    if time_base not in RESOLVED_TIME_BASES:
        raise ValueError(
            f"frame_steps needs a resolved time base ('frames' or 'steps'), got "
            f"{time_base!r}."
        )
    idx = np.asarray(frame_indices, dtype=np.float64).reshape(-1)
    if len(idx) == 0:
        return np.zeros(0, dtype=np.int64)
    if time_base == "steps":
        return np.arange(len(idx), dtype=np.int64)
    inc = np.asarray(steps_between(np.diff(idx), frame_step), dtype=np.int64)
    return np.concatenate([[0], np.cumsum(inc)]).astype(np.int64)

`frame_stride(frame_indices)` ¶

(frame_step, regular) of a sequence of frame indices.

See :func:stride_from_counts (applied to the positive consecutive deltas).

Source code in sleap_nn/tracking/mot/calibration.py

def frame_stride(frame_indices: Sequence[int]) -> Tuple[int, bool]:
    """``(frame_step, regular)`` of a sequence of frame indices.

    See :func:`stride_from_counts` (applied to the positive consecutive deltas).
    """
    deltas = np.diff(np.asarray(frame_indices, dtype=np.int64).reshape(-1))
    values, counts = np.unique(deltas[deltas > 0], return_counts=True)
    return stride_from_counts(dict(zip(values.tolist(), counts.tolist())))

`link_rule_tables(calibration, query_len=LINK_QUERY_LEN)` ¶

The calibrated linking rule for each query length.

Entry n - 1 is the (threshold, margin) for a query of n vectors (n = 1 .. query_len; longer calibrated tables are cut, shorter ones repeat their last entry). Without per-length tables (a hand-built or older calibration) the full-length link_thresh / link_margin apply to every length. A length whose margin is unknown (no grid margin reached the precision there) gets NaN for BOTH: appearance cannot decide such a query.

Parameters:

Name	Type	Description	Default
`calibration`	`Optional[VideoCalibration]`	The video's calibration, or `None`.	required
`query_len`	`int`	Longest query.	`LINK_QUERY_LEN`

Returns:

Type	Description
`Tuple[ndarray, ndarray]`	`(thresholds, margins)`, two `(query_len,)` float arrays (NaN where undecidable).

Source code in sleap_nn/tracking/mot/calibration.py

def link_rule_tables(
    calibration: Optional["VideoCalibration"], query_len: int = LINK_QUERY_LEN
) -> Tuple[np.ndarray, np.ndarray]:
    """The calibrated linking rule for each query length.

    Entry ``n - 1`` is the ``(threshold, margin)`` for a query of ``n`` vectors
    (``n = 1 .. query_len``; longer calibrated tables are cut, shorter ones
    repeat their last entry). Without per-length tables (a hand-built or older
    calibration) the full-length ``link_thresh`` / ``link_margin`` apply to every
    length. A length whose margin is unknown (no grid margin reached the
    precision there) gets NaN for BOTH: appearance cannot decide such a query.

    Args:
        calibration: The video's calibration, or ``None``.
        query_len: Longest query.

    Returns:
        ``(thresholds, margins)``, two ``(query_len,)`` float arrays (NaN where
        undecidable).
    """
    q = max(1, int(query_len))
    thresh = np.full(q, np.nan)
    margin = np.full(q, np.nan)
    if calibration is None:
        return thresh, margin

    def fill(out: np.ndarray, table: Sequence, scalar: Optional[float]) -> None:
        if len(table):
            for n in range(q):
                v = table[min(n, len(table) - 1)]
                out[n] = np.nan if v is None else float(v)
        elif scalar is not None:
            out[:] = float(scalar)

    fill(thresh, tuple(calibration.link_thresh_by_len), calibration.link_thresh)
    fill(margin, tuple(calibration.link_margin_by_len), calibration.link_margin)
    bad = ~(np.isfinite(thresh) & np.isfinite(margin))
    thresh[bad] = np.nan
    margin[bad] = np.nan
    return thresh, margin

`pair_geometry(prev, cur, max_dist=None)` ¶

Same-frame spacing of cur and its unambiguous links to prev.

The online counterpart of :func:calibrate's geometry (used by uncalibrated trackers), with the same rules: duplicate pairs (IoMin > 0.5) are not neighbors, and a prev -> cur link needs mutual nearest neighbors closer than max_dist whose runner-ups are at least twice farther.

Parameters:

Name	Type	Description	Default
`prev`	`Optional[FrameDetections]`	The previous processed frame's detections (`None`: no links).	required
`cur`	`FrameDetections`	This frame's detections. Pose pseudo boxes (degenerate geometry extracted with a `pseudo_box_size`) are replaced by the keypoint extent floored at 1 px, the geometry :func:`calibrate` sees.	required
`max_dist`	`Optional[float]`	Link gate (px). `None`: half the median same-frame spacing of the two frames when measurable, else no gate.	`None`

Returns:

Type	Description
`Tuple[ndarray, ndarray]`	`(nn, links)`: the finite nearest-neighbor distances of `cur`'s valid detections and the lengths (px) of the unambiguous links.

Source code in sleap_nn/tracking/mot/calibration.py

def pair_geometry(
    prev: Optional[FrameDetections],
    cur: FrameDetections,
    max_dist: Optional[float] = None,
) -> Tuple[np.ndarray, np.ndarray]:
    """Same-frame spacing of ``cur`` and its unambiguous links to ``prev``.

    The online counterpart of :func:`calibrate`'s geometry (used by uncalibrated
    trackers), with the same rules: duplicate pairs (IoMin > 0.5) are not
    neighbors, and a ``prev -> cur`` link needs mutual nearest neighbors closer
    than ``max_dist`` whose runner-ups are at least twice farther.

    Args:
        prev: The previous processed frame's detections (``None``: no links).
        cur: This frame's detections. Pose pseudo boxes (degenerate geometry
            extracted with a ``pseudo_box_size``) are replaced by the keypoint
            extent floored at 1 px, the geometry :func:`calibrate` sees.
        max_dist: Link gate (px). ``None``: half the median same-frame spacing
            of the two frames when measurable, else no gate.

    Returns:
        ``(nn, links)``: the finite nearest-neighbor distances of ``cur``'s valid
        detections and the lengths (px) of the unambiguous links.
    """
    frames = [_keypoint_extents(f) for f in ([cur] if prev is None else [prev, cur])]
    flat = _flatten(frames)
    flat.vectors = None  # Geometry only.
    nn_dist, duplicate, _ = _same_frame_pairs(flat)
    nn = nn_dist[flat.frame_pos == len(frames) - 1]
    nn = nn[np.isfinite(nn)]
    if prev is None:
        return nn, np.zeros(0, dtype=np.float64)
    if max_dist is None:
        finite = nn_dist[np.isfinite(nn_dist)]
        max_dist = (
            _ANCHOR_MAX_NN_FRACTION * np.median(finite) if finite.size else np.inf
        )
    nxt = _anchor_links(flat, duplicate, float(max_dist))
    src = np.flatnonzero(nxt >= 0)
    links = np.linalg.norm(flat.centroids[nxt[src]] - flat.centroids[src], axis=1)
    return nn, links

`resolve_body_size(measured, step_p99, nn_distance)` ¶

The body size (px) and its source (the shared fallback chain).

Used by :func:calibrate and by the uncalibrated trackers:

measured (median raw-box diagonal of sized detections) -> "measured".
Else (single-node detections have no extent) the per-step motion: MOTION_BODY_FACTOR * step_p99, at least :data:MIN_MOTION_BODY and at most nn_distance (when known) -> "motion". Inter-animal spacing is not a body length: in a sparse arena it would open the centroid gate (scale body / 2) most of the way to the next animal.
Else nn_distance -> "nn_distance".
Else :data:DEFAULT_BODY_SIZE (50 px) -> "default".

Parameters:

Name	Type	Description	Default
`measured`	`float`	Measured body size (NaN if none).	required
`step_p99`	`float`	99^th percentile per-step displacement (NaN if unknown).	required
`nn_distance`	`float`	Same-frame nearest-neighbor spacing (NaN if unknown).	required

Returns:

Type	Description
`Tuple[float, str]`	`(body_size, source)`.

Source code in sleap_nn/tracking/mot/calibration.py

def resolve_body_size(
    measured: float, step_p99: float, nn_distance: float
) -> Tuple[float, str]:
    """The body size (px) and its source (the shared fallback chain).

    Used by :func:`calibrate` and by the uncalibrated trackers:

    1. ``measured`` (median raw-box diagonal of sized detections) ->
       ``"measured"``.
    2. Else (single-node detections have no extent) the per-step motion:
       ``MOTION_BODY_FACTOR * step_p99``, at least :data:`MIN_MOTION_BODY` and at
       most ``nn_distance`` (when known) -> ``"motion"``. Inter-animal spacing is
       not a body length: in a sparse arena it would open the centroid gate
       (scale ``body / 2``) most of the way to the next animal.
    3. Else ``nn_distance`` -> ``"nn_distance"``.
    4. Else :data:`DEFAULT_BODY_SIZE` (50 px) -> ``"default"``.

    Args:
        measured: Measured body size (NaN if none).
        step_p99: 99th percentile per-step displacement (NaN if unknown).
        nn_distance: Same-frame nearest-neighbor spacing (NaN if unknown).

    Returns:
        ``(body_size, source)``.
    """

    def ok(v: float) -> bool:
        return v is not None and bool(np.isfinite(v)) and v > 0

    if ok(measured):
        return float(measured), "measured"
    if step_p99 is not None and np.isfinite(step_p99) and step_p99 >= 0:
        body = max(MOTION_BODY_FACTOR * float(step_p99), MIN_MOTION_BODY)
        if ok(nn_distance):
            body = min(body, float(nn_distance))
        return float(body), "motion"
    if ok(nn_distance):
        return float(nn_distance), "nn_distance"
    return DEFAULT_BODY_SIZE, "default"

`resolve_time_base(time_base, regular)` ¶

Resolve MOTParams.time_base to "frames" or "steps".

Parameters:

Name	Type	Description	Default
`time_base`	`str`	`"auto"`, `"frames"` or `"steps"`.	required
`regular`	`bool`	Whether the processed frames are regularly strided.	required

Returns:

Type	Description
`str`	`time_base` itself when explicit; for `"auto"`, `"frames"` when the stride is regular, else `"steps"`.

Raises:

Type	Description
`ValueError`	On an unknown `time_base`.

Source code in sleap_nn/tracking/mot/calibration.py

def resolve_time_base(time_base: str, regular: bool) -> str:
    """Resolve ``MOTParams.time_base`` to ``"frames"`` or ``"steps"``.

    Args:
        time_base: ``"auto"``, ``"frames"`` or ``"steps"``.
        regular: Whether the processed frames are regularly strided.

    Returns:
        ``time_base`` itself when explicit; for ``"auto"``, ``"frames"`` when the
        stride is regular, else ``"steps"``.

    Raises:
        ValueError: On an unknown ``time_base``.
    """
    if time_base == "auto":
        return "frames" if regular else "steps"
    if time_base not in RESOLVED_TIME_BASES:
        raise ValueError(
            f"time_base must be 'auto', 'frames' or 'steps', got {time_base!r}."
        )
    return time_base

`score_normalizer(calibration)` ¶

Per-video score normalizer of MOTParams.normalize_scores.

max(q95, SCORE_NORMALIZER_FLOOR) of the calibration's (raw) score quantiles; 1.0 without a calibration or scores. Dividing a detection score by it (then clipping to [0, 1], see :func:~sleap_nn.tracking.mot.detections.normalize_scores) maps a typical detection of the video to ~0.85-1.0 whatever the model's score scale.

Parameters:

Name	Type	Description	Default
`calibration`	`Optional[VideoCalibration]`	The video's calibration, or `None`.	required

Returns:

Type	Description
`float`	The normalizer (> 0).

Source code in sleap_nn/tracking/mot/calibration.py

def score_normalizer(calibration: Optional["VideoCalibration"]) -> float:
    """Per-video score normalizer of ``MOTParams.normalize_scores``.

    ``max(q95, SCORE_NORMALIZER_FLOOR)`` of the calibration's (raw) score
    quantiles; 1.0 without a calibration or scores. Dividing a detection score by
    it (then clipping to ``[0, 1]``, see
    :func:`~sleap_nn.tracking.mot.detections.normalize_scores`) maps a typical
    detection of the video to ~0.85-1.0 whatever the model's score scale.

    Args:
        calibration: The video's calibration, or ``None``.

    Returns:
        The normalizer (> 0).
    """
    if calibration is None or len(calibration.score_quantiles) < len(
        SCORE_QUANTILE_LEVELS
    ):
        return 1.0
    q95 = float(calibration.score_quantiles[SCORE_QUANTILE_LEVELS.index(0.95)])
    if not np.isfinite(q95):
        return 1.0
    return max(q95, SCORE_NORMALIZER_FLOOR)

`steps_between(delta, frame_step)` ¶

Time steps between two processed frames delta frame indices apart.

The "frames" time base: max(1, round(delta / frame_step)), rounding half to even (np.rint, like Python's round) -- the ONE rounding rule of the trackers, :func:calibrate and the identity linker. A non-positive delta (a repeated or backwards frame) counts one step. In the "steps" time base every processed frame is one step and this is not used.

Parameters:

Name	Type	Description	Default
`delta`	`Union[int, float, ndarray]`	Frame-index difference(s) to the previous processed frame.	required
`frame_step`	`int`	The stride (floored at 1).	required

Returns:

Type	Description
`Union[int, ndarray]`	An `int` for a scalar `delta`, else an `int64` array.

Source code in sleap_nn/tracking/mot/calibration.py

def steps_between(
    delta: Union[int, float, np.ndarray], frame_step: int
) -> Union[int, np.ndarray]:
    """Time steps between two processed frames ``delta`` frame indices apart.

    The ``"frames"`` time base: ``max(1, round(delta / frame_step))``, rounding
    half to even (``np.rint``, like Python's ``round``) -- the ONE rounding rule
    of the trackers, :func:`calibrate` and the identity linker. A non-positive
    ``delta`` (a repeated or backwards frame) counts one step. In the
    ``"steps"`` time base every processed frame is one step and this is not used.

    Args:
        delta: Frame-index difference(s) to the previous processed frame.
        frame_step: The stride (floored at 1).

    Returns:
        An ``int`` for a scalar ``delta``, else an ``int64`` array.
    """
    frame_step = max(1, int(frame_step))
    steps = np.maximum(
        1.0, np.rint(np.asarray(delta, dtype=np.float64) / frame_step)
    ).astype(np.int64)
    return int(steps) if steps.ndim == 0 else steps

`stride_from_counts(counts)` ¶

Processing stride from the counts of the positive frame-index deltas.

The stride (frame_step) is the most common positive delta (ties: the smallest); it is regular when at least half of the deltas equal it. This is the rule of :func:calibrate (over the whole video) and of an uncalibrated tracker (over the deltas seen so far).

Parameters:

Name	Type	Description	Default
`counts`	`Mapping[int, int]`	`{delta: count}`; non-positive deltas and counts are ignored.	required

Returns:

Type	Description
`Tuple[int, bool]`	`(frame_step, regular)`; `(1, True)` when there is no positive delta.

Source code in sleap_nn/tracking/mot/calibration.py

def stride_from_counts(counts: Mapping[int, int]) -> Tuple[int, bool]:
    """Processing stride from the counts of the positive frame-index deltas.

    The stride (``frame_step``) is the most common positive delta (ties: the
    smallest); it is regular when at least half of the deltas equal it. This is
    the rule of :func:`calibrate` (over the whole video) and of an uncalibrated
    tracker (over the deltas seen so far).

    Args:
        counts: ``{delta: count}``; non-positive deltas and counts are ignored.

    Returns:
        ``(frame_step, regular)``; ``(1, True)`` when there is no positive delta.
    """
    kept = {int(d): int(c) for d, c in counts.items() if int(d) > 0 and int(c) > 0}
    if not kept:
        return 1, True
    frame_step = min(kept, key=lambda d: (-kept[d], d))
    return frame_step, bool(kept[frame_step] >= 0.5 * sum(kept.values()))

calibration

sleap_nn.tracking.mot.calibration ¶

VideoCalibration ¶

as_dict() ¶

summary() ¶

calibrate(frames, *, track_buffer=30, gallery_size=30, query_len=LINK_QUERY_LEN, ema_alpha=0.8, max_trials=2000, seed=0, time_base='auto', body_size=None, appearance=True) ¶

frame_steps(frame_indices, time_base, frame_step=1) ¶

frame_stride(frame_indices) ¶

link_rule_tables(calibration, query_len=LINK_QUERY_LEN) ¶

pair_geometry(prev, cur, max_dist=None) ¶

resolve_body_size(measured, step_p99, nn_distance) ¶

resolve_time_base(time_base, regular) ¶

score_normalizer(calibration) ¶

steps_between(delta, frame_step) ¶

stride_from_counts(counts) ¶

`sleap_nn.tracking.mot.calibration` ¶

`VideoCalibration` ¶

`as_dict()` ¶

`summary()` ¶

`calibrate(frames, *, track_buffer=30, gallery_size=30, query_len=LINK_QUERY_LEN, ema_alpha=0.8, max_trials=2000, seed=0, time_base='auto', body_size=None, appearance=True)` ¶

`frame_steps(frame_indices, time_base, frame_step=1)` ¶

`frame_stride(frame_indices)` ¶

`link_rule_tables(calibration, query_len=LINK_QUERY_LEN)` ¶

`pair_geometry(prev, cur, max_dist=None)` ¶

`resolve_body_size(measured, step_p99, nn_distance)` ¶

`resolve_time_base(time_base, regular)` ¶

`score_normalizer(calibration)` ¶

`steps_between(delta, frame_step)` ¶

`stride_from_counts(counts)` ¶