Skip to content

postprocessing

sleap_nn.inference.postprocessing

Inference-level postprocessing filters for pose predictions.

This module provides filters that run after model inference but before tracking. These filters are independent of tracking configuration and can be used standalone.

Functions:

Name Description
filter_overlapping_instances

Filter overlapping instances using greedy non-maximum suppression.

filter_overlapping_instances(labels, threshold=0.8, method='iou')

Filter overlapping instances using greedy non-maximum suppression.

Removes duplicate/overlapping instances by applying greedy NMS based on either bounding box IOU or Object Keypoint Similarity (OKS). When two instances overlap above the threshold, the lower-scoring one is removed.

This filter runs independently of tracking and can be used to clean up model outputs before saving or further processing.

Parameters:

Name Type Description Default
labels Labels

Labels object with predicted instances to filter.

 required
threshold float

Similarity threshold for considering instances as overlapping. Instances with similarity > threshold are candidates for removal. Lower values are more aggressive (remove more). Typical values: 0.3 (aggressive) to 0.8 (permissive).

 0.8
method Literal['iou', 'oks']

Similarity metric to use for comparing instances. "iou": Bounding box intersection-over-union. "oks": Object Keypoint Similarity (pose-based).

 'iou'

Returns:

Type Description
Labels

The input Labels object with overlapping instances removed. Modification is done in place, but the object is also returned for convenience.
Example

Filter instances with >80% bounding box overlap

labels = filter_overlapping_instances(labels, threshold=0.8, method="iou")

Filter using OKS similarity

labels = filter_overlapping_instances(labels, threshold=0.5, method="oks")

Note
  • Only affects frames with 2+ predicted instances
  • Uses instance.score for ranking; higher scores are preferred
  • For IOU: bounding boxes computed from non-NaN keypoints
  • For OKS: uses standard COCO OKS formula with bbox-derived scale
Source code in sleap_nn/inference/postprocessing.py 
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
def filter_overlapping_instances(
    labels: sio.Labels,
    threshold: float = 0.8,
    method: Literal["iou", "oks"] = "iou",
) -> sio.Labels:
    """Filter overlapping instances using greedy non-maximum suppression.

    Removes duplicate/overlapping instances by applying greedy NMS based on
    either bounding box IOU or Object Keypoint Similarity (OKS). When two
    instances overlap above the threshold, the lower-scoring one is removed.

    This filter runs independently of tracking and can be used to clean up
    model outputs before saving or further processing.

    Args:
        labels: Labels object with predicted instances to filter.
        threshold: Similarity threshold for considering instances as overlapping.
            Instances with similarity > threshold are candidates for removal.
            Lower values are more aggressive (remove more).
            Typical values: 0.3 (aggressive) to 0.8 (permissive).
        method: Similarity metric to use for comparing instances.
            "iou": Bounding box intersection-over-union.
            "oks": Object Keypoint Similarity (pose-based).

    Returns:
        The input Labels object with overlapping instances removed.
        Modification is done in place, but the object is also returned
        for convenience.

    Example:
        >>> # Filter instances with >80% bounding box overlap
        >>> labels = filter_overlapping_instances(labels, threshold=0.8, method="iou")
        >>> # Filter using OKS similarity
        >>> labels = filter_overlapping_instances(labels, threshold=0.5, method="oks")

    Note:
        - Only affects frames with 2+ predicted instances
        - Uses instance.score for ranking; higher scores are preferred
        - For IOU: bounding boxes computed from non-NaN keypoints
        - For OKS: uses standard COCO OKS formula with bbox-derived scale
    """
    for lf in labels.labeled_frames:
        if len(lf.instances) <= 1:
            continue

        # Separate predicted instances (have scores) from other instances
        predicted = []
        other = []
        for inst in lf.instances:
            if isinstance(inst, sio.PredictedInstance):
                predicted.append(inst)
            else:
                other.append(inst)

        # Only filter predicted instances
        if len(predicted) <= 1:
            continue

        # Get scores
        scores = np.array([_instance_score(inst) for inst in predicted])

        # Apply greedy NMS with selected method
        if method == "iou":
            bboxes = np.array([_instance_bbox(inst) for inst in predicted])
            keep_indices = _nms_greedy_iou(bboxes, scores, threshold)
        elif method == "oks":
            points = [inst.numpy() for inst in predicted]
            keep_indices = _nms_greedy_oks(points, scores, threshold)
        else:
            raise ValueError(f"Unknown method: {method}. Use 'iou' or 'oks'.")

        # Reconstruct instance list: kept predicted + other instances
        kept_predicted = [predicted[i] for i in keep_indices]
        lf.instances = kept_predicted + other

    return labels