Skip to content

confidence_maps

sleap_nn.data.confidence_maps

Generate confidence maps.

Functions:

Name Description
generate_confmaps

Generate Confidence maps.
generate_multiconfmaps

Generate multi-instance confidence maps.
make_confmaps

Make confidence maps from a batch of points for multiple instances.
make_multi_confmaps

Make confidence maps for multiple instances through reduction.

generate_confmaps(instance, img_hw, sigma=1.5, output_stride=2)

Generate Confidence maps.

Parameters:

Name Type Description Default
instance Tensor

Input keypoints. (n_samples, n_instances, n_nodes, 2) or (n_samples, n_nodes, 2).

 required
img_hw Tuple[int]

Image size as tuple (height, width).

 required
sigma float

The standard deviation of the Gaussian distribution that is used to generate confidence maps. Default: 1.5.

 1.5
output_stride int

The relative stride to use when generating confidence maps. A larger stride will generate smaller confidence maps. Default: 2.

 2

Returns:

Type Description
Tensor

Confidence maps for the input keypoints.
Source code in sleap_nn/data/confidence_maps.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
def generate_confmaps(
    instance: torch.Tensor,
    img_hw: Tuple[int],
    sigma: float = 1.5,
    output_stride: int = 2,
) -> torch.Tensor:
    """Generate Confidence maps.

    Args:
        instance: Input keypoints. (n_samples, n_instances, n_nodes, 2) or
            (n_samples, n_nodes, 2).
        img_hw: Image size as tuple (height, width).
        sigma: The standard deviation of the Gaussian distribution that is used to
            generate confidence maps. Default: 1.5.
        output_stride: The relative stride to use when generating confidence maps.
            A larger stride will generate smaller confidence maps. Default: 2.

    Returns:
        Confidence maps for the input keypoints.
    """
    if instance.ndim != 3:
        instance = instance.view(instance.shape[0], -1, 2)
        # instances: (n_samples, n_nodes, 2)

    height, width = img_hw

    xv, yv = make_grid_vectors(height, width, output_stride)

    confidence_maps = make_confmaps(
        instance,
        xv,
        yv,
        sigma * output_stride,
    )  # (n_samples, n_nodes, height/ output_stride, width/ output_stride)

    return confidence_maps

generate_multiconfmaps(instances, img_hw, num_instances, sigma=1.5, output_stride=2, is_centroids=False)

Generate multi-instance confidence maps.

Parameters:

Name Type Description Default
instances Tensor

Input keypoints. (n_samples, n_instances, n_nodes, 2) or for centroids - (n_samples, n_instances, 2)

 required
img_hw Tuple[int]

Image size as tuple (height, width).

 required
num_instances int

Original number of instances in the frame.

 required
sigma float

The standard deviation of the Gaussian distribution that is used to generate confidence maps. Default: 1.5.

 1.5
output_stride int

The relative stride to use when generating confidence maps. A larger stride will generate smaller confidence maps. Default: 2.

 2
is_centroids bool

True if confidence maps should be generates for centroids else False. Default: False.

 False

Returns:

Type Description
Tensor

Confidence maps for the input keypoints.
Source code in sleap_nn/data/confidence_maps.py 
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
def generate_multiconfmaps(
    instances: torch.Tensor,
    img_hw: Tuple[int],
    num_instances: int,
    sigma: float = 1.5,
    output_stride: int = 2,
    is_centroids: bool = False,
) -> torch.Tensor:
    """Generate multi-instance confidence maps.

    Args:
        instances: Input keypoints. (n_samples, n_instances, n_nodes, 2) or
            for centroids - (n_samples, n_instances, 2)
        img_hw: Image size as tuple (height, width).
        num_instances: Original number of instances in the frame.
        sigma: The standard deviation of the Gaussian distribution that is used to
            generate confidence maps. Default: 1.5.
        output_stride: The relative stride to use when generating confidence maps.
            A larger stride will generate smaller confidence maps. Default: 2.
        is_centroids: True if confidence maps should be generates for centroids else False.
            Default: False.

    Returns:
        Confidence maps for the input keypoints.
    """
    if is_centroids:
        points = instances[:, :num_instances, :].unsqueeze(dim=-2)
        # (n_samples, n_instances, 1, 2)
    else:
        points = instances[
            :, :num_instances, :, :
        ]  # (n_samples, n_instances, n_nodes, 2)

    height, width = img_hw

    xv, yv = make_grid_vectors(height, width, output_stride)

    confidence_maps = make_multi_confmaps(
        points,
        xv,
        yv,
        sigma * output_stride,
    )  # (n_samples, n_nodes, height/ output_stride, width/ output_stride).
    # If `is_centroids`, (n_samples, 1, height/ output_stride, width/ output_stride).

    return confidence_maps

make_confmaps(points_batch, xv, yv, sigma)

Make confidence maps from a batch of points for multiple instances.

Parameters:

Name Type Description Default
points_batch Tensor

A tensor of points of shape (n_samples, n_nodes, 2) and dtype torch.float32 where the last axis corresponds to (x, y) pixel coordinates on the image for each instance. These can contain NaNs to indicate missing points.

 required
xv Tensor

Sampling grid vector for x-coordinates of shape (grid_width,) and dtype torch.float32. This can be generated by sleap.nn.data.utils.make_grid_vectors.

 required
yv Tensor

Sampling grid vector for y-coordinates of shape (grid_height,) and dtype torch.float32. This can be generated by sleap.nn.data.utils.make_grid_vectors.

 required
sigma float

Standard deviation of the 2D Gaussian distribution sampled to generate confidence maps.

 required

Returns:

Type Description
Tensor

Confidence maps as a tensor of shape (n_samples, n_nodes, grid_height, grid_width) of dtype torch.float32.
Source code in sleap_nn/data/confidence_maps.py 
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
def make_confmaps(
    points_batch: torch.Tensor, xv: torch.Tensor, yv: torch.Tensor, sigma: float
) -> torch.Tensor:
    """Make confidence maps from a batch of points for multiple instances.

    Args:
        points_batch: A tensor of points of shape `(n_samples, n_nodes, 2)` and dtype `torch.float32` where
            the last axis corresponds to (x, y) pixel coordinates on the image for each instance.
            These can contain NaNs to indicate missing points.
        xv: Sampling grid vector for x-coordinates of shape `(grid_width,)` and dtype
            `torch.float32`. This can be generated by
            `sleap.nn.data.utils.make_grid_vectors`.
        yv: Sampling grid vector for y-coordinates of shape `(grid_height,)` and dtype
            `torch.float32`. This can be generated by
            `sleap.nn.data.utils.make_grid_vectors`.
        sigma: Standard deviation of the 2D Gaussian distribution sampled to generate
            confidence maps.

    Returns:
        Confidence maps as a tensor of shape `(n_samples, n_nodes, grid_height, grid_width)` of
        dtype `torch.float32`.
    """
    samples, n_nodes, _ = points_batch.shape

    x = torch.reshape(points_batch[:, :, 0], (samples, n_nodes, 1, 1))
    y = torch.reshape(points_batch[:, :, 1], (samples, n_nodes, 1, 1))

    xv_reshaped = torch.reshape(xv, (1, 1, 1, -1))
    yv_reshaped = torch.reshape(yv, (1, 1, -1, 1))

    cm = torch.exp(-((xv_reshaped - x) ** 2 + (yv_reshaped - y) ** 2) / (2 * sigma**2))

    # Replace NaNs with 0.
    cm = torch.nan_to_num(cm)

    return cm

make_multi_confmaps(points_batch, xv, yv, sigma)

Make confidence maps for multiple instances through reduction.

Parameters:

Name Type Description Default
points_batch Tensor

A tensor of shape (n_samples, n_instances, n_nodes, 2) and dtype tf.float32 containing instance points where the last axis corresponds to (x, y) pixel coordinates on the image. This must be rank-3 even if a single instance is present.

 required
xv Tensor

Sampling grid vector for x-coordinates of shape (grid_width,) and dtype torch.float32. This can be generated by sleap.nn.data.utils.make_grid_vectors.

 required
yv Tensor

Sampling grid vector for y-coordinates of shape (grid_height,) and dtype torch.float32. This can be generated by sleap.nn.data.utils.make_grid_vectors.

 required
sigma float

Standard deviation of the 2D Gaussian distribution sampled to generate confidence maps.

 required

Returns:

Type Description
Tensor

Confidence maps as a tensor of shape (n_samples, n_nodes, grid_height, grid_width) of dtype torch.float32.

Each channel will contain the elementwise maximum of the confidence maps generated from all individual points for the associated node.
Source code in sleap_nn/data/confidence_maps.py 
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
def make_multi_confmaps(
    points_batch: torch.Tensor, xv: torch.Tensor, yv: torch.Tensor, sigma: float
) -> torch.Tensor:
    """Make confidence maps for multiple instances through reduction.

    Args:
        points_batch: A tensor of shape `(n_samples, n_instances, n_nodes, 2)`
            and dtype `tf.float32` containing instance points where the last axis
            corresponds to (x, y) pixel coordinates on the image. This must be rank-3
            even if a single instance is present.
        xv: Sampling grid vector for x-coordinates of shape `(grid_width,)` and dtype
            `torch.float32`. This can be generated by
            `sleap.nn.data.utils.make_grid_vectors`.
        yv: Sampling grid vector for y-coordinates of shape `(grid_height,)` and dtype
            `torch.float32`. This can be generated by
            `sleap.nn.data.utils.make_grid_vectors`.
        sigma: Standard deviation of the 2D Gaussian distribution sampled to generate
            confidence maps.

    Returns:
        Confidence maps as a tensor of shape `(n_samples, n_nodes, grid_height, grid_width)` of
        dtype `torch.float32`.

        Each channel will contain the elementwise maximum of the confidence maps
        generated from all individual points for the associated node.

    """
    samples, n_inst, n_nodes, _ = points_batch.shape
    w, h = xv.shape[0], yv.shape[0]
    cms = torch.zeros((samples, n_nodes, h, w), dtype=torch.float32)
    points = points_batch.reshape(samples * n_inst, n_nodes, 2)
    for p in points:
        cm_instance = make_confmaps(p.unsqueeze(dim=0), xv, yv, sigma)
        cms = torch.maximum(cms, cm_instance)
    return cms