instance_centroids
sleap_nn.data.instance_centroids
¶
Handle calculation of instance centroids.
Centroid methods (#586). A centroid can be derived from an instance's points in
several ways, and sleap-nn now exposes the same four-way vocabulary as
sleap_io's Instance.to_centroid / SegmentationMask.to_centroid:
===================== =========================================================
center_of_mass Mean of the visible nodes (the historical default).
bbox_center Midpoint of the visible nodes' bounding box.
geometric_median Weiszfeld geometric median of the visible nodes. The
least affected by a MISLOCALIZED node: with one node off by
a body length, the centroid moves ~1.7x less than the mean
and ~5x less than the bbox midpoint (measured on flies13 and
gerbil pose). It is NOT more stable than the mean when a
node goes MISSING -- a different perturbation, where it
measured slightly worse.
anchor A named node, with a reduce-method fallback when that node
is not visible.
===================== =========================================================
Two levels compute centroids and must agree: this batched torch op (training
targets, top-down crop centers, GT-centroid inference) and to_centroid at the
object level. :func:resolve_centroid_method is the single place that turns the
config pair (anchor_part, centroid_method/centroid_fallback) into the
(method, fallback) argument pair both levels take, so there is one spelling of
these concepts across sleap-nn and sleap-io. tests/data/test_instance_centroids.py
asserts the two levels agree per method.
Divergence from sleap_io, deliberate. sleap_io decides node visibility
from the x-coordinate alone (~isnan(pts[:, 0])); the reductions here count
non-NaN values per axis (find_points_mean, #584) or per point
(find_points_geometric_median). The two agree whenever a node's coordinates are
NaN together — which is what sleap_io itself writes — and differ only for a
half-NaN point, where this module's answer is the better-defined one.
Functions:
| Name | Description |
|---|---|
add_centroids_from_masks |
Derive |
centroid_method_from_config |
Read and resolve the centroid knobs off a head-config leaf. |
degrade_anchor_if_unresolved |
Degrade an |
find_points_bbox_midpoint |
Find the midpoint of the bounding box of a set of points. |
find_points_geometric_median |
Find the geometric median of a set of points via Weiszfeld's algorithm. |
find_points_mean |
Find the mean position of a set of points, ignoring NaNs. |
generate_centroids |
Return centroids derived from instance points by the configured method. |
reduce_points |
Reduce a set of points to one centroid by the named method. |
resolve_centroid_method |
Resolve the head config's centroid knobs into |
add_centroids_from_masks(labels, method='center_of_mass', overwrite=False)
¶
Derive UserCentroid annotations from a labels' segmentation masks.
Mask-only datasets (no pose annotations at all) cannot train a centroid model
today: the confmap target needs either pose keypoints or first-class centroid
annotations, and such labels have neither. sio.SegmentationMask.to_centroid
supplies the missing piece — one call per mask, carrying the mask's track /
identity / instance linkage — after which the ordinary
centroid_source="user" path takes over unchanged. Nothing downstream of
this function knows the centroids came from masks.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
An |
required | |
method
|
str
|
The derivation method, one of :data: |
'center_of_mass'
|
overwrite
|
bool
|
If |
False
|
Returns:
| Type | Description |
|---|---|
int
|
The number of centroids added. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in sleap_nn/data/instance_centroids.py
389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 | |
centroid_method_from_config(head_config)
¶
Read and resolve the centroid knobs off a head-config leaf.
Convenience wrapper over :func:resolve_centroid_method for the many callers
that hold a head-config leaf (head_configs.centroid.confmaps,
...centered_instance.confmaps, ...embedding.embedding, ...). Missing
keys resolve to None, so a config written before #586 — or a plain dict —
yields the historical behavior.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
head_config
|
A head-config leaf ( |
required |
Returns:
| Type | Description |
|---|---|
tuple
|
|
Source code in sleap_nn/data/instance_centroids.py
degrade_anchor_if_unresolved(method, fallback, anchor_ind)
¶
Degrade an "anchor" method to its fallback when the node is unresolvable.
anchor_part names a node that may be absent from the skeleton — the
centroid model deliberately tolerates this (an anchor is only a fallback path
there), and the embedding dataset resolves the index leniently. Rather than
raising from deep inside the batched op, degrade to the configured fallback,
which is what the pre-#586 code did implicitly, and say so once.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
method
|
str
|
The resolved method, as returned by :func: |
required |
fallback
|
Optional[str]
|
The resolved fallback. |
required |
anchor_ind
|
Optional[int]
|
The anchor node index, or |
required |
Returns:
| Type | Description |
|---|---|
tuple
|
|
Source code in sleap_nn/data/instance_centroids.py
find_points_bbox_midpoint(points)
¶
Find the midpoint of the bounding box of a set of points.
Retained as a utility for callers that explicitly want bbox-midpoint behavior.
The canonical anchor fallback used by :func:generate_centroids is
:func:find_points_mean (mean of visible nodes) — see that function for the
project-wide convention.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
Tensor
|
A torch.Tensor of dtype torch.float32 and of shape (..., n_points, 2), i.e., rank >= 2. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
The midpoints between the bounds of each set of points. The output will be of shape (..., 2), reducing the rank of the input by 1. NaNs will be ignored in the calculation. |
Notes
The midpoint is calculated as: xy_mid = xy_min + ((xy_max - xy_min) / 2) = ((2 * xy_min) / 2) + ((xy_max - xy_min) / 2) = (2 * xy_min + xy_max - xy_min) / 2 = (xy_min + xy_max) / 2
Source code in sleap_nn/data/instance_centroids.py
find_points_geometric_median(points, max_iter=100, tol=1e-06, eps=1e-12)
¶
Find the geometric median of a set of points via Weiszfeld's algorithm.
The geometric median minimizes the sum of Euclidean distances to the input
points, which makes it markedly more robust than the mean to a badly localized
node — a tracker that flings one node across the frame pulls
:func:find_points_mean off the body, but barely moves this.
Measured on real pose data (flies13, gerbil; one visible node displaced by one body length), the resulting centroid shift is:
================== =============== =============== method flies13 median gerbil median ================== =============== =============== geometric_median 3.7 px 4.4 px center_of_mass 6.6 px 6.9 px bbox_center 21.3 px 17.8 px ================== =============== ===============
This is robustness to a node in the WRONG PLACE, not to a node being absent: under single-node dropout the geometric median moved slightly MORE than the mean (4.8 px vs 3.4 px median on flies13), since removing a node can move the Fermat point it was pinned near.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
Tensor
|
A torch.Tensor of dtype torch.float32 and of shape (..., n_points, 2), i.e., rank >= 2. |
required |
max_iter
|
int
|
Maximum Weiszfeld iterations. |
100
|
tol
|
float
|
Convergence tolerance on the estimate's movement between iterations. Iteration stops once every slot has moved less than this. |
1e-06
|
eps
|
float
|
Distances below this are treated as coincident with the estimate and dropped from the reweighting (their weight would be unbounded). |
1e-12
|
Returns:
| Type | Description |
|---|---|
Tensor
|
The geometric medians. The output will be of shape (..., 2), reducing the rank of the input by 1. Slots whose points are all NaN return NaN. |
Notes
A point is used only when BOTH of its coordinates are non-NaN — the
estimate is a joint 2D quantity, so per-axis visibility (which
:func:find_points_mean uses) has no meaning here. Matches the algorithm
in sleap_io.model.centroid._geometric_median (initialized at the
arithmetic mean, inverse-distance reweighting) so the object level and
this batched op agree.
The iteration runs in float64 regardless of the input dtype, and the
result is cast back. Weiszfeld reweights by 1 / distance, so near a
Fermat point that sits on one of the input nodes the distances approach
zero and float32 loses all relative precision there: measured against the
numpy reference on random 13-node instances, float32 drifts up to 0.18 px
while float64 agrees to ~1e-13. The tensors involved are a few hundred
floats, so the promotion costs nothing worth measuring — and it is what
lets the object/tensor parity test assert equality rather than a loose
tolerance.
Source code in sleap_nn/data/instance_centroids.py
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 | |
find_points_mean(points)
¶
Find the mean position of a set of points, ignoring NaNs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
Tensor
|
A torch.Tensor of dtype torch.float32 and of shape (..., n_points, 2), i.e., rank >= 2. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
The NaN-ignoring mean across the |
Source code in sleap_nn/data/instance_centroids.py
generate_centroids(points, anchor_ind=None, method=None, fallback=None)
¶
Return centroids derived from instance points by the configured method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
Tensor
|
A torch.Tensor of dtype torch.float32 and of shape (..., n_nodes, 2), i.e., rank >= 2. |
required |
anchor_ind
|
Optional[int]
|
The index of the node to use as the anchor for the centroid.
Required by (and only used by) |
None
|
method
|
Optional[str]
|
One of :data: |
None
|
fallback
|
Optional[str]
|
The reduce method for a missing anchor, one of
:data: |
None
|
Returns:
| Type | Description |
|---|---|
Tensor
|
The centroids of the instances. The output will be of shape (..., 2), reducing the rank of the input by 1. NaNs will be ignored in the calculation. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Note
This op defines what a centroid means for training targets, top-down crop
centers and GT-centroid inference alike; it must stay in lockstep with the
object-level to_centroid (see the module docstring) and with the
sio.Centroid.source tag written by
sleap_nn.inference.centroid_convert.
Source code in sleap_nn/data/instance_centroids.py
reduce_points(points, method)
¶
Reduce a set of points to one centroid by the named method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
Tensor
|
A torch.Tensor of shape (..., n_points, 2), i.e., rank >= 2. |
required |
method
|
str
|
One of :data: |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
The centroids, of shape (..., 2). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in sleap_nn/data/instance_centroids.py
resolve_centroid_method(anchor_part=None, centroid_method=None, centroid_fallback=None)
¶
Resolve the head config's centroid knobs into (method, fallback).
The single place that maps sleap-nn's config fields onto sleap_io's
to_centroid vocabulary, so the object level, the batched tensor level
(:func:generate_centroids) and the recorded sio.Centroid.source tag can
never disagree about what a model's centroid means.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
anchor_part
|
Optional[str]
|
The configured anchor node name, or |
None
|
centroid_method
|
Optional[str]
|
One of :data: |
None
|
centroid_fallback
|
Optional[str]
|
The reduce method used when the anchor node is not
visible. One of :data: |
None
|
Returns:
| Type | Description |
|---|---|
tuple
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If a value is not in the vocabulary, if |