Changelog¶

v0.3.3 ¶

sleap-nn v0.3.3 Release Notes

Summary

SLEAP-NN v0.3.3 is a small correctness-focused follow-up to v0.3.2, fixing a top-down inference scale-sharing regression where the centered-instance stage silently inherited the centroid stage's scale instead of its own, closing an observability gap where predict's output provenance never recorded the scale/crop_size that actually ran, and fixing two training-config validation gaps — an unvalidated anchor_part that crashed confusingly deep, and an LR scheduler priority order that silently ignored its own documented precedence. 3 PRs since v0.3.2.

Installation

# Install / upgrade the CLI tool (auto-selects the right torch backend)
uv tool install sleap-nn --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: sleap-nn 0.3.3

Fixes

Fixed a regression where the top-down inference pipeline's centered-instance stage silently inherited the centroid stage's preprocessing.scale instead of using its own trained value, whenever the two models were trained at different scales (a common setup — centroid models are often trained at lower resolution for speed; this is literally the GUI's own default training-profile pairing). This corrupted confidence-map peak-finding badly enough to drop detections almost entirely: on a real 2560-frame project with mismatched scales (centroid=0.5, centered_instance=1.0), sleap-nn predict went from 5120 correct instances (v0.3.1) to 0, and sleap-nn track dropped from 5120 to 11. Both pipelines now correctly resolve and apply each stage's own scale by default, while still honoring an explicit --input_scale override applied uniformly to both stages (#725).
Fixed sleap-nn predict never recording scale/crop_size in its output provenance metadata at all, unlike the legacy track pipeline (which recorded only the raw CLI override, not necessarily the resolved value that actually ran). Provenance now reads the actual resolved scale off each stage's built inference layer; for top-down models it records centroid_scale/instance_scale/crop_size distinctly rather than collapsing to one shared value (#728).
Fixed anchor_part having no upfront validation for centered_instance/multi_class_topdown/centered_instance_segmentation models: a typo'd or nonexistent anchor_part passed config setup cleanly and only failed deep inside dataset construction, with an error message that misleadingly blamed part_names instead of the actual offending field. A clear, correctly-attributed error is now raised upfront. (centroid models are intentionally exempt — an unmatched anchor_part there is a documented fallback case, not an error.) (#729)
Fixed configure_optimizers()'s LR scheduler selection ignoring its own documented priority (cosine_annealing_warmup > linear_warmup_linear_decay > step_lr > reduce_lr_on_plateau): it iterated scheduler fields in declaration order instead, so any user who set cosine_annealing_warmup/linear_warmup_linear_decay without also explicitly nulling the always-populated-by-default reduce_lr_on_plateau silently got ReduceLROnPlateau instead — no error, no warning, training just ran with the wrong LR schedule indefinitely. The scheduler is now selected in the documented priority order (#729).

Dependencies & Build

sleap-io pin unchanged at >=0.9.2,<0.10.0 — 0.9.2 remains the latest release on both PyPI and GitHub, so no upper-bound audit was needed this cycle.

Changelog

#725: fix(infer): topdown centered-instance stage silently inherits centroid's scale (@gitttt-1234)
#728: fix(infer): predict never recorded input_scale/crop_size in provenance (@gitttt-1234)
#729: fix(train): anchor_part crashes confusingly deep; LR scheduler priority ignored (@gitttt-1234)
#730: chore: bump version to 0.3.3 (@gitttt-1234)

Contributors: @gitttt-1234

Full Changelog: v0.3.2...v0.3.3

v0.3.2 ¶

sleap-nn v0.3.2 Release Notes

Summary

SLEAP-NN v0.3.2 is a small correctness-focused follow-up to v0.3.1, fixing a cluster of predict/track parity bugs (--input_scale overrides, empty-frame retention, tracking-log fidelity), guarding eval against zero-matched-instance crashes/warning-spam, fixing sparse eval metrics in training_log.csv, adding a JSON sibling for evaluation metrics, fixing .pkg.slp inference outputs referencing the wrong video by default, and fixing legacy config conversion silently dropping flip augmentation. 10 PRs since v0.3.1.

Installation

# Install / upgrade the CLI tool (auto-selects the right torch backend)
uv tool install sleap-nn --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: sleap-nn 0.3.2

Breaking Changes

`sleap-nn predict` now retains empty-detection frames by default (#717)

Previously, non-tracking predict runs silently dropped every zero-detection frame from the output .slp, while track runs kept them. predict now retains one output frame per zero-detection frame by default, matching track and legacy SLEAP behavior. Pass --no_empty_frames to restore the old drop-empty-frames behavior.

`predict` now backreferences a `.pkg.slp` input to itself, not its source video (#724)

--restore_source_videos (and the predict()/save_predictions()/run_sam_segmentation() API kwarg of the same name) now defaults to false. Previously, saving predictions from a .pkg.slp input restored a reference to the pre-embedding source video by default — a file that's frequently not available (that's the point of a .pkg.slp), which could leave the output unable to display images at all. The output now backreferences the .pkg.slp itself by default, matching sleap-nn track's existing behavior. Pass --restore_source_videos to restore the old behavior. This also fixes a related bug where, for some inputs, the output could end up self-referentially pointing at itself instead of any video at all.

New Features

Evaluation

JSON sibling of pickled .npz metrics (#721) — training/eval now also writes a metrics.{split}.{idx}.json file next to the existing pickled .npz, so non-Python tooling (e.g. the sleap-app metrics UI) can read evaluation metrics directly without a numpy/pickle dependency. .npz output is unchanged.

CLI Updates

sleap-nn predict --gui mode hardening (#715) — log output is now cleanly routed to stderr so it can no longer interleave with and corrupt the --gui mode's JSON progress-line parsing; runtime failures during --gui runs now also emit a structured JSON error line before raising.
Duplicate --model_paths of the same model type now raises a clear error (#715) — previously silently discarded one of the paths.
--input_scale override fixed for both predict and track (#716) — see Fixes.
predict retains empty-detection frames by default (#717) — see Breaking Changes.
--restore_source_videos now defaults to false (#724) — see Breaking Changes.

Fixes

Fixed predict --tracking producing different track-ID assignments than track --tracking: empty-detection frames were dropped before tracking, skewing the candidate window's view of elapsed time; empty frames are now retained through tracking so identities flush/persist exactly as legacy does (#714).
Fixed --input_scale override being broken in both CLIs when set to something other than the training-time value: predict silently ignored the override entirely, while track applied it to the forward resize but not the coordinate rescale-back, producing out-of-frame keypoint coordinates. Both pipelines now correctly resolve and apply the override end-to-end (#716).
Fixed a tracking-logging bug that silently swallowed several tracking/model-resolution notices, and restored several legacy log lines (startup banner, per-filter config, save-path confirmation, tracking timing) so predict output parity-matches track for debugging (#717).
A fully collapsed eval split (0 matched instances) no longer spams RuntimeWarning: Mean of empty slice from OKS/PCK/VOC metric calculations in either the training eval loop or the standalone sleap-nn eval CLI — it now logs one clear message and skips cleanly (#720).
Fixed sparse eval metrics in training_log.csv: when trainer_config.eval.frequency runs eval only every N epochs, the CSV previously either omitted eval columns entirely or silently repeated the last-computed value on non-eval epochs. Eval columns are now always included and correctly show NaN on epochs where eval didn't run, so progress plots aren't misled by stale carry-forward values (#722).
Fixed predict (and run_sam_segmentation) saving a .pkg.slp input's output with a broken video reference: VideoProvider/LabelsProvider close the video's backend to cheaply copy it for the prefetch thread, and the closed backend was never reopened before the final save, so sleap-io's embedded-image detection missed it and the output could end up self-referentially pointing at itself instead of any real video. The backend is now reopened right after the thread-local copy is made (#724).
Fixed legacy SLEAP JSON training-config conversion silently dropping flip augmentation: data_mapper() never read the legacy optimization.augmentation_config.random_flip / flip_horizontal fields, so importing an old config with flip enabled produced a sleap-nn config with flip silently disabled. random_flip=True + flip_horizontal=True now correctly maps to GeometricConfig.flip_p=0.5; a legacy vertical flip (flip_horizontal=False, unsupported by sleap-nn's horizontal-only flip) now logs a warning instead of silently doing nothing (#723).

Dependencies & Build

sleap-io pin unchanged at >=0.9.2,<0.10.0 — 0.9.2 remains the latest release on both PyPI and GitHub, so no upper-bound audit was needed this cycle.

Upgrade Notes

If your workflow depends on predict silently dropping zero-detection frames, add --no_empty_frames (#717).
If your workflow depends on predict restoring the pre-embedding source video reference for .pkg.slp inputs, add --restore_source_videos (#724).
If you pin sleap-io, no change needed — >=0.9.2,<0.10.0 is still current.

Changelog

#714: Make predict --tracking see empty-detection frames, matching legacy (@gitttt-1234)
#715: Predict-pipeline robustness hardening (3 small fixes) (@gitttt-1234)
#716: Fix --input_scale override in both predict and track (@gitttt-1234)
#717: Empty-frame retention, tracking-logging bug, and legacy log-line parity (@gitttt-1234)
#720: Guard zero-matched-instance warnings + retire legacy predict pipeline from post-training eval (@gitttt-1234)
#721: Emit a JSON sibling of the pickled .npz metrics (@alicup29)
#722: NaN-fill sparse eval metrics in training_log.csv instead of stale carry-forward (@gitttt-1234)
#723: Convert legacy random_flip/flip_horizontal to flip_p (@gitttt-1234)
#724: Bump version to 0.3.2; fix .pkg.slp predict output referencing the wrong video by default (@gitttt-1234)

Contributors: @gitttt-1234, @alicup29

Full Changelog: v0.3.1...v0.3.2

v0.3.1 ¶

sleap-nn v0.3.1 Release Notes

Summary

SLEAP-NN v0.3.1 is a focused follow-up to v0.3.0, adding sliding-window tiling for high-res / small-object frames, a new whole-frame semantic_segmentation model type, pretrained HuggingFace backbones, and several centroid-training correctness fixes (single-source centroid targets, training directly from UserCentroid annotations, including pure-centroid frames). It also restores CPU/GPU overlap in sleap-nn predict (dropped silently in an earlier inference-pipeline refactor) and fixes a repo-wide sweep of predict-pipeline and training/evaluation correctness bugs (postprocess-override isolation, checkpoint-monitor metrics, OKS eval config forwarding, a ground-truth mutation bug, and more), and bumps the sleap-io dependency to >=0.9.2,<0.10.0 (re-ID/Category/Event annotations, large-project save hardening, O(N) merges). 23 PRs since v0.3.0.

⚠️ Breaking changes: whole-frame segmentation models (semantic_segmentation, bottomup_segmentation) trained before #693 have a baked-in mask/image misalignment and should be retrained; default mask-evaluation metrics change (predicted-instance-linked GT masks are now excluded); frame-caching failures during training now hard-fail instead of warning-and-continuing; and centroid training's default target-source resolution changed. Read Breaking Changes before upgrading.

Highlights:

Sliding-window tiling (data_config.preprocessing.tiling) for 4K++ frames with small objects — cut into overlapping tiles, run per-tile at native resolution, stitch via Gaussian-weighted merge. Opt-in; supported for single_instance and bottomup_segmentation. (#687)
New semantic_segmentation model type — whole-frame binary fg/bg mask, no instance grouping, with a matching-free --match_method semantic eval mode. (#688)
Pretrained HuggingFace backbones (sleap-nn[backbones]) — use any AutoBackbone (ConvNeXtV2, ResNet, Swinv2, DINOv2/v3, ...) as a model's encoder, frozen or fine-tuned. (#681)
Centroid training correctness: centroid_source config ("user"/"computed"/None) fixes a mixed-annotation footgun where the centroid head could train against two different centroid definitions in one run (#704); CentroidDataset can now train directly on UserCentroid annotations, including pure-centroid frames with no pose instance (#702, #703).
clDice mask metric for connectivity-aware segmentation quality, plus a configurable best-checkpoint monitor metric so segmentation runs can checkpoint on full-resolution quality instead of coarse val/loss. (#682, #691)
⚠️ sleap-io >=0.9.2,<0.10.0 — audited past the previous <0.9.0 ceiling (re-ID, Category, Event annotations; large-project save hardening; O(N) merges). See sleap-io's v0.9.0/v0.9.1/v0.9.2 release notes.

Installation

# Install / upgrade the CLI tool (auto-selects the right torch backend)
uv tool install sleap-nn --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: sleap-nn 0.3.1

Breaking Changes

⚠️ Whole-frame segmentation masks were misaligned with the image grid (#693)

semantic_segmentation and bottomup_segmentation models trained before this fix had their GT masks resized straight to the padded frame size instead of threaded through the same size-match/scale/stride-pad chain as the image — offsetting the foreground target by roughly half the padding (worse toward the bottom-right) whenever height/width weren't multiples of max_stride. This is a training-target bug, not an inference-geometry change; affected models should be retrained.

⚠️ Mask evaluation now excludes predicted-instance-linked GT masks by default (#694)

run_evaluation(match_method="mask") previously counted every mask in frame.masks as ground truth, even ones linked to a PredictedInstance, which imposed an artificial, unreachable recall/F1 ceiling on labels files containing stray predicted instances. exclude_predicted_instance_masks (driven by the existing user_labels_only argument, default True) now drops those from the GT side only — expect higher default recall/F1 on affected files. Predicted-side masks and match_method="semantic" are unaffected; pass user_labels_only=False to restore the old counting.

⚠️ Frame-caching failures now raise instead of warn (#701)

Training runs that previously continued past frame-caching errors (logged as warnings, then crashed later mid-epoch with a confusing FileNotFoundError/KeyError) now hard-fail immediately with a RuntimeError naming the bad frame(s) and a re-encode hint. A corrupt-video or disk-full condition will now stop training up front rather than silently producing an incomplete cache.

⚠️ Centroid target source is now resolved once per dataset, not per frame (#704)

model_config.head_configs.centroid.confmaps.centroid_source ("user", "computed", or None) replaces #702's per-frame fallback: the source is now inferred once for the whole dataset and applied consistently across train/val, and CentroidDataset drops frames that can't supply the chosen target (e.g. pose-only frames in "user" mode). This changes the effective training set for datasets that mix annotation styles. Leaving it unset infers a source with a loud warning recommending it be set explicitly.

New Features

Tiling, segmentation & pretrained backbones

Sliding-window tiling (#687) for high-res / small-object frames (4K microscopy, plant-root scans, ...). Tile geometry auto-sizes from labels, is written into the model config, and is parity-checked at inference; unsupported configs (pretrained-encoder backbones, ClassVectorsHead) get a clear error instead of a silent no-op. sleap-nn export now warns when exporting a tiled model to ONNX/TensorRT, since tiled export isn't supported yet and the exported model would otherwise silently run whole-frame instead of tiled.
semantic_segmentation model type (#688) — a lone SegmentationHead on the whole frame predicting one binary fg/bg mask, no instance grouping; matching-free --match_method semantic eval (whole-frame IoU/clDice/boundary-IoU).
Pretrained HuggingFace backbones (#681) via the optional sleap-nn[backbones] extra — any AutoBackbone as encoder for pose, centroid, or segmentation models, frozen or fine-tuned (model_config.backbone_config.pretrained).
clDice mask metric (#682) — centerline-Dice score (mean_cldice / eval/val/mask_mean_cldice) for connectivity-aware quality on thin/tubular structures where mask IoU is misleading.

Centroid training

Train from UserCentroid annotations (#702, #703) — CentroidDataset trains directly on first-class sio.UserCentroid annotations when present (enabling active-learning workflows with a user-seeded, non-node-tied centroid), falling back to keypoint-derived centroids otherwise. #703 is the follow-up that makes the pure-centroid seeding case (a frame with a UserCentroid but no pose instance at all — the Phase-1 active-learning workflow) actually reach the dataset: the train/val split previously filtered to has_user_instances before CentroidDataset ever saw the frame, silently producing 0 training frames for a labels file that was otherwise entirely valid.
Single-source centroid targets (#704) — see Breaking Changes.

Training diagnostics & robustness

Configurable best-checkpoint metric (#691) — ModelCkptConfig.monitor/mode (e.g. eval/val/fg_mean_cldice, mode: max) fixes segmentation best.ckpt selection, which previously used the coarse val/loss instead of full-resolution quality metrics.
Confmap fg/bg MSE diagnostic (#698) — {train,val}/confmap_loss_fg, confmap_loss_bg, confmap_fg_frac logged (no effect on training/checkpoints) to surface foreground-vs-background fit that a single blended loss number obscures.
Trainer-accelerator validation (#708) — a saved config's trainer_accelerator (e.g. mps, cuda) is now checked against actual device availability at training setup and falls back to "auto" with a log message instead of crashing deep in Trainer.train(); fixes reloading a config across machines (e.g. Mac → Linux/CUDA).

CLI Updates

No single PR this release is purely a CLI change, but several touch sleap-nn's command-line surface directly:

sleap-nn eval --match_method semantic (#688) — new matching-free evaluation mode for semantic_segmentation models (whole-frame foreground IoU/clDice/boundary-IoU, no instance matching).
sleap-nn export now warns on a tiled model (#687) — exporting a model trained with tiling enabled to ONNX/TensorRT logs a warning that tiled export isn't supported yet (the export runs whole-frame instead of failing silently).
sleap-nn predict warns on ignored frame-filter flags for video input (#712) — --only_predicted_frames / --only_suggested_frames / --exclude_user_labeled / --only_labeled_frames require a .slp source (they filter by annotation status); passing one with a video now logs a warning naming the ignored flags instead of silently running full-video inference.
sleap-nn predict --paf_workers warns on unsupported model types (#711) — previously a silent no-op if the model type couldn't use the pipelined CPU-grouping path.
--max_instances/-n fixed for centroid-only and top-down models (#709) — see Fixes; this documented predict-time flag was being silently ignored for these model types.
sleap-nn track crash fixed when combining --tracking_clean_instance_count with --post_connect_single_breaks (#707) — see Fixes.

Fixes

Segmentation ModelCheckpoint no longer crashes when a monitored mask metric is NaN on validation epochs with no matched instances (#692).
sleap-nn predict output built from a pre-constructed provider (--only_suggested_frames, --only_labeled_frames, --only_predicted_frames, --exclude_user_labeled, --video_index) no longer crashes on save due to a dropped source video (#700).
Fixed an UnboundLocalError in sleap-nn track's legacy pipeline when combining --tracking_clean_instance_count with --post_connect_single_breaks, and a related crash on empty frame lists (#707).
--max_instances/-n overrides at predict time are now honored for centroid-only and top-down models (CentroidLayer), matching the fix already applied to bottom-up models; the sleap-nn predict startup log also now reflects actual peak_threshold/max_instances values (#709).
Fixed a crash (ValueError: 'centroid' is not in list) when the configured anchor part isn't a node in the pose skeleton (#702).
Training a dataset with zero usable training samples (e.g. a .slp with labeled frames but no user-labeled instances/centroids) now fails fast with a clear, actionable error instead of a cryptic IndexError: list index out of range deep in the trainer's first log line (#706).
Restored CPU-decode / GPU-inference overlap in sleap-nn predict — an earlier inference-pipeline refactor silently replaced the legacy background-thread frame readers with synchronous ones, so CPU video/.slp decode no longer overlapped the GPU forward pass; most noticeable on long or high-resolution videos where decode time is non-trivial. Also warns (instead of silently doing nothing) when --paf_workers is set on a model type that can't use the pipelined CPU-grouping path (#711).
Predictor.predict_streaming() no longer leaks predict-time postprocess overrides (peak_threshold, max_instances, etc.) across two interleaved streaming calls on the same Predictor object (#712).
sleap-nn predict now warns (instead of silently doing nothing) when a label-status frame filter (--only_labeled_frames, --only_suggested_frames, --exclude_user_labeled, --only_predicted_frames) is set for a non-.slp (video) input, since those flags have no annotation data to filter on there (#712).
Fixed LabelsProvider's frame-status filter priority order (when more than one only_*/exclude_* flag is set) to match the legacy pipeline's precedence (#712).
model_ckpt.monitor can now target pose-model (eval/val/mOKS, etc.) or centroid-model (eval/val/centroid_dist_avg, etc.) eval metrics — previously only segmentation eval metrics worked as a checkpoint monitor target; the other two callbacks computed their metrics but never exposed them to ModelCheckpoint, which crashed the first time a run tried to monitor one (#713).
Post-training final-split evaluation for pose models now uses the configured oks_stddev/oks_scale (matching the per-epoch training eval) instead of silently falling back to defaults — previously the saved metrics.<split>.npz could disagree with the training curves for any run that customized these (#713).
run_evaluation(user_labels_only=True) no longer mutates the caller's ground-truth Labels object in place; a second evaluation on the same Labels (e.g. with user_labels_only=False) no longer silently sees fewer instances than it should (#713).

Dependencies & Build

sleap-io >=0.9.2,<0.10.0 (was >=0.8.0,<0.9.0) — audited through v0.9.0–v0.9.2; the two breaking changes there (Identity.color removed, .category promoted from str to a Category object) don't affect any sleap-nn code path.
macOS CI no longer hangs to the 45-minute timeout or fails on MPS backend out of memory — an opt-in SLEAP_NN_DISABLE_MPS=1 env var (honored at sleap_nn import) forces CPU device selection; also usable by end users as an escape hatch for flaky MPS drivers (#695).

Upgrade Notes

Retrain any semantic_segmentation / bottomup_segmentation model trained before this release (#693 mask/image alignment fix).
Expect mask-evaluation recall/F1 to increase by default on labels files containing predicted instances (#694); pass user_labels_only=False to restore old behavior.
A training run that hits a frame-caching error will now stop immediately with a RuntimeError instead of continuing (#701) — check for corrupt videos / disk space if you see this.
If training a centroid model on a dataset that mixes user-labeled poses and UserCentroid annotations, set model_config.head_configs.centroid.confmaps.centroid_source explicitly ("user" or "computed") rather than leaving it unset (#704).
If you pin sleap-io, move to >=0.9.2,<0.10.0.

Changelog

#681: Reuse pretrained HuggingFace encoders as backbones (#680) (@talmo)
#682: Add clDice metric to eval/wandb + fix bottom-up viz (@talmo)
#687: Sliding-window tiling for high-res / small-object frames (@talmo)
#688: Add semantic_segmentation model type (whole-frame fg/bg) (@talmo)
#691: Configurable best-checkpoint metric + segmentation viz cleanup (#690) (@talmo)
#692: Don't crash ModelCheckpoint when a monitored seg metric is NaN (@talmo)
#693: Register whole-frame masks to the image grid (thread through shared preprocessing) (@talmo)
#694: Exclude PredictedInstance-linked GT masks in mask evaluation (@talmo)
#695: Disable MPS on mac CI to stop hangs + MPS-OOM failures (@talmo)
#698: Log confmap fg/bg MSE split as a training diagnostic (@talmo)
#700: Attach provider source videos so predict output is saveable (#699) (@tom21100227)
#701: Raise instead of only logging when frame caching fails (@gitttt-1234)
#702: Train centroid model from UserCentroid annotations (@tom21100227)
#703: Centroid model trains on pure-centroid frames (no pose instance) (@tom21100227)
#704: Single-source centroid targets (no user/computed mix) (@tom21100227)
#706: Clear error on empty dataset instead of cryptic IndexError (@tom21100227)
#707: Fix unbound corrected_lfs when combining tracking_clean_instance_count with post_connect_single_breaks (@gitttt-1234)
#708: Verify trainer_accelerator is available before training (@gitttt-1234)
#709: Honor predict-time --max_instances override in CentroidLayer (@alicup29)
#710: Bump version to 0.3.1 and sleap-io pin to >=0.9.2,<0.10.0 (@gitttt-1234)
#711: Restore CPU decode / GPU inference overlap in predict pipeline (@gitttt-1234)
#712: Predict-pipeline correctness fixes (overrides, CLI filters, LabelsProvider priority) (@gitttt-1234)
#713: Training/evaluation correctness fixes (callback_metrics, OKS eval config, GT mutation) (@gitttt-1234)

Contributors: @talmo, @tom21100227, @gitttt-1234, @alicup29

Full Changelog: v0.3.0...v0.3.1

v0.3.0 ¶

Summary

SLEAP-NN v0.3.0 is a major release centered on a new unified sleap-nn predict inference command and a clean Predictor Python API, first-class centroid-only models, an experimental instance-segmentation stack (bottom-up, top-down, and SAM-prompted), the sleap-io v0.8.0 annotation architecture, and Kalman tracking. GPU (CUDA 13 / cu130) is now the default backend. 60+ PRs since v0.2.0.

⚠️ Breaking changes: sleap-nn predict now means pose inference (in v0.2.0 it was the exported-model runner); the sleap-io pin moves to >=0.8.0,<0.9.0; seed defaults to 42; and a fresh prediction .slp is no longer embedded by default. Read Breaking Changes and Upgrade Notes before updating automation or the SLEAP GUI.

Highlights:

New sleap-nn predict inference command + Predictor API. A single, unified entry point from model dir(s) + data to sio.Labels, with streaming, raw-tensor access, in-memory frames, and matching CLI ↔ Python ergonomics. (In v0.2.0 the inference command was sleap-nn track, which remains available as a legacy command.)
Top-level Python API: sleap_nn.predict(...), sleap_nn.Predictor, and sleap_nn.load_models(...) are now importable straight from sleap_nn for quick scripting and discoverability.
Centroid-only models are first-class. Train a lone centroid head and predict / evaluate / export it end-to-end; a single centroid directory auto-detects.
Instance segmentation (experimental): bottom-up, top-down (centered_instance_segmentation), and SAM-prompted backends, with mask evaluation, mask-IoU tracking, and training augmentation/viz/eval parity.
Kalman tracking (--use_kalman) alongside optical-flow shift, plus a per-node "keypoints" tracking mode.
⚠️ sleap-io >=0.8.0,<0.9.0 — the new annotation architecture (Centroid, PredictedSegmentationMask, PredictedROI, rendering helpers) plus a few behavior changes (read-only annotation views, identity-default track matching).
Exported ONNX/TensorRT inference is unified into sleap-nn predict --runtime onnx|tensorrt (CLI) and Predictor.from_export_dir(...) (Python).
GPU (cu130) is the default backend; remote-URL --data_path; repeatable --output_format; configurable output embedding/source-video controls.

Installation

# Install / upgrade the CLI tool (auto-selects the right torch backend)
uv tool install sleap-nn --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: sleap-nn 0.3.0

GPU builds now default to the cu130 (CUDA 13) backend. See the installation docs for CPU-only, specific CUDA versions, and project-dependency usage.

Breaking Changes

⚠️ `sleap-nn predict` now means pose inference

In v0.2.0, sleap-nn predict was the exported-model runner (positional EXPORT_DIR VIDEO --runtime ...), and pose inference was run with sleap-nn track. In v0.3.0, sleap-nn predict is the unified pose-inference command, and running an exported ONNX/TensorRT model is now a flag on it. sleap-nn track still exists as a legacy command, so existing track scripts keep working; new work should use predict.

# Pose inference (v0.2.0)
sleap-nn track   -i video.mp4 -m models/centroid/ -m models/centered_instance/
# Pose inference (v0.3.0)
sleap-nn predict -i video.mp4 -m models/centroid/ -m models/centered_instance/

# Exported ONNX/TRT model (v0.2.0): sleap-nn predict <export_dir> <video> --runtime onnx
# Exported ONNX/TRT model (v0.3.0):
sleap-nn predict -m exported_model/ -i video.mp4 -o predictions.slp --runtime onnx

The standalone exported-inference entry points were removed — including the sleap_nn.export.predictors.ONNXPredictor class. The Python replacement for running exported models is Predictor.from_export_dir, which supports both ONNX and TensorRT:

from sleap_nn.inference import Predictor

predictor = Predictor.from_export_dir("exported_model/", runtime="onnx")       # or runtime="tensorrt"
labels = predictor.predict("video.mp4")

⚠️ sleap-io pinned to `>=0.8.0,<0.9.0`

v0.3.0 adopts the sleap-io v0.8.0 annotation architecture (v0.2.0 capped sleap-io at <0.8.0). Notable downstream-visible changes: Labels annotation lists (instances, masks, …) are now read-only views (mutate via the documented APIs); and track matching in Labels.merge() / Labels.match() now defaults to identity, not name — pass track="name" to restore same-named-track collapsing. Analysis-HDF5 / save behavior was re-baselined (regression-guarded).

base.merge(other)                 # 0.8.0 default: same-named tracks stay separate
base.merge(other, track="name")   # restore pre-0.8.0 name-collapse behavior

⚠️ `seed` now defaults to `42`

Training previously left seed unset; it now defaults to 42, and runs warn on a seed mismatch when resuming. This changes the train/val split RNG vs v0.2.0, so a config that omits seed will produce a different split than before. All shipped sample configs were set to seed: 42. To restore fully-random behavior, set seed: null explicitly.

⚠️ Default prediction `.slp` is non-embedded

By default a prediction .slp is now written non-embedded, referencing the original source videos (--embed false, --restore_source_videos true) rather than a self-contained .pkg.slp. Pass --embed true for a self-contained file. The GUI and any downstream .slp loader must expect non-embedded outputs.

New Features

Unified `sleap-nn predict` + `Predictor` API

The inference stack gained a unified command and a clean Python surface. The one-call predict() returns sio.Labels; Predictor is reusable; raw model outputs and streaming are first-class.

from sleap_nn.inference import predict, Predictor

# One call -> sio.Labels (single-stage / bottom-up)
labels = predict("video.mp4", model_paths=["models/my_model/"])
# Top-down (centroid + centered-instance)
labels = predict("video.mp4", model_paths=["models/centroid/", "models/ci/"])

# Build once, predict many times
predictor = Predictor.from_model_paths(["models/bottomup/"], device="cuda")
labels = predictor.predict("video.mp4", peak_threshold=0.3)

# Raw outputs (confmaps / PAFs / centroids) without building Labels
for out in predictor.predict("video.mp4", make_labels=False, return_confmaps=True):
    cms = out.pred_confmaps

# Predict directly on in-memory frames: np.ndarray / torch.Tensor (N, H, W, C)
labels = predictor.predict(frames)

Top-level Python API for discoverability

predict, Predictor, and a new load_models(...) convenience are importable directly from sleap_nn:

import sleap_nn

labels = sleap_nn.predict("video.mp4", model_paths=["models/my_model/"])   # one-shot
predictor = sleap_nn.load_models(["models/bottomup/"], device="cuda")      # reusable Predictor
labels = predictor.predict("video.mp4")

Centroid-only models

Train a lone centroid head and use it end-to-end. A single centroid directory auto-detects; the output collapses to a single-node centroid skeleton emitting sio.Centroid; evaluation uses distance matching.

sleap-nn predict -m models/centroid/ -i video.mp4 -o centroids.slp
sleap-nn eval -g gt.slp -p centroids.slp --match_method centroid

Centroid-only models are an inference/predict feature; running one through the legacy track path raises a clear error (use predict). Standalone ONNX/TensorRT export is supported.

Instance segmentation — experimental

Bottom-up: training stack, masks carried through to sio.Labels (frame.masks), wired into predict with --min_mask_area/--fg_threshold.
Top-down (centered_instance_segmentation): offset-aware mask decode, training, inference, and eval routing/CLI.
SAM-prompted (opt-in, lazy heavy deps): SAM1 and SAM3 backends, mask-based reconciliation + re-tracking.
Tracking & post-processing: mask-IoU tracker; adaptive distance-gate + greedy RAG fragment-merge; mask-at-output-stride encoding with opt-in morphology and polygon ROI.
Eval & training parity: COCO mask AP/AR/boundary-IoU/fragmentation/per-size + PQ; post-training mask-IoU eval; augmentation/viz/eval parity; predicted-mask overlay in training viz.

Segmentation is experimental in 0.3.0 — see Known Limitations.

Tracking

KalmanShiftTracker (--use_kalman, requires a target instance count; uses pykalman) lands alongside optical-flow shift, with a per-node "keypoints" tracking mode. FlowShiftTracker no longer crashes on frames with no detections. Setting a track cap (--max_tracks) now auto-switches the candidate method to local_queues (logged at INFO) so the cap is actually honored — previously it was silently ignored under the default fixed_window (#670).

Inference UX & I/O

Repeatable --output_format — write several formats by repeating the flag: --output_format slp --output_format analysis_h5 writes both a .slp and a SLEAP Analysis HDF5 (one .analysis.h5 per video).
Remote URLs (http/https/s3/gs/…) accepted as --data_path.
Configurable output embedding / source-video controls for the .slp (--embed, --restore_source_videos).
Frame-based progress with a windowed FPS column, inference spin-up & run-summary logging, and a tracking progress bar.
--workspace-size-gb for TensorRT export.

Training

Symmetry-aware flip augmentation; a hard error on multi-instance frames in single-instance training; and negative-frame split metrics (val/loss_negative, val/n_negative) with a double-count fix. Out-of-bounds keypoints — off-frame annotations, or nodes pushed outside the crop/frame by augmentation — are now masked to empty targets instead of producing phantom Gaussian blobs at the image/crop edge (#666). Training visualizations can now be saved as JPG to shrink the local viz/ folder when training a battery of models (trainer_config.viz_img_format: jpg; #644).

Fixes

FlowShiftTracker no longer crashes on detection-less frames (#612).
A warning now fires when SizeMatcher silently resizes input frames (#561).
WandBRenderer peak-values shape fixed for the centroid case (#557).
Each Tracker gets its own _track_objects state (mutable-default bug) (#592).
Three bottom-up segmentation inference parity bugs fixed (#614).
Single-instance ONNX/TensorRT export now antialiases the input resize to match PyTorch inference, fixing a ~5 px keypoint discrepancy (#672).
Mixed-resolution .slp / .pkg.slp inputs no longer crash predict / post-training eval — LabelsProvider now grows each batch to a shared image shape and closes it at a video/resolution boundary (#678).

Dependencies & Build

sleap-io >=0.8.0,<0.9.0 from PyPI; 0.8.0 save/analysis behavior is regression-guarded.
GPU (cu130) is the default uv backend; gpu/cpu are first-class torch extras so --extra gpu attaches cuDNN.
pykalman is a new (lazy-imported) core dependency for Kalman tracking.

Documentation

Inference docs migrated to sleap-nn predict with nav/overview cleanup (plus multi-GPU & Windows uv fixes); a flies13 top-down training + tracking demo notebook; segmentation docs/config polish; and a docs-correctness sweep (fixed broken copy-paste examples, corrected eval defaults, and seed: 42 in all sample configs). The model reference also gained a reworked plain-language "Choosing a Model" picker and a new Supervised ID guide for the multi_class_topdown / multi_class_bottomup model types (#677).

Known Limitations (planned for 0.3.1)

The segmentation/SAM stack is experimental and intentionally limited; it is opt-in and does not affect the core pose train/predict/eval paths.

No config-generator/TUI scaffolding for the segmentation model types — hand-author YAML or copy a sample config (config_bottomup_segmentation_unet.yaml, config_topdown_centered_instance_segmentation_unet.yaml).
Mask tracking is an MVP; mask_output="polygon" writes only frame.rois (not re-trainable / mask-evaluable); SAM reconciliation/re-tracking has no CLI producer yet.
Inference API: no clean accessor for the underlying torch nn.Module / head-swap "model surgery" yet; the realtime layer.warmup() is not auto-invoked, so the first single-frame call pays cold-start.
Eval: evaluation.get_instances ignores LabeledFrame.centroids, so only the non-default emit_centroid="centroid" .slp is affected (the default instance emission evaluates fine).

Upgrade Notes

Pose inference: prefer sleap-nn predict … (the v0.2.0 sleap-nn track … still works as a legacy command).
Exported models: sleap-nn predict -m <export_dir> --runtime onnx|tensorrt, or in Python Predictor.from_export_dir(<export_dir>, runtime="onnx"|"tensorrt") (the ONNXPredictor class is gone).
Run centroid-only models with predict, not track.
Expect prediction .slp files to be non-embedded and to reference the original source videos by default (--embed true restores embedding).
If you pin sleap-io, move to >=0.8.0,<0.9.0; if you call Labels.merge() / .match() directly, pass track="name" to keep the pre-0.8.0 behavior.
A config that omits seed now defaults to 42 (different split RNG than v0.2.0).
The legacy from sleap_nn.predict import run_inference import moved to from sleap_nn.legacy_predict import run_inference (the sleap_nn.predict name is now the high-level inference function).
SLEAP GUI integrators: route centroid-only models to predict, stop using the removed exported-inference entry points, and tolerate the new default non-embedded .slp output.

Changelog

#530: New unified inference pipeline / Predictor API (#508) (@gitttt-1234)
#557: Fix WandBRenderer peak_values shape for the centroid case (@davorvr)
#558: Restrict Codecov upload to talmolab/sleap-nn (skip upload in forks) (@davorvr)
#559: Expose --workspace-size-gb to the export CLI (TensorRT) (@davorvr)
#560: Fix swint torch.fx.wrap so torch.compile works (#527) (@gitttt-1234)
#561: Warn when SizeMatcher silently resizes input frames (@tom21100227)
#562: Centroid-only inference + mean-of-visible-nodes anchor fallback (@gitttt-1234)
#563: Device-agnostic layer buffers + Linux spawn-context for the PAF pool (@gitttt-1234)
#564: Inference preprocessing parity with the legacy pipeline (@gitttt-1234)
#580: Inference loader/factory fork (independent of the legacy predictors module) (@gitttt-1234)
#585: Inference parity/correctness follow-ups (@talmo)
#587: Inference CLI/streaming/feature follow-ups (@talmo)
#588: Inference test-coverage + minor-correctness follow-ups (@talmo)
#589: Centroid-only models (1/3): core inference — collapse + sio.Centroid emission (@talmo)
#590: Centroid-only models (2/3): distance eval + single-point tracking + train post-eval (@talmo)
#591: Centroid-only models (3/3): authoring UX + export consistency + docs (@talmo)
#592: Give each Tracker its own _track_objects dict (mutable attrs default) (#574) (@talmo)
#593: Accept .ckpt / training_config paths in model_paths (#575) (@talmo)
#594: Validation-side negative-frame split metrics (val/loss_negative, val/n_negative) (#577) (@talmo)
#595: Add KalmanShiftTracker for legacy parity (#572) (@talmo)
#596: Fix KalmanShiftTracker algorithmic correctness (#572 follow-up) (@talmo)
#597: Fix negative-frame metric double-count; add weighted/unweighted + per-head split metrics (@talmo)
#599: Add keypoints (per-node pose) tracking mode for KalmanShiftTracker (#572) (@talmo)
#600: Default seed to 42 and warn on seed mismatch during resume (@gitttt-1234)
#601: Add a progress bar for tracking in the new inference pipeline (@gitttt-1234)
#603: Bottom-up instance segmentation — training stack (refresh of #501) (@talmo)
#604: Carry segmentation masks through Outputs → sio.Labels (@talmo)
#605: Wire segmentation into the new predict pipeline (train→predict) (@talmo)
#607: Rename the inference subcommand to predict (@gitttt-1234)
#608: Post-training mask-IoU evaluation for bottom-up segmentation (@talmo)
#612: Fix FlowShiftTracker crash on frames with no detections (#611) (@talmo)
#613: Add --output_format to save predictions directly as analysis HDF5 (@tom21100227)
#614: Fix three bottom-up segmentation inference parity bugs (@talmo)
#615: Unify exported-model inference into sleap-nn predict (@gitttt-1234)
#624: Segmentation polish — docs/config, PQ eval, postproc knobs, offset viz (@talmo)
#626: Frame-based inference progress + windowed FPS column (#610) (@gitttt-1234)
#628: Inference spin-up + run-summary logging (#610) (@gitttt-1234)
#629: COCO-style mask AP/AR/boundary-IoU/fragmentation/per-size eval (#616) (@talmo)
#630: Mask-IoU tracker MVP for bottom-up segmentation (#619) (@talmo)
#631: Encode masks at output-stride + opt-in morphology + polygon ROI (#618) (@talmo)
#633: Bump sleap-io pin to main (4ee1fb38) (@talmo)
#634: Error on multi-instance frames in single-instance training (@gitttt-1234)
#635: Make gpu/cpu first-class torch extras so --extra gpu attaches cuDNN (#632) (@talmo)
#636: Adaptive distance-gate + greedy RAG fragment-merge postproc (#617) (@talmo)
#637: Top-down segmentation — offset-aware mask decode + sleap-io bump (#622) (@talmo)
#638: centered_instance_segmentation model type — config, data, training (#622) (@talmo)
#639: Top-down (crop-centered) segmentation inference (#622) (@talmo)
#640: Top-down segmentation eval routing + CLI + docs (#622) (@talmo)
#641: Lower bottomup_segmentation center-head sigma default 10.0 → 4.0 (@talmo)
#645: Symmetry-aware flip augmentation (@gitttt-1234)
#646: Mask-based reconciliation + re-tracking for SAM inference (@talmo)
#647: SAM1 prompted inference segmentation core (@talmo)
#648: SAM3 prompted mask backend (opt-in, mask_backend="sam3") (@talmo)
#649: Augmentation, viz & eval parity for segmentation training (@talmo)
#650: SAM-inference tech-debt cleanup + predict CLI surface (@talmo)
#651: Make codecov coverage upload non-blocking (@talmo)
#653: Training viz — overlay predicted per-instance masks (#627) (@talmo)
#654: Configurable image-embedding & source-video controls for the prediction output .slp (#652) (@talmo)
#659: Lock in sleap-io 0.8.0 save/analysis behavior (compat regression guards) (@talmo)
#660: Pin sleap-io to >=0.8.0,<0.9.0 from PyPI (drop git override) (@talmo)
#661: Accept remote URLs (http/s3/gs/...) as --data_path (@talmo)
#662: Make GPU (cu130) the default uv sync/run backend (@talmo)
#663: Migrate inference docs to sleap-nn predict + nav/overview cleanup (multi-GPU & Windows uv fixes) (@gitttt-1234)
#664: flies13 top-down training + tracking demo notebook (@talmo)
#665: 0.3.0 version bump + seed-config fix + docs-correctness sweep (@talmo)
#666: Filter out-of-bounds points before training (NaN-mask off-crop/off-frame nodes) (#571) (@gitttt-1234)
#667: Route np.ndarray / torch.Tensor predict() source to NumpyProvider (@talmo)
#668: Top-level predict/Predictor/load_models + repeatable --output_format (@talmo)
#669: Add trainer_config.viz_img_format (png|jpg) for local training-viz images (#644) (@talmo)
#670: Auto-switch to local_queues when max_tracks is set so the track cap is honored (sleap#2720) (@gitttt-1234)
#672: Antialias the single-instance ONNX resize to match PyTorch inference (@talmo)
#677: Clarify model selection + add a Supervised ID guide for multi_class_topdown/bottomup (#570) (@gitttt-1234)
#678: Batch LabelsProvider frames by shape so mixed-resolution .slp inputs don't crash predict/eval (@gitttt-1234)

Contributors: @talmo, @gitttt-1234, @tom21100227, @davorvr

Full Changelog: v0.2.0...v0.3.0

v0.2.0 ¶

SLEAP-NN v0.2.0 Release Notes

Summary

SLEAP-NN v0.2.0 adopts sleap-io v0.7.0, which reorganizes the annotation model around a unified User*/Predicted* architecture and is API-breaking against earlier sleap-io versions. The release also brings the TUI config picker (sleap-nn config <slp>) to identical parity with the web-app config picker, switches default training to memory caching with 2 workers, simplifies the --pipeline CLI surface, and fixes bugs across negative-frame caching, multi-instance ConvNeXt export, and checkpoint metadata.

Key highlights:

sleap-io v0.7.0: pinned to >=0.7.0,<0.8.0; carries the unified annotation architecture (abstract BoundingBox/SegmentationMask/ROI/LabelImage bases with User*/Predicted* variants). This is the reason for the minor version bump.
TUI ↔ web-app parity: sleap-nn config <slp> now produces byte-identical YAML to the web-app picker for all six pipelines, with corrected augmentation/head/trainer schema emission and matching max_stride recommendations.
Smarter defaults: TUI/web defaults flipped to Cache to Memory + 2 workers; --pipeline topdown is now a first-class value that emits paired centroid + centered_instance configs.
ConvNeXt export fix: resolves the spatial-size mismatch in SimpleUpsamplingBlock that crashed ONNX export on certain input sizes.
Negative-frame robustness: negative frames are now cached during fill_cache, fixing crashes on Lustre / DDP / containerized setups.

Installation

# Upgrade to v0.2.0
uv tool install sleap-nn --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: 0.2.0

CLI Updates

TUI / web-app config-picker parity (#524)

sleap-nn config <slp> (TUI) and sleap-nn config <slp> --auto (CLI auto-config) now produce byte-identical YAML to the web-app config picker for all six pipelines. The bulk of the change is schema-emission correctness — many fields the canonical loader needs were either wrong or missing.

Highlights:

Augmentation: emits canonical brightness_min/max + brightness_p, contrast_min/max + contrast_p, per-axis rotation_p/scale_p/translate_p (was emitting non-canonical brightness_limit/contrast_limit).
Backbones: UNet now emits kernel_size: 3; ConvNeXt/SwinT emit pre_trained_weights and max_stride: 32.
Heads: emit part_names for every head that takes them; edges for PAFs; correct class_maps for multi_class_bottomup; full class_vectors block for multi_class_topdown.
Trainer: emits optimizer.amsgrad, model_ckpt.{save_top_k, save_last}, min_train_steps_per_epoch, the full lr_scheduler block with all four branches, online_hard_keypoint_mining, and conditional wandb/eval blocks.
max_stride recommendation matches the web app: switches to bbox diagonal (sqrt(w² + h²)) for avg_animal_size and stops rebucketing after the centroid-stage scale switch.
Top-down dual emit: pipeline() sets correct per-stage defaults; new build_centroid() helper.

Default to memory caching + 2 workers, simplified `--pipeline` (#526)

New defaults in TUI and web-app config picker: Cache to Memory with 2 workers (previously no caching, 0 workers). Underlying Python schema defaults are unchanged, so programmatic TrainingJobConfig() users are unaffected.

--pipeline CLI surface simplified — topdown is now a single user-facing value that emits paired centroid + centered_instance configs:

Old `--pipeline` value	New
`single_instance`	`single_instance`
`bottomup`	`bottomup`
`centroid`	(removed — use `topdown`)
`centered_instance`	(removed — use `topdown`)
`multi_class_bottomup`	`multi_class_bottomup`
`multi_class_topdown`	`multi_class_topdown`
(new)	`topdown` → produces both centroid & centered_instance configs

The internal PipelineType literal and gen.pipeline() Python API still accept all six canonical types — existing tests and power-user code paths are unchanged.

Bug Fixes

Architecture & Export

#525: Fixed off-by-one spatial-size mismatch in SimpleUpsamplingBlock that crashed ConvNeXt ONNX export on inputs producing odd intermediate feature maps. Decoder upsample is now resized to match the encoder skip connection before concat — no-op when shapes already match, so trained models are unaffected.

Training & Data

#505: Negative frames are now cached during _fill_cache and read from cache in _load_negative_sample(), fixing IndexError crashes when use_negative_frames: true is combined with torch_dataset_cache_img_disk on network filesystems (Lustre), under DDP, or inside containers with different mount paths. Closes #504.
#506: sleap_nn_version in saved initial_config.yaml and training_config.yaml is now overwritten with the running sleap_nn.__version__ at checkpoint time, so saved configs always reflect the version that produced them.

Web Config Picker (`docs/configuration/config-picker/app.html`)

#523: Twelve-plus bug fixes to the docs config picker — corrected centered-instance translate units (/100), || merge bugs that ignored explicit zero/false values, val_labels_path array stringification, data_pipeline_fw baseline fallthrough, crop_padding: 0 rejection, empty bottomup edges, eval match_threshold for centroid models, plus lifecycle/leak fixes (blob-URL revocation, ImageBitmap cleanup) and a rebuilt frame viewer with track-stable colors, zoom-to-instances, and shape-metadata-less SLP support.

Dependency Updates

sleap-io: >=0.6.5,<0.7.0 → >=0.7.0,<0.8.0 (#550, #551)
onnxruntime-gpu in the export-gpu extra now carries a platform marker — only resolved on linux/x86_64 and win/AMD64 where wheels exist (#550)

Changelog

CLI Updates

PR	Title
#524	TUI parity with web-app config picker
#526	Default memory caching + 2 workers; simplify --pipeline CLI

Bug Fixes

PR	Title
#505	Cache negative frames during fill_cache (closes #504)
#506	Overwrite sleap_nn_version in saved checkpoint configs
#525	Align upsampled spatial size to skip connection in SimpleUpsamplingBlock

Other

PR	Title
#523	Config picker bug fixes and frame viewer overhaul

Chores

PR	Title
#550	Bump to v0.2.0 and require sleap-io >=0.7.0
#551	Cap sleap-io at <0.8.0

Full Changelog: v0.1.3...v0.2.0

v0.1.3 ¶

SLEAP-NN v0.1.3 Release Notes

Summary

SLEAP-NN v0.1.3 introduces negative frame training support, a new sleap-nn info CLI command, simplified installation, and numerous bug fixes across inference, tracking, multi-GPU training, and the CLI.

Key highlights:

Negative Frame Training: Include user-confirmed empty frames during training to reduce false positive detections
sleap-nn info Command: Rich summary of trained models including architecture, training results, and evaluation metrics
Simplified Installation: torch is now a default dependency — no extras needed for uv tool install sleap-nn --torch-backend auto
Faster CLI Startup: Lazy imports reduce sleap-nn -h from ~8s to ~1.2s
TensorRT/ONNX Fixes: Bug fixes and expanded test coverage for the export pipeline

Installation

# Upgrade to v0.1.3
uv tool install sleap-nn --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: 0.1.3

New Features

Negative Frame Training (#484)

Train models to suppress false detections by including user-confirmed negative frames (frames with no animals):

data_config:
  use_negative_frames: true
  negative_loss_weight: 1.0  # scale negative sample loss relative to positives

Only frames explicitly marked as negative by the user are included — unlabeled frames are never sampled
Produces all-zero confidence maps, teaching the model not to hallucinate on empty backgrounds
Per-sample metrics logged: loss_positive, loss_negative, n_positive, n_negative
Supported models: SingleInstance, Centroid, BottomUp, BottomUpMultiClass
Not supported for instance-crop models (CenteredInstance, TopDownMultiClass) — warning logged, feature auto-disabled

`sleap-nn info` CLI Subcommand (#498)

Inspect trained models or config files from the command line:

sleap-nn info /path/to/model/

Displays:

Model architecture (type, backbone, head, parameter count, skeleton)
Data pipeline and training hyperparameters
Training results (from training_log.csv)
Evaluation metrics (from .npz files)
File listing with sizes

Simplified Installation (#499)

torch and torchvision are now default dependencies. New [cpu] and [gpu] convenience extras added. The [torch] extra is retained for backwards compatibility.

# Before
uv tool install sleap-nn[torch] --torch-backend auto

# Now (simpler)
uv tool install sleap-nn --torch-backend auto

Anchor Part in Export Metadata (#495)

ExportMetadata now includes an anchor_part field so downstream tools can determine which skeleton node is used as the centroid anchor without parsing the full training config.

Bug Fixes

CLI & Startup

#497: Added -h flag alias for --help and lazy imports for fast CLI startup (~8s → ~1.2s)
#488: Aligned CLI defaults with Python API defaults (oks_stddev, user_labels_only, ensure_rgb/ensure_grayscale)

Inference & Export

#469: Bug fixes and expanded test coverage for TensorRT/ONNX export and inference pipeline (closes #466, #464)
#489: Applied node count and confidence filters in track-only mode (previously skipped during re-tracking)

Training

#485: Standardized intensity augmentation defaults that were on incorrect scales (e.g., gaussian_noise_std: 1.0 mapped to 255-pixel std dev)
#487: Fixed check_memory() reading all HDF5 frames sequentially (~21 min) — now uses video.shape (< 1 sec)
#494: Fixed multi-GPU training failing when launched from GUIs by adding __main__.__spec__ re-spawn check (fixes talmolab/sleap#2656)

Tracking

#492: Fixed crash on all-NaN cost matrices in hungarian_matching() by replacing non-finite values with large finite placeholders (closes #491)

Web UI

#490: Fixed broken augmentation sliders and missing memory estimates in config picker web UI

Changelog

Features

PR	Title
#484	Add negative frames support
#495	Add anchor_part field to ExportMetadata
#498	Add `sleap-nn info` CLI subcommand
#499	Add torch as default dep with `--torch-backend` support and `[cpu]`/`[gpu]` extras

Bug Fixes

PR	Title
#469	Bug fixes and more tests for TensorRT/ONNX inference pipeline
#485	Standardize intensity augmentation default parameters
#487	Use video.shape in check_memory() to avoid reading all frames
#488	Align CLI defaults with Python API defaults
#489	Apply node count and confidence filters in track-only mode
#490	Expose missing functions to window in config picker
#492	Handle infeasible cost matrix in hungarian_matching
#494	Standardize multi-GPU re-spawn using `__main__.__spec__` check
#497	Add `-h` flag and lazy imports for fast CLI startup

Chores

PR	Title
#500	Bump version to 0.1.3

Full Changelog: v0.1.2...v0.1.3

v0.1.2 ¶

SLEAP-NN v0.1.2 Release Notes

Summary

SLEAP-NN v0.1.2 is a maintenance release focused on documentation improvements and dependency updates.

Key highlights:

Improved Inference Guide: New Quick Start section, expanded troubleshooting, and clearer parameter documentation
GPU FAQ: Comprehensive hardware compatibility guide including VRAM recommendations, GPU buying guide, and Apple Silicon support info
Dependency Updates: Bump sleap-io to 0.6.5 and pin skia-python for Python 3.13 compatibility

Installation

# Upgrade to v0.1.2
uv tool install sleap-nn[torch] --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: 0.1.2

Documentation

Improved Inference Guide (#482)

The inference guide has been significantly expanded:

New sections:

Quick Start - TL;DR box with basic commands and common fixes
Viewing Results - How to validate output using SLEAP GUI, Python, or exports
Recommended Starting Points - Problem → solution tuning table

Expanded content:

Detailed parameter behavior for each model type
Clarified peak_threshold behavior (centroids removed vs keypoints become NaN)
Instance score source explained for top-down vs bottom-up models
Expanded troubleshooting from 3 items to 10+ specific scenarios

GPU & Hardware FAQ (#482)

New GPU FAQ section with comprehensive hardware guidance:

VRAM recommendations table
GPU buying guide (budget to workstation)
NVIDIA GPU compatibility matrix (RTX 50/40/30/20 series, workstation, data center)
CUDA version guidance including RTX 50-series (Blackwell) requirements
Apple Silicon (MPS) support and limitations
GPU troubleshooting (OOM, detection, compute capability errors)

Dependency Updates

sleap-io 0.6.5 (#481)

Updated minimum sleap-io version from 0.6.2 to 0.6.5, which includes:

ROI and segmentation mask support (experimental)
numpy 2.x compatibility fix for suggestion frames
Memory improvements for video rendering

Python 3.13 Compatibility (#481)

Pinned skia-python to ≤138.0 for Python 3.13 compatibility (matches sleap-io constraint).

Changelog

Documentation

PR	Title
#482	Improve inference guide and add GPU FAQ section

Chores

PR	Title
#481	Bump sleap-io to 0.6.5 and pin skia-python for Python 3.13
#483	Bump version to 0.1.2

Full Changelog: v0.1.1...v0.1.2

v0.1.1 ¶

SLEAP-NN v0.1.1 Release Notes

Summary

SLEAP-NN v0.1.1 is a patch release focused on bug fixes and performance improvements, with several quality-of-life enhancements for multi-GPU training and post-processing workflows.

Key highlights:

6.7x Faster Bottom-Up Inference: Vectorized PAF grouping and async video prefetching
New Post-Processing Filters: Filter predictions by node count and confidence scores
Multi-GPU Stability Fixes: Resolved DDP collective mismatches and deadlocks
Tracking Bug Fixes: Fixed track stealing and spurious track creation issues
TUI Config Generator: New terminal-based wizard for remote/HPC users

Installation

# Upgrade to v0.1.1
uv tool install sleap-nn[torch] --torch-backend auto --upgrade

# Verify
sleap-nn --version
# Expected output: 0.1.1

New Features

Web-based Config picker

Check out the new web-based Config Picker in the documentation where you can upload your SLP file and customize your config interactively.

TUI Config Generator (#461)

⚠️ Experimental: This feature is experimental and may change in future releases.

New wizard-style Terminal User Interface for generating training configurations on remote systems (HPC clusters, SSH sessions):

# Interactive mode
sleap-nn config /path/to/labels.slp

# Auto-generate config
sleap-nn config --auto /path/to/labels.slp --model single_instance

Features:

4-step wizard: Load SLP → Select Model → Configure → Export
Smart model recommendations based on dataset analysis
Memory estimation for batch size and caching decisions
Support for all model types

Node Count and Confidence Filters (#477)

New post-processing filters to remove low-quality predictions:

sleap-nn track -i video.mp4 -m model/ \
    --filter_min_visible_nodes 3 \
    --filter_min_visible_node_fraction 0.5 \
    --filter_min_mean_node_score 0.3 \
    --filter_min_instance_score 0.5

Filter	Description
`--filter_min_visible_nodes`	Minimum number of visible keypoints
`--filter_min_visible_node_fraction`	Minimum fraction of skeleton nodes visible
`--filter_min_mean_node_score`	Minimum average confidence across visible nodes
`--filter_min_instance_score`	Minimum overall instance score

sleap-io.js Integration (#472)

The config picker web tool now uses sleap-io.js for cleaner SLP file parsing:

Improved model type selection UX
Fixed skeleton overlay alignment with external videos
Smart frame selection for mismatched video/label durations
Model-specific config filenames (e.g., single_instance_config.yaml)

Performance Improvements

6.7x Faster Bottom-Up Inference (#463)

Major optimizations to bottom-up inference pipeline:

Hardware	Before	After	Speedup
NVIDIA A40 (TRT FP16)	55.6 FPS	370.9 FPS	6.7x
NVIDIA DGX Spark (TRT FP16)	34.2 FPS	190.7 FPS	5.6x

Optimizations include:

Producer/Consumer post-processing logic to reduce GPU waiting
Vectorized PAF grouping for cost matrix creation
Async video prefetching to avoid I/O bottlenecks

Bug Fixes

Multi-GPU / DDP Fixes

#476: Fixed DDP collective mismatch crash during sanity check by moving callback_metrics access before is_global_zero guards
#471: Fixed 30-second NCCL deadlock on "Stop Early" command by replacing barrier() with reduce_boolean_decision()

Tracking Fixes

#470: Fixed spurious track creation when using --filter_overlapping with --tracking by running NMS before track assignment
#467: Fixed track stealing bug in connect_single_breaks when instances temporarily disappear (fixes sleap#2618)

Inference Fixes

#475: Fixed incorrect RGB channel ordering in Skia augmentation on Linux by using explicit RGBA color type
#462: Fixed double squeeze bug in SingleInstanceLightningModule.forward() causing channel mismatch during validation
#460: Aligned predict CLI defaults with track CLI for bottom-up models (peak_conf_threshold: 0.1→0.2, min_line_scores: -0.5→0.25)

Installation Fixes

#479: Fixed Windows users getting CPU-only PyTorch when using CUDA extras by adding AMD64 platform marker

Documentation

#468: Improved inference guide with device index docs, CLI reference with "Values" column, new config generator guide
#473: Updated docs to reflect that --filter_overlapping now runs before tracking

Changelog

Features

PR	Title
#461	Add TUI config generator for interactive training configuration
#472	Integrate sleap-io.js into config picker for SLP file loading
#477	Add node count and confidence score filters for post-processing

Performance

PR	Title
#463	Bottom-up inference optimization (6.7x speedup)

Bug Fixes

PR	Title
#460	Align predict CLI defaults with track CLI for bottom-up models
#462	Fix double squeeze bug in SingleInstanceLightningModule.forward()
#467	Fix track stealing bug in connect_single_breaks
#470	Run filter_overlapping before tracking to prevent spurious track creation
#471	Fix TrainingControllerZMQ DDP stop deadlock
#475	Handle platform-specific Skia surface pixel format (BGRA vs RGBA)
#476	Fix DDP collective mismatch in callback_metrics access
#479	Add AMD64 platform marker for Windows CUDA PyTorch

Documentation

PR	Title
#468	Improve documentation for inference, CLI reference, and config generator
#473	Update filter_overlapping order to reflect PR #470 changes

Chores

PR	Title
#474	Trigger docs rebuild

Full Changelog: v0.1.0...v0.1.1

v0.1.0 ¶

SLEAP-NN v0.1.0 Release Notes

Summary

We are excited to announce SLEAP-NN v0.1.0, the first stable release of the v0.1.x series! This major release brings significant improvements across the entire stack: simplified installation, faster data pipelines, multi-GPU training support, ONNX/TensorRT export, and comprehensive documentation.

Key highlights:

Simplified Installation: One-command install with automatic GPU detection via --torch-backend auto
2x Faster Data Pipeline: New Skia-based augmentation backend replaces Kornia
Multi-GPU Training: Full DDP support with synchronized caching and callbacks
ONNX/TensorRT Export: 3-6x faster inference with optimized model formats
51x Faster Peak Refinement: Optimized tensor indexing for centroid/instance finding
Parallel Image Caching: Multi-threaded caching for faster training startup
Real-time Evaluation: Epoch-end metrics logged directly to WandB
Revamped Documentation: Comprehensive guides, tutorials, and API reference

Installation

Install with uv (Recommended)

# Automatic GPU detection (CUDA, MPS or CPU)
uv tool install sleap-nn[torch] --torch-backend auto

Verify Installation

sleap-nn --version
# Expected output: 0.1.0

sleap-nn system
# Shows full system diagnostics including GPU info

Optional Dependencies

# ONNX export (CPU inference)
uv tool install "sleap-nn[torch,export]" --torch-backend auto

# ONNX export (GPU inference)
uv tool install "sleap-nn[torch,export-gpu]" --torch-backend auto

# TensorRT support (Linux/Windows only)
uv tool install "sleap-nn[torch,tensorrt]" --torch-backend auto

Breaking Changes

1. Crop Size Semantics for Top-Down Models

The scaling behavior for top-down (centered-instance) models has changed:

Aspect	Old Behavior	New Behavior
Order	Resize full image first, then crop	Crop first, then resize
`crop_size` meaning	Region size in scaled coordinates	Region size in original image coordinates

Migration: Review your crop_size configuration values. Previously trained models may produce different results.

2. Model Run Folder File Naming

File naming conventions have been standardized:

Old Pattern	New Pattern
`labels_train_gt_0.slp`	`labels_gt.train.0.slp`
`labels_val_gt_0.slp`	`labels_gt.val.0.slp`
`pred_train_0.slp`	`labels_pr.train.0.slp`
`pred_val_0.slp`	`labels_pr.val.0.slp`
`train_0_pred_metrics.npz`	`metrics.train.0.npz`
`val_0_pred_metrics.npz`	`metrics.val.0.npz`

3. `load_metrics()` API Changes

Change	Old	New
Parameter name	`model_path`	`path`
Default split	`"val"`	`"test"`

4. Video Path Mapping CLI Syntax

# Old syntax (no longer works)
sleap-nn train -c config --video-path-map "/old/path->/new/path"

# New syntax
sleap-nn train -c config --video-path-map /old/path /new/path

New Features

Simplified Installation (#405)

Install with automatic GPU detection:

uv tool install sleap-nn[torch] --torch-backend auto

Supports CUDA 11.8, 12.8, 13.0, Apple Silicon (MPS), and CPU-only installations.

Skia-Based Augmentation Backend (#431, #434)

Replaced Kornia with Skia-python for 2x faster augmentation:

Backend	Throughput	Relative
Kornia	142 samples/sec	1.0x
Skia	285 samples/sec	2.0x

The new pipeline also maintains uint8 images until GPU transfer, achieving 4x bandwidth savings.

Multi-GPU Training Support (#435, #436, #437, #453)

Full DDP (Distributed Data Parallel) support:

# Train on multiple GPUs
sleap-nn train config.yaml trainer_config.devices=4 trainer_config.strategy=ddp

Features:

Synchronized run_name generation across workers
Proper GPU device ordering to prevent NCCL errors
DDP-compatible callbacks with barrier synchronization
Subprocess-based launcher for reliable multi-GPU caching

ONNX/TensorRT Export (#418, #456)

Export trained models for optimized inference:

# Export to ONNX
sleap-nn export /path/to/model -o exports/my_model --format onnx

# Export to TensorRT FP16
sleap-nn export /path/to/model -o exports/my_model --format both

# Run inference on exported model
sleap-nn predict exports/my_model video.mp4 -o predictions.slp

Performance (NVIDIA RTX A6000, batch size 8):

Model	PyTorch	TensorRT FP16	Speedup
single_instance	3,111 FPS	11,039 FPS	3.5x
topdown	94 FPS	525 FPS	5.6x
bottomup	113 FPS	524 FPS	4.6x

Parallel Image Caching (#432)

Multi-threaded caching for faster training startup:

data_config:
  preprocessing:
    parallel_caching: true
    cache_workers: 4  # Number of parallel caching threads

Epoch-End Evaluation Metrics (#414, #449)

Real-time evaluation metrics logged to WandB during training:

trainer_config:
  eval:
    enabled: true
    frequency: 1  # Evaluate every epoch

Metrics include: mOKS, mAP, mAR, PCK@5, PCK@10, distance percentiles, and visibility precision/recall.

Post-Inference Filtering (#420)

Remove overlapping/duplicate predictions:

sleap-nn track -i video.mp4 -m model/ \
    --filter_overlapping \
    --filter_overlapping_method oks \
    --filter_overlapping_threshold 0.5

Simplified Train CLI (#429)

Training can now be started with a single config file path:

# Simple usage
sleap-nn train path/to/config.yaml

# With overrides
sleap-nn train config.yaml trainer_config.max_epochs=100

GUI Integration (#424)

New --gui flag for JSON progress output:

sleap-nn track --data_path video.mp4 --model_paths model/ --gui

Warmup Learning Rate Schedulers (#442)

New scheduler options:

linear_warmup_cosine_annealing: Linear warmup followed by cosine decay
linear_warmup_linear_decay: Linear warmup followed by linear decay

System Diagnostics (#391)

New diagnostic commands:

sleap-nn --version  # Show version
sleap-nn system     # Full system diagnostics

Provenance Metadata (#407)

Inference outputs now include full reproducibility metadata in SLP files.

Performance Improvements

51x Faster Peak Refinement (#426)

Replaced kornia's crop_and_resize with fast tensor indexing:

Platform	Before	After	Speedup
MPS (Apple Silicon)	21.45 ms	0.42 ms	51x
CUDA (RTX A6000)	2.64 ms	0.15 ms	17x

GPU-Accelerated Normalization (#406)

Image normalization now runs on GPU, reducing PCIe bandwidth by 4x:

Image Size	Before	After	Speedup
1024x1280 grayscale	55.2 FPS	64.7 FPS	17%
3307x3304 RGB	6.7 FPS	10.1 FPS	50%

Default Augmentations (#445, #447)

Augmentations are now enabled by default with sensible presets:

Rotation: -15 to +15 degrees
Scale: 0.9 to 1.1

Bug Fixes

#428: Fixed skip connection channel mismatch in ConvNext/SwinT decoders
#429: Fixed crop device mismatch during top-down inference
#423: Fixed CSV logger not capturing learning_rate
#436: Fixed multi-GPU DDP duplicate GPU detection error
#437: Fixed DDP synchronization in training callbacks
#439: Fixed confusing weight loading logging for legacy models
#440: Fixed cache memory estimation to account for DataLoader workers
#441: Fixed self.log warnings when no logger is configured
#451: Fixed critical bugs in weight verification
#454: Fixed caching progress bar not showing in subprocess
#382: Fixed max_instances handling in centroid-only inference
#385: Fixed crash on frames with empty instances
#392: Clean up run folder when training canceled via GUI
#395, #401, #402: Various WandB visualization fixes

Documentation

Revamped Documentation (#444, #455)

Completely restructured documentation at nn.sleap.ai:

Tutorials: Quick Start guide, Your First Model walkthrough
Guides: Training, Inference, Evaluation, Tracking, Export, Multi-GPU
Configuration Reference: Detailed docs for data, model, and trainer configs
API Reference: Auto-generated from docstrings
Evaluation Metrics Reference (#448): Comprehensive guide to all metrics

Changelog

Features

PR	Title
#405	Add CUDA 13 support and simplify installation with --torch-backend
#418	Add ONNX/TensorRT export module
#420	Add post-inference filtering for overlapping instances
#424	Add --gui flag for JSON progress output in inference
#429	Add --config flag for simpler train CLI
#431	Add Skia-based augmentation backend for faster data pipeline
#432	Add parallel image caching for faster dataset preparation
#433	Add UnifiedVizCallback for consolidated visualization outputs
#442	Add warmup learning rate schedulers
#445	Enable default augmentations with rotation and scale
#449	Add centroid-specific evaluation callback with distance-based metrics
#453	Add multi-GPU training support with subprocess-based run_name sync
#456	Add TensorRT as uv-managed optional dependency

Performance

PR	Title
#406	Optimize inference by deferring normalization to GPU
#426	Replace kornia crop_and_resize with fast tensor indexing (17-51x speedup)
#434	Fix uint8 pipeline to achieve 4x GPU bandwidth savings

Bug Fixes

PR	Title
#423	Fix CSV logger not capturing learning_rate
#428	Fix skip connection channel mismatch in ConvNext/SwinT decoders
#435	Fix multi-GPU disk caching run_name synchronization issue
#436	Fix multi-GPU DDP duplicate GPU detection error
#437	Fix DDP synchronization in training callbacks
#439	Fix confusing weight loading logging for legacy models
#440	Fix cache memory estimation to account for DataLoader workers
#441	Fix self.log warnings when no logger is configured
#451	Fix critical bugs in weight verification, docs, and tests
#454	Fix caching progress bar not showing in subprocess

Documentation

PR	Title
#390	Add CLI reference page and Colab notebooks
#419	Add Exporting guide to How-to guides section
#425	Add prerelease alias to docs deployment
#444	Revamp documentation structure and content
#448	Add evaluation metrics reference page to docs
#455	Fix docs issues and add installation instructions

Breaking Changes

PR	Title
#381	Fix crop size behavior for top-down models
#389	Fix train CLI path replacement syntax
#408	Standardize model run folder file naming conventions
#409	Improve load_metrics with format compatibility and flexible paths

Full Changelog: v0.0.5...v0.1.0

v0.1.0a4 ¶

v0.1.0a4 Release Notes

Summary

This pre-release focuses on bug fixes, performance improvements, and CLI usability enhancements:

Simpler Train CLI: New --config flag and positional config support for sleap-nn train
17-51x Faster Peak Refinement: Replaced kornia-based cropping with fast tensor indexing
ConvNext/SwinT Bug Fix: Fixed skip connection channel mismatch that broke training with these backbones
GUI Integration: New --gui flag for SLEAP frontend progress reporting

For the full list of major features, breaking changes, and improvements introduced in the v0.1.0 series, see the v0.1.0a0 release notes.

What's New in v0.1.0a4

Features

Simplified Train CLI (#429)

Training can now be started with a single config file path:

# NEW: Positional config path
sleap-nn train path/to/config.yaml

# NEW: --config flag
sleap-nn train --config path/to/config.yaml

# With Hydra overrides
sleap-nn train config.yaml trainer_config.max_epochs=100

# Legacy flags still work
sleap-nn train --config-dir /path/to/dir --config-name myconfig

The CLI now uses rich-click for styled help output with better formatting and readability.

GUI Progress Mode (#424)

New --gui flag enables JSON progress output for SLEAP GUI integration:

sleap-nn track --data_path video.mp4 --model_paths model/ --gui

Output format:

{"n_processed": 100, "n_total": 1410, "rate": 38.4, "eta": 34.1}
{"n_processed": 200, "n_total": 1410, "rate": 39.2, "eta": 30.8}

This enables real-time progress updates when running inference from the SLEAP GUI.

Performance

17-51x Faster Peak Refinement (#426)

Replaced kornia's crop_and_resize with fast tensor indexing for peak refinement:

Platform	Before	After	Speedup
MPS (M-series Mac)	21.45 ms	0.42 ms	51x
CUDA (RTX A6000)	2.64 ms	0.15 ms	17x

This also enables integral refinement on Mac - the MPS workaround that disabled it has been removed.

Bug Fixes

ConvNext/SwinT Skip Connection Fix (#428)

Fixed RuntimeError: Given groups=1, weight of size [X, Y, 3, 3], expected input to have Y channels when training with ConvNext or SwinT backbones.

What was broken: Training with ConvNext/SwinT backbones crashed during validation due to channel mismatch in skip connections. The decoder assumed skip channels matched computed decoder filters, but ConvNext/SwinT encoder stages have different channel counts.

Impact: Users can now successfully train models with ConvNext and SwinT backbones. All 24 architecture tests pass.

Crop Device Mismatch Fix (#429)

Fixed RuntimeError: indices should be either on cpu or on the same device as the indexed tensor during top-down inference when bboxes tensor was on GPU but images were on CPU.

CSV Learning Rate Logging Fix (#423)

Fixed regression from v0.1.0a2 where learning_rate column in training_log.csv was always empty.

What was broken: PR #417 changed learning rate logging from lr-Adam to train/lr, but the CSV logger only checked for the old format.

Now: The CSV logger checks for train/lr (new format), lr-* (legacy), and learning_rate (direct) in that order. Also adds model-specific loss columns for better parity with wandb logging.

GUI Progress 99% Fix (#429)

Fixed inference progress ending at 99% instead of 100% in GUI mode. The throttled progress reporting was skipping the final update.

Documentation

Prerelease Docs Alias (#425)

Pre-release documentation is now accessible at both:

Version-specific: https://sleap.ai/sleap-nn/v0.1.0a4/
Alias: https://sleap.ai/sleap-nn/prerelease/

Internal

Test Suite Optimization (#427)

Optimized the 10 slowest tests for faster CI runs:

Test	Before	After	Improvement
test_main_cli	54.44s	21.76s	60%
test_bottomup_predictor	6.71s	1.76s	74%
test_predict_main	15.97s	5.35s	67%

Total estimated savings: ~55% reduction for slowest tests.

Installation

This is an alpha pre-release. Pre-releases are excluded by default per PEP 440 - you must explicitly opt in.

Install with uv (Recommended)

# With --prerelease flag (requires uv 0.9.20+)
uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow

# Or pin to exact version
uv tool install "sleap-nn[torch]==0.1.0a4" --torch-backend auto

Run with uvx (One-off execution)

uvx --from "sleap-nn[torch]" --prerelease=allow --torch-backend auto sleap-nn system

Verify Installation

sleap-nn --version
# Expected output: 0.1.0a4

sleap-nn system
# Shows full system diagnostics including GPU info

Upgrading from v0.1.0a3

If you already have v0.1.0a3 installed with --prerelease=allow:

# Simple upgrade (retains original settings like --prerelease=allow)
uv tool upgrade sleap-nn

To force a complete reinstall:

uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow --force

Changelog

PR	Category	Title
#423	Bug Fix	Fix CSV logger not capturing learning_rate
#424	Feature	Add --gui flag for JSON progress output in inference
#425	Documentation	Add prerelease alias to docs deployment
#426	Performance	Replace kornia crop_and_resize with fast tensor indexing
#427	Internal	Optimize slow tests for faster CI runs
#428	Bug Fix	Fix skip connection channel mismatch in ConvNext/SwinT decoders
#429	Feature	Add --config flag for simpler train CLI + fix crop device mismatch

Full Changelog: v0.1.0a3...v0.1.0a4

v0.1.0a3 ¶

Summary

This pre-release adds powerful new capabilities for high-performance inference and post-processing:

ONNX/TensorRT Export: Export trained models to optimized formats for 3-6x faster inference
Post-Inference Filtering: Remove overlapping/duplicate predictions using IOU or OKS similarity
Improved WandB Logging: Better metrics organization and run naming

For the full list of major features, breaking changes, and improvements introduced in the v0.1.0 series, see the v0.1.0a0 release notes.

What's New in v0.1.0a3

Features

ONNX/TensorRT Export Module (#418)

A complete model export system for high-performance inference:

# Export to ONNX
sleap-nn export /path/to/model -o exports/my_model --format onnx

# Export to both ONNX and TensorRT FP16
sleap-nn export /path/to/model -o exports/my_model --format both

# Run inference on exported model
sleap-nn predict exports/my_model video.mp4 -o predictions.slp

Performance Benchmarks (NVIDIA RTX A6000):

Batch size 1 (latency-optimized):

Model	Resolution	PyTorch	ONNX-GPU	TensorRT FP16	Speedup
single_instance	192×192	1.8 ms	1.3 ms	0.31 ms	5.9x
centroid	1024×1024	2.5 ms	2.7 ms	0.77 ms	3.2x
topdown	1024×1024	11.4 ms	9.7 ms	2.31 ms	4.9x
bottomup	1024×1280	12.3 ms	9.6 ms	2.52 ms	4.9x
multiclass_topdown	1024×1024	8.3 ms	9.1 ms	1.84 ms	4.5x
multiclass_bottomup	1024×1024	9.4 ms	9.4 ms	2.64 ms	3.6x

Batch size 8 (throughput-optimized):

Model	Resolution	PyTorch	ONNX-GPU	TensorRT FP16	Speedup
single_instance	192×192	3,111 FPS	3,165 FPS	11,039 FPS	3.5x
centroid	1024×1024	453 FPS	474 FPS	1,829 FPS	4.0x
topdown	1024×1024	94 FPS	122 FPS	525 FPS	5.6x
bottomup	1024×1280	113 FPS	121 FPS	524 FPS	4.6x
multiclass_topdown	1024×1024	127 FPS	145 FPS	735 FPS	5.8x
multiclass_bottomup	1024×1024	116 FPS	120 FPS	470 FPS	4.1x

Speedup is relative to PyTorch baseline.

Supported model types:

Single Instance, Centroid, Centered Instance
Top-Down (combined centroid + instance)
Bottom-Up (multi-instance with PAF grouping)
Multi-class Top-Down and Bottom-Up (with identity classification)

New CLI commands:

sleap-nn export - Export models to ONNX/TensorRT
sleap-nn predict - Run inference on exported models

New optional dependencies:

uv pip install "sleap-nn[export]"      # ONNX CPU inference
uv pip install "sleap-nn[export-gpu]"  # ONNX GPU inference
uv pip install "sleap-nn[tensorrt]"    # TensorRT support

See the Export Guide for full documentation.

Post-Inference Filtering for Overlapping Instances (#420)

New capability to remove duplicate/overlapping pose predictions after model inference:

# Filter with IOU method (default)
sleap-nn track -i video.mp4 -m model/ --filter_overlapping

# Use OKS method with custom threshold
sleap-nn track -i video.mp4 -m model/ \
    --filter_overlapping \
    --filter_overlapping_method oks \
    --filter_overlapping_threshold 0.5

New CLI options for sleap-nn track:

Option	Default	Description
`--filter_overlapping`	`False`	Enable filtering using greedy NMS
`--filter_overlapping_method`	`iou`	Similarity method: `iou` (bbox) or `oks` (keypoints)
`--filter_overlapping_threshold`	`0.8`	Similarity threshold (lower = more aggressive)

Programmatic API:

from sleap_nn.inference.postprocessing import filter_overlapping_instances

labels = filter_overlapping_instances(labels, threshold=0.5, method="oks")

Why use this? Previously, IOU-based filtering only existed in the tracking pipeline. This feature allows filtering overlapping predictions without requiring --tracking.

Improvements

WandB Run Naming and Metrics Logging (#417)

Fixed run naming: WandB runs now correctly use auto-generated run names
Improved metrics organization: All metrics use / separator for automatic panel grouping in WandB UI:
- train/loss, train/lr - Training metrics (epoch x-axis)
- val/loss - Validation metrics (epoch x-axis)
- eval/val/ - Epoch-end evaluation metrics
- eval/test.X/ - Post-training test set metrics
New metrics logged:
- train/lr - Learning rate (useful for monitoring LR schedulers)
- PCK@5, PCK@10 - PCK at 5px and 10px thresholds
- distance/p95, distance/p99 - Additional distance percentiles

Documentation

Exporting Guide (#419): Added comprehensive export documentation to How-to guides navigation

Installation

This is an alpha pre-release. Pre-releases are excluded by default per PEP 440 - you must explicitly opt in.

Install with uv (Recommended)

# With --prerelease flag (requires uv 0.9.20+)
uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow

# Or pin to exact version
uv tool install "sleap-nn[torch]==0.1.0a3" --torch-backend auto

Run with uvx (One-off execution)

uvx --from "sleap-nn[torch]" --prerelease=allow --torch-backend auto sleap-nn system

Verify Installation

sleap-nn --version
# Expected output: 0.1.0a3

sleap-nn system
# Shows full system diagnostics including GPU info

Upgrading from v0.1.0a2

If you already have v0.1.0a2 installed with --prerelease=allow:

# Simple upgrade (retains original settings like --prerelease=allow)
uv tool upgrade sleap-nn

To force a complete reinstall:

uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow --force

Changelog

PR	Category	Title
#417	Improvement	Fix wandb run naming and improve metrics logging
#418	Feature	Add ONNX/TensorRT export module
#419	Documentation	Add Exporting guide to How-to guides section
#420	Feature	Add post-inference filtering for overlapping instances

Full Changelog: v0.1.0a2...v0.1.0a3

v0.1.0a2 ¶

Summary

This pre-release adds real-time evaluation metrics during training and improves video matching robustness in the evaluation pipeline:

Epoch-End Evaluation: New metrics logged to WandB at the end of each validation epoch (mOKS, mAP, mAR, PCK, distance metrics)
Robust Video Matching: Improved evaluation video matching using sleap-io's Labels.match() API

For the full list of major features, breaking changes, and improvements introduced in the v0.1.0 series, see the v0.1.0a0 release notes.

What's New in v0.1.0a2

Features

Epoch-End Evaluation Metrics (#414): Real-time evaluation metrics are now computed at the end of each validation epoch and logged to WandB. This enables monitoring training quality without waiting for post-training evaluation.

New metrics logged:

Metric	Description
`val_mOKS`	Mean Object Keypoint Similarity [0-1]
`val_oks_voc_mAP`	VOC-style mean Average Precision [0-1]
`val_oks_voc_mAR`	VOC-style mean Average Recall [0-1]
`val_avg_distance`	Mean Euclidean distance error (pixels)
`val_p50_distance`	Median Euclidean distance error (pixels)
`val_mPCK`	Mean Percentage of Correct Keypoints [0-1]
`val_visibility_precision`	Precision for visible keypoint detection
`val_visibility_recall`	Recall for visible keypoint detection

Enable in your training config:

trainer_config:
  eval:
    enabled: true      # Enable epoch-end evaluation
    frequency: 1       # Evaluate every epoch (or higher for less frequent)
    oks_stddev: 0.025  # OKS standard deviation parameter

Improvements

Robust Video Matching in Evaluation (#415): The evaluation module now uses sleap-io's Labels.match() API for more robust video matching between ground truth and prediction labels. This fixes several common failure scenarios:
- Embedded videos (.pkg.slp) with different internal paths
- Cross-platform path differences (Windows vs Linux)
- Renamed or moved video files

Bug Fixes

Embedded video handling (#414): get_instances() now correctly handles embedded videos that lack backend.filename attributes, preventing errors during evaluation.
Centroid model ground truth matching (#414): Centroid models now properly match centroids to ground truth instances for epoch-end evaluation.
Bottom-up training stability (#414): Added max_peaks_per_node=100 guardrail to prevent combinatorial explosion when noisy early-training confidence maps produce spurious peaks.

Dependencies

sleap-io: Minimum version bumped from >=0.6.0 to >=0.6.2 for Labels.match() API support

Installation

This is an alpha pre-release. Pre-releases are excluded by default per PEP 440 - you must explicitly opt in.

Install with uv (Recommended)

# With --prerelease flag (requires uv 0.9.20+)
uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow

# Or pin to exact version
uv tool install "sleap-nn[torch]==0.1.0a2" --torch-backend auto

Run with uvx (One-off execution)

uvx --from "sleap-nn[torch]" --prerelease=allow --torch-backend auto sleap-nn system

Verify Installation

sleap-nn --version
# Expected output: 0.1.0a2

sleap-nn system
# Shows full system diagnostics including GPU info

Upgrading from v0.1.0a1

If you already have v0.1.0a1 installed with --prerelease=allow:

# Simple upgrade (retains original settings like --prerelease=allow)
uv tool upgrade sleap-nn

To force a complete reinstall:

uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow --force

Changelog

PR	Category	Title
#414	Feature	Add epoch-end evaluation metrics to WandB logging
#415	Improvement	Use sleap-io Labels.match() API for robust video matching in evaluation

Full Changelog: v0.1.0a1...v0.1.0a2

v0.1.0a1 ¶

Summary

This pre-release is a minor update to v0.1.0a0 with quality-of-life improvements for training workflows:

Progress Feedback: Rich progress bar during dataset caching eliminates the "freeze" after startup
Disk Space Management: Automatic cleanup of WandB local logs (saves GB of disk space per run)

For the full list of major features, breaking changes, and improvements introduced in the v0.1.0 series, see the v0.1.0a0 release notes.

What's New in v0.1.0a1

Features

WandB Local Log Cleanup (#412): Added delete_local_logs option to WandBConfig that automatically deletes the local wandb/ folder after training completes. By default, logs are automatically deleted when syncing online and kept when logging offline. This can save several GB of disk space per training run. Set trainer_config.wandb.delete_local_logs=false to keep local logs.

Improvements

Training Startup Progress Bar (#411): Added a rich progress bar during dataset caching to provide visual feedback during training startup. Previously, there was no indication while images were being cached to disk or memory after the "Input image shape" log message.
Simplified Log Format (#411): Cleaned up log output by removing module names and log level fields for more user-friendly output.

Installation

This is an alpha pre-release. Pre-releases are excluded by default per PEP 440 - you must explicitly opt in.

Install with uv (Recommended)

# With --prerelease flag (requires uv 0.9.20+)
uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow

# Or pin to exact version
uv tool install "sleap-nn[torch]==0.1.0a1" --torch-backend auto

Run with uvx (One-off execution)

uvx --from "sleap-nn[torch]" --prerelease=allow --torch-backend auto sleap-nn system

Verify Installation

sleap-nn --version
# Expected output: 0.1.0a1

sleap-nn system
# Shows full system diagnostics including GPU info

Upgrading from v0.1.0a0

If you already have v0.1.0a0 installed with --prerelease=allow:

# Simple upgrade (retains original settings like --prerelease=allow)
uv tool upgrade sleap-nn

To force a complete reinstall:

uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow --force

Changelog

PR	Category	Title
#411	Improvement	Improve logging during training startup
#412	Feature	Add option to clean up wandb local logs after training

Full Changelog: v0.1.0a0...v0.1.0a1

v0.1.0a0 ¶

Summary

This pre-release introduces major improvements to sleap-nn including simplified installation, enhanced training controls, comprehensive inference provenance, and significant performance optimizations. It also includes several breaking changes that warrant testing before the stable v0.1.0 release.

Key highlights:

Simplified Installation: New --torch-backend auto flag for automatic GPU detection
CUDA 13.0 Support: Full support for latest CUDA version
GPU-accelerated Inference: Up to 50% faster inference via GPU normalization
Provenance Tracking: Full reproducibility metadata in output SLP files
Enhanced Training Controls: Independent augmentation probabilities, auto crop padding
System Diagnostics: New sleap-nn system command for troubleshooting

Installation

This is an alpha pre-release. Pre-releases are excluded by default per PEP 440 - you must explicitly opt in.

Install with uv (Recommended)

# With --prerelease flag (requires uv 0.9.20+)
uv tool install sleap-nn[torch] --torch-backend auto --prerelease=allow

# Or pin to exact version
uv tool install "sleap-nn[torch]==0.1.0a0" --torch-backend auto

Install with uvx (One-off execution)

uvx --from "sleap-nn[torch]" --prerelease=allow --torch-backend auto sleap-nn system

Install with pip

pip install --pre sleap-nn[torch] --index-url https://pypi.org/simple --extra-index-url https://download.pytorch.org/whl/cu128

Verify Installation

sleap-nn --version
# Expected output: 0.1.0a0

sleap-nn system
# Shows full system diagnostics including GPU info

Breaking Changes

1. Crop Size Semantics for Top-Down Models (PR #381)

Impact: High - affects model training and inference

The scaling behavior for top-down (centered-instance) models has changed:

Aspect	Old Behavior	New Behavior
Order	Resize full image first, then crop	Crop first, then resize
`crop_size` meaning	Region size in scaled coordinates	Region size in original image coordinates

Migration: Review your crop_size configuration values. Previously trained models may produce different results.

2. Model Run Folder File Naming (PR #408)

Impact: Medium - affects scripts that read model outputs

File naming conventions standardized:

Old Pattern	New Pattern
`labels_train_gt_0.slp`	`labels_gt.train.0.slp`
`labels_val_gt_0.slp`	`labels_gt.val.0.slp`
`pred_train_0.slp`	`labels_pr.train.0.slp`
`pred_val_0.slp`	`labels_pr.val.0.slp`
`train_0_pred_metrics.npz`	`metrics.train.0.npz`
`val_0_pred_metrics.npz`	`metrics.val.0.npz`

3. `load_metrics()` API Changes (PR #409)

Impact: Low - affects programmatic metrics loading

Change	Old	New
Parameter name	`model_path`	`path`
Default split	`"val"`	`"test"`

4. Video Path Mapping CLI Syntax (PR #389)

Impact: Low - affects CLI users with path remapping

# Old syntax (no longer works)
sleap-nn train -c config --video-path-map "/old/path->/new/path"

# New syntax
sleap-nn train -c config --video-path-map /old/path /new/path

Performance Improvements

GPU-Accelerated Normalization (PR #406)

Image normalization now runs on GPU, reducing PCIe bandwidth by 4x.

Image Size	Before	After	Speedup
1024x1280 grayscale	55.2 FPS	64.7 FPS	17%
3307x3304 RGB	6.7 FPS	10.1 FPS	50%

New Features

Simplified Installation (PR #405): uv tool install sleap-nn[torch] --torch-backend auto
CUDA 13.0 Support (PR #405): New --torch-backend cu130 option
System Diagnostics (PR #391): sleap-nn system command and --version flag
Provenance Metadata (PR #407): Full reproducibility tracking in output SLP files
Video Path Remapping (PR #387, #389): Remap paths at training time
Frame Filtering (PR #396, #397): --exclude_user_labeled and --only_predicted_frames
Enhanced Data Pipeline (PR #394): Auto crop padding, independent augmentation probabilities
Multiple Test Files (PR #383): Evaluate against multiple test datasets
Enhanced WandB (PR #393, #395, #401): Interactive visualizations, per-head loss logging
Centroid Confmaps (PR #386): Return centroid confidence maps in top-down inference

Bug Fixes

#382: Fixed max_instances handling in centroid-only inference
#385: Fixed crash on frames with empty instances
#395: Fixed WandB visualization issues
#397: Fixed --exclude_user_labeled being ignored with --video_index
#401: Fixed PAF visualization scaling
#402: Fixed WandB deprecation warning
#394: Fixed user_instances_only handling bugs

Improvements

#380: Use sleap-io built-in video matching methods
#390: Added CLI reference page and Colab notebooks to docs
#392: Run folders cleaned up when training canceled via GUI
#398: Comprehensive test coverage improvements
#400: WandB URL reported via ZMQ on train start
#403: Migrated dev deps to PEP 735 dependency-groups

Changelog

PR	Category	Title
#380	Improvement	Use sleap-io built-in video matching methods
#381	Breaking	Fix crop size behavior for top-down models
#382	Fix	Fix max_instances handling in centroid-only inference
#383	Feature	Support list of paths for test_file_path
#384	Feature	Add source image to FindInstancePeaksGroundTruth output
#385	Fix	Fix running inference on frames with empty instances
#386	Feature	Return centroid confmaps when running topdown inference
#387	Feature	Add video path remapping options to train CLI
#389	Breaking	Fix train CLI path replacement syntax
#390	Docs	Add CLI reference page and Colab notebooks
#391	Feature	Add system diagnostics command and --version flag
#392	Fix	Clean up run folder when training is canceled via GUI
#393	Feature	Improve wandb visualization with slider support
#394	Feature	Enhance data pipeline with auto crop padding
#395	Fix	Fix wandb visualization issues
#396	Feature	Add --exclude_user_labeled and --only_predicted_frames flags
#397	Fix	Fix --exclude_user_labeled flag being ignored with --video_index
#398	Tests	Add comprehensive test coverage
#400	Feature	Report WandB URL via ZMQ on train start
#401	Feature	Add per-head loss logging and fix PAF visualization
#402	Fix	Fix wandb deprecation warning
#403	Improvement	Move dev dependencies to PEP 735 dependency-groups
#405	Feature	Add CUDA 13 support and simplify installation
#406	Performance	Optimize inference by deferring normalization to GPU
#407	Feature	Add provenance metadata to inference output SLP files
#408	Breaking	Standardize model run folder file naming
#409	Breaking	Improve load_metrics with format compatibility

Full Changelog: v0.0.5...v0.1.0a0

v0.0.5 ¶

Summary

This release includes important bug fixes, usability improvements, and configuration enhancements. Key highlights include automatic video-specific output naming for multi-video predictions, improved progress tracking, better handling of edge cases in configuration files, and enhanced security for API key storage.

Major changes

New Features

Progress Bar for Tracking (#366)

Added visual progress tracking during tracking operations, providing real-time feedback on tracking progress for better user experience.

Video-Specific Output Paths (#378)

When running inference with the video_index parameter on multi-video .slp files, output files now automatically include the video name to prevent overwrites. Previously, all predictions would save to the same path (e.g., labels.predictions.slp), requiring users to
manually specify unique output paths. Now, predictions are saved with the format <labels_file>.<video_name>.predictions.slp, enabling seamless batch processing of multiple videos from the same project file.

Bug Fixes

Resume Checkpoint Mapping (#370)

Fixed checkpoint mapping when resuming training from PyTorch model checkpoints, ensuring proper state restoration for torch models.

Metrics Format Compatibility (#371)

Updated metrics saving format to match SLEAP 1.4 specifications and eliminated code duplication in metrics handling, ensuring cross-compatibility between SLEAP-NN and SLEAP 1.4.

Configuration Parameter Handling (#377)

Improved handling of run_name and ckpt_dir configuration parameters when set to empty strings or the string literal "None" in YAML files. This prevents unexpected behavior and ensures consistent defaults are applied.

Security Improvements

API Key Protection (#372)

WandB API keys are now automatically masked when saving initial_config.yaml files, preventing accidental exposure of sensitive credentials in saved configurations.

Configuration & Training Improvements

Optimized Default Parameters (#374, #375)

Updated default trainer configuration parameters based on extensive training experiments, improving training stability and convergence behavior out of the box.

Documentation

Dependency Update Instructions (#376)

Added comprehensive instructions for updating dependencies across all installation methods (GPU, CPU, and Apple Silicon), making it easier for users to maintain up-to-date environments.

Changelog

Add progress bar to tracker by @gitttt-1234 in #366
Fix resume checkpoint mapping for torch models only by @gitttt-1234 in #370
Fix metrics saving format to match SLEAP 1.4 and eliminate code duplication by @gitttt-1234 in #371
Mask wandb API key in initial_config.yaml by @gitttt-1234 in #372
Update default trainer configuration parameters for improved training stability by @gitttt-1234 in #374
Update default configuration values for improved training by @gitttt-1234 in #375
Add dependency update instructions for all installation methods by @gitttt-1234 in #376
Handle empty and "None" string values for run_name and ckpt_dir config parameters by @gitttt-1234 in #377
Append video name to output path when video_index is specified by @gitttt-1234 in #378
Bump version to 0.0.5 by @gitttt-1234 in #379

Full Changelog: v0.0.4...v0.0.5

v0.0.4 ¶

Summary

This release includes a dependency version bump and a critical bug fix for empty instance handling. The minimum torchvision version has been updated to 0.20.0, and sleap-io minimum version has been set to 0.5.7 to ensure compatibility with the latest features and improvements.

Major changes

Dependency Version Updates (#365)

Minimum torchvision version: Set to 0.20.0 across all torch extras (torch, torch-cpu, torch-cuda118, torch-cuda128)
Minimum sleap-io version: Updated to 0.5.7 for improved compatibility

Bug Fixes

Fixed empty instance handling (#364): Improved handling of instances with only NaN keypoints in the instance cropping method and CenteredInstanceDataset class. Previously, these instances would trigger "NaN values encountered" warnings when computing bounding boxes. The fix ensures only non-empty instances are processed for crop size computation and removes redundant filtering logic.

Changelog

Fix empty instance handling (#364)
Bump minimum torchvision version to 0.20.0 (#365)

Full Changelog: v0.0.3...v0.0.4

v0.0.3 ¶

Summary

This release delivers critical bug fixes for multiprocessing support, enhanced tracking capabilities, and significant improvements to the inference workflow. The v0.0.3 release resolves HDF5 pickling issues that prevented proper multiprocessing on macOS/Windows, fixes ID models, and introduces new track cleaning parameters for better tracking performance.

Major changes

Fixed Multiprocessing Bug with num_workers > 0 (#359)

Resolved HDF5 pickling issues that prevented proper multiprocessing on macOS/Windows systems. This fix enables users to utilize multiple workers for faster data loading during training and inference when caching is enabled.

Fixed ID Models (#345)

Fixed minor issues with TopDown and BottomUp ID models.

The ID models dataset classes were re-computing the tracks from the labels file. However, they should just grab it from the head config classes parameter.
Fix shape mismatch issue with BottomUp ID models

Added Track Cleaning Arguments (#349)

Added new parameters for better track management and cleanup:

tracking_clean_instance_count: Target number of instances to clean after tracking
tracking_clean_iou_threshold: IOU threshold for cleaning overlapping instances
tracking_pre_cull_to_target: Pre-culling instances before tracking
tracking_pre_cull_iou_threshold: IOU threshold for pre-culling

Updated Installation Documentation (#348, #351)

Added comprehensive uv add installation instructions for modern Python package management instead of uv pip install method. Added warning for 3.14 python version to prevent installation issues.

Inference workflow enhancements (#360, #361)

Enhanced bottom-up model inference capabilities with improved performance and stability. Fix logger encoding issues on windows and better handle integral refinement error on mps accelerator.

Changelog

Fix ID models by @gitttt-1234 in #345
Fix changelog.md by @gitttt-1234 in #346
Add warning for Python v3.14 by @gitttt-1234 in #348
Add track cleaning args by @gitttt-1234 in #349
Update uv add installation docs by @gitttt-1234 in #351
Fix marimo usage docs by @gitttt-1234 in #352
Fix target instance count parameter by @gitttt-1234 in #358
Fix multiprocessing bug with num_workers>0 by @gitttt-1234 in #359
Minor fixes to inference workflow by @gitttt-1234 in #360
Update bottomup inference and add note on num_workers by @gitttt-1234 in #361
Bump version to v0.0.3 by @gitttt-1234 in #362

v0.0.2 ¶

Summary

This release focuses on several bug fixes and improvements across the training, inference, and CLI components of sleap-nn. It includes bug fixes for model backbones and loaders, enhancements to the configuration and CLI experience, improved robustness in multi-GPU training, and new options for device selection and tracking. Documentation and installation guides have also been updated, along with internal refactors to streamline the code consistency.

Major changes

Backbones & Models:
- Fixed bugs in Swin Transformer and UNet backbone filter computations.
- Corrected weight mapping for legacy TopDown ID models.
Inference & Tracking:
- Removed unintended loading of pretrained weights during inference.
- Fixed inference with suggestion frames and improved stalling handling.
- Added option to run tracking on selected frames and video indices.
- Added thread-safe video access to prevent backend crashes.
- Added function to load metrics for better evaluation reporting.
Training Pipeline:
- Fixed bugs in the training workflow with the infinite dataloader handling.
- Improved seeding behavior for reproducible label splits in multi-GPU setups.
- Fixed experiment run name generation across multi-GPU workers.
CLI & Config:
- Introduced unified sleap-nn CLI with subcommands (train, track, eval) and more robust help injection.
- Removed deprecated CLI commands and cleaned up legacy imports.
- Added option to specify which devices to use, with auto-selection of GPUs based on available memory.
- Updated sample configs and sleap-io skeleton function usage.
- Minor parameter name and default updates for consistency with SLEAP.
Documentation & Installation:
- Fixed broken documentation pages and improved menu structure.
- Updated installation instructions with CUDA support for uv-based workflows.

What's Changed

Fix Bug for SwinT Backbone Model by @7174Andy in #304
More robust help injection in CLI by @tom21100227 in #303
Remove loading pretrained weights during inference pipeline by @gitttt-1234 in #305
Remove back import in lightning module by @gitttt-1234 in #312
Fix compute filters in unet by @gitttt-1234 in #313
Update CLI commands by @gitttt-1234 in #314
Update sleap-io skeleton functions usage by @gitttt-1234 in #315
Minor updates to config parameters by @gitttt-1234 in #316
Minor bug fixes by @gitttt-1234 in #317
Fix Inference on SuggestionFrames by @7174Andy in #318
Add pck to voc metrics by @gitttt-1234 in #320
Fix bugs in training pipeline by @gitttt-1234 in #322
Add option to specify which devices to use by @gitttt-1234 in #327
Fix bug in infinite data loader by @gitttt-1234 in #325
Add thread-safe video access by @gitttt-1234 in #326
Fix bugs in docs by @gitttt-1234 in #319
Change zmq address to port arguments by @gitttt-1234 in #328
Add option to run tracking on select frames by @gitttt-1234 in #329
Fix seeding in training workflow by @gitttt-1234 in #330
Fix inference stalling by @gitttt-1234 in #331
Make wandb artifact logging optional by @gitttt-1234 in #332
Auto-select GPUs by @gitttt-1234 in #333
Add function to load metrics by @gitttt-1234 in #334
Fix experiment run name in multi-gpu training by @gitttt-1234 in #336
Add option to pass labels and video objects by @gitttt-1234 in #337
Fix mapping for legacy topdown id models by @gitttt-1234 in #339
Modify uv installation docs for cuda support by @gitttt-1234 in #340
Update sample configs by @gitttt-1234 in #338
Bump up sleap-nn version for v0.0.2 by @gitttt-1234 in #341

Full Changelog: v0.0.1...v0.0.2

v0.0.1 ¶

SLEAP-NN v0.0.1 - Initial Release

SLEAP-NN is a PyTorch-based deep learning framework for pose estimation, built on top of the SLEAP (Social LEAP Estimates Animal Poses) platform. This framework provides efficient training, inference, and evaluation tools for multi-animal pose estimation tasks.

Documentation: https://nn.sleap.ai/

Quick start

# Install with PyTorch CPU support
pip install sleap-nn[torch-cpu]

# Train a model
sleap-nn train --config-name config.yaml --config-dir configs/

# Run inference
sleap-nn track --model_paths model.ckpt --data_path video.mp4

# Evaluate predictions
sleap-nn eval --ground_truth_path gt.slp --predicted_path pred.slp

What's Changed

Core Data Loader Implementation by @davidasamy in #4
Add centroid finder block by @davidasamy in #7
Add DataBlocks for rotation and scaling by @gitttt-1234 in #8
Refactor datapipes by @talmo in #9
Instance Cropping by @davidasamy in #13
Add more Kornia augmentations by @alckasoc in #12
Confidence Map Generation by @davidasamy in #11
Peak finding by @alckasoc in #14
UNet Implementation by @alckasoc in #15
Top-down Centered-instance Pipeline by @alckasoc in #16
Adding ruff to ci.yml by @alckasoc in #21
Implement base Model and Head classes by @alckasoc in #17
Add option to Filter to user instances by @gitttt-1234 in #20
Add Evaluation Module by @gitttt-1234 in #22
Add metadata to dictionary by @gitttt-1234 in #24
Added SingleInstanceConfmapsPipeline by @alckasoc in #23
modify keys by @gitttt-1234 in #31
Small fix to find_global_peaks_rough by @alckasoc in #28
Add trainer by @gitttt-1234 in #29
PAF Grouping by @alckasoc in #33
Add predictor class by @gitttt-1234 in #36
Edge Maps by @alckasoc in #38
Add ConvNext Backbone by @gitttt-1234 in #40
Add VideoReader by @gitttt-1234 in #45
Refactor model pipeline by @gitttt-1234 in #51
Add BottomUp model pipeline by @gitttt-1234 in #52
Remove Part-names and Edge dependency in config by @gitttt-1234 in #54
Refactor model config by @gitttt-1234 in #61
Refactor Augmentation config by @gitttt-1234 in #67
Add minimal pretrained checkpoints for tests and fix PAF grouping interpolation by @gqcpm in #73
Fix augmentation in TopdownConfmaps pipeline by @gitttt-1234 in #86
Implement tracker module by @gitttt-1234 in #87
Resume training and automatically compute crop size for TopDownConfmaps pipeline by @gitttt-1234 in #88
LitData Refactor PR1: Get individual functions for data pipelines by @gitttt-1234 in #90
Add function to load trained weights for backbone model by @gitttt-1234 in #95
Remove IterDataPipe from Inference pipeline by @gitttt-1234 in #96
Move ld.optimize to a subprocess by @gitttt-1234 in #100
Auto compute max height and width from labels by @gitttt-1234 in #101
Fix sizematcher in Inference data pipline by @gitttt-1234 in #102
Convert Tensor images to PIL by @gitttt-1234 in #105
Add threshold mode in config for learning rate scheduler by @gitttt-1234 in #106
Add option to specify .bin file directory in config by @gitttt-1234 in #107
Add StepLR scheduler by @gitttt-1234 in #109
Add config to WandB by @gitttt-1234 in #113
Add option to load trained weights for Head layers by @gitttt-1234 in #114
Add option to load ckpts for backbone and head for running inference by @gitttt-1234 in #115
Add option to reuse .bin files by @gitttt-1234 in #116
Fix Normalization order in data pipelines by @gitttt-1234 in #118
Add torch Dataset classes by @gitttt-1234 in #120
Fix Pafs shape by @gitttt-1234 in #121
Add caching to Torch Datasets pipeline by @gitttt-1234 in #123
Remove random_crop augmentation by @gitttt-1234 in #124
Generate np chunks for caching by @gitttt-1234 in #125
Add group to wandb config by @gitttt-1234 in #126
Fix crop size by @gitttt-1234 in #127
Resize images before cropping in Centered-instance model by @gitttt-1234 in #129
Check memory before caching by @gitttt-1234 in #130
Replace eval with an explicit mapping dictionary by @gitttt-1234 in #131
Add CyclerDataLoader to ensure minimum steps per epoch by @gitttt-1234 in #132
Fix running inference on Bottom-up models with CUDA by @gitttt-1234 in #133
Fix caching in datasets by @gitttt-1234 in #134
Save .slp file after inference by @gitttt-1234 in #135
Add option to reuse np chunks by @gitttt-1234 in #136
Filter instances while generating indices by @gitttt-1234 in #138
Fix config format while logging to wandb by @gitttt-1234 in #144
Add multi-gpu support by @gitttt-1234 in #145
Implement Omegaconfig PR1: basic functionality by @gqcpm in #97
Move all params to config by @gitttt-1234 in #146
Add output stride to backbone config by @gitttt-1234 in #147
Change backbone config structure by @gitttt-1234 in #149
Add an entry point train function by @gitttt-1234 in #150
Add logger by @gqcpm in #148
Fix preprocessing during inference by @gitttt-1234 in #156
Add CLI for training by @gitttt-1234 in #155
Specify custom anchor index in Inference pipeline by @gitttt-1234 in #157
Fix lr scheduler config by @gitttt-1234 in #158
Add max stride to Convnext and Swint backbones by @gitttt-1234 in #159
Fix length in custom datasets by @gitttt-1234 in #160
Add scale argument to custom datasets by @gitttt-1234 in #166
Fix size matcher by @gitttt-1234 in #167
Fix max instances in TopDown Inference by @gitttt-1234 in #168
Move lightning modules by @gitttt-1234 in #169
Save config with chunks by @gitttt-1234 in #174
Add profiler and strategy parameters by @gitttt-1234 in #175
Add docker img for remote dev by @gitttt-1234 in #176
Save files only in rank: 0 by @gitttt-1234 in #177
Minor changes to validate configs by @gitttt-1234 in #179
Fix multi-gpu training by @gitttt-1234 in #184
Cache only images by @gitttt-1234 in #186
Add a new data pipeline strategy without caching by @gitttt-1234 in #187
Minor fixes to lightning modules by @gitttt-1234 in #189
Fix caching when imgs path already exist by @gitttt-1234 in #191
Ensure caching of images to disk in rank:0 by @gitttt-1234 in #193
Fix bug in caching images to disk by @gitttt-1234 in #194
Close videos before creating data loaders by @gitttt-1234 in #195
Update instance creation for sleap-io v0.3.0 compatibility by @gitttt-1234 in #196
Fix up block computation for swint and convnext by @gitttt-1234 in #197
Bump up to python 3.11 by @gitttt-1234 in #200
Map legacy SLEAP json configs to SLEAP-NN OmegaConf objects by @gqcpm in #162
Add option to get validation data from train labels by @gitttt-1234 in #201
Fix anchor part in config by @gitttt-1234 in #203
Minor fixes to config mapper by @gitttt-1234 in #204
Save labels with centroid inference by @gitttt-1234 in #205
Add custom callbacks to publish metrics during training by @gitttt-1234 in #207
Add visualizer by @gitttt-1234 in #208
Add CLI for inference by @gitttt-1234 in #209
Add option to parse frame ranges for videos by @gitttt-1234 in #211
Add control flags to run inference on select LabeledFrames by @gitttt-1234 in #212
Add support to run inference on specific video in a .slp file by @gitttt-1234 in #213
Minor fixes to tracking by @gitttt-1234 in #214
Fix bug in evaluation by @gitttt-1234 in #215
Save train, val, test predictions after training by @gitttt-1234 in #216
Add option to auto-select device for inference by @gitttt-1234 in #217
Add option to pass multiple .slp files for training by @papamanu in #218
Add logs by @gitttt-1234 in #219
Bug fixes to model architecture and trainer by @gitttt-1234 in #220
Remove nested tensors to support mps for BottomUp models by @gitttt-1234 in #221
Modify viz functions by @gitttt-1234 in #223
Add ensure_grayscale parameter by @gitttt-1234 in #224
Fix bugs with zmq config by @gitttt-1234 in #225
Log part-wise losses by @gitttt-1234 in #226
Fix trainer config mappings and add option to load config from json str by @gitttt-1234 in #227
Refactor ModelTrainer class by @gitttt-1234 in #228
Fix infinite data loader and update steps per epoch by @gitttt-1234 in #229
Add online hard keypoint mining by @gitttt-1234 in #222
Add more features to Tracker by @gitttt-1234 in #231
Remove litdata and iterdatapipe pipelines by @gitttt-1234 in #232
Add CLAUDE.md and update .gitignore for Claude Code integration by @talmo in #233
Add length parameter to InfiniteDataLoader by @gitttt-1234 in #237
Get sleap-nn pip package ready to publish by @eberrigan in #236
Map sleap (json) skeleton to sleap-nn (yaml) format by @gitttt-1234 in #238
Enable tracking on user-labeled instances by @gitttt-1234 in #239
Add ID models by @gitttt-1234 in #234
Add codespell workflow for spell checking by @talmo in #241
Make torch dependencies optional by @eberrigan in #243
Reorganize assets and revise checkpoints by @gitttt-1234 in #242
Refactor architectures per SLEAP by @gitttt-1234 in #245
Add keep-viz parameter by @gitttt-1234 in #246
Format config.md by @gitttt-1234 in #249
Fix minor bugs by @gitttt-1234 in #250
Setup docs by @talmo in #251
Remove broken Docker image by @gitttt-1234 in #254
Import legacy SLEAP model weights by @talmo in #235
Fix wandb logging by @gitttt-1234 in #255
Fix preprocess config in inference by @gitttt-1234 in #257
Update ckpts and cfgs by @gitttt-1234 in #259
Minor bug fixes in training pipeline by @gitttt-1234 in #260
Revert "Minor bug fixes in training pipeline" by @gitttt-1234 in #261
Fix minor bugs by @gitttt-1234 in #262
Update in channels for torch with keras weights by @gitttt-1234 in #263
Move convnext/ swint pretrained weights by @gitttt-1234 in #264
Refactor lightning module parameters by @gitttt-1234 in #265
Fix skeletons structure in config by @gitttt-1234 in #266
Ensure consistent types for augmentation parameters by @gitttt-1234 in #267
Add CLI entry-point functions and shortcuts by @gitttt-1234 in #270
Ensure only rank-0 handles writing files in ddp training by @gitttt-1234 in #271
Fix bug in centered-instance dataset by @gitttt-1234 in #272
Add eff_scale to dataset by @gitttt-1234 in #273
Update docs by @gitttt-1234 in #256
Add self-hosted runner to CI by @talmo in #277
Minor changes to data pipeline and training guide notebook by @gitttt-1234 in #280
Add support to load keras weights for model init by @gitttt-1234 in #285
Self-hosted Runners Tests and Trainer Accelerator by @alicup29 in #281
Check memory with source images by @eberrigan in #283
Fix @oneof Validation and Add Support for None for train_labels_path by @7174Andy in #282
Parallelizing dataset caching by @emdavis02 in #284
Remove Permanent File Creations After Testing Locally by @7174Andy in #292
Add file existence tracking to trainer and related tests by @tom21100227 in #291
Fix transitive torch/torchvision installation & platform compatilibility by @alicup29 in #268
Add more documentation by @gitttt-1234 in #287
Update build CI by @talmo in #295
Add build ci option to release to testpypi by @gitttt-1234 in #296
Add testpypi index to toml by @gitttt-1234 in #297
Minor fixes to pyproject.toml by @gitttt-1234 in #298
Add Hyphen to Checkpoint Paths when Duplication Found by @7174Andy in #299
Add helpful CLI message to sleap-nn-train by @tom21100227 in #294

New Contributors

@davidasamy made their first contribution in #4
@gqcpm made their first contribution in #73
@papamanu made their first contribution in #218
@eberrigan made their first contribution in #236
@alicup29 made their first contribution in #281
@7174Andy made their first contribution in #282
@emdavis02 made their first contribution in #284
@tom21100227 made their first contribution in #291

Full Changelog: https://github.com/talmolab/sleap-nn/commits/v0.0.1

Changelog¶

sleap-nn v0.3.3 Release Notes

Summary

Installation

Fixes

Dependencies & Build

Changelog

sleap-nn v0.3.2 Release Notes

Summary

Installation

Breaking Changes

sleap-nn predict now retains empty-detection frames by default (#717)

predict now backreferences a .pkg.slp input to itself, not its source video (#724)

New Features

Evaluation

CLI Updates

Fixes

Dependencies & Build

Upgrade Notes

Changelog

sleap-nn v0.3.1 Release Notes

Summary

Installation

Breaking Changes

⚠️ Whole-frame segmentation masks were misaligned with the image grid (#693)

⚠️ Mask evaluation now excludes predicted-instance-linked GT masks by default (#694)

⚠️ Frame-caching failures now raise instead of warn (#701)

⚠️ Centroid target source is now resolved once per dataset, not per frame (#704)

New Features

Tiling, segmentation & pretrained backbones

Centroid training

Training diagnostics & robustness

CLI Updates

Fixes

Dependencies & Build

Upgrade Notes

Changelog

Summary

Installation

Breaking Changes

⚠️ sleap-nn predict now means pose inference

⚠️ sleap-io pinned to >=0.8.0,<0.9.0

⚠️ seed now defaults to 42

⚠️ Default prediction .slp is non-embedded

New Features

Unified sleap-nn predict + Predictor API

Top-level Python API for discoverability

Centroid-only models

Instance segmentation — experimental

Tracking

Inference UX & I/O

Training

Fixes

Dependencies & Build

Documentation

Known Limitations (planned for 0.3.1)

Upgrade Notes

Changelog

SLEAP-NN v0.2.0 Release Notes

Summary

Installation

CLI Updates

TUI / web-app config-picker parity (#524)

Default to memory caching + 2 workers, simplified --pipeline (#526)

Bug Fixes

Architecture & Export

Training & Data

Web Config Picker (docs/configuration/config-picker/app.html)

Dependency Updates

Changelog

CLI Updates

Bug Fixes

Other

Chores

SLEAP-NN v0.1.3 Release Notes

Summary

Installation

New Features

Negative Frame Training (#484)

sleap-nn info CLI Subcommand (#498)

`sleap-nn predict` now retains empty-detection frames by default (#717)

`predict` now backreferences a `.pkg.slp` input to itself, not its source video (#724)

⚠️ `sleap-nn predict` now means pose inference

⚠️ sleap-io pinned to `>=0.8.0,<0.9.0`

⚠️ `seed` now defaults to `42`

⚠️ Default prediction `.slp` is non-embedded

Unified `sleap-nn predict` + `Predictor` API

Default to memory caching + 2 workers, simplified `--pipeline` (#526)

Web Config Picker (`docs/configuration/config-picker/app.html`)

`sleap-nn info` CLI Subcommand (#498)

3. `load_metrics()` API Changes