Skip to content

Data Config

Configure data loading, preprocessing, and augmentation.

Data Paths

data_config:
  train_labels_path:
    - train.slp
    - more_train.slp      # Multiple files supported
  val_labels_path:
    - val.slp             # Optional
  validation_fraction: 0.1  # Used if val_labels_path not set
  test_file_path: test.slp  # For post-training evaluation
Option Description Default
train_labels_path Training data files Required
val_labels_path Validation data files null
validation_fraction Fraction for auto-split 0.1
test_file_path Test data for evaluation null
user_instances_only Only use user-labeled instances true
use_negative_frames Include user-confirmed negative frames (see guide) false
negative_loss_weight Loss weight multiplier for negative samples 1.0

Data Pipeline

data_config:
  data_pipeline_fw: torch_dataset  # or cache options
  cache_img_path: /path/to/cache   # For disk caching
Option Description When to Use
torch_dataset Load on demand Default, works everywhere
torch_dataset_cache_img_memory Cache in RAM Faster, needs enough RAM
torch_dataset_cache_img_disk Cache to disk Large datasets, multi-GPU

With caching, you can use num_workers > 0

Preprocessing

data_config:
  preprocessing:
    ensure_rgb: false          # Force 3 channels
    ensure_grayscale: false    # Force 1 channel
    scale: 1.0                 # Resize factor (0.5 = half size)
    max_height: null           # Pad to this height
    max_width: null            # Pad to this width
    crop_size: null            # For centered-instance models
    min_crop_size: 100         # Minimum auto crop size
    crop_padding: null         # Extra padding for crops

Common Patterns

preprocessing:
  scale: 0.5  # Half resolution, 4x faster

preprocessing:
  max_height: 512
  max_width: 512

preprocessing:
  crop_size: 256  # Fixed crop size
  # or
  crop_size: null  # Auto-compute from data
  min_crop_size: 100
  crop_padding: 20

Augmentation

data_config:
  use_augmentations_train: true
  augmentation_config:
    intensity:
      # ... intensity settings
    geometric:
      # ... geometric settings

Intensity Augmentation

augmentation_config:
  intensity:
    # Noise
    uniform_noise_p: 0.0
    gaussian_noise_p: 0.0
    gaussian_noise_mean: 0.0
    gaussian_noise_std: 0.02

    # Contrast
    contrast_min: 0.9
    contrast_max: 1.1
    contrast_p: 0.0

    # Brightness
    brightness_min: 0.9
    brightness_max: 1.1
    brightness_p: 0.0

Geometric Augmentation

augmentation_config:
  geometric:
    # Rotation (degrees)
    rotation_min: -15.0
    rotation_max: 15.0

    # Scale
    scale_min: 0.9
    scale_max: 1.1

    # Translation (fraction of image size)
    translate_width: 0.0
    translate_height: 0.0

    # Combined probability (used when individual *_p values are null)
    affine_p: 0.0

    # Independent probabilities (override affine_p when set)
    rotation_p: 1.0   # Always apply rotation by default
    scale_p: 1.0      # Always apply scaling by default
    translate_p: null # Uses affine_p if null

    # Random erase
    erase_p: 0.0
    erase_scale_min: 0.0001
    erase_scale_max: 0.01

    # Mixup
    mixup_p: 0.0
    mixup_lambda_min: 0.01
    mixup_lambda_max: 0.05

    # Flip (mirror image + keypoints left/right, swapping symmetric node pairs)
    flip_p: 0.0  # 0 disables; >0 enables left/right flip (uses skeleton symmetries)

Flip & skeleton symmetries: when flip_p > 0, the image and keypoints are mirrored left/right and left/right symmetric body parts (e.g. left_paw ↔ right_paw) are swapped so labels stay correct. Symmetry pairs are read from the skeleton. If the skeleton has no symmetries, flipping is only correct when the animal is truly left/right symmetric in labeling — a warning is logged otherwise.

Examples

No Augmentation

data_config:
  use_augmentations_train: false

Geometric Only (using defaults)

data_config:
  use_augmentations_train: true
  # Default augmentation_config applies rotation ±15° and scale 0.9-1.1 with p=1.0

Geometric Only (explicit)

data_config:
  use_augmentations_train: true
  augmentation_config:
    intensity: null
    geometric:
      rotation_min: -30.0
      rotation_max: 30.0
      rotation_p: 0.5        # 50% chance of rotation
      scale_min: 0.8
      scale_max: 1.2
      scale_p: 0.5           # 50% chance of scaling

Full Augmentation

data_config:
  use_augmentations_train: true
  augmentation_config:
    intensity:
      contrast_p: 0.3
      brightness_p: 0.3
      gaussian_noise_p: 0.2
      gaussian_noise_std: 0.1
    geometric:
      rotation_min: -30.0
      rotation_max: 30.0
      scale_min: 0.8
      scale_max: 1.2
      affine_p: 0.7

Full Reference

DataConfig

Option Type Default Description
train_labels_path list null List of paths to training .slp files
val_labels_path list null List of paths to validation .slp files
validation_fraction float 0.1 Fraction of training data for auto-split validation
use_same_data_for_val bool false Use same data for both training and validation (useful for intentional overfitting)
test_file_path str/list null Path(s) to test data for post-training evaluation
provider str LabelsReader Data provider class (only LabelsReader supported)
user_instances_only bool true Only use user-labeled instances (not predicted)
data_pipeline_fw str torch_dataset Data loading framework: torch_dataset, torch_dataset_cache_img_memory, torch_dataset_cache_img_disk
cache_img_path str null Path for disk caching (used with torch_dataset_cache_img_disk)
use_existing_imgs bool false Use existing cached images instead of regenerating
delete_cache_imgs_after_training bool true Delete cached images after training completes
parallel_caching bool true Use parallel processing for caching (faster for large datasets)
cache_workers int 0 Number of workers for parallel caching (0 = auto: min(4, cpu_count))
use_augmentations_train bool true Apply augmentations during training
bbox_source str auto Ground-truth boxes for bbox_* models: auto (the file's user boxes if it has any, else the tight box of each user instance's visible points, else of each mask), bboxes, instances or masks. Ignored by other model types
bbox_padding float 0.0 Pixels added on every side of boxes derived from instances or masks

PreprocessingConfig

Option Type Default Description
ensure_rgb bool false Convert images to 3 channels (RGB). Single-channel images are replicated
ensure_grayscale bool false Convert images to 1 channel (grayscale)
max_height int null Pad images to this height
max_width int null Pad images to this width
scale float 1.0 Resize factor (e.g., 0.5 = half size)
crop_size int null Crop size for centered-instance models. If null, auto-computed from data
min_crop_size int 100 Minimum crop size when auto-computing
crop_padding int null Extra padding around crops. If null, auto-computed from augmentation settings

IntensityConfig (augmentation_config.intensity)

Option Type Default Description
uniform_noise_min float 0.0 Minimum uniform noise value (0-1 scale, multiplied by 255 internally)
uniform_noise_max float 0.04 Maximum uniform noise value (0-1 scale, multiplied by 255 internally)
uniform_noise_p float 0.0 Probability of applying uniform noise
gaussian_noise_mean float 0.0 Mean of Gaussian noise distribution (0-1 scale, multiplied by 255 internally)
gaussian_noise_std float 0.02 Standard deviation of Gaussian noise (0-1 scale, multiplied by 255 internally)
gaussian_noise_p float 0.0 Probability of applying Gaussian noise
contrast_min float 0.9 Minimum contrast factor
contrast_max float 1.1 Maximum contrast factor
contrast_p float 0.0 Probability of applying contrast adjustment
brightness_min float 0.9 Minimum brightness factor
brightness_max float 1.1 Maximum brightness factor (max 2.0)
brightness_p float 0.0 Probability of applying brightness adjustment

GeometricConfig (augmentation_config.geometric)

Option Type Default Description
rotation_min float -15.0 Minimum rotation angle in degrees
rotation_max float 15.0 Maximum rotation angle in degrees
rotation_p float 1.0 Probability of rotation (independent). If null, uses affine_p
scale_min float 0.9 Minimum scale factor
scale_max float 1.1 Maximum scale factor
scale_p float 1.0 Probability of scaling (independent). If null, uses affine_p
translate_width float 0.0 Maximum horizontal translation as fraction of width
translate_height float 0.0 Maximum vertical translation as fraction of height
translate_p float null Probability of translation (independent). If null, uses affine_p
affine_p float 0.0 Probability of bundled affine transform (rotation + scale + translate)
erase_scale_min float 0.0001 Minimum erased area as proportion of image
erase_scale_max float 0.01 Maximum erased area as proportion of image
erase_ratio_min float 1.0 Minimum aspect ratio of erased area
erase_ratio_max float 1.0 Maximum aspect ratio of erased area
erase_p float 0.0 Probability of random erasing
mixup_lambda_min float 0.01 Minimum mixup strength
mixup_lambda_max float 0.05 Maximum mixup strength
mixup_p float 0.0 Probability of applying mixup
flip_p float 0.0 Probability of mirroring image + keypoints left/right (swaps symmetric node pairs)