Skip to content

Conversation

maximpavliv
Copy link
Collaborator

@maximpavliv maximpavliv commented Jul 25, 2025

Summary

This Pull Request introduces a new FilteredDetector class, which wraps a generic object detector (typically trained on COCO) and filters its outputs to retain only detections for a specified class_id.

It also adds a utility function, get_filtered_coco_detector_inference_runner(), which returns an inference runner using a FilteredDetector wrapping one of the following torchvision models:

  • fasterrcnn_resnet50_fpn
  • fasterrcnn_resnet50_fpn_v2
  • fasterrcnn_mobilenet_v3_large_fpn

Main changes

  • Replaces the existing superanimal_humanbody inference pipeline with a FilteredDetector-based DetectorInferenceRunner.

  • Solves multiple issues in the current superanimal_humanbody integration:

    • Fixes temporal shift in predictions
      Screenshot from 2025-07-22 12-11-52

    • Prevents CTD inference pipeline crashes (which solves this comment)

    • Resolves detector training bug where loss remained at 0.0

    • Removes unnecessary debug prints

Structural improvements

  • Integrates superanimal_humanbody more cleanly into the inference flow: it is now handled via deeplabcut.pose_estimation_pytorch.modelzoo.inference._video_inference_superanimal(), like other PyTorch-based superanimal models.

  • Exposes FilteredDetector as a reusable utility for users who want to focus on specific class_ids.

  • Improves separation of concerns:

    • Preprocessor handles input data preparation
    • Postprocessor handles filtering and final outputs

Related PR

This PR is paired with DLClibrary PR #42, which extends dlclibrary.download_huggingface_model() to rename the humanbody pose estimation snapshot upon download.

@maximpavliv maximpavliv requested a review from AlexEMG July 31, 2025 12:34
@MMathisLab MMathisLab self-requested a review August 24, 2025 20:30
@AlexEMG AlexEMG merged commit 29479fa into main Sep 4, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants