Skip to content
This repository was archived by the owner on Aug 22, 2024. It is now read-only.
This repository was archived by the owner on Aug 22, 2024. It is now read-only.

Body tracking is too slow and inaccurate #514

Closed
@rfilkov

Description

@rfilkov

Describe the bug

By all means Azure Kinect is the best Kinect so far, and will be probably the best depth sensor on the market. The sensor SDK is pretty stable and good, providing almost everything an average user would want. But the body tracking subsystem is ruining this positive user experience. In means of API this SDK is great too, but the DNN model performance is much worse than the body tracking of Kinect-v2. The joint positions are inaccurate by fast movements. The body index map is not very accurate, as well. It does not fully match the user's silhouette on the depth frame. On my GTX 1060 it takes 2-3 depth frame cycles to process a body frame. Hence, it works at about 10 fps.

To Reproduce

  1. Run Azure Kinect Body Tracking Viewer.
  2. Stand in front of the sensor.
  3. Make fast arm movements.
  4. Look at the arm joint positions with regard to the real arms.
  5. Look at the colorized body-index map with regard to the real body.

Expected behavior

  1. I expect the body tracking to work at least at 30 fps or more, i.e. faster than the depth frames arrive.
  2. I expect the body joint positions to match as precisely as possible the user's joints on the depth frame.
  3. I expect the body index map to match as precisely as possible the user's silhouette on the depth frame.
  4. I expect the body tracking to be less demanding, in means of hardware and 3rd party software requirements. GTX 1070 + CUDA + cuDNN + manually setting paths would be too much for the average user.

Please consider at least providing some option to the users, who don't have high end graphics cards and would like to get Kinect body tracking out of the box, without (or with minimum) extra installations. As far as I remember, Kinect-v2 used random forest model(s) for its body tracking. The performance was great and no extra installations were needed, back then in 2013/14.

Logs

Screenshots

Desktop (please complete the following information):

  • OS with Version: Windows 1809
  • Kinect SDK Version: 1.1.0
  • Body tracking SDK Version: 0.9.0

Additional context

I believe most Kinect users would expect better, more accurate and more performant body tracking experience, not worse. And now, with Apple adding people segmentation and body tracking to their AR-Kit 3.0 I would expect Kinect (with all these years of experience) to provide a better user experience in all aspects than anybody else.

Metadata

Metadata

Assignees

Labels

Body TrackingIssue related to the Body Tracking SDKEnhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions