Skip to content

Conversation

@lucasreljic
Copy link
Contributor

Adding deep_ort_gpu_backend_plugin to allow for inference with onnxruntime with GPU-based execution providers. Uses the onnxruntime_gpu_vendor package.

  • Tested with object_detection branch on CUDA v12.2
  • Currently, only tested with CUDA as the execution provider; through further debugging, TensorRT might be possible
  • Passes in inputs under CPU memory just like deep_ort_backend_plugin, since onnxruntime converts to GPU memory anyway. There are methods for passing in GPU memory, which could be explored in the future when ROS Nitros is implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants