Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

quic-apreetam · 2025-04-29T20:19:22Z

Detailed Description

This PR introduces FastCV DSP Extension APIs within the 'cv::fastcv::dsp' namespace.
The following APIs have been added:

fcvdspinit: Initializes the FastCV DSP environment.
fcvdspdeinit: Deinitializes the FastCV DSP environment.
sumOfAbsoluteDiffs: Computes the sum of absolute differences of an image against an 8x8 template.
thresholdOtsu: Binarizes a grayscale image using Otsu's method.
FFT: Computes the 1D or 2D Fast Fourier Transform of a real-valued matrix.
IFFT: Computes the 1D or 2D Inverse Fast Fourier Transform of a complex-valued matrix.
canny: Applies the Canny edge detector to an 8-bit grayscale image.
filter2D: Applies a generic 2D filter to an image.

The QcAllocator has been added to manage memory allocations on Qualcomm's Chipsets. This allocator ensures that matrices are allocated using the Qualcomm hardware memory allocator, providing efficient DSP operations.

Requires updated binary from: opencv/opencv_3rdparty#97
Requires binary from opencv/opencv_3rdparty#95

Lib Hash Update: opencv/opencv#27403

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

modules/fastcv/include/opencv2/fastcv/edges_dsp.hpp

modules/fastcv/src/allocator.cpp

modules/fastcv/src/blur_dsp.cpp

modules/fastcv/src/precomp.hpp

modules/fastcv/src/sad_dsp.cpp

asmorkalov · 2025-04-30T09:30:37Z

Java bindings generator does not handle namespaces well:

[255/3358] Generating opencv-4120.jar
FAILED: CMakeFiles/dephelper/opencv_java_jar bin/opencv-4120.jar 
cd /home/ci/build/modules/java/jar/opencv && /bin/ant -noinput -k jar && /usr/bin/cmake -E touch /home/ci/build/CMakeFiles/dephelper/opencv_java_jar
Buildfile: /home/ci/build/modules/java/jar/opencv/build.xml

jar:
    [javac] Compiling 299 source files to /home/ci/build/modules/java/jar/opencv/build/classes
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:715: error: method filter2D(Mat,Mat,int,Mat) is already defined in class Fastcv
    [javac]     public static void filter2D(Mat _src, Mat _dst, int ddepth, Mat _kernel) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:798: error: method FFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void FFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:818: error: method IFFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void IFFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 3 errors
Target 'jar' failed with message 'Compile failed; see the compiler error output for details.'.

BUILD FAILED

If I understand correctly, DSP part is not usable from Java code (no way to manage allocators in Java). I propose to replace CV_EXPORTS_W with CV_EXPORTS.

quic-apreetam · 2025-05-02T09:35:57Z

Java bindings generator does not handle namespaces well:

[255/3358] Generating opencv-4120.jar
FAILED: CMakeFiles/dephelper/opencv_java_jar bin/opencv-4120.jar 
cd /home/ci/build/modules/java/jar/opencv && /bin/ant -noinput -k jar && /usr/bin/cmake -E touch /home/ci/build/CMakeFiles/dephelper/opencv_java_jar
Buildfile: /home/ci/build/modules/java/jar/opencv/build.xml

jar:
    [javac] Compiling 299 source files to /home/ci/build/modules/java/jar/opencv/build/classes
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:715: error: method filter2D(Mat,Mat,int,Mat) is already defined in class Fastcv
    [javac]     public static void filter2D(Mat _src, Mat _dst, int ddepth, Mat _kernel) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:798: error: method FFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void FFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:818: error: method IFFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void IFFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 3 errors
Target 'jar' failed with message 'Compile failed; see the compiler error output for details.'.

BUILD FAILED

If I understand correctly, DSP part is not usable from Java code (no way to manage allocators in Java). I propose to replace CV_EXPORTS_W with CV_EXPORTS.

sure @asmorkalov , will change CV_EXPORTS_W with CV_EXPORTS

quic-apreetam · 2025-05-05T08:07:12Z

hi @asmorkalov changes updated. Please review

modules/fastcv/src/blur_dsp.cpp

modules/fastcv/test/test_blur_dsp.cpp

quic-apreetam

.

quic-apreetam

Corrected Typo and removed redundant break @asmorkalov thanks

qc-apreetam

.

qc-apreetam · 2025-05-06T07:34:07Z

hi @asmorkalov , we are getting undefined reference to `dlerror'/'dlsym'/'dlopen'/'dlclose' errors https://github.com/opencv/opencv_contrib/actions/runs/14832679408/job/41640009265

looks like the library libdl is missing on the build machine

asmorkalov · 2025-05-06T12:16:50Z

It's system linker and cannot be missing. It should be mentioned in linkage list in CMake.

quic-apreetam · 2025-05-06T18:47:58Z

It's system linker and cannot be missing. It should be mentioned in linkage list in CMake.
hi @asmorkalov
updating line:8 in https://github.com/opencv/opencv_contrib/blob/4.x/modules/fastcv/CMakeLists.txt
to
ocv_target_link_libraries(${the_module} ${FASTCV_LIBRARY} dl) should be sufficient?

asmorkalov · 2025-05-07T05:35:58Z

I recommend to add the dependency, when FastCV is added to cmake here: https://github.com/opencv/opencv/blob/4.x/cmake/OpenCVFindLibsPerf.cmake#L200. HAL links the same library and we may get similar linkage issues in other places.

See similar discussion on Stackoverflow with ready solution:
https://stackoverflow.com/questions/36922635/cmake-add-dependency-to-imported-library

quic-apreetam · 2025-05-07T07:17:52Z

I recommend to add the dependency, when FastCV is added to cmake here: https://github.com/opencv/opencv/blob/4.x/cmake/OpenCVFindLibsPerf.cmake#L200. HAL links the same library and we may get similar linkage issues in other places.

See similar discussion on Stackoverflow with ready solution: https://stackoverflow.com/questions/36922635/cmake-add-dependency-to-imported-library

hi @asmorkalov , thanks for the suggestion.

updated "cmake/OpenCVFindLibsPerf.cmake" to link libdl
this change is a part of : opencv/opencv#27269

quic-apreetam · 2025-05-07T12:01:35Z

hi @asmorkalov , we still see undefined reference to `dlerror'/'dlsym'/'dlopen'/'dlclose' errors in https://github.com/opencv/opencv_contrib/actions/runs/14832679408/job/41789805574?pr=3931

I think the opencv changes are not merged before Build and Test
Run OPENCV_FORK=$(git ls-remote --heads "https://github.com/quic-apreetam/opencv" "apreetam_5thPost") || true
fatal: could not read Username for 'https://github.com/': No such device or address
No merge since quic-apreetam/opencv does not have branch apreetam_5thPost

Also, as we already updated https://github.com/opencv/opencv/blob/4.x/cmake/OpenCVFindLibsPerf.cmake#L200

we did not make any change in https://github.com/opencv/opencv_contrib/blob/4.x/modules/fastcv/CMakeLists.txt
should we still update this file?

asmorkalov · 2025-05-07T13:08:15Z

Hm.., it's not your repo, it's CodeLinaro :(

quic-apreetam · 2025-05-07T13:57:51Z

Hm.., it's not your repo, it's CodeLinaro :(

hi @asmorkalov please help to explain, not sure if I understand the problem, anything required to be done from my side?

asmorkalov · 2025-05-07T14:11:01Z

Nothing from your side. Our script looks for extra branches to merge in your own namespace, but not CodeLinaro. We will test things manually.

gursimarsingh · 2025-05-20T11:41:41Z

I have tested the PR manually on QCS6490, RB3 Gen2 Vision Kit board with x86-qcom-6.6.65-QLI.1.4-Ver.1.1_qim-product-sdk-1.1.2 image binary.

Source Branch:

git clone https://github.com/CodeLinaro/opencv.git
git checkout apreetam_5thPost
git clone https://github.com/CodeLinaro/opencv_contrib.git
git checkout apreetam_5thPost

Build Config:

cmake -G Ninja \
  -DCMAKE_SYSTEM_NAME=Linux \
  -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
  -DCMAKE_TOOLCHAIN_FILE=../opencv/platforms/linux/aarch64-gnu.toolchain.cmake \
  -DWITH_FASTCV=ON \
  -DPYTHON_DEFAULT_EXECUTABLE=/usr/bin/python3 \
  -DBUILD_SHARED_LIBS=OFF \
  -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++" \
  -DBUILD_EXAMPLES=OFF \
  -DOPENCV_ENABLE_NONFREE=ON \
  -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules \
  ../opencv

Host system: Ubuntu 24.04

Note: -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++" was required otherwise there is a mismatch in version of libstdc++

All fastcv performance and other tests pass. Logs are attached.

opencv_perf_fastcv_logs.txt
opencv_test_fastcv_logs.txt

cc: @asmorkalov

asmorkalov · 2025-05-21T11:54:21Z

modules/fastcv/test/test_blur_dsp.cpp

+    int initStatus = cv::fastcv::dsp::fcvdspinit();
+    ASSERT_EQ(initStatus, 0) << "Failed to initialize FastCV DSP";


It's too strong requirement. We get a lot of test failures on Android, even with Qualcomm hardware. See CI failures for example.
I propose softer solution:

if (initStatus != 0) throw SkipTestException("Cannot initialize DSP! Test skipped.");

In case if you need strong testing mode you can use cv::utils::getConfigurationParameterBool to get external option from environment variable and add it to the condition.

hi @asmorkalov, yeah we will see test failures on older chipsets. we can skip test in case dsp init fails

hi @asmorkalov, i am planning to add this check in all perf/accuracy tests. If it looks fine, will add. Please suggest
`

bool dspEnabled = cv::utils::getConfigurationParameterBool("FASTCV_DSP", false); if (!dspEnabled) { throw SkipTestException("FASTCV_DSP is not enabled. Skipping DSP test."); } // Initialize FastCV DSP int initStatus = cv::fastcv::dsp::fcvdspinit(); // If initialization fails, assert failure ASSERT_EQ(initStatus, 0) << "Failed to initialize FastCV DSP";

`

Ok, In case if you want to have the tests disabled by default, I propose to use test tags mechanism in our test system. I added Extra tag for you in opencv/opencv#27360 and applied it to the DSP tests in #3947. It makes the marked tests disabled by default. In case you want to run tests for DSP, you need to add --test_tag=dsp to the command line. It selects and runs tests by tag.

thanks @asmorkalov, I believe with test tags mechanism, we don't need the checks to skip dsp. We can run the tests only on supported chipset

@qc-apreetam I talked to colleagues and reworked the tags approach to make the tag local without test system modification. See #3947.

I see three scenarios:

Linux with DSP support. All test should pass, including DSP.

Linux without DSP support, e.g. CI system. We need to a way to disable DSP tests to have CI green.

Android with/without DSP. DSP is not available on non-rooted devices and I suppose, that the DSP tests should be disabled there.

I propose to add tags to all DSP tests (my PR) and add logic to test_main.cpp to have DSP enabled for Linux only. We can disable the DSP tag in CI by default to have CI green. What do you think?

asmorkalov · 2025-05-26T07:36:02Z

Documentation warning:

  Warning: /home/ci/opencv_contrib/modules/fastcv/include/opencv2/fastcv/blur_dsp.hpp:29: warning: unbalanced grouping commands

gursimarsingh · 2025-06-04T13:18:11Z

@asmorkalov The PR has been tested on QCS6490, RB3 Gen2 Vision Kit board with x86-qcom-6.6.65-QLI.1.4-Ver.1.1_qim-product-sdk-1.1.2 image binary with and without building shared libraries.

Source Branch:

git clone https://github.com/CodeLinaro/opencv.git
git checkout apreetam_6thPost
git clone https://github.com/CodeLinaro/opencv_contrib.git
git checkout apreetam_5thPost

With shared libs off

Build Config:

cmake -G Ninja \
  -DCMAKE_SYSTEM_NAME=Linux \
  -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
  -DCMAKE_TOOLCHAIN_FILE=../opencv/platforms/linux/aarch64-gnu.toolchain.cmake \
  -DWITH_FASTCV=ON \
  -DPYTHON_DEFAULT_EXECUTABLE=/usr/bin/python3 \
  -DBUILD_SHARED_LIBS=OFF \
  -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++" \
  -DBUILD_EXAMPLES=OFF \
  -DOPENCV_ENABLE_NONFREE=ON \
  -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules \
  ../opencv

With shared libs on
Build Config:

cmake -G Ninja \
  -DCMAKE_SYSTEM_NAME=Linux \
  -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
  -DCMAKE_TOOLCHAIN_FILE=../opencv/platforms/linux/aarch64-gnu.toolchain.cmake \
  -DWITH_FASTCV=ON \
  -DPYTHON_DEFAULT_EXECUTABLE=/usr/bin/python3 \
  -DBUILD_SHARED_LIBS=ON \
  -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++" \
  -DBUILD_EXAMPLES=OFF \
  -DOPENCV_ENABLE_NONFREE=ON \
  -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules \
  ../opencv

Host system: Ubuntu 22.04, GLibc version: 2.35
binding_logs.txt

All fastcv performance and other tests pass. Logs are attached.

asmorkalov · 2025-06-04T14:06:28Z

Thanks a lot! The last step is to fix CI and skip tests on unsupported platforms. I'm waiting for it.

…am_5thPost - includes test tag for FastCV DSP

quic-apreetam · 2025-06-04T16:54:10Z

Thanks a lot! The last step is to fix CI and skip tests on unsupported platforms. I'm waiting for it.

hi @asmorkalov ,
to run dsp tests using test tag,
I had to use --test_tag_force in the command:opencv_perf_fastcv --test_tag_force=fastcv_skip_dsp --gtest_filter=\*DSP\*

if i use --test_tag in the command: opencv_perf_fastcv --test_tag=fastcv_skip_dsp --gtest_filter=\*DSP\*
`
i see the following message:

[ RUN ] FastCV_Extension/FFT_DSPExtPerfTest.inverse/1, where GetParam() = 512x512

[ SKIP ] Test tags don't pass required tags list (--test_tag parameter)

[ OK ] FastCV_Extension/FFT_DSPExtPerfTest.inverse/1 (0 ms)

Is this how it should work?

and I’ve merged your changes from as/test_tag_dsp into my branch apreetam_5thPost.

asmorkalov · 2025-06-05T08:26:43Z

Is this how it should work?

Yes, it's expected behaviour.

quic-apreetam · 2025-06-06T09:41:03Z

thanks @asmorkalov and @gursimarsingh

Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs opencv#3931 Merge with opencv/opencv#27290 **Detailed Description** This PR introduces FastCV DSP Extension APIs within the '**cv::fastcv::dsp**' namespace. The following APIs have been added: 1. **fcvdspinit**: Initializes the FastCV DSP environment. 2. **fcvdspdeinit**: Deinitializes the FastCV DSP environment. 3. **sumOfAbsoluteDiffs**: Computes the sum of absolute differences of an image against an 8x8 template. 4. **thresholdOtsu**: Binarizes a grayscale image using Otsu's method. 5. **FFT**: Computes the 1D or 2D Fast Fourier Transform of a real-valued matrix. 6. **IFFT**: Computes the 1D or 2D Inverse Fast Fourier Transform of a complex-valued matrix. 7. **canny**: Applies the Canny edge detector to an 8-bit grayscale image. 8. **filter2D**: Applies a generic 2D filter to an image. The **QcAllocator** has been added to manage memory allocations on Qualcomm's Chipsets. This allocator ensures that matrices are allocated using the Qualcomm hardware memory allocator, providing efficient DSP operations. Requires updated binary from: opencv/opencv_3rdparty#97 Requires binary from opencv/opencv_3rdparty#95 Lib Hash Update: opencv/opencv#27403 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [ ] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

quic-apreetam added 6 commits April 30, 2025 00:16

Add QcAllocator and DSP Init and Deinit Functions

7c342d6

Added sumOfAbsoluteDiffs DSP API with Accuracy and Perf Test

2bed4bc

Add thresholdOtSu DSP API with Accuracy and Perf Test

2ea338a

Add FFT and IFFT DSP API with Accuracy and Perf Test

51f92b0

Add canny DSP API with Accuracy and Perf Test

ffa4c84

Add filter2D DSP API with Accuracy and Perf Test

350f7f9

asmorkalov reviewed Apr 30, 2025

View reviewed changes

asmorkalov added the category: fastcv label Apr 30, 2025

asmorkalov assigned gursimarsingh Apr 30, 2025

Remove parallel_for_ from filter2D dsp API, update mat allocation check

a0c5c8c

quic-apreetam requested review from asmorkalov and qc-apreetam May 5, 2025 08:07

asmorkalov reviewed May 5, 2025

View reviewed changes

modules/fastcv/src/blur_dsp.cpp Outdated Show resolved Hide resolved

modules/fastcv/test/test_blur_dsp.cpp Outdated Show resolved Hide resolved

Remove redundant break and correct typo

a0e609f

quic-apreetam commented May 5, 2025

View reviewed changes

quic-apreetam requested a review from asmorkalov May 5, 2025 09:05

qc-apreetam reviewed May 5, 2025

View reviewed changes

asmorkalov reviewed May 21, 2025

View reviewed changes

Added extra test tag for Qualcomm DSP (disabled by default)

b6a49a8

asmorkalov mentioned this pull request May 26, 2025

Added extra test tag for Qualcomm DSP (disabled by default) #27360 #3947

Closed

6 tasks

quic-apreetam and others added 2 commits May 27, 2025 12:05

Fix Documentation Warn

e5c7e07

Switch to locally defined skip tag in FastCV tests.

015033d

Merge remote-tracking branch 'asmorkalov/as/test_tag_dsp' into apreet…

9300c5f

…am_5thPost - includes test tag for FastCV DSP

asmorkalov approved these changes Jun 6, 2025

View reviewed changes

asmorkalov merged commit 6e8ce30 into opencv:4.x Jun 6, 2025
22 of 25 checks passed

asmorkalov self-assigned this Jun 6, 2025

asmorkalov mentioned this pull request Jun 11, 2025

5.x merge 4.x #3951

Merged

		int initStatus = cv::fastcv::dsp::fcvdspinit();
		ASSERT_EQ(initStatus, 0) << "Failed to initialize FastCV DSP";

Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

Uh oh!

Conversation

quic-apreetam commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asmorkalov commented Apr 30, 2025

Uh oh!

quic-apreetam commented May 2, 2025

Uh oh!

quic-apreetam commented May 5, 2025

Uh oh!

Uh oh!

Uh oh!

quic-apreetam left a comment

Choose a reason for hiding this comment

Uh oh!

quic-apreetam left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qc-apreetam left a comment

Choose a reason for hiding this comment

Uh oh!

qc-apreetam commented May 6, 2025

Uh oh!

asmorkalov commented May 6, 2025

Uh oh!

quic-apreetam commented May 6, 2025

Uh oh!

asmorkalov commented May 7, 2025

Uh oh!

quic-apreetam commented May 7, 2025

Uh oh!

quic-apreetam commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asmorkalov commented May 7, 2025

Uh oh!

quic-apreetam commented May 7, 2025

Uh oh!

asmorkalov commented May 7, 2025

Uh oh!

gursimarsingh commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asmorkalov May 21, 2025

Choose a reason for hiding this comment

Uh oh!

quic-apreetam May 21, 2025

Choose a reason for hiding this comment

Uh oh!

quic-apreetam May 22, 2025

Choose a reason for hiding this comment

Uh oh!

asmorkalov May 26, 2025

Choose a reason for hiding this comment

Uh oh!

quic-apreetam May 26, 2025

Choose a reason for hiding this comment

Uh oh!

asmorkalov May 28, 2025

Choose a reason for hiding this comment

Uh oh!

asmorkalov commented May 26, 2025

Uh oh!

gursimarsingh commented Jun 4, 2025

Uh oh!

asmorkalov commented Jun 4, 2025

Uh oh!

quic-apreetam commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

quic-apreetam commented Apr 29, 2025 •

edited

Loading

quic-apreetam left a comment •

edited

Loading

quic-apreetam commented May 7, 2025 •

edited

Loading

gursimarsingh commented May 20, 2025 •

edited

Loading

quic-apreetam commented Jun 4, 2025 •

edited

Loading

quic-apreetam commented Jun 6, 2025 •

edited

Loading