Skip to content

Conversation

@jac0626
Copy link
Contributor

@jac0626 jac0626 commented Aug 19, 2025

feat: Automate Python Wheel Builds and Optimize Dependencies

This PR modernizes the Python packaging process by introducing a fully automated CI/CD pipeline to build
and publish wheels, while also optimizing the binary dependencies.

Key Changes:

  • Automated Wheel Builds: Implemented a new GitHub Actions workflow using cibuildwheel to create manylinux
    wheels for x86_64 and aarch64 architectures. The workflow automatically publishes to TestPyPI on pushes
    to main and to the official PyPI when a new release is created.
  • Modernized Packaging: Migrated the build system to use pyproject.toml (following PEP 517/518 standards),
    replacing the previous setup.py-based scripts for a more robust and reliable process.

Note for Maintainers: Required Secrets

To enable the new automated publishing workflow (build_release_wheel.yml), the following secrets must be
added to the repository's settings under Settings > Secrets and variables > Actions:

  • PYPI_API_TOKEN: The API token for publishing packages to the official PyPI repository. This is used when
    a new release is published.
  • TEST_PYPI_API_TOKEN: The API token for the TestPyPI repository, used for testing the release process on
    pushes to main.

These secrets are essential for the workflow to publish packages automatically. For more details, please
see the GitHub documentation on creating secrets
(https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions).

Future work should include adding build support for Windows and macOS
close #1020

Summary by Sourcery

Modernize and automate the Python packaging workflow by replacing setup.py with a pyproject.toml-based build, adding CMake support for static MKL linking, and implementing a GitHub Actions pipeline with cibuildwheel to build and publish manylinux wheels across multiple architectures.

New Features:

  • Automate wheel building and publishing for x86_64 and aarch64 using GitHub Actions and cibuildwheel
  • Migrate packaging to PEP 517/518 with pyproject.toml and dynamic versioning via setuptools_scm
  • Reduce runtime mkl dependencies and wheel size
  • Add build_cpp_for_cibw.sh to configure, build, strip, and package Python extension modules for cibuildwheel

Enhancements:

  • Simplify setup.py to minimal extension declaration and remove legacy packaging logic
  • Enhance CMakeLists to detect Python executables, emit configuration messages, and apply static linking flags for the Python module
  • Remove obsolete Makefile and build scripts in favor of the new CI/CD pipeline

CI:

  • Introduce a build_release_wheel GitHub Actions workflow to build wheels, upload artifacts, and publish to TestPyPI and PyPI based on branch or release events

@gemini-code-assist
Copy link

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

@sourcery-ai
Copy link

sourcery-ai bot commented Aug 19, 2025

Reviewer's Guide

This PR implements a complete CI/CD workflow for building and publishing manylinux Python wheels via GitHub Actions and cibuildwheel, migrates Python packaging to PEP 517/518 with pyproject.toml, and refactors the CMake infrastructure to support static MKL linking and enhanced Python bindings.

Sequence diagram for automated wheel publishing on release

sequenceDiagram
    participant Dev as Developer
    participant GH as GitHub Actions
    participant PyPI as PyPI
    Dev->>GH: Create new release
    GH->>GH: Build wheels (cibuildwheel)
    GH->>GH: Upload artifacts
    GH->>PyPI: Publish wheels (using PYPI_API_TOKEN)
    PyPI-->>GH: Success/Failure response
    GH-->>Dev: Notify of publish result
Loading

Sequence diagram for automated wheel publishing on push to main (TestPyPI)

sequenceDiagram
    participant Dev as Developer
    participant GH as GitHub Actions
    participant TestPyPI as TestPyPI
    Dev->>GH: Push to main branch
    GH->>GH: Build wheels (cibuildwheel)
    GH->>GH: Upload artifacts
    GH->>TestPyPI: Publish wheels (using TEST_PYPI_API_TOKEN)
    TestPyPI-->>GH: Success/Failure response
    GH-->>Dev: Notify of publish result
Loading

Class diagram for the updated Python packaging structure

classDiagram
    class pyvsag {
        <<package>>
        _pyvsag*.so
        _version.py
    }
    class pyproject.toml {
        build-system
        project
        tool.setuptools
        tool.setuptools_scm
        tool.cibuildwheel
    }
    pyvsag <.. pyproject.toml : defined in
Loading

File-Level Changes

Change Details Files
Automate wheel builds and releases
  • Add GitHub Actions workflow for matrix builds, TestPyPI and PyPI publishing
  • Configure cibuildwheel via pyproject.toml for manylinux x86_64 and aarch64
  • Introduce build_cpp_for_cibw.sh to orchestrate CMake build and artifact handling
  • Remove legacy Makefile target and obsolete build scripts
.github/workflows/build_release_wheel.yml
python/pyproject.toml
scripts/build_cpp_for_cibw.sh
Makefile
scripts/build_pyvsag_multiple_version.sh
Migrate packaging to pyproject.toml
  • Eliminate setup.py content in favor of PEP 517/518 build-system section
  • Define project metadata, dependencies, package data, and dynamic versioning in toml
  • Integrate setuptools_scm for version management
python/setup.py
python/pyproject.toml
Refactor CMake for static MKL linking
  • Implement dynamic search paths and break loops upon first match
  • Link MKL and OpenMP statically using .a libraries and linker flags
  • Provide clear status messages and graceful fallback to OpenBLAS
extern/mkl/mkl.cmake
Enhance Python binding configuration in CMake
  • Allow custom Python executable overrides and verbose interpreter discovery
  • Add static linking flags for _pyvsag extension
  • Emit detailed build status messages for Python version and output naming
CMakeLists.txt

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

sourcery-ai[bot]
sourcery-ai bot previously requested changes Aug 19, 2025
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!

Blocking issues:

  • An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload. (link)
  • An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload. (link)
  • An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload. (link)
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments

### Comment 1
<location> `extern/mkl/mkl.cmake:56` </location>
<code_context>
+        message(FATAL_ERROR "Could not find Intel MKL in standard locations; use -DMKL_PATH to specify the install location")
+    endif()
+
+    if(NOT MKL_INCLUDE_PATH)
+        message(WARNING "Could not find MKL headers, continuing without include path")
+    endif()
</code_context>

<issue_to_address>
Build continues with a warning if MKL headers are missing.

If MKL headers are required, consider treating their absence as a fatal error. Otherwise, clearly document what functionality will be unavailable when headers are missing.

Suggested implementation:

```
    if(NOT MKL_INCLUDE_PATH)
        message(FATAL_ERROR "Could not find MKL headers. MKL headers are required for building MKL-dependent functionality. Please specify the header location with -DMKL_INCLUDE_PATH.")
    endif()

```

If MKL headers are optional, use:
message(WARNING "Could not find MKL headers. MKL-dependent functionality will be disabled. Specify the header location with -DMKL_INCLUDE_PATH to enable full MKL support.")
and ensure the rest of the build system disables or excludes MKL-dependent features when headers are missing.
</issue_to_address>

### Comment 2
<location> `extern/mkl/mkl.cmake:63` </location>
<code_context>
+
+    
+    link_directories(${MKL_PATH})
+    if(OMP_PATH)
+        link_directories(${OMP_PATH})
+    endif()
</code_context>

<issue_to_address>
OMP_PATH is now optional for linking.

If OMP_PATH is unset, please ensure the build system checks for OpenMP availability or falls back to system libraries to prevent runtime errors.

Suggested implementation:

```


    link_directories(${MKL_PATH})
    if(OMP_PATH)
        link_directories(${OMP_PATH})
    else()
        find_package(OpenMP)
        if(OpenMP_FOUND)
            message(STATUS "OpenMP found: ${OpenMP_C_FLAGS}")
            # Link OpenMP libraries if found
            if(TARGET OpenMP::OpenMP_CXX)
                set(OPENMP_LIBRARIES OpenMP::OpenMP_CXX)
            elseif(OpenMP_CXX_LIBRARIES)
                set(OPENMP_LIBRARIES ${OpenMP_CXX_LIBRARIES})
            elseif(OpenMP_C_LIBRARIES)
                set(OPENMP_LIBRARIES ${OpenMP_C_LIBRARIES})
            endif()
        else()
            message(WARNING "OMP_PATH not set and OpenMP not found; build may fail or run without OpenMP support")
        endif()
    endif()

```

You may need to ensure that `${OPENMP_LIBRARIES}` is actually linked to your targets later in your CMake configuration, e.g., using `target_link_libraries(your_target ${OPENMP_LIBRARIES})`. Adjust as needed for your project's structure.
</issue_to_address>

### Comment 3
<location> `extern/mkl/mkl.cmake:95` </location>
<code_context>
-    message ("enable intel-mkl as blas backend")
-else ()
+
+    add_compile_options(-m64)
+    
+   
</code_context>

<issue_to_address>
Explicitly adds -m64 compile option.

Consider applying -m64 only when the target architecture is 64-bit to avoid compatibility issues on platforms that require 32-bit builds or lack support for this flag.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
+    add_compile_options(-m64)
+    
+   
=======
+    # Add -m64 only if building for 64-bit architecture
+    if(CMAKE_SIZEOF_VOID_P EQUAL 8)
+        add_compile_options(-m64)
+    endif()
>>>>>>> REPLACE

</suggested_fix>

### Comment 4
<location> `scripts/build_cpp_for_cibw.sh:119` </location>
<code_context>
+    echo "Found libvsag.so at: $LIBVSAG_SO"
+    
+    
+    if ldd "$PYVSAG_CPYTHON_SO" 2>/dev/null | grep -q "libvsag.so.*=> not found"; then
+        echo "libvsag.so not in standard path, copying to package"
+        strip --strip-unneeded "$LIBVSAG_SO" || true
</code_context>

<issue_to_address>
Script checks for missing libvsag.so and copies it if needed.

If multiple libvsag.so versions exist, the script may not select the correct one. Match the version or build timestamp to avoid inconsistencies.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
LIBVSAG_SO=$(find $CMAKE_BUILD_DIR -name "libvsag.so*" -type f | head -n1)

if [ -n "$LIBVSAG_SO" ]; then
    echo "Found libvsag.so at: $LIBVSAG_SO"


    if ldd "$PYVSAG_CPYTHON_SO" 2>/dev/null | grep -q "libvsag.so.*=> not found"; then
        echo "libvsag.so not in standard path, copying to package"
        strip --strip-unneeded "$LIBVSAG_SO" || true
        cp "$LIBVSAG_SO" python/pyvsag/
    else

        strip --strip-unneeded "$LIBVSAG_SO" || true
        echo "libvsag.so is accessible, letting auditwheel handle it"
    fi
fi
=======
# Find all libvsag.so* files and select the most recently modified one
LIBVSAG_SO=$(find "$CMAKE_BUILD_DIR" -name "libvsag.so*" -type f -printf "%T@ %p\n" | sort -nr | head -n1 | awk '{print $2}')

if [ -n "$LIBVSAG_SO" ]; then
    echo "Selected libvsag.so for packaging: $LIBVSAG_SO"

    if ldd "$PYVSAG_CPYTHON_SO" 2>/dev/null | grep -q "libvsag.so.*=> not found"; then
        echo "libvsag.so not in standard path, copying to package"
        strip --strip-unneeded "$LIBVSAG_SO" || true
        cp "$LIBVSAG_SO" python/pyvsag/
    else
        strip --strip-unneeded "$LIBVSAG_SO" || true
        echo "libvsag.so is accessible, letting auditwheel handle it"
    fi
fi
>>>>>>> REPLACE

</suggested_fix>

## Security Issues

### Issue 1
<location> `.github/workflows/build_release_wheel.yml:31` </location>

<issue_to_address>
**security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha):** An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

*Source: opengrep*
</issue_to_address>

### Issue 2
<location> `.github/workflows/build_release_wheel.yml:110` </location>

<issue_to_address>
**security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha):** An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

*Source: opengrep*
</issue_to_address>

### Issue 3
<location> `.github/workflows/build_release_wheel.yml:143` </location>

<issue_to_address>
**security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha):** An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

*Source: opengrep*
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

message(FATAL_ERROR "Could not find Intel MKL in standard locations; use -DMKL_PATH to specify the install location")
endif()

if(NOT MKL_INCLUDE_PATH)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Build continues with a warning if MKL headers are missing.

If MKL headers are required, consider treating their absence as a fatal error. Otherwise, clearly document what functionality will be unavailable when headers are missing.

Suggested implementation:

    if(NOT MKL_INCLUDE_PATH)
        message(FATAL_ERROR "Could not find MKL headers. MKL headers are required for building MKL-dependent functionality. Please specify the header location with -DMKL_INCLUDE_PATH.")
    endif()

If MKL headers are optional, use:
message(WARNING "Could not find MKL headers. MKL-dependent functionality will be disabled. Specify the header location with -DMKL_INCLUDE_PATH to enable full MKL support.")
and ensure the rest of the build system disables or excludes MKL-dependent features when headers are missing.



link_directories(${MKL_PATH})
if(OMP_PATH)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): OMP_PATH is now optional for linking.

If OMP_PATH is unset, please ensure the build system checks for OpenMP availability or falls back to system libraries to prevent runtime errors.

Suggested implementation:



    link_directories(${MKL_PATH})
    if(OMP_PATH)
        link_directories(${OMP_PATH})
    else()
        find_package(OpenMP)
        if(OpenMP_FOUND)
            message(STATUS "OpenMP found: ${OpenMP_C_FLAGS}")
            # Link OpenMP libraries if found
            if(TARGET OpenMP::OpenMP_CXX)
                set(OPENMP_LIBRARIES OpenMP::OpenMP_CXX)
            elseif(OpenMP_CXX_LIBRARIES)
                set(OPENMP_LIBRARIES ${OpenMP_CXX_LIBRARIES})
            elseif(OpenMP_C_LIBRARIES)
                set(OPENMP_LIBRARIES ${OpenMP_C_LIBRARIES})
            endif()
        else()
            message(WARNING "OMP_PATH not set and OpenMP not found; build may fail or run without OpenMP support")
        endif()
    endif()

You may need to ensure that ${OPENMP_LIBRARIES} is actually linked to your targets later in your CMake configuration, e.g., using target_link_libraries(your_target ${OPENMP_LIBRARIES}). Adjust as needed for your project's structure.

Comment on lines 95 to 97
add_compile_options(-m64)


Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Explicitly adds -m64 compile option.

Consider applying -m64 only when the target architecture is 64-bit to avoid compatibility issues on platforms that require 32-bit builds or lack support for this flag.

Suggested change
add_compile_options(-m64)
+ # Add -m64 only if building for 64-bit architecture
+ if(CMAKE_SIZEOF_VOID_P EQUAL 8)
+ add_compile_options(-m64)
+ endif()

Comment on lines +113 to +128
LIBVSAG_SO=$(find $CMAKE_BUILD_DIR -name "libvsag.so*" -type f | head -n1)

if [ -n "$LIBVSAG_SO" ]; then
echo "Found libvsag.so at: $LIBVSAG_SO"


if ldd "$PYVSAG_CPYTHON_SO" 2>/dev/null | grep -q "libvsag.so.*=> not found"; then
echo "libvsag.so not in standard path, copying to package"
strip --strip-unneeded "$LIBVSAG_SO" || true
cp "$LIBVSAG_SO" python/pyvsag/
else

strip --strip-unneeded "$LIBVSAG_SO" || true
echo "libvsag.so is accessible, letting auditwheel handle it"
fi
fi
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Script checks for missing libvsag.so and copies it if needed.

If multiple libvsag.so versions exist, the script may not select the correct one. Match the version or build timestamp to avoid inconsistencies.

Suggested change
LIBVSAG_SO=$(find $CMAKE_BUILD_DIR -name "libvsag.so*" -type f | head -n1)
if [ -n "$LIBVSAG_SO" ]; then
echo "Found libvsag.so at: $LIBVSAG_SO"
if ldd "$PYVSAG_CPYTHON_SO" 2>/dev/null | grep -q "libvsag.so.*=> not found"; then
echo "libvsag.so not in standard path, copying to package"
strip --strip-unneeded "$LIBVSAG_SO" || true
cp "$LIBVSAG_SO" python/pyvsag/
else
strip --strip-unneeded "$LIBVSAG_SO" || true
echo "libvsag.so is accessible, letting auditwheel handle it"
fi
fi
# Find all libvsag.so* files and select the most recently modified one
LIBVSAG_SO=$(find "$CMAKE_BUILD_DIR" -name "libvsag.so*" -type f -printf "%T@ %p\n" | sort -nr | head -n1 | awk '{print $2}')
if [ -n "$LIBVSAG_SO" ]; then
echo "Selected libvsag.so for packaging: $LIBVSAG_SO"
if ldd "$PYVSAG_CPYTHON_SO" 2>/dev/null | grep -q "libvsag.so.*=> not found"; then
echo "libvsag.so not in standard path, copying to package"
strip --strip-unneeded "$LIBVSAG_SO" || true
cp "$LIBVSAG_SO" python/pyvsag/
else
strip --strip-unneeded "$LIBVSAG_SO" || true
echo "libvsag.so is accessible, letting auditwheel handle it"
fi
fi

fetch-depth: 0

- name: Build wheels
uses: pypa/[email protected]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha): An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

Source: opengrep


- name: Publish to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha): An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

Source: opengrep


- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha): An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

Source: opengrep

@codecov
Copy link

codecov bot commented Aug 19, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main    #1064      +/-   ##
==========================================
+ Coverage   91.43%   92.00%   +0.56%     
==========================================
  Files         317      312       -5     
  Lines       18956    18889      -67     
==========================================
+ Hits        17333    17378      +45     
+ Misses       1623     1511     -112     
Flag Coverage Δ
cpp 92.00% <ø> (+0.56%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
common 91.48% <ø> (ø)
datacell 93.03% <ø> (+0.49%) ⬆️
index 90.81% <ø> (+0.57%) ⬆️
simd 100.00% <ø> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9148850...9743e6c. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jac0626 jac0626 force-pushed the build_wheel branch 6 times, most recently from 941cc58 to 6793c3c Compare August 19, 2025 06:35
@jac0626
Copy link
Contributor Author

jac0626 commented Aug 19, 2025

https://pypi.org/project/pyvsag-test-jc/ this is the test wheels uploaded by my test branch

@wxyucs
Copy link
Collaborator

wxyucs commented Aug 27, 2025

Hi @jac0626, thanks for your contribution. This change is large, so I need more time to review and validate it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LHT129 @ShawnShawnYou, please review changes in this file to make sure it's correct for the library packaging (include our internal usage)

@jac0626 jac0626 changed the title feat: Automate Python Wheel Build And Release (WIP)feat: Automate Python Wheel Build And Release Sep 29, 2025
@jac0626 jac0626 force-pushed the build_wheel branch 7 times, most recently from ed82b13 to 429179b Compare October 12, 2025 03:23
@jac0626 jac0626 force-pushed the build_wheel branch 6 times, most recently from 09f64f4 to c078185 Compare October 15, 2025 13:36
@jac0626 jac0626 force-pushed the build_wheel branch 2 times, most recently from 468f9f4 to d8d976c Compare October 15, 2025 13:44
@jac0626 jac0626 changed the title (WIP)feat: Automate Python Wheel Build And Release feat: Automate Python Wheel Build And Release Oct 16, 2025
@jac0626 jac0626 requested a review from wxyucs October 16, 2025 02:14
@jac0626 jac0626 requested a review from LHT129 October 16, 2025 12:57
@wxyucs wxyucs added the kind/feature New feature or request label Oct 20, 2025
Makefile Outdated
Comment on lines 136 to 146
# Target to build a specific Python version wheel.
# Usage: make pyvsag PY_VERSION=3.10
pyvsag:
@echo "Building wheel for Python $(PY_VERSION)..."
bash ./scripts/local_build_wheel.sh $(PY_VERSION)

.PHONY: pyvsag
pyvsag: ## Build pyvsag wheel.
bash ./scripts/build_pyvsag_multiple_version.sh $(PARAM1) $(PARAM2) $(PARAM3)
# Target to build wheels for all supported versions.
# Usage: make pyvsag-all
pyvsag-all:
@echo "Building wheels for all supported versions..."
bash ./scripts/local_build_wheel.sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comments that start with ##. so that we can use the make help command to show the command list.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

esac
echo " - pyproject.toml patched."

echo "✅ Build preparation complete for Python ${PY_VERSION}." No newline at end of file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a newline at EOF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

echo ""
echo "✅ All tasks completed."
echo "📦 Wheels have been generated in the 'wheelhouse' directory:"
ls -l wheelhouse No newline at end of file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline at EOF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done



echo ""
echo "Build script completed successfully!" No newline at end of file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline at EOF

[tool.cibuildwheel.linux]
repair-wheel-command = "auditwheel repair -w {dest_dir} {wheel}"
manylinux-x86_64-image = "manylinux2014"
manylinux-aarch64-image = "manylinux2014" No newline at end of file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a newline at EOF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

endif ()
endif()

set(BLAS_LIBRARIES "${BLAS_LIBRARIES}" CACHE STRING "Final list of BLAS libraries to link against.")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a newline at EOF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


.PHONY: pyvsag pyvsag-all

pyvsag: ## Build a specific Python version wheel. Usage: make pyvsag PY_VERSION=3.10
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use <space> instead of <tab>, and keep the comments aligned in make help output.

.PHONY: pyvsag
pyvsag: ## Build pyvsag wheel.
bash ./scripts/build_pyvsag_multiple_version.sh $(PARAM1) $(PARAM2) $(PARAM3)
pyvsag-all: ## Build wheels for all supported versions. Usage: make pyvsag-all
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@@ -0,0 +1,158 @@
#!/bin/bash

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this blank line, keep only one blank line between statements

Comment on lines +8 to +10



Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove these blank lines, make VARS definitions together



set -eo pipefail

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build and publish Python packages for ARM64 (aarch64)

3 participants