Skip to content

[DRAFT] Allow compilation on Apple Silicon #212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ashao
Copy link

@ashao ashao commented Feb 8, 2025

The changes here get most of the way to building a wheel for ml-metadata for MacOS on Apple Silicon. This currently does not succeed and I would ask for some help from the maintainers to understand what the way forward should be:

  1. Cleanup modifications to the bazel configuration to properly define compiling and linking flags
  2. Resolve errors in linking associated with metadata_store_extension.

See the following sections for more details

Modifying the CC toolchain

The following manual steps need to be done to by modifying files directly in the external bazel directory.

  • Install libiconv via brew (or compile from scratch): brew install libiconv
  • Modify bazel_tools/tools/cpp/unix_cc_toolchain_config.bzl to remove the -static -s flag group
  • Remove undefined dynamic_lookup from local_config_cc/BUILD
  • Add -L$(ICONV_PATH)/lib -liconv to link_libs in local_config_cc/BUILD
  • Try to build with USE_BAZEL_VERSION=6.1.0 ICONV_PATH=/opt/homebrew/opt/libiconv python setup.py build

What is the most bazel-like way to to embed these modifications?

Error during linking stage

The build fails at the very end due to some symbols not being found (see attached for the full output) with the salient part of the error posted here:

  "__Py_NotImplementedStruct", referenced from:
      pybind11::cpp_function::dispatcher(_object*, _object*, _object*) in metadata_store_extension.pic.o
  "__Py_TrueStruct", referenced from:
      pybind11::detail::type_caster<bool, void>& pybind11::detail::load_type<bool, void>(pybind11::detail::type_caster<bool, void>&, pybind11::handle const&) in metadata_store_extension.pic.o
  "_init_metadata_store_extension", referenced from:
      <initial-undefines>
  "_initmetadata_store_extension", referenced from:
      <initial-undefines>

I'm guessing that this has something to do with the way that pybind11 prepends function names to objects, but I am definitely out of my depth on this one.

Does this problem arise from incorrect name mangling or does it require some other modification to the link instructions?

link_error.txt

Copy link

google-cla bot commented Feb 8, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@ashao ashao marked this pull request as draft February 8, 2025 01:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant