Problem with v0.13.64 and v0.13.65: `ModuleNotFoundError: No module named 'pyodbc'`

[amotl]

Problem

When running ingestr using uvx --prerelease=allow 'ingestr>=0.13.64' ingest ..., it fails.

ModuleNotFoundError: No module named 'pyodbc'

Interim solution

⚠ Workaround ⚠

Please downgrade to a previous version of ingestr, e.g. by skipping the offending versions v0.13.64 and v0.13.65,

uv pip install --prerelease=allow --upgrade 'ingestr!=0.13.64,!=0.13.65'

or install ingestr including its odbc extra.

uv pip install --prerelease=allow --upgrade 'ingestr[odbc]'

Permanent solution

Please yank the ingestr v0.13.64 and ingestr v0.13.65 packages from PyPI.

Details

  File "/Users/amo/.cache/uv/archive-v0/XoWBb3yE3D5q7XjrnFwMr/lib/python3.13/site-packages/ingestr/main.py", line 291, in ingest
    from ingestr.src.destinations import AthenaDestination
  File "/Users/amo/.cache/uv/archive-v0/XoWBb3yE3D5q7XjrnFwMr/lib/python3.13/site-packages/ingestr/src/destinations.py", line 22, in <module>
    from dlt.destinations.impl.mssql.mssql import (
    ...<2 lines>...
    )
  File "/Users/amo/.cache/uv/archive-v0/XoWBb3yE3D5q7XjrnFwMr/lib/python3.13/site-packages/dlt/destinations/impl/mssql/mssql.py", line 15, in <module>
    from dlt.destinations.impl.mssql.sql_client import PyOdbcMsSqlClient
  File "/Users/amo/.cache/uv/archive-v0/XoWBb3yE3D5q7XjrnFwMr/lib/python3.13/site-packages/dlt/destinations/impl/mssql/sql_client.py", line 6, in <module>
    import pyodbc
ModuleNotFoundError: No module named 'pyodbc'

[amotl]

Workaround

When adding the odbc extra and going back to Python 3.12, using uvx --python=3.12 --prerelease=allow 'ingestr[odbc]>=0.13.64' ingest ..., it works.

Thoughts

It would be sweet to isolate this dependency so the application does not fail if one of the optional extra packages is not installed.

[turtleDev]

This only happens for specific source/destination.

Our docs will mention it's requirement on the appropriate pages.

Maybe we could improve our error messages in these scenarios. Something along the lines of:

(Name) requires pyodbc to work. Install it via:
$ pip install ingestr[odbc]

[amotl]

This only happens for specific source/destination.

Hi. It looks like it currently happens every time.

$ uv pip uninstall pyodbc

$ python

>>> import ingestr.src.destinations
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    import ingestr.src.destinations
  File "/path/to/ingestr/ingestr/src/destinations.py", line 22, in <module>
    from dlt.destinations.impl.mssql.mssql import (
    ...<2 lines>...
    )
  File "/path/to/ingestr/.venv/lib/python3.13/site-packages/dlt/destinations/impl/mssql/mssql.py", line 15, in <module>
    from dlt.destinations.impl.mssql.sql_client import PyOdbcMsSqlClient
  File "/path/to/ingestr/.venv/lib/python3.13/site-packages/dlt/destinations/impl/mssql/sql_client.py", line 6, in <module>
    import pyodbc
ModuleNotFoundError: No module named 'pyodbc'

[amotl]

To make sure we are not chasing ghosts, we exercised the procedure using a fresh virtualenv and used a data source different than MSSQL.

Install

uv venv --python 3.12 --seed .venv312
source .venv312/bin/activate
uv pip install --prerelease=allow --upgrade ingestr

Invoke

$ ingestr version
v0.13.64

$ ingestr ingest --source-uri "csv://" --dest-uri "csv://" --source-table foo

  File "/Users/amo/dev/tmp/testdrive/.venv312/lib/python3.12/site-packages/dlt/destinations/impl/mssql/mssql.py", line 15, in <module>
    from dlt.destinations.impl.mssql.sql_client import PyOdbcMsSqlClient
  File "/Users/amo/dev/tmp/testdrive/.venv312/lib/python3.12/site-packages/dlt/destinations/impl/mssql/sql_client.py", line 6, in <module>
    import pyodbc
ModuleNotFoundError: No module named 'pyodbc'

Observations

It still works using ingestr v0.13.63, only the most recent v0.13.64 displays the error. In that spirit, this is a viable workaround.

uv pip install --prerelease=allow --upgrade "ingestr<0.13.64"

[amotl]

This patch adds references to dlt.destinations.impl.mssql to the main module namespace in destinations.py. That spot would need to be isolated again after the recent changes.

• mssql: add support for token auth #281 (review)

The regression went unnoticed on CI, because pyodbc is added per requirements-dev.txt. In this spirit, the flaw only materializes for downstream users.

$ ag odbc requirements*

requirements-dev.txt
9:pyodbc==5.2.0

Let us know if you would like to receive a little one-time support about a) isolating this dependency again and b) improving dependency governance and better quality assurance on behalf of CI/GHA, tangentially also looking at GH-42 in this context. 🍀

[amotl]

Because only the most recent version of ingestr is affected, I think it is a viable procedure to just yank the v0.13.64 package from PyPI, to reduce downstream impact, and give more time for a fix.

[amotl]

Hi again. This is our proposal to isolate the pyodbc dependency.

• Dependencies: Isolate pyodbc package #296

[turtleDev]

hey @amotl

We've released a fix in #298

Could you check if the issue still persists with ingestr v0.13.66?

[amotl]

Hi @turtleDev. It works well using v0.13.67. Thank you very much!