Re: ImportError: Failed to load PyTorch C extensions

[Andrius Merkys <merkys@debian.org>]

Hi,

On 2023-12-06 08:58, Andrius Merkys wrote:
> On 2023-12-05 15:35, Andreas Tille wrote:
> > when I pushed python-array-api-compat to new two weeks ago this has
> > built nicely on Salsa CI[1] but now it fails[2] when trying to load
> > pytorch:
> >
> > =========================== short test summary info 
> > ============================
> > FAILED tests/test_array_namespace.py::test_array_namespace[None-torch] 
> > - ImportError: Failed to load PyTorch C extensions:
> >      It appears that PyTorch has loaded the `torch/_C` folder
> >      of the PyTorch repository rather than the C extensions which
> >      are expected in the `torch._C` namespace. This can occur when
> >      using the `install` workflow. e.g.
> >          $ python setup.py install && python -c "import torch"
> >      This error can generally be solved using the `develop` workflow
> >          $ python setup.py develop && python -c "import torch"  # This 
> > should succeed
> >      or by running Python from a different directory.
> > FAILED 
> > tests/test_array_namespace.py::test_array_namespace[2021.12-torch] - 
> > ImportError: Failed to load PyTorch C extensions:
> > ...
> >
> > Any idea what might be wrong here?
>
> I ran into the same yesterday when building pytorch-geometric 
> (arch:all). My wild guess is that pytorch needs a binNMU to include 
> support for Python 3.12.

Simply rebuilding pytorch does not help, alas. Pytorch's source builds 
for the default Python version only. Thus the appropriate fix would be 
to build python-array-api-compat for the default Python version only as 
well. To do so, replace python3-all dependency with just python3.

> > [1] 
> > https://salsa.debian.org/python-team/packages/python-array-api-compat/-/pipelines/602473
> > [2] 
> > https://salsa.debian.org/python-team/packages/python-array-api-compat/-/jobs/5003765

Hope this helps,
Andrius