Tests fail: KeyError: 'LLVMPY_AddRefPrunePass'

[yurivict]

Tests fail, see:

=========================================================================================== ERRORS ===========================================================================================
______________________________________________________________________ ERROR collecting llvmlite/tests/test_binding.py _______________________________________________________________________
llvmlite/binding/ffi.py:95: in __getattr__
    return self._fntab[name]
E   KeyError: 'LLVMPY_AddRefPrunePass'

During handling of the above exception, another exception occurred:
llvmlite/tests/test_binding.py:16: in <module>
    from llvmlite import binding as llvm
llvmlite/binding/__init__.py:10: in <module>
    from .passmanagers import *
llvmlite/binding/passmanagers.py:397: in <module>
    ffi.lib.LLVMPY_AddRefPrunePass.argtypes = [ffi.LLVMPassManagerRef, c_int,
llvmlite/binding/ffi.py:98: in __getattr__
    cfn = getattr(self._lib, name)
/usr/local/lib/python3.9/ctypes/__init__.py:387: in __getattr__
    func = self.__getitem__(name)
/usr/local/lib/python3.9/ctypes/__init__.py:392: in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
E   AttributeError: Undefined symbol "LLVMPY_AddRefPrunePass"
_________________________________________________________________________ ERROR collecting llvmlite/tests/test_ir.py _________________________________________________________________________
llvmlite/binding/ffi.py:95: in __getattr__
    return self._fntab[name]
E   KeyError: 'LLVMPY_AddRefPrunePass'

During handling of the above exception, another exception occurred:
llvmlite/tests/test_ir.py:14: in <module>
    from llvmlite import binding as llvm
llvmlite/binding/__init__.py:10: in <module>
    from .passmanagers import *
llvmlite/binding/passmanagers.py:397: in <module>
    ffi.lib.LLVMPY_AddRefPrunePass.argtypes = [ffi.LLVMPassManagerRef, c_int,
llvmlite/binding/ffi.py:98: in __getattr__
    cfn = getattr(self._lib, name)
/usr/local/lib/python3.9/ctypes/__init__.py:387: in __getattr__
    func = self.__getitem__(name)
/usr/local/lib/python3.9/ctypes/__init__.py:392: in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
E   AttributeError: Undefined symbol "LLVMPY_AddRefPrunePass"
______________________________________________________________________ ERROR collecting llvmlite/tests/test_refprune.py ______________________________________________________________________
llvmlite/binding/ffi.py:95: in __getattr__
    return self._fntab[name]
E   KeyError: 'LLVMPY_AddRefPrunePass'

During handling of the above exception, another exception occurred:
llvmlite/tests/test_refprune.py:3: in <module>
    from llvmlite import binding as llvm
llvmlite/binding/__init__.py:10: in <module>
    from .passmanagers import *
llvmlite/binding/passmanagers.py:397: in <module>
    ffi.lib.LLVMPY_AddRefPrunePass.argtypes = [ffi.LLVMPassManagerRef, c_int,
llvmlite/binding/ffi.py:98: in __getattr__
    cfn = getattr(self._lib, name)
/usr/local/lib/python3.9/ctypes/__init__.py:387: in __getattr__
    func = self.__getitem__(name)
/usr/local/lib/python3.9/ctypes/__init__.py:392: in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
E   AttributeError: Undefined symbol "LLVMPY_AddRefPrunePass"
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================================================== 3 errors in 1.41s ======================================================================================

Version: 0.39.1
llvm-11
Python-3.9
FreeBSD 13.1

[esc]

@yurivict thank you for submitting this. May I ask how you installed Python and how you installed llvmlite?

FWIW: the "thing" that appears to be missing should come from here: https://github.com/numba/llvmlite/blob/main/ffi/custom_passes.cpp#L1171 -- so my guess would be, that you are experiencing some sort of linking issue.

[yurivict]

Both Python and llvmlite are installed from FreeBSD ports.

[esc]

Both Python and llvmlite are installed from FreeBSD ports.

OK; thank you for following up. Is there a way to submit this issue to the folks that maintain this port? The Numba team doesn't officially support/test-on FreeBSD (it's a best effort) and we didn't build the ports/packages you are installing, so we don't know exactly how they were built and thus our chances of success to debug this are somewhat limited.

[yurivict]

The C++ part looks like this:

===>  Building for py39-llvmlite-0.39.1
running build
got version from file /disk-samsung/freebsd-ports/devel/py-llvmlite/work-py39/llvmlite-0.39.1/llvmlite/_version.py {'version': '0.39.1', 'full': '31f808fa7a5796bb8d4b14e8a9e27c6dd15f2b6c'}
running build_ext
/usr/local/bin/python3.9 /disk-samsung/freebsd-ports/devel/py-llvmlite/work-py39/llvmlite-0.39.1/ffi/build.py
LLVM version... --- libllvmlite.so ---
c++ -shared -fno-omit-frame-pointer -fno-omit-frame-pointer -fPIC  -fPIC -fno-omit-frame-pointer -I/usr/local/llvm11/include -std=c++14 -fno-exceptions -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -g assembly.cpp bitcode.cpp core.cpp initfini.cpp module.cpp value.cpp  executionengine.cpp transforms.cpp passmanagers.cpp targets.cpp dylib.cpp  linker.cpp object_file.cpp -o libllvmlite.so  -Wl,-rpath=/usr/local/lib/gcc11  -L/usr/local/lib/gcc11 -B/usr/local/bin -fstack-protector-strong  -L/usr/local/llvm11/lib -lLLVM-11

It doesn't include custom_passes.cpp.

custom_passes.cpp isn't mentioned anywhere in the log.

It seems that GNU make is used. How to switch to cmake?

[yurivict]

custom_passes.cpp is missing in ffi/Makefile.freebsd.

[stuartarchibald]

custom_passes.cpp is missing in ffi/Makefile.freebsd.

Seems like it is indeed missing, which I think would explain the immediate problem. A patch to fix would be welcomed! Many thanks!

[esc]

@yurivict @stuartarchibald thank you for diagnosing. I have updated the labels to bug and build.

[ke6jjj]

As the automated system shows, I have submitted a patch for this issue. I've run it on 13.2-RELEASE-p2 for amd64. It does indeed resolve the missing symbol and the ensuing library seems to work enough that I can finally run OpenAI's Whisper project under FreeBSD. (This was a blocking issue there).

[esc]

Closing now as #997 has been merged.