ya make -DUSE_LOCAL_PYTHON=yes leads to compile failure because ya SDK is not compatible with local python SDK:
------- [CC] {pic, FAILED} $(S)/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp
command /root/.ya/tools/v4/6495238978/bin/clang++ --target=x86_64-linux-gnu --sysroot=/root/.ya/tools/v4/243881345 -B/root/.ya/tools/v4/243881345/usr/bin -c -o /root/.ya/build/build_root/sr1c/0000d2/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp.py3.pic.o -I/root/.ya/build/build_root/sr1c/0000d2 -I/ydb -I/ydb/contrib/libs/linux-headers -I/ydb/contrib/libs/linux-headers/_nf -I/ydb/contrib/libs/cxxsupp/libcxx/include -I/ydb/contrib/libs/cxxsupp/libcxxrt/include -I/ydb/contrib/libs/clang16-rt/include -I/ydb/contrib/libs/zlib/include -I/ydb/contrib/libs/double-conversion -I/ydb/contrib/libs/libc_compat/include/readpassphrase -I/ydb/contrib/libs/libc_compat/reallocarray -I/ydb/contrib/libs/libc_compat/random -I/ydb/contrib/libs/protobuf/src -I/ydb/contrib/restricted/abseil-cpp -I/ydb/contrib/libs/sparsehash/src -I/ydb/contrib/libs/brotli/include -I/ydb/contrib/libs/snappy/include -I/root/.ya/build/build_root/sr1c/0000d2/library/cpp/build_info -pipe -m64 -g -ggnu-pubnames -fexceptions -fno-common -ffunction-sections -fdata-sections -fuse-init-array -fcolor-diagnostics -faligned-allocation -fdebug-default-version=4 -fstack-protector -Wno-everything -DARCADIA_ROOT=/ydb -DARCADIA_BUILD_ROOT=/root/.ya/build/build_root/sr1c/0000d2 -D_THREAD_SAFE -D_PTHREADS -D_REENTRANT -D_LARGEFILE_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_YNDX_LIBUNWIND_ENABLE_EXCEPTION_BACKTRACE -D__LONG_LONG_SUPPORTED -DSSE_ENABLED=1 -DSSE3_ENABLED=1 -DSSSE3_ENABLED=1 -DSSE41_ENABLED=1 -DSSE42_ENABLED=1 -DPOPCNT_ENABLED=1 -DCX16_ENABLED=1 -fno-omit-frame-pointer -D_libunwind_ -DLIBCXX_BUILDING_LIBCXXRT -UNDEBUG -fPIC -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -DCATBOOST_OPENSOURCE=yes -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mpopcnt -mcx16 -I/usr/include/python3.8 -I/usr/include/python3.8 -UNDEBUG -fPIC -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -std=c++20 -Wno-everything -DUDF_ABI_VERSION_MAJOR=2 -DUDF_ABI_VERSION_MINOR=27 -DUDF_ABI_VERSION_PATCH=0 -nostdinc++ -DCATBOOST_OPENSOURCE=yes -nostdinc++ /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp failed with exit code 1 in /root/.ya/build/build_root/sr1c/0000d2
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp:1:
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.h:3:
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_ptr.h:3:
In file included from /usr/include/python3.8/Python.h:8:
/usr/include/python3.8/pyconfig.h:3:12: fatal error: 'x86_64-linux-gnu/python3.8/pyconfig.h' file not found
# include <x86_64-linux-gnu/python3.8/pyconfig.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Failed
How to reproduce:
- Checkout ydb-platform-ydb to
$HOME/ydb
- Run the following command:
docker run --rm -it -v $HOME/ydb:/ydb ubuntu:focal /bin/bash -c "apt update && apt install -y python3-dev python3.8-dev; /ydb/ya make -DUSE_ARCADIA_PYTHON=no -DUSE_LOCAL_PYTHON -DPYTHON_CONFIG=python3.8-config -DPYTHON_BIN=python3.8 /ydb/ydb/library/yql/udfs/common/python/system_python/python3_8"
We had to implement a workaround because of it: https://github.com/ytsaurus/ytsaurus/pull/793/files#diff-fce526c9dfb5ba1a3e79350a2715244f3414cf4363ee61a8b049d81151cf82e9R29-R41
One other workaround is to use -DOS_SDK=local, but seems like it may do more harm than benefit
ya make -DUSE_LOCAL_PYTHON=yesleads to compile failure because ya SDK is not compatible with local python SDK:How to reproduce:
$HOME/ydbdocker run --rm -it -v $HOME/ydb:/ydb ubuntu:focal /bin/bash -c "apt update && apt install -y python3-dev python3.8-dev; /ydb/ya make -DUSE_ARCADIA_PYTHON=no -DUSE_LOCAL_PYTHON -DPYTHON_CONFIG=python3.8-config -DPYTHON_BIN=python3.8 /ydb/ydb/library/yql/udfs/common/python/system_python/python3_8"We had to implement a workaround because of it: https://github.com/ytsaurus/ytsaurus/pull/793/files#diff-fce526c9dfb5ba1a3e79350a2715244f3414cf4363ee61a8b049d81151cf82e9R29-R41
One other workaround is to use
-DOS_SDK=local, but seems like it may do more harm than benefit