Skip to content

USE_LOCAL_PYTHON is incompatible with ya SDK and leads to failure #21

@Krock21

Description

@Krock21

ya make -DUSE_LOCAL_PYTHON=yes leads to compile failure because ya SDK is not compatible with local python SDK:

------- [CC] {pic, FAILED} $(S)/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp
command /root/.ya/tools/v4/6495238978/bin/clang++ --target=x86_64-linux-gnu --sysroot=/root/.ya/tools/v4/243881345 -B/root/.ya/tools/v4/243881345/usr/bin -c -o /root/.ya/build/build_root/sr1c/0000d2/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp.py3.pic.o -I/root/.ya/build/build_root/sr1c/0000d2 -I/ydb -I/ydb/contrib/libs/linux-headers -I/ydb/contrib/libs/linux-headers/_nf -I/ydb/contrib/libs/cxxsupp/libcxx/include -I/ydb/contrib/libs/cxxsupp/libcxxrt/include -I/ydb/contrib/libs/clang16-rt/include -I/ydb/contrib/libs/zlib/include -I/ydb/contrib/libs/double-conversion -I/ydb/contrib/libs/libc_compat/include/readpassphrase -I/ydb/contrib/libs/libc_compat/reallocarray -I/ydb/contrib/libs/libc_compat/random -I/ydb/contrib/libs/protobuf/src -I/ydb/contrib/restricted/abseil-cpp -I/ydb/contrib/libs/sparsehash/src -I/ydb/contrib/libs/brotli/include -I/ydb/contrib/libs/snappy/include -I/root/.ya/build/build_root/sr1c/0000d2/library/cpp/build_info -pipe -m64 -g -ggnu-pubnames -fexceptions -fno-common -ffunction-sections -fdata-sections -fuse-init-array -fcolor-diagnostics -faligned-allocation -fdebug-default-version=4 -fstack-protector -Wno-everything -DARCADIA_ROOT=/ydb -DARCADIA_BUILD_ROOT=/root/.ya/build/build_root/sr1c/0000d2 -D_THREAD_SAFE -D_PTHREADS -D_REENTRANT -D_LARGEFILE_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_YNDX_LIBUNWIND_ENABLE_EXCEPTION_BACKTRACE -D__LONG_LONG_SUPPORTED -DSSE_ENABLED=1 -DSSE3_ENABLED=1 -DSSSE3_ENABLED=1 -DSSE41_ENABLED=1 -DSSE42_ENABLED=1 -DPOPCNT_ENABLED=1 -DCX16_ENABLED=1 -fno-omit-frame-pointer -D_libunwind_ -DLIBCXX_BUILDING_LIBCXXRT -UNDEBUG -fPIC -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -DCATBOOST_OPENSOURCE=yes -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mpopcnt -mcx16 -I/usr/include/python3.8 -I/usr/include/python3.8 -UNDEBUG -fPIC -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -std=c++20 -Wno-everything -DUDF_ABI_VERSION_MAJOR=2 -DUDF_ABI_VERSION_MINOR=27 -DUDF_ABI_VERSION_PATCH=0 -nostdinc++ -DCATBOOST_OPENSOURCE=yes -nostdinc++ /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp failed with exit code 1 in /root/.ya/build/build_root/sr1c/0000d2
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.cpp:1:
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_dict.h:3:
In file included from /ydb/ydb/library/yql/udfs/common/python/bindings/py_ptr.h:3:
In file included from /usr/include/python3.8/Python.h:8:
/usr/include/python3.8/pyconfig.h:3:12: fatal error: 'x86_64-linux-gnu/python3.8/pyconfig.h' file not found
#  include <x86_64-linux-gnu/python3.8/pyconfig.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Failed

How to reproduce:

  1. Checkout ydb-platform-ydb to $HOME/ydb
  2. Run the following command: docker run --rm -it -v $HOME/ydb:/ydb ubuntu:focal /bin/bash -c "apt update && apt install -y python3-dev python3.8-dev; /ydb/ya make -DUSE_ARCADIA_PYTHON=no -DUSE_LOCAL_PYTHON -DPYTHON_CONFIG=python3.8-config -DPYTHON_BIN=python3.8 /ydb/ydb/library/yql/udfs/common/python/system_python/python3_8"

We had to implement a workaround because of it: https://github.com/ytsaurus/ytsaurus/pull/793/files#diff-fce526c9dfb5ba1a3e79350a2715244f3414cf4363ee61a8b049d81151cf82e9R29-R41

One other workaround is to use -DOS_SDK=local, but seems like it may do more harm than benefit

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions