Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sonic-ycabled]: Fix swsscommon import protobuf error #407

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Pterosaur
Copy link

@Pterosaur Pterosaur commented Nov 28, 2023

Description

After the PR: sonic-net/sonic-swss-common#837 , the protobuf will be used by swsscommon.
Explicitly import swsscommon before grpc and pytest so that the protobuf global variable will be initialized only once.

Motivation and Context

Current thread 0x00007f4ce5a01740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1108 in create_module
  File "<frozen importlib._bootstrap>", line 565 in module_from_spec
  File "<frozen importlib._bootstrap>", line 666 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1058 in _handle_fromlist
  File "/usr/lib/python3/dist-packages/swsscommon/swsscommon.py", line 13 in <module>
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 790 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1058 in _handle_fromlist
  File "/sonic/src/sonic-platform-daemons/sonic-ycabled/tests/test_y_cable_helper.py", line 3 in <module>
  File "/usr/lib/python3/dist-packages/_pytest/assertion/rewrite.py", line 170 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1030 in _gcd_import
  File "/usr/lib/python3.9/importlib/__init__.py", line 127 in import_module
  File "/usr/lib/python3/dist-packages/_pytest/pathlib.py", line 520 in import_path
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 552 in _importtestmodule
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 484 in _getobj
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 288 in obj
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 500 in _inject_setup_module_fixture
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 487 in collect
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 324 in <lambda>
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 294 in from_call
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 324 in pytest_make_collect_report
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83 in <lambda>
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 441 in collect_one_node
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 768 in genitems
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 771 in genitems
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 568 in _perform_collect
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 516 in perform_collect
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 306 in pytest_collection
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83 in <lambda>
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 295 in _main
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 240 in wrap_session
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 289 in pytest_cmdline_main
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83 in <lambda>
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 157 in main
  File "/usr/local/lib/python3.9/dist-packages/ptr.py", line 220 in run_tests
  File "/usr/local/lib/python3.9/dist-packages/ptr.py", line 209 in run
  File "/usr/lib/python3.9/distutils/dist.py", line 985 in run_command
  File "/usr/lib/python3.9/distutils/dist.py", line 966 in run_commands
  File "/usr/lib/python3.9/distutils/core.py", line 148 in setup
  File "/usr/local/lib/python3.9/dist-packages/setuptools/__init__.py", line 163 in setup
  File "/sonic/src/sonic-platform-daemons/sonic-ycabled/setup.py", line 30 in <module>
Aborted (core dumped)

By GDB and get the error trace of protobuf like :

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7c44537 in __GI_abort () at abort.c:79
#2  0x00007ffff7c4440f in __assert_fail_base (fmt=0x7ffff7dbb688 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7ffff7faa110 "new_prio == -1 || (new_prio >= fifo_min_prio && new_prio <= fifo_max_prio)", file=0x7ffff7faa104 "tpp.c", line=82, function=<optimized out>)
    at assert.c:92
#3  0x00007ffff7c53662 in __GI___assert_fail (assertion=assertion@entry=0x7ffff7faa110 "new_prio == -1 || (new_prio >= fifo_min_prio && new_prio <= fifo_max_prio)", file=file@entry=0x7ffff7faa104 "tpp.c", line=line@entry=82,
    function=function@entry=0x7ffff7faa1c0 <__PRETTY_FUNCTION__.0> "__pthread_tpp_change_priority") at assert.c:101
#4  0x00007ffff7fa6e59 in __pthread_tpp_change_priority (previous_prio=previous_prio@entry=-1, new_prio=new_prio@entry=0) at tpp.c:82
#5  0x00007ffff7f9d290 in __pthread_mutex_lock_full (mutex=0xb4ab78) at ../nptl/pthread_mutex_lock.c:545
#6  0x00007ffff463a468 in __gthread_mutex_lock (__mutex=0xb4ab78) at /usr/include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:749
#7  std::mutex::lock (this=0xb4ab78) at /usr/include/c++/10/bits/std_mutex.h:100
#8  google::protobuf::internal::WrappedMutex::Lock (this=0xb4ab78) at ./google/protobuf/stubs/mutex.h:126
#9  google::protobuf::internal::MutexLock::MutexLock (mu=0xb4ab78, this=<synthetic pointer>) at ./google/protobuf/stubs/mutex.h:148
#10 google::protobuf::internal::OnShutdownRun (f=0x7ffff4631d10 <google::protobuf::internal::DestroyString(void const*)>, arg=0x7ffff484a200 <google::protobuf::internal::fixed_address_empty_string[abi:cxx11]>) at google/protobuf/message_lite.cc:583
#11 0x00007ffff4631e1e in google::protobuf::internal::OnShutdownDestroyString (ptr=0x7ffff484a200 <google::protobuf::internal::fixed_address_empty_string[abi:cxx11]>) at ./google/protobuf/generated_message_util.h:205
#12 google::protobuf::internal::InitProtobufDefaultsImpl () at google/protobuf/generated_message_util.cc:75
#13 google::protobuf::internal::InitProtobufDefaultsSlow () at google/protobuf/generated_message_util.cc:83
#14 0x00007ffff7fe1fe2 in call_init (l=<optimized out>, argc=argc@entry=3, argv=argv@entry=0x7fffffffe638, env=env@entry=0xe20d00) at dl-init.c:72
#15 0x00007ffff7fe20e9 in call_init (env=0xe20d00, argv=0x7fffffffe638, argc=3, l=<optimized out>) at dl-init.c:30
#16 _dl_init (main_map=0x10ade80, argc=3, argv=0x7fffffffe638, env=0xe20d00) at dl-init.c:119
#17 0x00007ffff7d57aed in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:182
#18 0x00007ffff7fe6058 in dl_open_worker (a=a@entry=0x7fffffff46e0) at dl-open.c:758
#19 0x00007ffff7d57a90 in __GI__dl_catch_exception (exception=0x7fffffff46c0, operate=0x7ffff7fe5ca0 <dl_open_worker>, args=0x7fffffff46e0) at dl-error-skeleton.c:208
#20 0x00007ffff7fe58fa in _dl_open (file=0x7ffff57e7fb0 "/usr/lib/python3/dist-packages/swsscommon/_swsscommon.so", mode=-2147483646, caller_dlopen=0x61b611 <_PyImport_FindSharedFuncptr+113>, nsid=-2, argc=3, argv=0x7fffffff46c0, env=0xe20d00) at dl-open.c:837
#21 0x00007ffff7f8e258 in dlopen_doit (a=a@entry=0x7fffffff4900) at dlopen.c:66
#22 0x00007ffff7d57a90 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffff48a0, operate=0x7ffff7f8e200 <dlopen_doit>, args=0x7fffffff4900) at dl-error-skeleton.c:208
#23 0x00007ffff7d57b4f in __GI__dl_catch_error (objname=0xa91230, errstring=0xa91238, mallocedp=0xa91228, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:227
#24 0x00007ffff7f8ea65 in _dlerror_run (operate=operate@entry=0x7ffff7f8e200 <dlopen_doit>, args=args@entry=0x7fffffff4900) at dlerror.c:170
#25 0x00007ffff7f8e2e4 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#26 0x000000000061b611 in _PyImport_FindSharedFuncptr (prefix=0x727825 "PyInit", shortname=0x7ffff57dbc50 "_swsscommon", pathname=0x7ffff57e7fb0 "/usr/lib/python3/dist-packages/swsscommon/_swsscommon.so", fp=0x0) at ../Python/dynload_shlib.c:100
#27 0x000000000061a2ca in _PyImport_LoadDynamicModuleWithSpec (fp=0x0,

How Has This Been Tested?

Check by Azp build

Additional Information (Optional)

@Pterosaur Pterosaur marked this pull request as ready for review November 29, 2023 01:26
@Pterosaur Pterosaur changed the title Fix swsscommon import protobuf error [sonic-ycabled]: Fix swsscommon import protobuf error Nov 29, 2023
@prgeor prgeor requested a review from vdahiya12 December 1, 2023 20:20
@prgeor
Copy link
Collaborator

prgeor commented Dec 1, 2023

@vdahiya12 could you review

@prgeor prgeor added the Bug label Dec 1, 2023
```
Current thread 0x00007f4ce5a01740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1108 in create_module
  File "<frozen importlib._bootstrap>", line 565 in module_from_spec
  File "<frozen importlib._bootstrap>", line 666 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1058 in _handle_fromlist
  File "/usr/lib/python3/dist-packages/swsscommon/swsscommon.py", line 13 in <module>
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 790 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1058 in _handle_fromlist
  File "/sonic/src/sonic-platform-daemons/sonic-ycabled/tests/test_y_cable_helper.py", line 3 in <module>
  File "/usr/lib/python3/dist-packages/_pytest/assertion/rewrite.py", line 170 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1030 in _gcd_import
  File "/usr/lib/python3.9/importlib/__init__.py", line 127 in import_module
  File "/usr/lib/python3/dist-packages/_pytest/pathlib.py", line 520 in import_path
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 552 in _importtestmodule
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 484 in _getobj
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 288 in obj
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 500 in _inject_setup_module_fixture
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 487 in collect
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 324 in <lambda>
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 294 in from_call
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 324 in pytest_make_collect_report
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83 in <lambda>
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 441 in collect_one_node
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 768 in genitems
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 771 in genitems
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 568 in _perform_collect
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 516 in perform_collect
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 306 in pytest_collection
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83 in <lambda>
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 295 in _main
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 240 in wrap_session
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 289 in pytest_cmdline_main
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83 in <lambda>
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 157 in main
  File "/usr/local/lib/python3.9/dist-packages/ptr.py", line 220 in run_tests
  File "/usr/local/lib/python3.9/dist-packages/ptr.py", line 209 in run
  File "/usr/lib/python3.9/distutils/dist.py", line 985 in run_command
  File "/usr/lib/python3.9/distutils/dist.py", line 966 in run_commands
  File "/usr/lib/python3.9/distutils/core.py", line 148 in setup
  File "/usr/local/lib/python3.9/dist-packages/setuptools/__init__.py", line 163 in setup
  File "/sonic/src/sonic-platform-daemons/sonic-ycabled/setup.py", line 30 in <module>
Aborted (core dumped)
```

By GDB and get the error trace of protobuf like :
```
    at assert.c:92
    function=function@entry=0x7ffff7faa1c0 <__PRETTY_FUNCTION__.0> "__pthread_tpp_change_priority") at assert.c:101

```

Signed-off-by: Ze Gan <[email protected]>
@Pterosaur
Copy link
Author

Redesign the dependency with swss_common , So this change isn't needed.

@Pterosaur Pterosaur closed this Mar 11, 2024
@Pterosaur Pterosaur reopened this Mar 22, 2024
@Pterosaur
Copy link
Author

This issue is raised in a new requirement with sairedis.
@prgeor @vdahiya12 Could you please help to re-sign and merge it?

@prgeor
Copy link
Collaborator

prgeor commented Apr 17, 2024

Redesign the dependency with swss_common , So this change isn't needed.

@Pterosaur is this change needed?

@Pterosaur
Copy link
Author

Redesign the dependency with swss_common , So this change isn't needed.

@Pterosaur is this change needed?

Yes, It's needed in the vs platform again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants