[question] CMakeDeps and inter-package dependencies when using shared libraries

[bukulin]

What is your question?

Hi!

I ran into a problem with CMakeDeps generator when shared libraries are used and the consuming project has a unit-test-like application. Trying to describe the situation as briefly as I could, but I have to dive deep. This might be due to my misunderstanding of the advanced dependency handling, but I try to do my best.

So let's consider the following things:

• conan/2 is used: 2.7
• every library is used as shared (*:shared=True)
• the dependency graph is the following:

  graph LR;
  rest{{deps. of log4cxx}};
  game-->engine;
  engine-->log4cxx;
  log4cxx-->rest;
  

  Loading
  Names borrowed from here: https://www.youtube.com/watch?v=kKGglzm5ous . math substituted with log4cxx to have some more dependencies.

Everything works correctly as described in the aforementioned video until I add a unit test to the engine package, just like that:

engine/CMakeLists.txt:

cmake_minimum_required(VERSION 3.15)
project(engine CXX)

include(CTest)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_VERBOSE_MAKEFILE ON)

find_package(log4cxx REQUIRED CONFIG)

add_library(engine src/engine.cpp)
target_include_directories(engine PUBLIC include)

target_link_libraries(engine PRIVATE log4cxx)

set_target_properties(engine PROPERTIES PUBLIC_HEADER "include/engine.h")
install(TARGETS engine)

if (BUILD_TESTING)
  add_subdirectory(test)
endif()

engine/test/CMakeLists.txt:

add_executable(engine_test
  test_main.cpp)

target_link_libraries(engine_test
  PRIVATE engine )

add_test(NAME engine_test
  COMMAND engine_test )

So, engine depends privately on log4cxx and engine_test depends on engine, in turn.
At this point when I try to compile the whole engine package, the link step of engine_test fails:

/usr/bin/c++ -m64 -O3 -DNDEBUG -m64 CMakeFiles/engine_test.dir/test_main.cpp.o -o engine_test  -Wl,-rpath,/home/user/some/path/engine/build/Release ../libengine.so 
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libaprutil-1.so.0, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libapr-1.so.0, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libiconv.so.2, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libcharset.so.1, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15: undefined reference to `apr_socket_connect'
...

Interestingly liblog4cxx.so.15 is found but its dependencies not. The linker command line lacks here the library paths (-L) (and RPATH definitions (-Wl,--rpath)) of the transitive dependencies of engine, however they were present when engine library was built:

/usr/bin/c++ -fPIC -m64 -O3 -DNDEBUG -m64 -shared -Wl,-soname,libengine.so -o libengine.so CMakeFiles/engine.dir/src/engine.cpp.o   -L/home/nbukuli/.conan2/p/b/log4c3762b28bed3ae/p/lib  -L/home/nbukuli/.conan2/p/apr-u46962cbae52f2/p/lib  -L/home/nbukuli/.conan2/p/apr-u46962cbae52f2/
p/lib/apr-util-1  -L/home/nbukuli/.conan2/p/apr544b7ac059c68/p/lib  -L/home/nbukuli/.conan2/p/libic63a94c7c34484/p/lib  -L/home/nbukuli/.conan2/p/b/expat439bcb6038711/p/lib  -Wl,-rpath,/home/nbukuli/.conan2/p/b/log4c3762b28bed3ae/p/lib:/home/nbukuli/.conan2/p/apr-u46962cbae52f2/p/li
b:/home/nbukuli/.conan2/p/apr-u46962cbae52f2/p/lib/apr-util-1:/home/nbukuli/.conan2/p/apr544b7ac059c68/p/lib:/home/nbukuli/.conan2/p/libic63a94c7c34484/p/lib:/home/nbukuli/.conan2/p/b/expat439bcb6038711/p/lib: /home/nbukuli/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so -lm 

When engine is built its dependencies are all come from the config files generated by CMakeDeps however dependencies of engine_test handled internally by CMake.

Everything works well, if *:shared=False, because in that case every static library is listed in the linker commandline when linking engine_test.

This issue might be related to this one: #16911

What do I wrong? If it is possible, I try to use modern CMake and rely on target_link_libraries with local targets in this case. I suppose a workaround could be written, where I query all the link-time dependencies of engine and apply them to engine_test but I try to avoid that.

Any guidance will be highly appreciated. Unfortunately I cannot find corresponding points in the documentation.
Thank you in advance!

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[bukulin]

As a possible workaround log4cxx could be linked with LINK_ONLY to engine_test:

target_link_libraries(engine_test
  PRIVATE
  engine
  $<LINK_ONLY:log4cxx>
)

That solves the linking problems, but something weird happens at the same time. Despite the LINK_ONLY, include paths of log4cxx appear in the compiler commandline with -isystem:

/usr/bin/c++  -I/home/user/some/path/engine/include -isystem /home/user/.conan2/p/b/log4c3762b28bed3ae/p/include -m64 -O3 -DNDEBUG -std=gnu++14 -MD -MT test/CMakeFiles/engine_test.dir/test_main.cpp.o -MF CMakeFiles/engine_test.dir/test_main.cpp.o.d -o CMakeFiles/engine_test.dir/test_main.cpp.o -c /home/user/some/path/engine/test/test_main.cpp

If I link apr::apr or apr-util::apr-util or even expat::expat with LINK_ONLY the compiler commandline remains intact.

Recipe of log4cxx is a bit suspicious now.

[memsharded]

Hi @bukulin

Thanks for your question and thanks specially for your detailed report.

Can you please do a quick check? It would be in your test CMakeLists.txt:

target_link_libraries(engine_test
  PUBLIC engine )

This shouldn't have a bad impact, being the engine_test and executable that is later no further reused by anyone, the way it links engine shouldn't affect.

If this work, this is kind of a known issue, that happens in Linux like shared libraries (it doesn't happen in Windows shared libraries). It is kind of challenging to fix it in CMakeDeps at the moment with the current information we have, but we are working to improve and expand that information and planning an improved CMakeDeps generator (it will take some time, there are still other higher priorities)

#16911

I think this was a very specific issue of the grpc recipe, and the way it works with regards to generated code having direct dependencies to other transitive dependencies, so it doesn't seem related.

[bukulin]

Hi @memsharded,

Thank you for the quick reply!

target_link_libraries(engine_test
  PUBLIC engine )

Unfortunately the public dependency did not help here, I have got the same linker error and the same linker commandline.

However, if engine depends publicly on log4cxx:

target_link_libraries(engine PUBLIC log4cxx)

, then the link step of engine_test works, but that is unfortunately not acceptable. Private dependency required there.

---

it will take some time, there are still other higher priorities

Thank you for the effort. We can live with a workaround that does not harm the required dependency chain.

---

#16911
Understood, thank you.

[memsharded]

Maybe this is related to the package_type of the different packages involved.
Could you please share your engine conanfile.py? Does it declare package_type or shared option?

Is the log4cxx package the one from ConanCenter as-is? Or is it your own custom package?

[bukulin]

engine is created with the following commands:
conan new -d name=engine -d version=1.0.0 cmake_lib
thus, the package_type is library and a shared option is defined.

The conanfile.py of engine:

from conan import ConanFile
from conan.tools.cmake import CMakeToolchain, CMake, cmake_layout, CMakeDeps


class engineRecipe(ConanFile):
    name = "engine"
    version = "1.0.0"
    package_type = "library"

    # Optional metadata
    license = "MIT"
    author = "Norbert Bukuli"
    url = "-"
    description = "middle engine library"

    # Binary configuration
    settings = "os", "compiler", "build_type", "arch"
    options = {
        "shared": [True, False],
        "fPIC": [True, False]
    }
    default_options = {
        "shared": False,
        "fPIC": True
    }

    # Sources are located in the same place as this recipe, copy them to the recipe
    exports_sources = "CMakeLists.txt", "src/*", "include/*"

    def requirements(self):
        self.requires("log4cxx/1.2.0")

    def config_options(self):
        if self.settings.os == "Windows":
            self.options.rm_safe("fPIC")

    def configure(self):
        if self.options.shared:
            self.options.rm_safe("fPIC")

    def layout(self):
        cmake_layout(self)

    def generate(self):
        deps = CMakeDeps(self)
        deps.generate()
        tc = CMakeToolchain(self)
        tc.generate()

    def build(self):
        cmake = CMake(self)
        cmake.configure(cli_args=["--graphviz", "deps.dot"])
        cmake.build()
        cmake.ctest(cli_args=["--output-on-failure"])

    def package(self):
        cmake = CMake(self)
        cmake.install()

    def package_info(self):
        self.cpp_info.libs = ["engine"]

---

log4cxx is used from ConanCenter:
log4cxx/1.2.0#95be70abf50d3c18e43b56697a2acdd5:60d416a34bb3a6d305ac643316beb9dea8dcd7be#ab45281aded77ea3bbe27b7af61f0f3b