Using RPATH for external libraries with CMake and Cocos2d-x

cocos2d-x C++
#1

I’ve encountered a serious roadblock trying to use Steam API (libsteam_api.so) with my Cocos2d-x 3.17 project when building on Linux.

According to the documentation supplied by Steam, I have to link steam_api library to my project as well as supply the .so file with the executable, but on Linux executable’s directory isn’t searched for shared libraries by default.

Searching various forums, I found out this is supposed to be solved setting “$ORIGIN” rpath during link time. This is where it gets problematic with Cocos2d-x.

When I run readelf -h on the executable built with Cocos2d-x 3.17 CMakeLists.txt, I noticed something very peculiar. Relevant output line is:

Type: DYN (Shared object file)

This is a bit puzzling to me, I expected to see:

Type: EXEC (Executable file)

This is not the most troubling thing as the file can still be executed just fine, but I would like to know if there’s some setting in Cocos’ CMakeLists.txt or anywhere else that I could change to make the output type EXEC.

I’ve tried various commands I found in the CMake docs when trying to make this work, among other things:

set(CMAKE_BUILD_RPATH_USE_ORIGIN true)

set_target_properties(${APP_NAME} PROPERTIES CMAKE_BUILD_RPATH "$ORIGIN")
set_target_properties(${APP_NAME} PROPERTIES CMAKE_INSTALL_RPATH "$ORIGIN")

None of them worked. Trying to run the app, I got following error message every time:
libsteam_api.so: cannot open shared object file: No such file or directory
And the relevant .so file was sitting in the same folder, next to the executable.

When trying to inspect the actual rpath used with readelf AppName -d | grep path, I got:

0x000000000000000f (RPATH) Library rpath: [/opt/cocos2d-x/external/linux-specific/fmod/prebuilt/64-bit]

Now this seems really strange to me, it appears some sort of Cocos2d-x’s inner workings already set this rpath (to accommodate libraries that cocos uses I guess?).

First question: I think I need to somehow append $ORIGIN to this path list: How?

Second (way more disturbing) question: If this rpath is set to some external library that cocos2d-x uses, the end-user of my application (who obviously isn’t going to have cocos2d-x engine installed on their machine) will most likely get an error message. How do I prevent this? How do I “bake-in” whatever cocos2d-x uses (in particular this fmod, whatever that is) into what I ship?

#2

For anyone interested, I’ve managed to “sort of” solve it with a following workaround in my project’s CMakeLists.txt:

if(LINUX)
    if(NOT EXISTS "${APP_BIN_DIR}/libs")
        add_custom_command(TARGET ${APP_NAME} PRE_BUILD
                            COMMAND ${CMAKE_COMMAND} -E make_directory
                            ${APP_BIN_DIR}/libs)
    endif()

    add_custom_command(TARGET ${APP_NAME} POST_BUILD
                        COMMAND patchelf --set-rpath '\$$ORIGIN/libs' ${APP_BIN_DIR}/${APP_NAME}
                        ${APP_BIN_DIR}/libs)

    add_custom_command(TARGET ${APP_NAME} PRE_BUILD
                        COMMAND ${CMAKE_COMMAND} -E copy_if_different
                        ${COCOS2DX_ROOT_PATH}/external/linux-specific/fmod/prebuilt/64-bit/libfmod.so
                        ${APP_BIN_DIR}/libs/libfmod.so)

    add_custom_command(TARGET ${APP_NAME} PRE_BUILD
                        COMMAND ${CMAKE_COMMAND} -E create_symlink
                        ${APP_BIN_DIR}/libs/libfmod.so
                        ${APP_BIN_DIR}/libs/libfmod.so.6)

    if(USE_STEAM_API)
        add_custom_command(TARGET ${APP_NAME} PRE_BUILD
                           COMMAND ${CMAKE_COMMAND} -E copy_if_different
                           ${CMAKE_CURRENT_SOURCE_DIR}/SteamAPI/redistributable_bin/linux64/libsteam_api.so
                           ${APP_BIN_DIR}/libs/libsteam_api.so)
    endif()
endif()

It feels very “hacky” to me though and not like it’s a proper way to deal with this problem.