Just as a reminder for myself the next time I need it, and for anyone else... using iOS:
After prebuilding the library file(s) as described above, we only want to keep the header files (.hpp and .h) in the cocos2d source, we don't need the .cpp etc any longer.
First copy these folders from cocos2d-x3.9/ to some other temporary new folder:
Then cd into that temp folder and run this:
find . -type f -not -name "*.hpp" -not -name "*.h" -not -name "*.inl" -delete
That line will find any file which does not end with .h or .hpp and delete those, recursively.
Now you can copy that header-only-folder to where you have your project. My setup is like this:
"mygamefolder" with subfolders like "Classes", "proj.ios_mac", "proj.linux" etc.
Put that temp folder from above next to Classes etc. You can rename it cocos2d-x-3.9 if that's the version you use.
Then add a user-defined Build Setting to the project in Xcode:
Value: ../cocos2d-x-3.9 (or wherever you copy the temp folder above with the subfolders that now only contain .h/.hpp files)
Because I do add the cocos2d header files also into the repository of each project.