Creating a shared library that statically includes ffmpeg

Question

I'm having hard time trying to create a shared library that has ffmpeg libraries "baked in" as static ones.

Consider the following directory schema:

include/
  my own .h files
  ext/
    ffmpeg .h files
lib/
  libav*.a archive files (softlinks to the actual .a files)
  libValkka.so (my shared library)
test/
  mytest.cpp
bin/
  (binaries appear here)

I've come a long way (see Including objects to a shared library from a C++ archive (.a) ) and the library compiles ok with this: ([STUFF] has been omitted for brevity)

/usr/bin/c++ -fPIC -std=c++14 -pthread -Iinclude/ext -I/usr/include/libdrm -g -shared -Wl,-soname,libValkka.so -o lib/libValkka.so CMakeFiles/Valkka.dir/src/avthread.cpp.o CMakeFiles/Valkka.dir/src/opengl.cpp.o CMakeFiles/Valkka.dir/src/openglthread.cpp.o [STUFF] CMakeFiles/Valkka.dir/src/filters.cpp.o -lX11 -lGLEW -lGLU -lGL -Wl,--allow-multiple-definition -Wl,-Bsymbolic -Wl,--whole-archive -Wreorder lib/libavdevice.a lib/libavfilter.a lib/libavformat.a lib/libavcodec.a lib/libavutil.a lib/libswscale.a lib/libswresample.a -Wl,--no-whole-archive

However, when creating executables - their source code does not use any ffmpeg api (just my own api) - with:

c++ -std=c++14 -pthread -Iinclude -Iinclude/ext -Llib test/mytest.cpp -lValkka -g -o bin/mytest

I get a hoard of errors about missing ffmpeg dependencies. Not everything is missing, just some weird stuff:

lib/libValkka.so: undefined reference to `pa_stream_get_index'
lib/libValkka.so: undefined reference to `deflateInit_'
lib/libValkka.so: undefined reference to `pa_stream_get_state'
lib/libValkka.so: undefined reference to `lzma_stream_decoder'
lib/libValkka.so: undefined reference to `BZ2_bzDecompress'
lib/libValkka.so: undefined reference to `vaInitialize'
lib/libValkka.so: undefined reference to `pa_stream_unref'
lib/libValkka.so: undefined reference to `deflateInit2_'
lib/libValkka.so: undefined reference to `snd_pcm_close'
...
lib/libValkka.so: undefined reference to `vaGetDisplayDRM'
lib/libValkka.so: undefined reference to `vaMaxNumEntrypoints'
lib/libValkka.so: undefined reference to `uncompress'
lib/libValkka.so: undefined reference to `pa_stream_drop'
lib/libValkka.so: undefined reference to `pa_context_connect'
lib/libValkka.so: undefined reference to `FT_Get_Kerning'
lib/libValkka.so: undefined reference to `ass_free_track'
lib/libValkka.so: undefined reference to `pa_operation_unref'
lib/libValkka.so: undefined reference to `FT_Stroker_Done'
lib/libValkka.so: undefined reference to `vaTerminate'
lib/libValkka.so: undefined reference to `ass_new_track'
lib/libValkka.so: undefined reference to `jack_client_close'
...
lib/libValkka.so: undefined reference to `xcb_xfixes_query_version'
lib/libValkka.so: undefined reference to `xcb_shape_rectangles'
lib/libValkka.so: undefined reference to `pa_mainloop_free'
lib/libValkka.so: undefined reference to `snd_device_name_hint'
lib/libValkka.so: undefined reference to `vaCreateImage'
lib/libValkka.so: undefined reference to `vaBeginPicture'
lib/libValkka.so: undefined reference to `DtsSetColorSpace'
lib/libValkka.so: undefined reference to `vaDestroyConfig'
lib/libValkka.so: undefined reference to `pa_stream_writable_size'
lib/libValkka.so: undefined reference to `snd_pcm_hw_params_get_buffer_size_max'
lib/libValkka.so: undefined reference to `ass_read_file'

This is pretty frustrating, especially when I can see that those names are included in the shared library..!

nm lib/libValkka.so | grep "vaBeginPicture"

gives

U vaBeginPicture

etc. I thought it might be a problem regarding the dependency order the archive .a files, and also tried with:

..... -Wl,--allow-multiple-definition -Wl,-Bsymbolic -Wl,--start-group -Wl,--whole-archive -Wreorder lib/libavdevice.a lib/libavfilter.a lib/libavformat.a lib/libavcodec.a lib/libavutil.a lib/libswscale.a lib/libswresample.a -Wl,--no-whole-archive -Wl,--end-group

But the problem persists.

I have succesfully created a shared library that does not "bake in" those .a archives, i.e. that just depends dynamically on ffmpeg libraries, and there are no such problems.

I am baffled.. Have I misunderstood something fundamental, forgot some annoying linked option, or both? Help appreciated!

score 3 · Accepted Answer · answered Oct 08 '17 at 20:35

3

You need to link your shared library with 3rd party/system libraries required by ffmpeg: libbz2, libva, libxcb, libass, freetype2 etc. Actual list should be somewhere in ffmpeg distribution/build artifacts (automake's .pc files)

ignore-all is not a good idea; your application might run OK but these unresolved items are still there; it will crash as soon as it hits any of them. My guess is that most of them won't be hit, ever, since they are for libavdevice which you might not be even using, but still a bad idea. Also, check if you actually need that libavdevice library - you might trim a list of required libraries quite a bit if you get rid of that one.

answered Oct 08 '17 at 20:35

Andrey Turkin

1,787
9
18

Tried removing libavdevice, but it didn't resolve the problem. So, any reference from the statically compiled ".a" archives (baked into my shared library) to another library will be broken then..? I.e., the "dependency chain" will be broken, in general, always when there is a static library in the chain? Here is some discussion .. https://stackoverflow.com/questions/7841920/how-do-static-libraries-do-linking-to-dependencies .. any tip on ffmpeg's configure script to create a library with (almost) no dependencies? – El Sampsa Oct 09 '17 at 07:46
ffmpeg as a static library has a lot of dependencies on other libraries which are not explicitly present in the archive files; it's the case for any static library with external references. Guess this is what you meant by broken dependency chain? Anyway, you need to link your shared library with those dependencies manually (or use automake/cmake/whatever build system which can do it for you). Removing libavdevice won't remove all those dependencies but it will remove some (at least to jack, oss, pulseaudio). You can build ffmpeg without most of the libraries but it will be pretty minimal build – Andrey Turkin Oct 09 '17 at 10:25
So when creating my shared library, I have to put there all those "-L" and "-l" switches that link to, say, pulseaudio, oss, vdpau-thingies, etc.? Ok.. thx! – El Sampsa Oct 09 '17 at 13:43

El Sampsa · Answer 2 · 2017-10-10T05:57:13.450

Telling linker to ignore unresolved symbols when creating the executable does the trick:

c++ -std=c++14 -pthread -Iinclude -Iinclude/ext -Llib test/mytest.cpp -lValkka -g -o bin/mytest -Wl,--unresolved-symbols=ignore-all

The resulting executable also runs OK.

However.. using such a linker flag rubs me the wrong way. Maybe there is a better option? And why those symbols are not found in the first place?

EDIT

Following Andrey's suggestion, I stripped all external libraries from ffmpeg, with the aid of ffmpeg's configure script. That was a bit awkward process, so I created a nice python script that does it automagically. It might be a bit of an overkill, but here it goes:

#!/usr/bin/python3
"""
* Creates script "run_configure.bash" that launches ffmpeg's "configure" script with correct parameters (enabling/disabling stuff)
* Run in the same directory where you have ffmpeg's configure script
"""
import subprocess
import os
import re

def features(switch, adstring="", remove=[]):
  p=subprocess.Popen(["./configure",switch],stdout=subprocess.PIPE)
  st=p.stdout.read()
  fst=""
  for s in st.split():
    ss=s.decode("utf-8")
    ok=True
    for rem in remove:
      if (ss.find(rem)!=-1):
        ok=False
    if ok: fst+=adstring+ss+" "
  return fst


def disable_external():  
  p=subprocess.Popen(["./configure","-h"],stdout=subprocess.PIPE)
  st=p.stdout.read().decode("utf-8")

  # find some text tags from the configure output:
  # i1=st.find("External library support:")
  i1=st.find("themselves, not all their features will necessarily be usable by FFmpeg.")
  i2=st.find("Toolchain options:")
  st=st[i1:i2]
  """ # debugging ..
  print(st)
  stop
  """
  p=re.compile('--(enable|disable)-(\S*)')
  switches=[]
  for sw in p.findall(st):
    if (sw[1] not in switches):
      # print(sw[1]) # debugging
      switches.append(sw[1])
  fst=""
  for sw in switches:
    fst+="--disable-"+sw+" "
  return fst

st ="./configure "
st+="--disable-everything --disable-doc --disable-gpl --disable-pthreads --enable-static --enable-shared "
st+= disable_external()
st+= features("--list-decoders",adstring="--enable-decoder=", remove=["vdpau","crystalhd","zlib"])
st+= features("--list-muxers",  adstring="--enable-muxer=")
st+= features("--list-demuxers",adstring="--enable-demuxer=")
st+= features("--list-parsers", adstring="--enable-parser=")

f=open("run_configure.bash","w")
f.write("#!/bin/bash\n")
f.write(st+"\n")
f.close()
os.system("chmod a+x run_configure.bash")
print("\nNext run ./run_configure.bash\n")

"""
For cleaning up .a and .so files, use
find -name *.a -exec ls {} \;
find -name *.so* -exec ls {} \;
"""

Hopefully someone finds it useful. Run with python3.

Creating a shared library that statically includes ffmpeg

2 Answers2

Linked