1

I am trying to work with Nyquist (a music programming platform, see: https://www.cs.cmu.edu/~music/nyquist/ or https://www.audacityteam.org/about/nyquist/) as a standalone program and it utilizes libsndfile (a library for reading and writing sound, see: http://www.mega-nerd.com/libsndfile/). I am doing this on an i686 GNU/Linux machine (Gentoo).

After successful set up and launching the program without errors, I tried to generate sound via one of the examples, "(play (osc 60))", and was met with this error: 

*** Fatal error : sizeof (off_t) != sizeof (sf_count_t)
*** This means that libsndfile was not configured correctly.

Investigating this further (and emailing the author) has proved somewhat helpful, but the solution is still far from my grasp. The author recommended looking at /usr/include/sndfile.h to see how sf_count_t is defined, and (this portion of) my file is identical to his:

/* The following typedef is system specific and is defined when libsndfile is
** compiled. sf_count_t will be a 64 bit value when the underlying OS allows
** 64 bit file offsets.
** On windows, we need to allow the same header file to be compiler by both GCC
** and the Microsoft compiler.
*/

#if (defined (_MSCVER) || defined (_MSC_VER))
typedef __int64         sf_count_t ;
#define SF_COUNT_MAX          0x7fffffffffffffffi64
#else
typedef int64_t sf_count_t ;
#define SF_COUNT_MAX            0x7FFFFFFFFFFFFFFFLL
#endif

In the above the author notes there is no option for a "32 bit offset". I'm not sure how I would proceed. Here is the particular file the author of Nyquist recommend I investigate: https://github.com/erikd/libsndfile/blob/master/src/sndfile.h.in , and here is the entire source tree: https://github.com/erikd/libsndfile

Here are some relevant snippets from the authors email reply:

"I'm guessing sf_count_t must be showing up as 32-bit and you want libsndfile to use 64-bit file offsets. I use nyquist/nylsf which is a local copy of libsndfile sources -- it's more work keeping them up to date (and so they probably aren't) but it's a lot easier to build and test when you have a consistent library."

"I use CMake and nyquist/CMakeLists.txt to build nyquist."

"It may be that one 32-bit machines, the default sf_count_t is 32 bits, but I don't think Nyquist supports this option."

And here is the source code for Nyquist: http://svn.code.sf.net/p/nyquist/code/trunk/nyquist/

This problem is difficult for me to solve because it's composed of an niche use case of relatively obscure software. This also makes the support outlook for the problem a bit worrisome. I know a little C++, but I am far from confident in my ability to solve this. Thanks for reading and happy holidays to all. If you have any suggestions, even in terms of formatting or editing, please do not hesitate!

Matthieu
  • 2,736
  • 4
  • 57
  • 87
steve-blobs
  • 21
  • 4
  • Can you post what is `off_t` defined to on your machine? How is `sf_count_t` defined on your machine? (just include relevant headers and use [such trick](https://stackoverflow.com/questions/2160635/is-there-a-way-to-print-out-the-type-of-a-variable-pointer-in-c) to check it). What happens if you remove [the check](https://github.com/svn2github/nyquist/blob/master/nylsf/file_io.c#L541) from psf_open_fd and recompile? What if you just `typedef off_t sf_count_t;` and recompile? libsndfile has a [different check](https://github.com/erikd/libsndfile/blob/master/src/file_io.c#L529). – KamilCuk Dec 26 '18 at 23:53
  • Could you clarify whether a system libsndfile is present (from gentoo/portage) and how exactly you are trying to build nyquist? –  Dec 27 '18 at 00:09
  • @Kamil Cuk, I'm afraid I don't completely understand your request or the information in the link. In this file on line 19: https://github.com/svn2github/nyquist/blob/36e55e16da89d4ef87c361920bd16f34c06fb8f3/snd/wxsnd.h#L19" , off_t is defined as an int, and navigating to my own program where I cloned the source and compiled, it seems unchanged (I don't know how to get it's value though, if any). I know you want to know what value these variables have been assigned, and I am looking into getting this information in the mean time. I will attempt the relevant recompile momentarily. – steve-blobs Dec 27 '18 at 00:09
  • `off_t` is not supposed to be defined in the nyquist or libsndfile sources, it is part of the POSIX standard and defined in the libc. So it might also be helpful to know whether you are using glibc (gentoo default) or e.g. musl or uclibc. The linked definition of `off_t` is a replacement for non-POSIX systems. `off_t` and `sf_count_t` are both _types_ btw, not functions or variables. –  Dec 27 '18 at 00:18
  • @user10605163, this file and library is from Gentoo/Portage, yes. I was building Nyquist from source using ccmake, as per the instructions here: https://github.com/svn2github/nyquist/tree/36e55e16da89d4ef87c361920bd16f34c06fb8f3/sys/unix. As for your second comment, it seems I am using libc but also have glibc installed. And thanks for clarifying my improper use of the word functions, I will edit the OP to reflect this. – steve-blobs Dec 27 '18 at 00:57
  • @steve-blobs I have added another option based on the README in `nyquist/nylsf` that I overlooked. Also note that `glibc` (GNU C library) is one possible implementation of the C library (or `libc`). Having both of them is normal. –  Dec 27 '18 at 01:55

1 Answers1

4

If you look at the sources for the bundled libsndfile in nyquist, i.e. nylsf, then you see that sndfile.h is provided directly. It defines sf_count_t as a 64-bit integer.

The libsndfile sources however do not have this file, rather they have a sndfile.h.in. This is an input file for autoconf, which is a tool that will generate the proper header file from this template. It has currently the following definition for sf_count_t for linux systems (and had it since a while):

typedef @TYPEOF_SF_COUNT_T@ sf_count_t ;

The @TYPEOF_SF_COUNT_T@ would be replaced by autoconf to generate a header with a working type for sf_count_t for the system that is going to be build for. The header file provided by nyquist is therefore already configured (presumably for the system of the author).

off_t is a type specified by the POSIX standard and defined in the system's libc. Its size on a system using the GNU C library is 32bit if the system is 32bit.

This causes the sanity check in question to fail, because the sizes of sf_count_t and off_t don't match. The error message is also correct, as we are using an unfittingly configured sndfile.h for the build.

As I see it you have the following options:

  1. Ask the nyquist author to provide the unconfigured sndfile.h.in and to use autoconf to configure this file at build time.

  2. Do not use the bundled libsndfile and link against the system's one. (This requires some knowledge and work to change the build scripts and header files, maybe additional unexpected issues)

  3. If you are using the GNU C library (glibc): The preprocessor macro _FILE_OFFSET_BITS can be set to 64 to force the size of off_t and the rest of the file interface to use the 64bit versions even on 32bit systems.

    This may or may not work depending on whether your system supports it and it is not a clean solution as there may be additional misconfiguration of libsndfile going unnoticed. This flag could also introduce other interface changes that the code relies on, causing further build or runtime errors/vulnerabilities.

    Nonetheless, I think the syntax for cmake would be to add:

    add_compile_definitions(_FILE_OFFSET_BITS=64)

    or depending on cmake version:

    add_definitions(-D_FILE_OFFSET_BITS=64)

    in the appropriate CMakeLists.txt.

  4. Actually the README in nyquist/nylsf explains how the files for it were generated. You may try to obtain the source code of the same libsndfile version it is based on and repeat the steps given to produce an nylsf configured to your system. It may cause less further problems than 2. and 3. because there wouldn't be any version/interface changes introduced.

  • I will certainly follow up with the author, with all due credit to user10605163 of course, by asking for the aforementioned file and sharing any solution (he requested I follow up if it was resolved). Solution #2 also seems like a nice way to go about it and might lead to a more portable solution, but that's for another day if solution 3 will work. Anyway, running "$ ccmake ." in the directory with a freshly cloned repo and the add_compile_definitions line added end of file resulted in: "Unknown CMake Command "add_compile_definitions", so perhaps the syntax was off a bit? – steve-blobs Dec 27 '18 at 02:00
  • Following up on that, I think the desired syntax is add_definitions, seen here: https://cmake.org/cmake/help/v3.0/command/add_definitions.html. It passed ccmake like that, but gcc failed with: "gcc: error: _FILE_OFFSET_BITS=64: No such file or directory". I see now you've provided a solution 4 which seems a bit more elegant and I'll start taking a look at that. – steve-blobs Dec 27 '18 at 02:10
  • @steve-blobs The syntax on non-recent cmake versions is `add_definitions(-D_FILE_OFFSET_BITS=64)`, see the answers and edit to the accepted answer in https://stackoverflow.com/questions/9017573/define-preprocessor-macro-through-cmake I overlooked how recent the change was. –  Dec 27 '18 at 02:15
  • Do you have any idea how I'd find version 1.0.17 of libsndfile (the version mentioned in the README.txt)? – steve-blobs Dec 27 '18 at 02:19
  • I see, thank you for the syntax correction. I haven't figured out any of the solutions 1-4 yet but as soon as I do and confirm they work I'll mark it as accepted, thank you *very* much for the detailed and extremely helpful answer. I was shocked to get any help at all on this, let alone such quick, detailed and sufficiently verbose help. – steve-blobs Dec 27 '18 at 02:22
  • @steve-blobs There doesn't seem to be a release tag in the git-repo and the author's website doesn't seem to have a link to older releases, but I think you can easily guess the correct link name if you take the download link on the author's site and adjust it a bit to fit the version. There are also going to be mirrors of linux distributions out there that have the copy. –  Dec 27 '18 at 02:29
  • Solution #3 has worked, and I will work toward a more portable and elegant solution like the other ones you outlined. Thank you so much!! – steve-blobs Dec 27 '18 at 02:35