sclite (SCTK) install, file not recognized, file format not recognized, Cygwin

Question

I am currently trying to install NIST's sclite, which is part of SCTK 2.4.0 (github or current version). I am attempting the install on Cygwin in bash. The installation is done using make.

What I've Done

I made a directory for the install and navigated to that directory

mkdir sctk2.4.0
cd sctk2.4.0

( You'll possibly need $ cd /path/to/dir/sctk2.4.0 .)

I cloned the project from github

git clone https://github.com/chinshr/sctk.git

navigated into the base folder

cd sctk

then I started following the instructions in the INSTALL file.

Running

make config

worked fine, but after typing

make all

I got the output that follows

(mkdir -p bin)
(cd src; make all)
make[1]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src'
(cd asclite; make all)
make[2]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite'
(cd core; make all)
make[3]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite/core'
g++ -o asclite -g -Os  alignment.o segment.o sgml_reportgenerator.o alignedsegmentiterator.o reportgenerator.o speechset.o segmentsgroup.o logger.o tokenalignment.o sgml_generic_reportgenerator.o recording.o statistics.o compressedlevenshteinmatrix.o segmentor.o id.o trntrn_segmentor.o linestyle_inputparser.o inputparser.o levenshteinmatrix.o levenshtein.o uemfilter.o speakermatch.o spkrautooverlap.o graphalignedsegment.o rawsys_reportgenerator.o graphalignedtoken.o timedobject.o stt_scorer.o aligner.o arraylevenshteinmatrix.o graph.o main.o trn_inputparser.o alignedspeech.o token.o alignedsegment.o graph_coordinate.o rttm_inputparser.o scorer.o properties.o ctmstmrttm_segmentor.o filter.o speech.o alignedspeechiterator.o stm_inputparser.o checker.o ctm_inputparser.o lzma/LzFind.o lzma/LzmaEnc.o lzma/Alloc.o lzma/LzmaLib.o lzma/LzmaDec.o -lm
alignment.o: file not recognized: File format not recognized
collect2: error: ld returned 1 exit status
make[3]: *** [makefile:62: asclite] Error 1
make[3]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite/core'
make[2]: *** [makefile:12: all] Error 2
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite'
make[2]: *** [makefile:12: all] Error 2
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src'
make: *** [makefile:20: all] Error 2

I've looked at this SO post, but I've determined that the alignment.o file is not corrupted. Just in case, I tried a few make clean and even re-cloned the project from github, but I still get the same error.

Can anyone help me complete this installation, or at least get to the next step?

System Details

$ uname -a
CYGWIN_NT-6.1 MyMachine 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin) ...
$ gcc --version
gcc (GCC) 6.4.0 ...
$ g++ --version
g++ (GCC) 6.4.0 ...
$ make --version
GNU Make 4.2.1
Built for x86_64-unknown-cygwin ...
$ systeminfo | sed -n 's/^OS\ *//p'
Name:                   Microsoft Windows 7 Enterprise
Version:                6.1.7601 Service Pack 1 Build 7601
Manufacturer:           Microsoft Corporation
Configuration:          Member Workstation
Build Type:             Multiprocessor Free

Note

I'm asking about this problem and then giving an answer to my own question. (I like that StackOverflow is allowing us to do that.) Hopefully, this will make it easier for people to help me with the problems I ran into further in the installation.

The next problem I ran into is discussed here. You can see this next problem in the answer to this problem.

There is a way that I think is easier. Basically, you install the `tools` part of [kaldi](http://kaldi-asr.org/). You can also install the `src` part and use it if you want (unless you're on Cygwin under Windows 7 as I am, in which case there are more problems.) If you just want the `sclite` stuff, copy `/path/to/kaldi/tools/sctk-2.4.10/*` to a place that's not inside the `kaldi` tree. I did `mkdir $HOME/sctk-2.4.10 && cp /path/to/kaldi/tools/sctk-2.4.10/* $HOME/sctk2.4.10/`. I'll probably elaborate later. — bballdave025, Jun 06 '18 at 21:52
Note that the fixes described here and in the other solutions to which I've linked ([link2](https://stackoverflow.com/q/50243691/6505499), [link3](https://stackoverflow.com/q/50283077/6505499)) are also needed if one uses a more current version of sclite available at jaguar.ncsl.nist.gov/pub/sctk-2.4.10-20151007-1312Z.tar.bz2 — bballdave025, Jun 06 '18 at 21:55
That more-current-version link might change, so here's the hosting site: [https://www.nist.gov/itl/iad/mig/tools](https://www.nist.gov/itl/iad/mig/tools). — bballdave025, Jul 16 '18 at 17:23

bballdave025 · Answer 1 · 2023-03-20T21:05:49.063

This is the 'EASIER' solution.

Here are the details on what I called "the kaldi solution". Right now, it's just a list of commands without details. As shown here, these commands will install a sclite-2.4.10 directory under the $HOME (~) directory:

$ cd
$ git clone https://github.com/kaldi-asr/kaldi.git
$ cd kaldi/tools
$ extras/check_dependencies.sh
$ make -j $(nproc --all)
$ cp -R sctk-2.4.10 ~/
$ cd
$ rm -rf kaldi
$ cd sctk-2.4.10/
$ cp $HOME/.bashrc "${HOME}/.bashrc.$(date +%s_%Y%m%dT%H%M%S).bak"
$ echo -e "\n\n## Allow access to sclite, rfilter, etc" >> $HOME/.bashrc
$ echo 'export PATH='"$(pwd)/bin"':$PATH' >> $HOME/.bashrc
$ source ~/.bashrc

See this question/answer for details on how to use it on Windows.

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

AN EVEN EASIER SOLUTION - Taking advantage of a new, edited version of sclite

This is in case someone finds the answer here useful. I know there are no votes here, but I just got a trophy telling me this is my first question to get 1000 views, so I'll update my answer to show the easiest way to get things done.

TL;DR

https://www.nist.gov/itl/iad/mig/tools

https://github.com/usnistgov/SCTK

% cd /the/dir/where/i/want/to/install
% git clone https://github.com/usnistgov/SCTK.git
% cd SCTK

From the git master README (quoted) with some comments I've put in.

% make config

% sed -i 's#[-]Os#-m64#g' src/asclite/core/makefile

% sed -i 's#[-]Os#-m64#g' src/asclite/test/makefile

% sed -i 's#[-]Os#-m64#g' src/sclite/makefile

% make clean

% ## Possible edit to the `rfilter1 makefile`, which are
% ## described at the end of the answer but were not necessary
% ## for me.

% make all

% make check

% make install

% make doc

I also add the executables' directory to my PATH and make the documentation available via the man command.

% pwd
/the/dir/where/i/want/to/install/SCTK
% # back up your `.bashrc`
% cp $HOME/.bashrc "${HOME}/.bashrc.$(date +%Y%m%d-%H%M%S).bak"
% # persistent path changes
% echo -e "\n\n## Allow access to sclite, rfilter, etc" >> $HOME/.bashrc
% # your machine might use something other than `export` for this. CHECK!
% echo 'export PATH='"$(pwd)/bin"':$PATH' >> $HOME/.bashrc
% # make changes availabel this session
% source ~/.bashrc
% # man stuff
% cd doc
% cp -r ./* /usr/man/man1

END OF THE TL;DR SECTION

Details

Since when I posted this in May 2018, there have finally been some updates made to the software:

I posted my question in May 2018, and the updates were made in Fall 2018. They partly fix the problems I ran into here, but some of the information in the README and some makefiles are useful to note here.

The 32- to 64- bit issue (changing -Os to -m64, as done above) was find-able via the README.

% cat -n README.md | grep -A 4 "64 bits Compilation"
    61  **64 bits Compilation**:
    62  With big alignments, sctk needs to be compiled in 64 bits. By default, the C/C++ software are compiled in 32 bits with the options (`-Os`) but can be compiled in 64 bits. To do so, `-m64` is added to the CFLAGS variable in `src/asclite/core/makefile`, `src/asclite/test/makefile`, `src/rfilter1/makefile` and `src/sclite/makefile`.
    63
    64  Example of `CFLAGS` for OSX 10.4+: `-fast -m64 -arch x86_64 -arch ppc64`
    65

Here is line 62 with word wrap

With big alignments, sctk needs to be compiled in 64 bits. By default, the C/C++ software are compiled in 32 bits with the options (-Os) but can be compiled in 64 bits. To do so, -m64 is added to the CFLAGS variable in src/asclite/core/makefile, src/asclite/test/makefile, src/rfilter1/makefile and src/sclite/makefile.

Since there was no -Os in src/rfilter1/makefile, I didn't make any changes.

I was able to finish the installation with no problem (including no failed tests). Here is my system info.

$ uname -a
CYGWIN_NT-10.0 MyMachine 3.0.7(0.338/5/3) 2019-04-30 18:08 x86_64 Cygwin
$ bash --version | head -n 1
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
$ gcc --version | head -n 1
gcc (GCC) 7.4.0
$ g++ --version | head -n 1
g++ (GCC) 7.4.0
$ make --version | head -n 2
GNU Make 4.2.1
Built for x86_64-unknown-cygwin
$ systeminfo | sed -n 's/^OS\ *//p'
Name:                   Microsoft Windows 10 Enterprise
Version:                10.0.17134 N/A Build 17134
Manufacturer:           Microsoft Corporation
Configuration:          Member Workstation
Build Type:             Multiprocessor Free

However, it seems that some other people trying to compile on Cygwin have had issues. Here is some more info from the README

% grep "Special Note to Cygwin users" README.md
*Special Note to Cygwin users:* it has been reported that compilation of `rfilter1` can fail in some case, please read the OPTIONS part of the `rfilter1/makefile` and adapt accordingly before retrying compilation.

Well, let's look at the makefile for rfilter1, and see what some of you might need to do.

% head -n 15 src/rfilter1/makefile | tail -7
###########################   Options for compilation  #########################
#######  If you have an very new version of GCC, the strcmp* family of functions
#######  is included in the distribution.  Changing the value of OPTIONS to
#######  be blank will diable the use of supplied versions of these functions.
#######  In particular, this behavior has been noted on some versions of cygwin
OPTIONS=-DNEED_STRCMP=1 $(CPPFLAGS) $(CFLAGS) $(LDFLAGS)

So, if you have rfilter1 compilation problems, change the non-commented line to

OPTIONS= $(CPPFLAGS) $(CFLAGS) $(LDFLAGS)

bballdave025 · Answer 3 · 2018-07-16T17:15:08.683

(See my comment under the question for the kaldi solution.)

The solution to this problem was in the README, as solutions often are. Note: There was another problem which came up after this problem was solved. See the bottom of this answer for help with that.

Here is the command I used to get the pertinent info from the README.

cat README | tail -13

and here is the pertinent info


                       64 bits Compilation

With big alignments, sctk needs to be compiled in 64 bits.
By default, the C/C++ software are compiled in 32 bits with the options (-Os)
but can be compiled in 64 bits, -m64 is added to the CFLAGS variable in:
src/asclite/core/makefile
src/asclite/test/makefile
src/rfilter1/makefile
src/sclite/makefile
Example of CFLAGS:
For OSX 10.4+: -fast -m64 -arch x86_64 -arch ppc64

So, I went to the makefiles listed (except rfilter1, see below) and changed the code there, replacing each -Os with -m64. Do this ONLY for the makefiles that are listed. I'll give an example for one of the listed files, but note you will have to do it for the others.

cd sctk
vim src/asclite/core/makefile

When the file was open, I found the line:

CFLAGS = -g -Os

which I changed to

CFLAGS = -g -m64

(pressed "i" to get into INSERT mode, made the change, pressed "Esc", then pressed ":wq" (Write and Quit) followed by "Enter")

I made the changes in all the listed files EXCEPT src/rfilter1/makefile, because that file had no -Os in it. This ended up being important, as the install wouldn't work if I had changed this file at all.

After this was completed, I ran make clean, but I DID NOT run make config, because this would have undone the changes we just make. I went directly to:

make all

This gets us past where we were before.

This problem was taken care of, but there was another problem:

In file included from main.cpp:20:0:
recording.h:122:36: error: template argument 2 is invalid
         map<string, Filter::Filter*> filters;
                                    ^
recording.h:122:36: error: template argument 4 is invalid

which I asked about here.

There is more about 32-bit and 64-bit stuff in Cygwin [here](https://stackoverflow.com/q/44477951/6505499). — bballdave025, May 10 '18 at 18:22