HTK - What do MFCCs of an HMM model and Input WAV File represent?

Question

While creating MFCCs following Voxforge's tutorial for a Speech to Text System using HTK (Hidden Markov Model Tool Kit), we are required to define a prototype model for our phones. I am trying to wrap my head around this this file.

~o <VecSize> 25 <MFCC_0_D_N_Z>
~h "proto"
<BeginHMM>
  <NumStates> 5
  <State> 2
    <Mean> 25
      0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0  
    <Variance> 25
      1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 
 <State> 3
    <Mean> 25
      0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0  
    <Variance> 25
      1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
 <State> 4
    <Mean> 25
      0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
    <Variance> 25
      1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
 <TransP> 5
  0.0 1.0 0.0 0.0 0.0
  0.0 0.6 0.4 0.0 0.0
  0.0 0.0 0.6 0.4 0.0
  0.0 0.0 0.0 0.7 0.3
  0.0 0.0 0.0 0.0 0.0
<EndHMM>

In this case, we are using a feature vector of Length 25 to represent every state of the HMM. However, I don't quite understand why we have 25 "Means" and "Variances" for every state. Do they represent the Mean and Variance of every Feature Vector?

Furthermore, why Do we have 3 states when is 5? Are <State>1 and <State>5 simply entry and exit points so they do not require a Mean and Variance?

Also, while taking sample wav files, I printed the MFCCs which displayed as below:

  0:     -15.769  -2.168   8.605   4.979   5.283   1.012   9.631  -0.619   3.622  10.977
             5.733   3.260  44.447  -0.153  -0.281  -0.810  -1.176   0.363  -0.658   0.676
            -1.569   1.363  -1.221   0.815  -0.759   1.427
    1:     -18.345  -3.220   7.177   0.293   7.232   3.111  17.942  -6.957   8.197   6.579
             9.102  -0.569  49.537   0.378  -0.337  -1.277  -1.709   0.623  -0.450   0.162
             0.315   2.088  -1.175   0.624   0.762   1.018
    2:     -15.244  -3.046   5.269   1.441   6.121  -3.326   8.854  -5.297   8.151   7.072
             8.122   1.379  49.036   0.543  -0.119  -1.162  -1.263   1.261  -0.388  -0.234
             0.816   1.195  -1.237  -0.288   1.600   0.244
    3:     -14.143  -3.413   3.887  -1.796   7.981   0.930  10.826   3.294  11.797   7.055
             7.661   8.011  47.243   0.613  -0.020  -0.568  -0.364   1.034  -0.165  -0.812
             2.525   0.351  -1.670  -1.086   1.493  -0.716
    4:     -15.156  -2.669   4.440  -0.293  11.213   0.162  12.020  -1.667   7.794   4.553
             5.013   6.968  46.813  -0.050  -0.092  -0.050  -0.329   0.325   0.585   0.751
             1.253  -0.008  -1.852  -0.845   0.058  -0.430
    5:     -15.323  -3.510   4.750  -0.660   9.856   0.545  12.301   3.855  10.132  -0.511
             5.224   4.104  47.068   0.073   0.151   0.163  -0.180  -0.186  -0.242  -0.335
            -0.577  -0.479  -0.745  -0.167  -1.565   0.013

For every "window", why do we have 26 coeffieincts instead of 25? What do they all represent? I believe:

1-12 are Cepstral Coeffiecients
14-25 are Delta Coefficients
26 is also a Delta Coeffieienct for the 13th number

But I have no idea what 13th number in each of these samples represent. They should be of the format <MFCC_0_D_N_Z> as defined in the prototype file displayed in the beginning, which is not explained well in the HTK Manual. But I can garner from page 80 of the Manual that :

MFCC_0 : MFCC Coefficients
_D : Delta Coefficients
_N : absolute Energy Suppressed
_Z : has Zero Mean Static Coef.

Any explanations would be appreciated.

score 2 · Accepted Answer · answered May 24 '17 at 07:45

Furthermore, why Do we have 3 states when is 5? Are 1 and 5 simply entry and exit points so they do not require a Mean and Variance?

Yes, boundary states are dummy.

For every "window", why do we have 26 coeffieincts instead of 25? What do they all represent? I believe:

MFCC type is MFCC_0_D as in Tutorial step 5, so those are 13 ceps and 13 deltas. You can also use HList -o -h to print the exact layout:

---------------------------------- Source: ar-03.mfc -----------------------------------
  Sample Bytes:  52       Sample Kind:   MFCC_D_C_K_0
  Num Comps:     26       Sample Period: 10000.0 us
  Num Samples:   648      File Format:   HTK
-------------------------------- Observation Structure ---------------------------------
x:      MFCC-1  MFCC-2  MFCC-3  MFCC-4  MFCC-5  MFCC-6  MFCC-7  MFCC-8  MFCC-9 MFCC-10
       MFCC-11 MFCC-12      C0   Del-1   Del-2   Del-3   Del-4   Del-5   Del-6   Del-7
         Del-8   Del-9  Del-10  Del-11  Del-12   DelC0

The type of features stored in mfc file might differ from the type of features used in HMM training, the HMM features are computed from mfc on the fly according to the proto specification, so on the disk you have 26 MFCC_0_D and when you compute you convert it to 25 coefficients MFCC_0_D_N_Z by dropping the energy and normalizing the mean.

I don't quite understand why we have 25 "Means" and "Variances" for every state. Do they represent the Mean and Variance of every Feature Vector?

Means and variances are gaussian parameters of the HMM emitting distribution for every HMM state, they are not the mean of feature vector. Check what HMM is.

Oh Okay. In the 26 coefficients, the 13th is `C0`. What does `C0` represent? And why is this number much higher than the MFCCs? — Ajay H, May 25 '17 at 04:35
c0 is a zero cepstrum coefficient. It is expected to be higher, cepstrum is usually decreazing — Nikolay Shmyrev, May 26 '17 at 07:52

HTK - What do MFCCs of an HMM model and Input WAV File represent?

1 Answers1