11

In the Matlab Statistics toolbox there are several functions for handling Hidden Markov Models (HMM), but they all work with discrete observation symbols. Does anyone know if there are toolboxes or functions (perhaps from a third party) that can handle continuous observation variables?

robguinness
  • 16,266
  • 14
  • 55
  • 65
  • 1
    You could use [hmm toolbox](http://www.cs.ubc.ca/~murphyk/Software/HMM/hmm.html) or its successor [pmtk](http://code.google.com/p/pmtk3/). – Junuxx Nov 23 '12 at 10:07
  • I have looked at hmm_toolbox, but I can't figure out how to use it for continuous observation variables. Can you (or someone) provide an example of this usage? – robguinness Nov 23 '12 at 10:29
  • 3
    @robguinness: What exactly could you not figure out? Did you read Kevin Murphy's [tutorial](http://www.cs.ubc.ca/~murphyk/Software/HMM/hmm_usage.html)? I used this toolbox a couples years ago and remember it working well for my case. – jerad Nov 23 '12 at 17:07
  • So, I'd suggest working through that little tutorial and then if you get stuck on anything specific, I could probably help you. – jerad Nov 23 '12 at 17:13
  • If I understand correctly, in the second example, the observation variables are modeled as a Gaussian mixture, but to be honest, I don't understand this example very deeply. In my usage, I'm not sure that my observation variables can be modeled very well as a Gaussian mixture. I was thinking to use a histogram approach to estimate the emission probabilities. Do you know if the toolbox is flexible enough to allow this kind of approach? Also, do you know what the "m" stands for in "mhmm"? Is it "mixture"? – robguinness Nov 26 '12 at 11:31
  • The last ‘m’ in “mhmm” stands for “mixture”, yes. – JesseBikman Dec 05 '12 at 16:39
  • 1
    @robguinness: The use of a Gaussian mixture model for the observation model was only for demonstration. Theoretically you could use whatever distribution you want, as long as you can compute the likelihood of an observation under it. I'm not sure what you mean have in mind when you say "histogram" approach. How would you then compute the likelihood of an observation given that distribution? – jerad Dec 05 '12 at 20:51
  • 1
    Have you considered using WEKA for this? Since MATLAB is a java interpreter, you can essentially make direct calls to the WEKA api from MATLAB to pass information and get results. – Will Faithfull Dec 10 '12 at 13:58
  • Hi, I found one in this link [here](http://www.cs.nyu.edu/~roweis/code.html) they claim that's it's for discrete and continues. not sure though. if it helps let me know so I can post it as an answer. – mamdouh alramadan Jan 02 '13 at 00:05
  • Sorry, I had to step away from this mini-project for awhile, but now I am back to it. @jerad: As far as "histogram" approach, what I meant is that I constructed a histogram to estimate the probability density p(X|S), where X is the observed signal and S is the state. Is this not the likelihood that you refer to? – robguinness Jan 04 '13 at 09:24
  • @Will Faithfull: Thanks for that suggestion. I am strongly considering WEKA. Do you know of any examples or tutorials related to making calls to WEKA in MATLAB? – robguinness Jan 04 '13 at 11:25
  • Oops...I should have searched myself before asking. I found this helpful example of using WEKA in MATLAB: http://www.mathworks.com/matlabcentral/fileexchange/19260-using-weka-in-matlab – robguinness Jan 04 '13 at 11:47

2 Answers2

4

We came to an acceptable solution in the comments, so I'll post it here for future reference:

WEKA has appropriate functions for handling HMMs, and as it has a Java API it is an ideal candidate for use with MATLAB. MATLAB itself is a Java interpreter, so you can make direct calls to the WEKA api, passing and retrieving data.

Here is a matlab fileexchange example demonstrating the use of WEKA through MATLAB.

Here is a Java example showing how to use a generic WEKA classifier, which should be applicable to the third party HMM

Will Faithfull
  • 1,868
  • 13
  • 20
  • Hi Will- I've been playing around with Weka for about a week now. It has an impressive number of different classifiers and is pretty easy to use. However, I don't find any implementation of HMMs included. I did find on the net one third-party HMM plug-in for Weka, but it seems to be in a fairly nebulous state: http://www.doc.gold.ac.uk/~mas02mg/software/hmmweka/index.html – robguinness Jan 15 '13 at 16:29
  • Aha, evidently I hadn't realised it was third-party. However, I did find what appears to be the source code on [GitHub](https://github.com/marcogillies/HMMWeka) – Will Faithfull Jan 23 '13 at 12:08
  • Thanks Will. I checked this library, but I had a little trouble figuring out how to format my data according to the relational structure that it requires. I didn't find the instructions on the author's website very thorough. If you managed to get it to work, would you mind posting an example code snippet, or some more details about the format? – robguinness Jan 25 '13 at 11:25
  • Hi Rob, It seems to have the usual WEKA inheritance tree for a classifier [according to the API](http://www.doc.gold.ac.uk/~mas02mg/software/hmmweka/HMM/doc/index.html). If you look at the top level weka.classifiers.[AbstractClassifier](http://www.doc.gold.ac.uk/~mas02mg/software/hmmweka/HMM/doc/index.html) then it seems that you need to put your data into [Instances](http://www.doc.gold.ac.uk/~mas02mg/software/hmmweka/HMM/doc/index.html). There should be code samples in the links, and more on the actual [WEKA API](http://www.cs.waikato.ac.nz/ml/weka/documentation.html) – Will Faithfull Jan 25 '13 at 12:57
  • Added a java example in the answer. – Will Faithfull Jan 25 '13 at 14:53
  • I don't think it works, unless you have specifically formatted your data as sequence data. From the author's website: "The HMM classifiers only work on sequence data, which in Weka is represented as a relational attribute." I will try to play around with it today and see if I can't get it to work. – robguinness Feb 04 '13 at 07:55
  • I finally was able to format my data into the multi-instance relational structure that WekaHMM requires, but when I ran the classifier, I got a java exception. Did you manage to get WekaHMM to actually work? Snippet from log: starting build classifier Exception in thread "Thread-15" java.lang.NoSuchMethodError: weka.estimators.MultivariateNormalEstimator.setCovarianceType(I)V weka.estimators.MultivariateNormalHMMEstimator.setNumStates(MultivariateNormalHMMEstimator.java:95) – robguinness Feb 05 '13 at 12:01
0

Prof. Zoubin Ghahramani has written code for the EM algorithm:

http://mlg.eng.cam.ac.uk/zoubin/software.html

sk1ll3r
  • 295
  • 4
  • 15