4

I am developing a speech application. It takes input in form of speech and then performs the actions available on GUI. I am using sphinx4-1.0beta 6 src and eclipse IDE.

I have created the GUI, now what I want to do is use speech to perform button click. For example when I say "SEARCH", the search button is clicked. Basically the idea is to access sphinx4 in my gui so that I can start speech recognition on my button and perform search according to the speech input.

I also want to listen for speech input after the button is clicked in order to determine what to search and perform action accordingly. Somebody please suggest what needs to be done.

Below is the modified file HelloWorld.

Here is the button's actionPerformed method:

 public void jButton3ActionPerformed(java.awt.event.ActionEvent evt){
      try {
        Class.forName("oracle.jdbc.OracleDriver");
        con=DriverManager.getConnection("jdbc:oracle:thin:@127.0.0.1:1521:xe","system","manhattan");

        PreparedStatement ps=con.prepareStatement("select * from records where stid=?");

       // ps.setInt(6,id);

        ResultSet rs=ps.executeQuery();

        if (rs.next()){
            //TODO add functionality.
        }

    }catch(Exception e){
        e.printStackTrace();
    }
   }

Can someone suggest how to perform click on this button using speech. I checked out that doClick() can be used to handle listeners . But can't make out how to actually achieve the goal.

And secondly how to make sphinx4 listen for speech input after click as well? That is how to link and use sphinx4 in GUI..?

For example I say"SEARCH" then search button is clicked and next I say "java", it retrieves result for java from database. I have modified the HelloWorld to include notepad and browser so that it can open notepad on saying notepad. But how to link this to button click?

Here is the modified HelloWorld demo from sphinx4:

package edu.cmu.sphinx.demo.helloworld;


import java.io.IOException;

import edu.cmu.sphinx.frontend.util.Microphone;
import edu.cmu.sphinx.recognizer.Recognizer;
import edu.cmu.sphinx.result.Result;
import edu.cmu.sphinx.util.props.ConfigurationManager;

import java.io.IOException;





public class HelloWorld {



public static void main(String[] args)
{


    ConfigurationManager cm;
    Process pr[]=new Process[2];

    if (args.length > 0) {
        cm = new ConfigurationManager(args));
    } else {
        cm = new ConfigurationManager(HelloWorld.class.getResource("helloworld.config.xml"));
    }

    Recognizer recognizer = (Recognizer) cm.lookup("recognizer");
    recognizer.allocate();



    // start the microphone or exit if the programm if this is not possible
    Microphone microphone = (Microphone) cm.lookup("microphone");
    if (!microphone.startRecording()) {
        System.out.println("Cannot start microphone.");
        recognizer.deallocate();
        System.exit(1);
    }

    System.out.println("Say: (Good morning | Hello ) ( Steve | Andrew | Paul | Phili p | Rita | Will | Mark )");
    System.out.print(" (Hi | Bye )* ( Tony ) \n( Search | Mail | Browser | Notepad ) \n (Open | Close) ( Search | Mail | Browser | Paint | Notepad ) \n");

    // loop the recognition until the programm exits.
    while (true) {
        System.out.println("Start speaking. Press Ctrl-C to quit.\n");

        Result result = recognizer.recognize();

        if (result != null) 
        {
            String resultText = result.getBestFinalResultNoFiller();

              if((resultText.equalsIgnoreCase("Notepad"))||(resultText.equalsIgnoreCase("Open Notepad")))
               { 
                try
                {
                //Process pr[]=new Process[2];

                 pr[0] = new ProcessBuilder("notepad.exe").start();

                //p.command("notepad.exe");

            //try {
                //p.start();
            }

            catch (IOException e) 

            {
                    e.printStackTrace();
            }
            }

               if(resultText.equalsIgnoreCase("Close Notepad"))
                { 
                    try{
                        pr[0].destroy();
                    }
                    catch(Exception ee)
                    {
                        ee.printStackTrace();


                    }
                    }

               if((resultText.equalsIgnoreCase("Browser"))||(resultText.equalsIgnoreCase("Open Browser")))
               { 
                try
                {
                //Process pr[]=new Process[2];

                 pr[1] = new ProcessBuilder("C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe").start();

                //p.command("notepad.exe");

            //try {
                //p.start();
            }

            catch (IOException e) 

            {
                    e.printStackTrace();
            }
               }


            System.out.println("You said: " + resultText + '\n');
        }

        else {
            System.out.println("I can't hear what you said.\n");
        }
     }
  }
 }
MANU
  • 1,358
  • 1
  • 16
  • 29
  • Cannot start microphone. 03:19:56.125 SEVERE microphone Can't open microphone line with format PCM_SIGNED 16000.0 Hz, 16 bit, mono, 2 bytes/frame, big-endian not supported. # --------------- Summary statistics --------- Total Time Audio: 0.00s Proc: 0.00s Speed: 0.00 X real time .............. This is what I get when I move back from a frame to previous one using speech..The application stops and I get this error on console...As the linking and controlling recognition is still not what it should be, so the question stands as it is. – MANU May 11 '15 at 21:53

0 Answers0