26

In Matlab I can start external .exe files that sometime have a pop up that requires an enter key pressed. For example:

system('C:\Program Files (x86)\WinZip\WINZIP32.EXE')

will start Winzip, and then in order to use it you need to pass the "buy now" pop up window by pressing enter. Now my problem is not with winzip, I only gave it as an example (i use winrar anyway :).

How can I programmatically press an enter key in Matlab in such cases ? (I use win 7)

Can an event listener be used to solve that?

EDIT: The java.awt.Robot class indeed works on explorer, but not on any software that has a pop up window with an OK button that needs to be pressed. I don't know why it doesn't work for that. I gave the winzip example because I assume everybody has winzip/winrar installed in their machine. The actual software I have is different and irrelevant for the question.

Amro
  • 123,847
  • 25
  • 243
  • 454
bla
  • 25,846
  • 10
  • 70
  • 101
  • 3
    Have you considered automation scripting languages like [AutoHotkey](http://en.wikipedia.org/wiki/AutoHotkey) and [AutoIt](http://en.wikipedia.org/wiki/AutoIt). The idea is to create a script to launch/control your program (WinZip or whatever), which you would run from MATLAB. Several of these language even provide embedding capabilities so you could call their APIs using COM or whatever wrapper/interop they provide... These scripting languages are used all the time to create game bots, form-filling apps, and other similar UI automation tasks. – Amro Jan 17 '15 at 04:02
  • 3
    ... You'll find them much easier to use than automation APIs from .NET or Java. You can do things like find program windows by title, send key messages to windows, wait on them, simulate mouse, and much more... – Amro Jan 17 '15 at 04:03
  • thanks Amro, I wasn't aware of these options! – bla Jan 17 '15 at 22:06

5 Answers5

29

There is a way using Java from Matlab, specifically the java.awt.Robot class. See here.

Apparently there are two types of programs, regarding the way they work when called from Matlab with system('...'):

  1. For some programs, Matlab waits until the program has finished before running the next statement. This happens for example with WinRAR (at least in my Windows 7 machine).

  2. For other programs this doesn't happen, and Matlab proceeds with the next statement right after the external program has been started. An example of this type is explorer (the standard Windows file explorer).

Now, it is possible to return execution to Matlab immediately even for type 1 programs: just add & at the end of the string passed to system. This is standard in Linux Bash shell, and it also works in Windows, as discussed here.

So, you would proceed as follows:

robot = java.awt.Robot;
command = '"C:\Program Files (x86)\WinRAR\WinRAR"'; %// external program; full path
system([command ' &']); %// note: ' &' at the end
pause(5) %// allow some time for the external program to start
robot.keyPress (java.awt.event.KeyEvent.VK_ENTER); %// press "enter" key
robot.keyRelease (java.awt.event.KeyEvent.VK_ENTER); %// release "enter" key
Luis Mendo
  • 110,752
  • 13
  • 76
  • 147
  • @bla: can you edit your question to explain why java.awt.Robot wouldn't work? – Jonas Jan 16 '15 at 19:45
  • I did edit the question, but I have no clue why the java.awt.Robot works for explorer window and not for a pop up window type software. – bla Jan 16 '15 at 20:17
  • I think I found it. It turns out it works as in Linux: use `&` to fork a process. So for example start WinRAR with `dos('"C:\Program Files (x86)\WinRAR\WinRAR" &')`, and Matlab goes on while WinRAR is running. I'm going to edit my question now – Luis Mendo Jan 16 '15 at 21:54
  • well spotted for the `&` – Hoki Jan 16 '15 at 22:36
  • Pity I almost got it working the first time. I was only missing the `&` – Luis Mendo Jan 18 '15 at 02:57
  • 4
    Hey, I got my first bounty! Now I can [live up to my avatar](http://www.starwars.com/databank/boba-fett)! :-) – Luis Mendo Jan 22 '15 at 09:58
  • robot has delay – JGFMK Oct 09 '22 at 22:26
  • @JGFMK Can you be a little more specific? Do you mean that using the `robot` class is slow, or that it has a [`delay` method](https://docs.oracle.com/javase/6/docs/api/java/awt/Robot.html#delay%28int%29)? And how is that related to my answer? I don't really see what you eman with your comment – Luis Mendo Oct 10 '22 at 08:27
  • Nope - pause(5) - that's not normal Java AFAIK. But you can use robot.delay(milliseconds) per the Java APIs. https://docs.oracle.com/javase/8/docs/api/java/awt/Robot.html#delay-int- Other than that there is Thread.sleep() too. But robot seems easier. – JGFMK Oct 10 '22 at 11:36
  • @JGFMK `pause(5)` may not be normal Java, but it is normal Matlab :-) The code in the answer is Matlab. It's just that Matlab can call Java classes – Luis Mendo Oct 10 '22 at 16:33
13

If your applications are only on Windows platform, you can try using .net objects.

The SendWait method of the SendKeys objects allows to send virtually any key, or key combination, to the application which has the focus, including the "modifier" keys like Alt, Shift, Ctrl etc ...

The first thing to do is to import the .net library, then the full syntax to send the ENTER key would be:

NET.addAssembly('System.Windows.Forms');
System.Windows.Forms.SendKeys.SendWait('{ENTER}'); %// send the key "ENTER"

If you only do it once the full syntax is OK. If you plan to make extensive use of the command, you can help yourself with an anonymous helper function.

A little example with notepad

%% // import the .NET assembly and define helper function
NET.addAssembly('System.Windows.Forms');
sendkey = @(strkey) System.Windows.Forms.SendKeys.SendWait(strkey) ;

%% // prepare a few things to send to the notepad
str1 = 'Hello World' ;
str2 = 'OMG ... my notepad is alive' ;
file2save = [pwd '\SelfSaveTest.txt'] ;
if exist(file2save,'file')==2 ; delete(file2save) ; end %// this is just in case you run the test multiple times.

%% // go for it
%// write a few things, save the file then close it.
system('notepad &') ;   %// Start notepad, without matlab waiting for the return value
sendkey(str1)           %// send a full string to the notepad
sendkey('{ENTER}');     %// send the {ENTER} key
sendkey(str2)           %// send another full string to the notepad
sendkey('{! 3}');       %// note how you can REPEAT a key send instruction
sendkey('%(FA)');       %// Send key combination to open the "save as..." dialog
pause(1)                %// little pause to make sure your hard drive is ready before continuing
sendkey(file2save);     %// Send the name (full path) of the file to save to the dialog
sendkey('{ENTER}');     %// validate
pause(3)                %// just wait a bit so you can see you file is now saved (check the titlebar of the notepad)
sendkey('%(FX)');       %// Bye bye ... close the Notepad

As explained in the Microsoft documentation the SendKeys class may have some timing issues sometimes so if you want to do complex manipulations (like Tab multiple times to change the button you actually want to press), you may have to introduce a pause in your Matlab calls to SendKeys.

Try without first, but don't forget you are managing a process from another without any synchronization between them, so timing all that can require a bit of trial and error before you get it right, at least for complex sequences (simple one should be straightforward).

In my case above for example I am running all my data from an external hard drive with an ECO function which puts it into standby, so when I called the "save as..." dialog, it takes time for it to display because the HDD has to wake up. If I didn't introduce the pause(1), sometimes the file path would be imcomplete (the first part of the path was send before the dialog had the focus).


Also, do not forget the & character when you execute the external program. All credit to Luis Mendo for highlighting it. (I tend to forget how important it is because I use it by default. I only omit it if I have to specifically wait for a return value from the program, otherwise I let it run on its own)


The special characters have a special code. Here are a few:

Shift          +
Control (Ctrl)  ^
Alt            %

Tab            {TAB}
Backspace      {BACKSPACE}, {BS}, or {BKSP}
Validation     {ENTER} or ~ (a tilde)
Ins Or Insert  {INSERT} or {INS}
Delete         {DELETE} or {DEL}

Text Navigation {HOME} {END} {PGDN} {PGUP}
Arrow Keys      {UP} {RIGHT} {DOWN} {LEFT}

Escape          {ESC}
Function Keys   {F1} ... {F16}
Print Screen    {PRTSC}
Break           {BREAK}

The full list from Microsoft can be found here

Community
  • 1
  • 1
Hoki
  • 11,637
  • 1
  • 24
  • 43
  • The limitations in my answer were caused by Matlab's `system` command, not by Java's `robot` class. Anyway I have been able to avoid those limitations using ` &` when calling `system`. I see you also use that, and your solution does work on my system. +1 – Luis Mendo Jan 16 '15 at 22:28
  • @LuisMendo. Yes, the `&` was the trick to release the launched program on its own (_without getting annoying matlab error message saying the program wouldn't take windows event_). I talked about same limitations because i suspect internally the java robot and the .net object use the same windows API call. So if one solution works for the OP undisclosed program, the other will work too (and unfortunately vice-versa). But i'm just speculating here, let's wait the feedback from the OP. – Hoki Jan 16 '15 at 22:44
  • 1
    @LuisMendo, actually, thinking about it, if anything have more limitations it will be the `.net` objects ... they will only work in Windows, while the `java` implementation will work in Linux too. – Hoki Jan 16 '15 at 23:30
  • Yes, it should. I can't test it, though – Luis Mendo Jan 16 '15 at 23:46
  • You deserve some credit for the `&` too, you had an answer already with that – Luis Mendo Jan 17 '15 at 11:31
  • And many many thanks to you @bla for this pretty nice bounty :-) Makes me want to improve the answer further but I'm not sure how. Let me know if you'd like more details on specific aspects. Cheers! – Hoki Jul 01 '15 at 08:15
  • your answer is great, don't worry. also its just virtual unicorn points, so don't let it go to your head :) – bla Jul 01 '15 at 20:49
4

There is a small javascript utility that simulates keystrokes like this on the Windows javascript interpreter.

Just create a js file with following code:

var WshShell = WScript.CreateObject("WScript.Shell");
WshShell.SendKeys(WScript.Arguments(0));

then call it from Matlab after the necessary timeout like this:

system('c:\my\js\file\script.js {Enter}');

Can't test here now, but I think this should work...

reverse_engineer
  • 4,239
  • 4
  • 18
  • 27
3

If you need to run a console-only program in a context that permits full DOS redirection, you can create a file called, say, CR.txt containing a carriage return and use the '<' notation to pipe the value into the program.

This only works if you can provide all the keyboard input can be recorded in the file. It fails dismally if the input has to vary based on responses.

An alternative is to duplicate the input (and possibly output) stream(s) for the program and then pipe data into and out of the program. This is more robust and can permit dynamic responses to the data, but will also likely require substantial effort to implement a robot user to the application.

Rog-O-Matic is an example of a large application completely controlled by a program that monitors screen output and simulates keyboard input to play an early (1980s) ASCII graphic adventure game.

The other responses will be required for GUI-based applications.

Pekka
  • 3,529
  • 27
  • 45
2

Python package pywinauto can wait any dialog and click buttons automatically. But it's capable for native and some .NET applications only. You may have problems with pressing WPF button (maybe QT button is clickable - not checked), but in such case code like app.DialogTitle.wait('ready').set_focus(); app.DialogTitle.type_keys('{ENTER}') may help. Your case is quite simple and probably some tricks with pywinauto are enough. Is your "app with popup" 64-bit or 32-bit?

wait and wait_not functions have timeout parameter. But if you need precisely listener with potentially infinite loop awaiting popups, good direction is global Windows hooks (pyHook can listen mouse and keybd events, but cannot listen dialog opening). I'll try to find my prototype that can detect new windows. It uses UI Automation API event handlers... and... ops... it requires IronPython. I still don't know how to set UI Automation handler with COM interface from standard CPython.


EDIT (2019, January): new module win32hooks was implemented in pywinauto a while ago. Example of usage is here: examples/hook_and_listen.py.

Vasily Ryabov
  • 9,386
  • 6
  • 25
  • 78
  • Official pywinauto 0.4.2 supports 32-bit Python and 32-bit apps only. That's why I'm asking about bitness. But for simple key press it may be enough. Just take it into account if something fails with 64-bit. There are 64-bit compatible clones in the Internet. – Vasily Ryabov Jan 17 '15 at 22:26
  • Java is also not a Matlab (see first answer). Python from MatLAB: http://www.mathworks.com/help/matlab/call-python-libraries.html Java from MatLAB: http://www.mathworks.com/help/matlab/using-java-libraries-in-matlab.html What is the principle difference? – Vasily Ryabov Jan 20 '15 at 08:47
  • 1
    One difference is that Matlab supports Java natively, whereas to run Python from Matlab "you need to install a supported version of the reference implementation (CPython) for Python". – Luis Mendo Jan 20 '15 at 09:11
  • 1
    I'm fully aware of its difference (hence my gold badge in MATLAB). Also, the answer from @LuisMendo is correct because MATLAB supports Java natively, as the driving engine behind MATLAB is the JRE. Your answer is not constructive as it requires the installation of CPython. The OP wants to natively do this in MATLAB without any third-party support. However, I'll remove my downvote as the OP may find the answer useful, but I'm not sure if I do. – rayryeng Jan 20 '15 at 09:23
  • Okay, thank you for detailed explanation. My suggestion is not the best, I agree. But it's not off-topic, just another way to solve the problem. – Vasily Ryabov Jan 20 '15 at 09:45