Using a piano keyboard as a computer keyboard

Question

I have RSI problems and have tried 30 different computer keyboards which all caused me pain. Playing piano does not cause me pain. I have played piano for around 20 years without any pain issues. I would like to know if there is a way to capture MIDI from a MIDI keyboard and output keyboard strokes. I know nothing at all about MIDI but I would like some guidance on how to convert this signal into a keystroke.

why to capture midi , if you can use a synthesizer keyboard as a keyboard? — shevski, May 08 '11 at 14:38
@I__: What operating system do you need to support? It will probably matter. — T.J. Crowder, May 08 '11 at 14:51
@T.J. Crowder you're trying to convert sound into keystrokes? — shevski, May 08 '11 at 15:16
@shevski: I'm not trying to do anything, @I__ is the person asking the question. He/she isn't looking to convert *sound*, but rather the digital keyboard event signals sent by electronic keyboards via the [Musical Instrument Digital Interface](http://en.wikipedia.org/wiki/Midi). These are very much like computer keyboard events: "Note on" corresponds nicely to keydown, "note off" to keyup, etc. You could even use velocity data to perform different actions depending on how "loudly" the chord was played. I love the idea. — T.J. Crowder, May 08 '11 at 15:23
If you get this to work, you can code using any kind of midi device. It probably takes some practice, but using an orchestra as development team should be fun. — Toon Krijthe, May 10 '11 at 10:24
@I__: The [most upvoted answer](http://stackoverflow.com/questions/5928061/using-a-piano-keyboard-as-a-computer-keyboard/5928462#5928462) points you to the Java sound API's MIDI support, which should get you going receiving MIDI data, and describes in some detail the theory involved in turning that into serieses of keystrokes. The [second most upvoted answer](http://stackoverflow.com/questions/5928061/using-a-piano-keyboard-as-a-computer-keyboard/5978553#5978553) talks about a very viable way you could then connect that to a computer using an Arduino, including... *(cont'd)* — T.J. Crowder, Jun 08 '11 at 17:28
*(continuing)* ...including referencing the software to emulate the USB keyboard with the device (if you look at that project, sending keystrokes is *really* easy). I don't think you're going to get much more than that in *this* question because of the compound nature of it. If you like the Arduino idea (and I certainly like it for this project, so many OS-specific hassles go away if you do that), what I'd probably do is create a community wiki answer referencing the two answers as being jointly "the" answer, accept it, and... *(cont'd)* — T.J. Crowder, Jun 08 '11 at 17:29
*(continuing)* ...and then dig into the docs and such referenced. If you have specific follow-up questions about MIDI processing details, the Arduino firmware and programming, the USB keyboard emulator software, etc., ask them as separate, specific questions. That will help you get answers. Not that I *mind* the question staying open (and not that you should care whether I do or not), it keeps getting activity and people keep voting on it, it just seems like you're not going to get much further in *this* question. Best, — T.J. Crowder, Jun 08 '11 at 17:30
Are you on windows / mac / linux? You're going to need to build a keyboard simulator, so that's a critical piece of information. — Leopd, Jun 16 '11 at 21:25

score 430 · Answer 1 · edited Jan 20 '12 at 18:58

I haven't done any MIDI programming in years, but your fundamental idea is very sound (no pun).

MIDI is a stream of "events" (or "messages"), two of the most fundamental being "note on" and "note off" which carry with them the note number (0 = C five octaves below middle C, through 127 = G five octaves above the G above middle C, in semi-tones). These events carry a "velocity" number on keyboards that are velocity sensitive ("touch sensitive"), with a force of (you guessed it) between 0 and 127.

Between velocity, chording, and the pedals, I'd think you could come up with quite a good "typing" interface for the piano keyboard. Chording in particular could be a very powerful technique — as I mentioned in the comments, it's why rank-and-file stenographers can use a stenotype machine to keep up with people talking for hours in a row, when even top-flight typists wouldn't be able to for any length of time via normal typewriter-style keyboards. As with machine stenography, you'd need a "dictionary" of the meanings of chords and sequences of chords. (Can you tell I used to work in the software side of machine stenography?)

To do this, the fundamental pieces are:

Receiving MIDI input. Don't try to do this yourself, use a library. Edit: Apparently, the Java Sound API supports MIDI, including receiving events from MIDI controllers. Cool. This page may also be useful.
Converting that data into the keystrokes you want to send, e.g. via the dictionary I mentioned above.
Outputting the keystrokes to the computer.

To be most broadly-compatible with software, you'd have to write this as a keyboard device driver. This is a plug-in to the operating system that serves as a source for keyboard events, talking to the underlying hardware (in your case, the piano keyboard). For Windows and Linux, you're probably going to want to use C for that.

However, since you're just generating keystrokes (not trying to intercept them, which I was trying to do years ago), you may be able to use whatever features the operating system has for sending artificial keystrokes. Windows has an interface for doing that (probably several, the one I'm thinking of is SendInput but I know there's some "journal" interface that does something similar), and I'm sure other operating systems do as well. That may well be sufficient for your purposes — it's where I'd start, because the device driver route is going to be awkward and you'd probably have to use a different language for it than Java. (I'm a big fan of Java, but the interfaces that operating systems use to talk to device drivers tend to be more easily consumed via C and similar.)

Update: More about the "dictionary" of chords to keystrokes:

Basically, the dictionary is a trie (thanks, @Adam) that we search with longest-prefix matching. Details:

In machine stenography, the stenographer writes by pressing multiple keys on the stenotype machine at the same time, then releasing them all. They call this a "stroke" of the keyboard; it's like playing a chord on the piano. Strokes frequently (but not always) correspond to a syllable of spoken language. Like syllables, sometimes one stroke (chord) has meaning all on its own, other times it only has meaning combined with following strokes. (Think "good" vs. "good" followed by "bye"). Although they'll be heavily influenced by the school at which they studied, each stenographer will have their own "dictionary" of what strokes they use to mean what, a dictionary they will continuously hone over the course of their working lives. The dictionary will have entries where the stenographic part ("steno", for short) is one stroke long, or multiple strokes long. Frequently, there will be several entries with the same starting stroke which are differentiated by their length and by the subsequent strokes. For instance (and I won't use real steno here, just placeholders), there may be these entries:

A     = alpha
A/B   = alphabet
A/B/C = alphabetic
A/C   = air conditioning
B     = bee
B/C   = because
C     = sea
D     = dog
D/D   = Dee Dee

(Those letters aren't meant to be musical notes, just abstract markers.)

Note that A starts multiple entries, and also note that how you translate a C stroke depends on whether you've previously seen an A, a B, or you're starting fresh.

Also note that (although not shown in the very small sample above), there may be multiple ways to "play" the same word or phrase, rather than just one. Stenographers do that to make it easier to flow from a preceding word to the next depending on hand position. There's an obvious analogy to music there, and you could use that to make your typing flow more akin to playing music, in order to both prevent this from negatively affecting your piano playing and to maximize the likelihood of this actually helping with the RSI.

When translating steno into standard text, again we use a "longest-prefix match" search: The translation algorithm starts with the first stroke ever written, and looks for entries starting with that stroke. If there is only one entry, and it's one stroke long, then we can reliably say "that's the entry to use", output the corresponding text, and then start fresh with the next stroke. But more likely, that stroke starts multiple entries of varying lengths. So we look at the next stroke and see if there are entries that start with those two strokes in order; and so on until we get a match.

So with the dictionary above, suppose we saw this sequence:

A C B B C A B C A B D

Here's how we'd translate it:

A is the start of three entries of varying lengths; look at next stroke: C
A/C matches only one entry; output "air conditioning" and start fresh with next stroke: B
B starts two entries; look at next stroke: B
B/B doesn't start anything; take the longest previous match (B) and output that ("bee")
Having output B = "bee", we still have a B stroke in our buffer. It starts two entries, so look at the next stroke: C
B/C matches one entry; output "because" and start fresh with the next stroke: A
A starts three entries; look at the next stroke: B
A/B starts two entries; look at the next stroke: C
A/B/C only matches one entry; output "alphabetic" and start fresh with the next stroke: A
A starts three entries; look at next stroke: B
A/B starts two entries; look at next stroke: D
A/B/D doesn't match anything, so take the longest previous match (A/B) and use it to output "alphabet". That leaves us with D still in the buffer.
D starts two entries, so we would normally look at the next stroke — but we've processed all the strokes, so consider it in isolation. In isolation, it translates as "dog" so output that.

Aspects of the above to note:

You have a buffer of strokes you've read but haven't translated yet.
You always want to match the most strokes against a single entry that you can. A/B should be translated as "alphabet", not "alpha" and "bee".
(Not shown above) You may well have sequences of strokes that you can't translate, because they don't match anything in the dictionary. (Steno people use the noun "untranslate" -- e.g., with our dictionary, the strokes E would be an "untranslate".)
(Not shown above) Some theories of steno allow the same set of strokes to mean more than one thing, based on a broader context. Steno people call these "conflicts". You probably want to disallow them in your project, and in fact when steno used to be translated manually by the stenographer, conflicts were fine because they'd know just by where in the sentence they were what the right choice was, but with the rise of machine translation, conflict-free theories of steno arose specifically to avoid having to go through the resulting translated text and "fix" conflicts.
Translating in real time (which you'd be doing) means that if you receive a partial match, you'll want to hold onto it while waiting for the next chord — but probably only up to a timeout, at which point you'd translate what you have in the buffer as best you can. (Or maybe you don't want a timeout; it's your call.)
Probably best to have a stroke that says "disregard the previous stroke"
Probably best to have a stroke that says "completely clear the buffer without outputting anything"

@t.j. beautiful response thanks so much. can you please tell me what is the reason that i will need to use a keyboard driver instead of just the sendkeys/sendinput — Alex Gordon, May 08 '11 at 16:04
@I__: Some programs try to talk directly to the keyboard device and so don't play nicely with "sendkeys"-like interfaces. Hopefully this is even less common now than some years ago when I last dealt with this, and so as I say, I'd start with "sendkeys" like mechanisms and only worry about the other if and when you needed to. (Probably best to do a quick program testing sending keys [on its own, not by receiving them from MIDI] -- including special key sequences -- and then testing that with the software you want to control before you get too far down the path.) — T.J. Crowder, May 08 '11 at 16:09
@t.j. thank you very much. can you recommend to me a .NET library and also can you elaborate on the dictionary of mapping the keystrokes — Alex Gordon, May 08 '11 at 16:25
@I__: I'm afraid I don't know any .Net MIDI libraries. I've updated the answer to go into more detail on the dictionary thing, along with rough notes on the algorithm (as I say, can you tell I've done steno software?). It's rough and I'm running out the door, hopefully I haven't missed out one of the strokes when walking about the translation or something, but I'm really running... — T.J. Crowder, May 08 '11 at 17:05
For those who never heard of this, dictionary you describe is basically a [Trie](http://en.wikipedia.org/wiki/Trie) (pronounced 'try') — Adam, May 17 '11 at 01:26
@I__: Interesting to write *now* to say that. What more do you want? I think I've addressed many of the important aspects of what you need to do. I haven't *done* it, nor pointed to anyone who has, because I have other things to do and don't happen to know of a pre-built solution. But in terms of *building* a solution, I think I've pointed you in the right direction. — T.J. Crowder, Dec 30 '11 at 23:15
Note that you could jam all the chords in a linear array and do traversal every time you receive a midi event, and still not experience any noticable slowdown. Using a trie smells slightly of premature optimization. — FeepingCreature, Jan 03 '12 at 02:38
@FeepingCreature: When I was doing this with stenographers and court reporters, their dictionaries could readily run to the hundreds of thousands of entries (I remember one who broke the million mark). There was nothing remotely premature about using an optimizing strategy, we were dealing with 12 MHz 386s. In I___'s case, if he goes with the Arduino idea (which is a really cool idea), he too will be dealing with fairly underpowered hardware for the modern era, though of course stuff ridiculously faster than what we had. I think his dictionary would tend to be dramatically smaller, but... — T.J. Crowder, Jan 03 '12 at 09:50

Brian O'Dell · Answer 2 · 2011-05-13T00:26:08.600

184

Consider doing something in hardware that emulates a usb (or ps/2?) keyboard. You will no longer be dependent on a specific OS, or specific OS API. A hardware solution will stand the test of time. Don't be stuck using an old API in Windows 7 when everyone else is running Windows 11! Arduino is pretty easy to learn.

There is a ton of info and help out there for Arduino. It is a hardware hacking platform built for newbies. It will only get bigger now that Google is pushing Arduino.

EDIT: Virtual USB Keyboard software and hardware

edited May 13 '11 at 00:26

answered May 12 '11 at 13:17

Brian O'Dell

3,029
1
20
23

I love the idea of using an Arduino for this. Memory could be a problem, though. It looks like the largest pre-designed boards have 256k of Flash (of which 8k is used by the bootloader). Back in the day, we crammed realtime translation / display / editing of machine shorthand into 1M under DOS (e.g., the usual 640k plus either EMS or extended memory), but we were writing tight C code and assembly and the dictionary (which could easily be >250k on its own) was on disk. Here there's be no display or simultaneous editing, so that's good, but the dictionary will have to be small... – T.J. Crowder May 15 '11 at 12:34
Oh, that's nothing. I routinely dealt with court reporters who had dictionaries that, even stored in the very compact form we used, were multiple megabytes in size. The biggest I personally saw was 4.5MB -- and again, that was stored in a highly-compressed form. Combine that with the fact that there are fewer keys on a steno keyboard than a piano keyboard... Still, though, the Arduino has a USB port, so the dictionary could be on a USB flash adapter, or even an (externally-powered) SSD or HDD. – T.J. Crowder May 22 '11 at 08:50
2

@you may want to consider a teensy instead of an arduino (http://www.pjrc.com/teensy/ ) significantly more memory and can directly emulate a keyboard. It's compatible with many arduino libraries already and the incompatible ones just require pin reassignment. -- alternately, atmega also makes a demo board that has 16 MB of built in flash and costs about $30 (at90usbkey) – Marc Jun 23 '11 at 14:16
8

Designing a useful piano interface for a computer is not an easy task. IMHO, the OP should first prototype it in his/her current environment, then, IF it satisfies his/her needs, worry about portability. Unless, of course, the portable alternative is not noticeably harder. – Emilio M Bumachar Sep 19 '11 at 20:06

score 111 · Answer 3 · edited Aug 30 '13 at 16:05

It sounds to me like you're looking less for advice on how to build this yourself and more asking what resources are already out there to accomplish what you want. Depending on your OS, there are many ways to accomplish this without having to write your own program from scratch:

MIDI Stroke

Free. For Mac OS X 10.3 and up. This one specifically comes with "the ability to use any MIDI keyboard as a full blown computer keyboard replacement."

Bome's MIDI Translator

Free/Postcardware (it's a bit odd). For Windows 2000 and up, and Mac OS X. It initially appears to be more geared towards AutoHotkey-type usage, but on further looking I think it could do what you want nicely.

Max and aka.keyboard

Free. For Mac OS X. Not exactly a "ready out of the box" solution, but if you are comfortable with basic device configuration, it shouldn't be too bad.

score 49 · Answer 4 · edited Nov 24 '12 at 16:42

You can access the hardware with source code samples in .NET in MIDI DotNet.

A complete working sample as sourcecode to create MIDI notes data stream is in VB 5/6-Tipp 0521: MIDI-Töne erzeugen (Visual Basic 6.0, somewhere is .NET version too)

A way to simulate keyboard strokes is in VB 5/6-Tipp 0155: Tastaturereignisse simulieren (Visual Basic 6.0, somewhere is .NET version too)

And recognize keystrokes is describedin Tipp-Upload: VB.NET 0266: Globaler KeyHook.

Enter image description here

Then, just use a good working matrix for a piano player Timo Boehme www.goldengel.ch

On piano and when you're a good player, you can have 10 fingers on the keyboard and if the matrix is usable you can be much more quickly that any computer keyboard user I think. :-)

In that case, if I understand your question right, it should not be a big thing.

score 41 · Answer 5 · edited Nov 24 '12 at 16:30

I studied piano performance in college and then got into interaction design, programming, and using Vim, so I have actually spent a lot of time prototyping things like this.

You can get this working pretty quick in Linux by using the graphical programming language for multimedia artists, "Pure Data," along with the x11key external by Alex Andre.

On Mac, you can use MidiStroke. I believe a method on Windows involved the MidiOx and AutoHotKey tools. At another time I had a version going using the Java plugin for Max/MSP. I believe Patrice Colet made a windows external for Pure Data that worked as well, but I can't seem to locate it anymore. Also, there's an external for MaxMSP that can do this on Windows. Finally, the non-free but awesome Osculator can do what you want - see the features page.

When I got it working, I never stuck with it, because I couldn't stop tooling with the layout. It was cool just having my monitor on my electric keyboard, though! Good luck.

score 37 · Answer 6 · edited Jun 20 '20 at 09:12

About MIDI

You stated that you "know nothing at all about MIDI". MIDI technology is fairly straight-forward once you grasp it, but it can be confusing at the outset. One of the resources that has been tremendously helpful for me in understanding the foundations for MIDI (which are certainly necessary if you want to program MIDI interactions), is a book called MIDI for the Technophobe. It's an easy book to read and is very helpful.

Pure Data & Max

In my experience developing interactive multimedia, there are two very similar programs I have encountered that facilitate connecting and mapping signals/inputs from any device.

These are Max for a Mac environment and Pure Data for a PC environment. Both have a plethora of online documentation and YouTube tutorials. The video Max/MSP Tutorial 1 - using your computer keyboard as midikeyboard (ableton style) demonstrates a program built in Max that maps a computer keyboard to a MIDI keyboard's inputs (which is basically the exact opposite of what you are trying to do). You could get your intended results by using the same pattern, but reversing the signals/mappings.

AutoHotKey

AutoHotKey is a free open source utility for Windows that allows you to remap keys and buttons on your devices to macros. It natively supports QWERTY keyboards, joysticks and mouse macros.

However, I was able to find an implementation supporting the specific mapping you are looking for. These two threads explain the process:

MIDI IN support in AutoHotkey , the discussion of the use case. The author was looking for a program that could detect MIDI IN input and translate that to keypresses.
MIDI input library , the solution to the author's problem and the posted code/patch to AutoHotKey which actually implements your intended result.

Basically, it looks like AutoHotKey, along with this user's custom patch, will provide exactly what you need to create a mapping from a MIDI keyboard to a QWERTY keyboard's input signal. All you would have to do is install, configure and define your mappings.

Anything else?

Some of the other answers have given you much more extensive information on MIDI and MIDI programming, in general, but as your post states that doesn't seem to be quite what you are looking for. I would like to help you more if possible, but it would be easier if you could be more specific about the type of information you are looking for. For instance, are you more interested in how to convert a MIDI keyboard's input signals to a QWERTY keyboard's signals, or is your primary interest finding an out of the box solution to your specific problem? What are you looking for that has not yet been addressed?

score 27 · Answer 7 · answered May 13 '11 at 22:41

27

You could hack your own USB keyboard pretty quickly using a Teenys micro controller.

In fact, they have example code for how to make a USB keyboard.

You could approach this two ways:

Get an old piano and wire up switches directly to the teensy
Add the additional logic to connect to the MIDI port and necessary decoding.

answered May 13 '11 at 22:41

cmcginty

113,384
42
163
163

This is the most promising answer because once you complete the devices, it will work on any computer and any OS. Take it in to work or just be happy that you can upgrade your computer without worrying about the keyboard. I would use two Teensy devices, and connect them together with their I/O pins. Since Teensy already supports USB MIDI http://www.pjrc.com/teensy/td_midi.html your code just needs to take care of moving signals between the keyboard teensy and the midi teensy. – Michael Dillon Jun 18 '11 at 18:14

score 24 · Answer 8 · edited May 23 '17 at 10:30

Actually, I worked on this a while ago, trying to capture Rock Band drum inputs into my computer (making a little Java homemade drum emulator) Anyway, I asked a question on here about that, Time delay problem (there is polling code in there, along with what I was attempting to do.). And if I can find my program I can give you the code, it uses a third-party API (JInput).

Good luck either way.

score 23 · Answer 9 · edited Dec 31 '12 at 12:46

Try Bome's MIDI translator.

It works cross platform, can convert any MIDI input to a keystoke easily, quick to setup and configure, plus it's free for personal use.

There is a tutorial, Quick Tip: MIDI Translation – MIDI to Keystrokes, of how to easily set it up:

Basically, there are infinite possibilities of what you can do, including chording and modifier keys. I use it for my live audio rig to control my DAW using my piano and have never had an issue.

score 18 · Answer 10 · edited Nov 24 '12 at 16:28

In Java, you can use JMF (Java Media framework) to convert MIDI signals. Basic of keyboard design is easy to use, that is, the user interface; and place frequently used charcter/symbol handy.

The sample code and API in Java Sound Resources: Examples: Digital Signal Processing (DSP) help to understand how to process the signal.

Some more references:

score 16 · Accepted Answer · answered May 10 '11 at 22:06

A good library in .NET with full midi support (BASS), go to http://www.un4seen.com.

And for the other part, translating keyboard midi notes to keys and more, I would go for AutoItX, the ActiveX/COM and DLL interface to autoIt. Info and download, go to http://www.autoitscript.com/site/autoit/

No need to write keyboard driver.

score 15 · Answer 12 · edited Nov 24 '12 at 16:25

15

There is a program called GlovePIE. You can program it in a simple scripting language, and I believe it supports MIDI. I'm not sure if this fits under the "Java" category, but still, it is a great program. I've used it to control robots using PS3 controllers. It's very powerful.

edited Nov 24 '12 at 16:25

Peter Mortensen

30,738
21
105
131

answered May 13 '11 at 14:54

tew

2,723
5
23
35

score 15 · Answer 13 · edited Dec 21 '13 at 16:20

15

Many keyboards have a serial port (RS-232) connector to send MIDI signals to the computer. I use a program called Girder to convert serial port communication into keyboard strokes.

Girder has a "mapping" feature that lets you map each key, one by one, to the corresponding keystroke.

This might be the simple solution you're looking for!

edited Dec 21 '13 at 16:20

Peter Mortensen

30,738
21
105
131

answered Jun 17 '11 at 16:34

Scott Rippey

15,614
5
70
85

score 12 · Answer 14 · edited Jun 20 '20 at 09:12

Just learn stenography!

It's clear from all the discussion on your part. You don't want to re-invent any wheels, from a technical standpoint. But once you have a connection made (what this question is asking) and up and working, you still have most of the work ahead of you: You have to train your brain. You also have to invent the cleverest, most efficient way to do that - a design issue totally out of the rhealm of computer techies. You or any of us would fall short.

Fortunately, the problem has been solved and honed though centuries of maturing...

Learn stenography!

Yes, this will set you back some jack. But what are hundreds of hours of your own time worth, with at the end, a less favorable result? Besides, the stenography Wikipedia article says, 'it looks more like a piano keyboard'.

Unless, of course, you want to have a sideshow effect going. I would have to admit, I never thought of this possibility, it it would be really entertaining to see somebody bust out a text from a piano keyboard!

You could start with a USB keyboard with touchpad (or a pointing stick would be more ergonomic?), use Plover to translate it (I'm sure it can be configured to let the non-letter keys retain their functionality as they are critical for programming), or, follow the thread Re: Plover keyboard to roll your own USB stenography keyboard, or, buy a stenotype.

Good luck!

Where are { and } on the stenography keyboard? How are you going to type names that are not words, like `pthread`? — bobobobo, Apr 14 '13 at 04:55
The wikipedia link is wrong and should be http://en.wikipedia.org/wiki/Stenography — joelhoro, Mar 06 '14 at 13:16

score 11 · Answer 15 · edited Dec 21 '13 at 16:28

Take a look at MAME arcade gaming. They have built hardware devices to allow input from any number of different items. The iPac, for example, converts signals from input devices into USB that the computer can then use to emulate keys. You could use any combination of input devices arranged any way that seems comfortable with no crazy programming logic required--because the software to interpret input is already done and well tested.

I've seen flight simulator cockpit inputs, custom kiosks, and voting systems built in this method.....and the price is right!

score 10 · Answer 16 · edited Dec 31 '12 at 13:36

To solve this you will need a few things:

A way to capture MIDI data from your keyboard. Depending upon the interface: MIDI interface (classic) or USB MIDI interface (modern) the most likely interface is to a computer as it provides the most options. USB host microcontrollers are not as simple as just using a computer.
A scheme to convert MIDI data into keystrokes. Like one user pointed out, chords are the way to go as the number of keys will not be dependent upon the number of piano keys.
A way to inject a key into the operating system. This will require a low-level driver to be accurate. I have played around with applications that inject keyboard and mouse data into applications in Windows 7, and it can be flaky and depend upon whether an application is currently in focus. This is hardest part of the interface. What may work is to create a HID USB keyboard microcontroller that also has a serial interface.

The serial interface would create a virtual serial port. The software that reads the MIDI data and produces the keystrokes could send a serial message to the virtual serial port. The microcontroller would send a keystroke so it would look like a standard keyboard input. This would allow interfacing both MIDI ports and USB MIDI keyboards.

Hmmm, with this type of interface you could also simulate a mouse and use some piano keys setup for the mouse axis and buttons. The pressure could be used to determine mouse pointer velocity. So you could eliminate the mouse as well. Another benefit of this approach is any type of input device you connect could talk to the virtual serial port to produce keyboard and mouse events. So if you wanted to add other hardware such as drum pedals or a joystick it would be a matter of adjusting the program that talks to the serial interface.

Another take on the above is like some posted above to use an Arduino, but also include USB Host Shield from Sparkfun to handle USB based music keyboards. This allows the Arduino to be programmed as a keyboard or keyboard mouse combo in the boot loader chip and allows the device to act a USB host for the USB based music keyboard. Then you are covered for both types. Although, I still think the virtual serial port method is more flexible and would be easier to program in the long run. The Arduino device will be harder to change than a desktop program or service.

There is another possibility: Chorded one handed keyboards already exist. I have seen videos on them, but you would have to determine if those hurt your hands or not.

score 7 · Answer 17 · edited Dec 31 '12 at 12:52

You'll get better and happier results (regardless what operating system and/or DAW program you like to use) by playing any external MIDI keyboard as a controller through your sound card. Then route that into your GB software (or whatever) and tone generate the many sounds they have supplied you that way in real-time.

If your sound card does not support MIDI I/O's (ins / outs /thrus), that's not a problem. You can consider researching and investing in an external MIDI table top converter. Many are equipped to further convert MIDI outs to USB 2.0 (by- passing an existing sound card altogether).

For example: it's pretty tough getting "human like" grace note results via a Z and X change key option using a computer keyboard and pencil tool. When, instead, your own fingers can just play that with a MIDI keyboard from its own physical octave register ranges—immediately!

I realize budgetary constraints may be involved. But, some of these seemingly cheap "Casio" type 5 octave keyboards sold at Radio Shack for under $100.00 U.S. Dollars (*or less) is all you would need (plus, some of their on-board sound patches and sequencer modules sound and handle amazingly well for other things too).

RadioShack MIDI keyboard options.

As for external MIDI converters for existing sound cards, I've run some Google searches for you as follows with Mac platforms specifically in mind:

A lot of this external MIDI conversion information may be cumbersome to you at first, so I've broken down things more as "user friendly" for your considerations & budget:

MIDI sound cards

There's nothing wrong with facilitating virtual keyboards as VST's when using DAW. They have their place.

But, you sound like an accomplished keyboardist. So, why not consider the external MIDI conversion / keyboard options I just mentioned for yourself?

Good luck and I hope this gave you some ideas that can and will work for you!

score 7 · Answer 18 · edited Nov 22 '11 at 16:31

It should be fairly easy using something like the .NET DirectSound interface to hook into an MIDI device. You will need to identify your target MIDI device and then get the code to listen in on the incoming messages (there are articles about doing this via Google).

Since you are using the MIDI in as a keyboard there are basically only two MIDI messages that you need to detect, namely note on and note off. The MIDI message is three data bytes specifying the command, the note and the velocity. The note off is just the note number (sometimes 'bad' MIDI stacks send a note on with zero velocity which you also have to expect).

Once you have the messages translating them the keyboard output should be fairly simple from .NET.

There is plenty of advice in the other answers about the technicalities; I just wanted to give you an idea of the actual MIDI messages. Good luck!

As a side note the document 'Universal Serial Bus Device Class Definition for MIDI Devices' gives a very tidy overview of MIDI messages: http://www.usb.org/developers/devclass_docs/midi10.pdf — Simon Inns, Nov 23 '11 at 07:30

score 6 · Answer 19 · answered Apr 30 '12 at 07:23

6

If you don't want to do any programming yourself but just want the problem solved you can just buy a USB-MIDI-keyboard where you can re-assign any key to send a QWERTY keyboard output signal instead of a MIDI-output, for example M-Audio Axiom Pro

This method will work with any OS and any computer that supports standard USB-keyboards since the MIDI-keyboard will identify itself as a standard QWERTY keyboard.

answered Apr 30 '12 at 07:23

Martin Odhelius

990
6
15

do you know if it would be possible to assign chords to specific keystrokes as well? for example if i play C MAJOR chord can i have that be translated to "the fox jumped" – Alex Gordon Apr 30 '12 at 15:19
Sorry for not answering so fast, I have had vacation for 4 days;) No, unfortunately you can only assign one QWERTY-code per key, so it pretty much work as a standard keyboard. – Martin Odhelius May 03 '12 at 09:03
Btw, if you buy one, just remember to buy the PRO-one, because I do not think the non-PRO-version has the QWERTY-assignment functionality. – Martin Odhelius May 03 '12 at 09:17
I do not think it is possible to assign chords to ASCII-codes, just one key per code, but I'll check this out on my own Axiom tonight when I get home from work. – Martin Odhelius May 07 '12 at 15:48
I just noticed that it is only possible to assign qwerty codes to the control buttons (function keys, drum pads, numeric keys etc), not to the actual piano keys on the keyboard. I own this keyboard myself and was sure it was possible to assign ascii codes to any key, but now when I tried it out I notice that was wrong! I am very sorry for this, is it possible to revert the reward in some way? – Martin Odhelius May 07 '12 at 22:55

score 6 · Answer 20 · edited Nov 24 '12 at 18:37

You can use a simple AutoIt script to both read MIDI events, see MIDI Input.

You'll also need MIDI UDF and simulate key presses.

Reading MIDI events should be easy, but different MIDI controllers (instruments) have different features. Try to find out what your MIDI piano can do first, then see how you can best map those features to simulated QWERTY-keyboard presses.

If you want, you could have something on screen or in the tray to help you see what you are doing (that is, for Shift, Ctrl and Alt simulation).

score 5 · Answer 21 · edited Dec 31 '12 at 13:30

5

You might take a look at chorded keyboards. They have the advantage that you don't need to write a driver for them before you can use them, and some are similar to the layout of a piano keyboard.

edited Dec 31 '12 at 13:30

Peter Mortensen

30,738
21
105
131

answered Jan 03 '12 at 11:41

Perkins

2,409
25
23

score 5 · Answer 22 · edited Dec 31 '12 at 14:45

If you know coding in Java, you could use this way:

First, implement a javax.sound.midi.Receiver with a send(..) method that is mapping the 'Note on' events to keystrokes like you want.

You would need to get the MidiMessage's content with its getMessage method and intepret it in your fashion. The meaning of the message bytes can be found in MIDI Messages.

After receiving a 'note on' or 'note off' for a certain keyboard key, you may map that to a key you like by assigning it a constant of the KeyEvent class, something like C#4 -> KeyEvent.VK_A and so on. This key code can then be used by java.awt.Robot's keyPress and keyRelease methods to actually send the keystroke to the OS and thus to other applications.

score 4 · Answer 23 · edited Nov 24 '12 at 18:44

I agree with Brian O'Dell's answer - if this were my project, I'd do it in hardware. It has the advantage of being platform and hardware independent - your box replaces the need for a MIDI-USB interface and a PC API.

mbed is a fast-prototyping platform that is very easy to learn, and has multiple advantages over Arduino IMHO (online compiler, 512 KB flash, 96 MHz, C++ language). It has a USB keyboard interface and a USB Midi interface pre-written for you.

The community is very friendly and willing to help, and there are a lot of existing projects using both MIDI and USB hid emulation - search Youtube for "mbed MIDI" or similar.

score 4 · Answer 24 · edited Nov 24 '12 at 18:50

4

If you use Linux have a take at Footware.

It should be exactly what you're looking for - if you adjust the MIDI pitches to a keymapping of your liking...

I never thought this could be useful for anyone but me ;o)

edited Nov 24 '12 at 18:50

Peter Mortensen

30,738
21
105
131

answered Mar 28 '12 at 07:11

FlashKorten

51
1

2

At my previous job as an admin I had a foot-controller under my desk - to minimize leaving the home-row in vim – FlashKorten Mar 31 '12 at 09:14

score 2 · Answer 25 · edited Dec 31 '12 at 14:49

2

Try using a microcontroller-based system, like Arduino.

edited Dec 31 '12 at 14:49

Peter Mortensen

30,738
21
105
131

answered Jun 15 '12 at 23:51

score 1 · Answer 26 · edited Dec 21 '13 at 16:35

This wouldn't be too tough.

I'm assuming you're on Windows, not sure about that though. I've written a MIDI sequencer, http://pianocheetah.com, in plain old C++, and it lets you use the piano keyboard to run commands. There isn't any reason you couldn't do the same thing to push keys into the keyboard input stream.

But come on now. You remember how long it took you to learn the keyboard in the first place, right? Are you willing to go through that again? And are you willing to pollute your blessed keyboard with a bunch of stupid looking key symbols all over it?

You'll need to use at least 26 alpha, 10 numeric, 11 punctuation, and at least 12 function keys AND their shifted states. So that's 60 keys plus shifted states. That'll burn up a full 5 octaves of keys. You will be doing piano "hops" =all= the time. Say goodbye to touch typing.

You may save yourself from RSI, but you've created another different type of nightmare for yourself.

And good luck getting your boss to buy you a MIDI keyboard at work.

If you've learned to truly play piano, you've learned how to play stress free. Do that on the QWERTY keyboard. No tension. Start slow.

Thanks for the kind words, but I've worked my butt off on this thing and don't plan on making it open source - sorry. But what ya'd do is use midiInOpen(); provide it with a callback to process midi input; ignore everything except for notedown events, map the midi note to some keyboard key and insert that into the keyboard input stream (I forget the api for that, but i'm PRETTY sure there is one, at least). — Stephen Hazel, Feb 13 '13 at 20:35

Using a piano keyboard as a computer keyboard

26 Answers26

About MIDI

Pure Data & Max

AutoHotKey

Anything else?

Just learn stenography!

Linked