25

Introduction

I heard something about writing device drivers in Java (heard as in "with my ears", not from the internet) and was wondering... I always thought device drivers operated on an operating system level and thus must be written in the same language as the OS (thus mostly C I suppose)

Questions

  1. Am I generally wrong with this assumption? (it seems so)
  2. How can a driver in an "alien" language be used in the OS?
  3. What are the requirements (from a programming language point of view) for a device driver anyway?

Thanks for reading

Thomas Owens
  • 114,398
  • 98
  • 311
  • 431
Philip Stark
  • 598
  • 1
  • 4
  • 12

13 Answers13

23

There are a couple of ways this can be done.

First, code running at "OS level" does not need to be written in the same language as the OS. It merely has to be able to be linked together with OS code. Virtually all languages can interoperate with C, which is really all that's needed.

So language-wise, there is technically no problem. Java functions can call C functions, and C functions can call Java functions. And if the OS isn't written in C (let's say, for the sake of argument that it's written in C++), then the OS C++ code can call into some intermediate C code, which forwards to your Java, and vice versa. C is pretty much a lingua franca of programming.

Once a program has been compiled (to native code), its source language is no longer relevant. Assembler looks much the same regardless of which language the source code was written in before compilation. As long as you use the same calling convention as the OS, it's no problem.

A bigger problem is runtime support. Not a lot of software services are available in the OS. There usually is no Java virtual machine, for example. (There is no reason why there technically couldn't be, but usually, but usually, it's safe to assume that it's not present).

Unfortunately, in its "default" representation, as Java bytecode, a Java program requires a lot of infrastructure. It needs the Java VM to interpret and JIT the bytecode, and it needs the class library and so on.

But there are two ways around this:

  • Support Java in the kernel. This would be an unusual step, but it could be done.
  • Or compile your Java source code to a native format. A Java program doesn't have to be compiled to Java bytecode. You could compile it to x86 assembler. The same goes for whatever class libraries you use. Those too could be compiled all the way to assembler. Of course, parts of the Java class library requires certain OS features that won't be available, but then use of those classes could be avoided.

So yes, it can be done. But it's not straightforward, and it's unclear what you'd gain.

Of course another problem may be that Java won't let you access arbitrary memory locations, which would make a lot of hardware communication pretty tricky. But that could be worked around too, perhaps by calling into very simple C functions which simply return the relevant memory areas as arrays for Java to work on.

jalf
  • 243,077
  • 51
  • 345
  • 550
  • 5
    +1: Very insightful but you forgot to mention that while it's possible it's still pretty insane in most cases. – Adam Hawes Mar 26 '09 at 00:40
  • 1
    In the land of computers, if you can imagine it, it is possible! – Chii Mar 26 '09 at 12:19
  • If somebody else would do the effort to port a JVM in kernel and optimized it to have decent performance, it would make a lot of sense. I don't think you're fully aware of what is the average driver programmer. – Blaisorblade Jan 24 '11 at 22:57
  • About H/W communication, even in Linux you use library functions for that - not on all platforms you can write on a memory location, and when you do, that's hidden from you. About compiling to native code, you need at least in-kernel GC, as I wrote elsewhere, and that's hard. Otherwise you lose most Java advantages. – Blaisorblade Jan 24 '11 at 23:00
  • 1
    @Blaisorblade: what's your point? Sounds like you're saying pretty much exactly what I wrote in my post already. – jalf Jan 25 '11 at 13:19
  • Compiling Java to assembler is not possible, as that doesn't obviate the need for GC. That's a runtime service you can't avoid, which you don't mention. Therefore, supporting Java at all is a huge work upfront, and is impossible for a driver programmer. You seem to suggest it would be possible. Then, for H/W communication, you suggest writing to arrays rather than writing to memory. Since you can't write to memory to talk to devices on many architectures, you would need a _different_ API, which gives function for I/O to devices. – Blaisorblade Feb 16 '11 at 14:20
  • @Blaisorblade: languages that compile to assembly rely on various runtime services as well. What do you think the C runtime is for, for example? Likewise, there is no *theoretical* reason why a GC couldn't be supplied for a Java program compiled to assembly. Like I already said, it'd be a lot of work, for no gain, but it is *possible*. And pretty much every modern CPU (that I know of) relies on memory mapping for hardware I/O. Anyway, you seem to have missed that the entire point of my answer is that "it's *theoretically* possible -- but would be painful to actually do. – jalf Feb 16 '11 at 17:34
  • Regarding the GC, it could be included in the driver code itself. It doesn't *have* to be provided for you by the OS or by some predefined runtime. And if you need to go through a separate API for I/O, then you just expose those C functions to your Java app. C/Java interop isn't a new thing. Once again, it would theoretically be possible to write a driver in Java. It'd just be a ton of work, it's hard to see a single advantage. But it is *possible*. Which is what the OP asked. – jalf Feb 16 '11 at 17:37
  • On one side, I didn't get your point because it was not fully clear in your answer (as remarked in the comment by Adam Hawes). On the other side, I was answering to Chii's comment. I'm convinced it would make sense if somebody supported writing drivers in a better language. You might have missed my point: it is not that you can't support Java, just that some strategies suggested in your answer don't work. You wrote that one could compile Java to assembler and avoid using runtime services which are not available, but avoiding GC is impossible. Memory access is instead simpler than you wrote. – Blaisorblade Feb 21 '11 at 13:56
  • @Blaisorblade: you're still wrong, in at least two ways: the GC *could* be avoided. The app just has to act as *if* there was a GC. With sufficient static analysis, the compiler could generate code to manage memory without relying on a real general-purpose GC. And more realistically, the GC code could be embedded in the driver code itself. The runtime services a Java program relies on don't have to be provided separately. They could all be embedded into the program (in this case, into the driver) – jalf Feb 21 '11 at 14:00
  • Exact static analysis algorithms are undecidable, therefore static analysis produces approximate answers. In some cases, researchers made the results precise enough to be useful and published a paper about that. Can you point to a paper confirming your conjecture? Otherwise, I can point to http://bit.ly/i55fVU, which shows what has been done - basically, if you program according to certain guidelines, you can get away without GC and with little explicit annotations in your source code. – Blaisorblade Feb 25 '11 at 19:15
  • Writing a GC as part of a driver is maybe not impossible in principle; my point is that it is just not cost-effective to do it for one driver. Furthermore, you probably need to modify the core kernel in subtle ways to write a GC. How do you handle indirect references from the C heap to the Java one? If you need to modify the kernel core, you're not writing a normal driver. – Blaisorblade Feb 25 '11 at 19:22
  • @Blaisorblade: Cost effective? I never claimed it was. Anyway, a GC doesn't have to be able to relocate objects or do heap compaction. Just use a simpler GC whic doesn't move objects once they've been allocated. No reason why you'd have to modify the kernel for that. But again, I never claimed it was cost effective, a good idea, or even sane human behavior. You'd have to be insane to even try this. I'm just saying it is technically possible. – jalf Feb 25 '11 at 19:25
  • 2
    Let me ask once again though, is there a *point* to you going on and on about this? What I'm saying is trivially true: you *could* embed the necessary runtime services into the driver, and then you *could* write your driver in Java. You could replace the GC with a simple reference counting mechanism (and add a mechanism to break cycles). What exactly are you trying to prove? What exactly are you arguing against? Unless you have something *new* to bring to this discussion, I think we're done here. – jalf Feb 26 '11 at 00:31
  • Whenever unsafe languages such as C can be avoided, it is useful, as programming language theory explains. That's fundamentally different from your point of view. Of course, the scenario I imagine is that the OS author to support a safe language, and the poor driver author to simply write his driver. Otherwise your conclusion is right. That's why I discussed about how to do that well and cost-effectively (but Singularity is maybe better). Meanwhile, I'm happy of your final solution to the GC problem. Generally, nothing of what I wrote was meant personally at you, so no need to be emotional. – Blaisorblade Mar 23 '11 at 22:35
21

Writing Solaris Device Drivers in Java covers a A RAM disk device written in Java.

Another one for Linux. Goes more in depth on why you might want a DD in Java as well (since some people were wondering by the looks of the other posts and comments)

TofuBeer
  • 60,850
  • 18
  • 118
  • 163
7

A device driver could be a lot of things

I actually write device drivers in java for a living: drivers for industrial devices, such as scales or weighing devices, packaging machines, barcode scanners, weighing bridges, bag and box printers, ... Java is a really good choice here.

Some examples

Industrial devices are very different from your home/office devices (e.g. scanners, printers). Especially in manufacturing (e.g. food), companies opt more and more for a centralized server which runs an MES application (e.g. developed in Java) The MES server needs to interface with the devices of the production line, but also contains business logic. Java is a language that can do both.

Where your home/office devices are often built-in to your computer or connected with an USB cable, these industrial devices usually use Ethernet or RS232 connectors. So, in essence, pretty much every language could do the job.

There is not much standardisation in this area yet. Most vendors prefer to create their own protocol for their devices. After all they are hardware builders, not software geniuses. The result is that there is a high diversity of protocols. Some vendors prefer simple plain-text protocols, but others prefer complex binary protocols with CRC codes, framing, ... Sometimes they like to stack multiple protocols (e.g. a vendor specific handshaking algorithm on top of an OPC layer). A strong OOP language has a lot of advantages here.

E.g. I've seen java print at a continuous speed of 100ms/cycle. This includes generating a unique label, sending it to the printer, receiving a confirmation, printing it on paper and applying it to the product using air pressure.

In summary, the power of java:

  • It is useful for both business logic as complex interfacing.
  • It is just as reliable in communication with sockets as C.
  • Some drivers can benifit from Java's OOP power.
  • Java is fast enough.
bvdb
  • 22,839
  • 10
  • 110
  • 123
4

It's not impossible, but possibly hard and possibly makes not much sense.

Possible is it, because Java is a normal programming language, as long as you have some way to access the data, it's no problem. Normally in a modern OS the kernel has a layer to allow raw access to hardware in some way. Also already exist drivers in userspace, at least the userspace-part should be no problem to implement in Java.

It makes possibly not too much sense, because the kernel has to start a JVM to execute the driver. Also JVM-implementations normally eat up much memory.

You could also use Java-code compiled to be executed natively on the platform (not with the help of a JVM). This is usually not that efficient, but it could be suitable for a device-driver.

The question is, does it make sense to implement the driver in Java? Or stated in another way: What is the benefit you hope for, if you use Java for implementing the driver instead of another alternative? If you can answer this question, you should find a way to make it possible.

At the end the hint to JNode, a project that tries to implement a complete OS purely based on Java.

Mnementh
  • 50,487
  • 48
  • 148
  • 202
  • I believe that the unstated assumption here [see point 2 in the question] is that we are talking about your typical machine (e.g. x86), possibly with your typical OS (e.g. Linux or Windows), NOT on picojava, NOT in a Java OS, NOT in a JVM-aware kernel etc. – vladr Mar 26 '09 at 00:03
  • Yes, possible. But you can compile Java to native code. I don't say it is easy with Java. That's why I asked, what the expected benefit is. – Mnementh Mar 27 '09 at 13:38
  • Once you get up to a certain number of operations/sec (say 500k ops/s), interrupts and kernel-user context switching become the most significant barriers to performance. Pure user space drivers do not need to perform context switches, so that goes away. Interrupts are only seen by kernel-side drivers so a relay mechanism like UIO is needed if you want them to arrive in user space. Drivers don't always need interrupts - polling and memory barriers can help. – Ross Judson Nov 13 '13 at 20:05
3

For the motivation, please remember that there is plenty of fast languages which are better than C for programming; they might not be as fast as C, but they are safe languages: if you make a mistake you don't get undefined behavior. And "undefined behavior" includes executing arbitrary code supplied by some attacker which formats your HD. Many functional languages are usually compiled to native code.

Device drivers contain the most bugs in an OS kernel - I know that for Linux (Linus Torvalds and others keep saying so) and I heard that for Windows. While for a disk or Ethernet driver you need top-notch performance, and while in Linux drivers today are the bottleneck for 10G Ethernet or SSD disks, most drivers don't need that much speed - all computers wait at the same speed.

That's why there are various projects to allow writing drivers which run outside of the kernel, even if that causes a slowdown; when you can do that, you can use whatever language you want; you will just then need Java bindings for the hardware control library you use - if you were writing the driver in C, you would still have a library with C bindings.

For drivers in kernel mode proper, there are two problems that I've not yet seen mentioned:

  • Garbage Collection, and that's a tough requirement. You need to write an in-kernel Garbage Collector; some GC algorithms rely on Virtual Memory, and you cannot use them. Moreover, you probably need to scan the whole OS memory to find roots for the GC. Finally, I would only trust an algorithm guaranteeing (soft) real-time GC, which would make the overhead even bigger. Reading the paper which was mentioned about Java Device Drivers on top of Linux, they just give up, and require programmers to manually free memory. They try to argue that this will not compromise safety, but I don't think their argument is convincing - it's not even clear whether they understand that Garbage Collection is needed for a safe language.

  • Reflection and class loading. A full Java implementation, even when running native code, needs to be able to load new code. This is a library you can avoid, but if you have an interpreter or JIT compiler in kernel (and there's no real reason that makes it technically impossible).

  • Performance. The paper about a JVM on Linux is very bad, and their performance numbers are not convincing - indeed, they test a USB 1.1 network driver, and then show that performance is not so bad! However, given enough effort something better can surely be done.

Two last things:

  • I'd like to mention Singularity, which is a complete OS written in a C# variant, with just a Hardware Abstraction Layer in a native language.
  • About picoJava, it's a bad idea to use it unless your system is a really memory constrained one, like a smart card. Cliff Click already explained why: it gives better performance to write a good JIT, and nowadays even smartphones can support that.
Blaisorblade
  • 6,438
  • 1
  • 43
  • 76
3

Have you perhaps heard a reference to the JDDK?

Writing a device driver 100% in Java is not possible without native code to provide the interaction between (1) the OS-specific driver entry points and conventions, and (2) the JVM instance. The JVM instance could be started "in-process" (and "in-process" may have different meanings depending on the OS and on whether the driver is a kernel-mode or user-mode driver), or as a separate user-land process with which a thin, native driver adaptation layer can communicate and onto which the said driver adaptation layer can offload actual user-land work.

vladr
  • 65,483
  • 18
  • 129
  • 130
  • Are you so sure about the 'not possible'-part, that you make it bold? You can compile java-code to a native-binary, you can have a Java-Processor as hardware (mobile phone) or your kernel contains a JVM. All of that are ways to allow pure Java-device-drivers. – Mnementh Mar 25 '09 at 22:27
  • there is no java cpu and so it is impossible to get down to the hardware. you need machine code to access the bits and bytes. – Peter Parker Mar 25 '09 at 23:01
  • @mnemeth, extremely sure; compiling java to native (non-java) code gets you nowhere without native-code wrappers; in the case of picojava note that I bolded more than just "not possible": my statement still logically hods (the native code is java bytecode -- *then* it is possible.) – vladr Mar 25 '09 at 23:38
  • @mnemeth, also note that "your phone" is most likely not running java bytecode natively, or at least not on the CPU :) (in the best of cases they will use a coprocessor to help with some of the bytecode - see jazelle) – vladr Mar 25 '09 at 23:44
  • You are right, you need native code. But you always need native code to execute code (even non-native). So, I can say: "It is not possible to write a device driver with C without native code." That's true the same. (and I didn't downvoted you, I only asked) – Mnementh Mar 27 '09 at 13:37
  • You are correct, I should have qualified that: without *custom* native code (i.e. facilities not already provided in the JVM or in some abstraction library -- something one would, as of today, have to write from scratch.) – vladr Mar 27 '09 at 16:00
  • Sorry to downvote, but this is flat-out wrong, at least for PCIe. There exist pure Java user-space drivers for Infiniband hardware, today (google JVerbs). The key technique here is map the memory regions of the device into the jvm's address space (/dev/mem or UIO); from there they can be directly manipulated as ByteBuffers. UIO provides an abstraction for receiving interrupts as well. – Ross Judson Nov 13 '13 at 19:59
  • @RossJudson, so how do you map memory into the JVM's address space using pure Java? Your so-called Java "driver" will ALWAYS need native assistance, be it JNI, or some other plumbing that exposes the device at the filesystem level -- you name it. – vladr Jan 22 '14 at 05:08
  • @vladr If the driver makes its control registers and data available at the filesystem level (as you point out), then memory-mapping that file in allows a device driver to be written in an unchanged JVM, without JNI. The difficulty there is not having direct access to interrupts. UIO maps generic PCIe devices to the file system, and provides an endpoint for retrieving interrupts as well. You are right -- the file system is the means by which the memory can be mapped, but it does not take JNI or a modified JVM to do it. – Ross Judson Feb 12 '14 at 18:22
  • @vladr I have used this technique to draw directly on the Linux frame buffer, for example. The frame buffer's file can be mapped in (with significant difficulty, because Java's libraries try to alter the length of the file, and that must be worked around). – Ross Judson Feb 12 '14 at 18:22
  • @vladr I should clarify precisely that I am stating it is possible to *drive devices* from user space Java. I am not stating that it is possible to build a *device driver* in user space Java that is accessible to *other* programs via standard mechanisms... – Ross Judson Feb 12 '14 at 18:25
  • Some hardware just communicates using sockets ; why would you need any native code in that case? - Conclusion: it really depends on what kind of hardware you are dealing with. – bvdb Jan 06 '16 at 00:18
3

You have a too narrow view of device drivers.

I have written such device drivers on top of MOST in an automotive application. A more widespread use might be drivers for USB devices if Java ever gets a decent USB library.

In these cases there is a generic low-level protocol which is handled in native code, and the Java driver handles the device specifics (data formats, state machines, ...).

starblue
  • 55,348
  • 14
  • 97
  • 151
2

It is possible to compile java code to hardware native (i.e. not JVM bytecode) instructions. See for instance GCJ. With this in hand, you're a lot closer to being able to compile device drivers than you were before.

I don't know how practical it is, though.

dmckee --- ex-moderator kitten
  • 98,632
  • 24
  • 142
  • 234
2

Possible?

Yes but only in special circumstances. Because you can write an operating system in Java and C#, and then, should be able to write device drivers for it. The memory hit to these drivers and operating systems would be substantial.

Probable?

Not likely. Atleast not in the world of Windows or MacOS or even Linux... At least not anytime soon. Because languages like C# and Java depend on the CLR and JVM. The way these languages work means that they cannot effectively be loaded into ring0.

Also, the performance hit would be rather large if managed languages were employed in device drivers.

bdd
  • 3,436
  • 5
  • 31
  • 43
1

The Windows Driver Foundation (WDF) is a Microsoft API that does allow both User and Kernel mode device drivers to be written. This is being done today, and it is now compatible with w2k and later (used to not have w2k as a supported target). There is no reason that JNI calls can't be made to do some work in the JRE . . . ( assuming that JNI is still the way to call Java from C/C++ . . . my knowledge is dated in that arena). This could be an interesting way to have high level algorithms directly munch on data from a USB pipe for something to that effect . . . cool stuff!

Jon B
  • 51,025
  • 31
  • 133
  • 161
TheEruditeTroglodyte
  • 360
  • 1
  • 4
  • The problem with Oracle's JNI, is that the C++ code has to be structured in a very specific way. i.e. your C++ method names should match the name of your consuming java class, (imho a bit awkward). So, if you want to use existing DLL's, you will still need to write a wrapper in C++. -- Fortunately, there is another way. There are commercial alternatives to JNI which are more powerful. (e.g. jacozoom and comjni). But since there are so many types of DLL files, with little transparency, these tools don't always work. It is a process of "trial and error", succes is not guarenteed. – bvdb Aug 05 '17 at 10:01
1

PCIe user space device drivers can be written in Pure Java. See JVerbs for details about memory-based direct hardware access, in the context of OFED. This is a technique that can be used to create very high performance systems.

You can examine the PCI bus to determine the memory regions for a given device, what ports it has, etc. The memory regions can be mapped into the JVM's process.

Of course, you're responsible for implementing everything yourself.

I didn't say easy. I said possible. ;)

See also Device Drivers in User Space, which discusses using the UIO framework to build a user space driver.

Ross Judson
  • 1,132
  • 5
  • 11
1

Device drivers have to be written in a language which can execute in the kernel, either compiled into it, or loaded as a module at runtime. This usually precludes writing device drivers in Java, but I suppose you theoretically could implement a JVM inside a device driver and let it execute Java code. Not that any sane person would want to do that.

On Linux there are several user-land (i.e. non-kernel) implementations of filesystems which uses a common abstraction layer called (fuse) which allows user-land programs to implement things which are typically done in the kernel.

JesperE
  • 63,317
  • 21
  • 138
  • 197
  • Some hardware just communicates using sockets.So, it really depends on what kind of hardware you are dealing with. – bvdb Jan 06 '16 at 00:19
-1

First of all, note that I'm not an expert on device drivers (though I wrote a few myself back in the day), much less an expert on Java.

Let's leave the fact that writing device drivers in a high-level language is not a good idea (for performance and possibly many other reasons) aside for a moment, and answer your question.

You can write device drivers in almost any language, at least in theory.

However, most device drivers need to do plenty of low-level stuff like handling interrupts and communicating with the OS using the OS APIs and system calls, which I believe you can't do in Java.

But, if your device communicates using, say, a serial port or USB, and if the OS doesn't necessarily need to be aware of the device (only your application will access the device*), then you can write the driver in any language that provides the necessary means to access the device.

So for example you probably can't write a SCSI card driver in Java, but you can write a driver for a proprietary control device, USB lava lamp, license dongle, etc.

* The obvious question here is, of course, does that count as a driver?

Can Berk Güder
  • 109,922
  • 25
  • 130
  • 137