12

I've heard of Java programs with strict latency requirements where "new" instructions are never -or very rarely- used (because no new=>no objects->no GC=>improved latency)... instead all business logic is handled using primitives.

I couldnt find references to this technique online though... any link to a webpage discussing this technique or snippets of code would be much appreciated.

andersoj
  • 22,406
  • 7
  • 62
  • 73
Eleco
  • 3,194
  • 9
  • 31
  • 40
  • 3
    Business logic using primitives? Is today April 1st? – Sean Patrick Floyd Nov 01 '10 at 15:27
  • 2
    That sounds like a super terrible idea! – ColinD Nov 01 '10 at 15:27
  • 2
    Seriously: this sounds like an awful case of Premature Optimization – Sean Patrick Floyd Nov 01 '10 at 15:28
  • wondering if it is 2010? – Mark Baijens Nov 01 '10 at 15:29
  • 7
    It's not necessarily premature optimisation. If your requirements say that late == wrong (hard real time) you do need to do something like this. Or use a JVM with a real-time GC pause guarantee (see, for instance, http://domino.research.ibm.com/comm/research_projects.nsf/pages/metronome.metronomegc.html) – The Archetypal Paul Nov 01 '10 at 15:31
  • 2
    @ColinD, Seanizer: the question is from a performance viewpoint only... it is agreed that using primitives instead of objects wont result in the most beautiful code ever ;) I'm talking about apps with very low latency requirements < 1 ms. – Eleco Nov 01 '10 at 15:37
  • @elec: I'm more or less in agreement with sleske that if you want to use Java like C, just use C instead. – ColinD Nov 01 '10 at 15:38
  • If you aren't able to find references to this technique online, perhaps that's a sign as to how bad of an idea it is? – matt b Nov 01 '10 at 15:44
  • 2
    Apart from the use of Java, it's a common technique when programming real time systems – The Archetypal Paul Nov 01 '10 at 15:45
  • @Paul: Do you have any references for that? I'd be interested to look at them. – sleske Nov 01 '10 at 21:27
  • Not really, I tried googling for them, but the idea of using static allocation (which is all these technique is, really) is common. There doesnt seem to be a particular term for it, though. – The Archetypal Paul Nov 01 '10 at 21:33
  • 1
    @Paul, @sleske: http://www.cotsjournalonline.com/articles/view/101217 in general, safety critical systems meeting DO-178B are not allowed to do dynamic memory allocation. – andersoj Nov 01 '10 at 21:59
  • @Paul, @sleske: See also http://stackoverflow.com/questions/2469904/resources-for-memory-management-in-embedded-application – andersoj Nov 01 '10 at 22:02
  • "new" is pretty fast nowadays – cherouvim Nov 02 '10 at 07:59
  • 4
    It's nothing to do with how fast new is. It's being able to *know* what the latency of response to some event is, which depends on having a garbage colllector that has a hard upper limit on gc pauses. Most JVMs do not have such a limit, and so (for hard real time) you need to avoid new. – The Archetypal Paul Nov 02 '10 at 08:26
  • 1
    For some reason, the distinction between "fast" and "predictable" is a major barrier when talking with people about real-time and safety-critical systems. Building on what @Paul points out, note that RT/SC system designers routinely make decisions that pay *major* penalties in throughput (orders of magnitude) like turning off caches, using dead-simple data structures, etc... in order to obtain predictability (or, in the extreme case, determinism). **In real-time systems, the time at which a computation completes is part of its correctness condition.** Throughput is secondary. – andersoj Nov 02 '10 at 13:42
  • 1
    Since I ran out of space... if the time at which a computation completes is part of its correctness, then you need to be able to demonstrate/verify that with the same fervor that you might bring to unit testing, for instance. Since the time-correctness of the system as a whole is easiest to reason about in terms of the aggregate time-behavior of unit operations, the designer needs the timing of those operations to be well understood. Hence the obsession with determinism. Just extend the mentality behind unit testing to measuring/assuring timeliness, and you'll get the drift. – andersoj Nov 02 '10 at 13:46
  • 2
    @ColinD: Your point about abusing java to behave like C is well-taken, but a counterpoint is to note that many time-critical systems have a very small subset of truly low-latency behaviors, alongside a much larger body of less stringently constrained code. The RT Java approaches (promise to) reduce cost and complexity by allowing you to do both in the same codebase, with well-defined coupling between them. For specific classes of systems, this makes a great deal of sense (e.g., defense and aviation), though it's far from established whether or not it will be successful in the long run. – andersoj Nov 02 '10 at 13:51
  • 1
    Why is *new* unpredictable ? As long as you reference your objects, they won't be deleted, so no GC, only static allocation, but use of classes, structs, enums = faster design (while maintaining RealTime constraints). – Aurelien Ribon Nov 02 '10 at 14:39
  • 1
    Do you know the complexity that may come into play when you call `new`? Certainly it has little impact on most applications, but what level of certainty do you have? What are all the potential cascading effects of asking for a new instance in Java? Classloaders? Static initializers? Calls to the OS for more pages? Here's an article that might provide some food for thought, just on the topic of memory allocation http://www.nojitter.com/feature/210600038?pgno=4 – andersoj Nov 02 '10 at 15:33
  • Here's an example of a special-purpose allocator for RT systems; there are others. My guess is that your garden variety JVM just uses the platform `malloc()`, with perhaps some additional magic on top. http://rtportal.upv.es/rtmalloc/ – andersoj Nov 02 '10 at 15:42
  • Some actual data showing instruction counts for several memory allocators in the last bit of these slides: http://rtportal.upv.es/rtdsa/files/rtportal.upv.es.rtdsa/documents/tlsf_slides.pdf – andersoj Nov 02 '10 at 15:49
  • 1
    @Aurélien Ribon, a GC isn't triggered by objects being referenced, but by more memory being required, or after some time limit, or some other trigger. It then discovers the unreferenced objects (over simplfying a bit there). So the unpredictability of new is in whether it causes a CD then or subseqently.. – The Archetypal Paul Nov 02 '10 at 17:47
  • @Paul: Also, there is no specification for the time complexity of `new`, so a RT programmer/analyst has to assume the worst (unless they are able to establish otherwise). One of the benefits of the RTSJ and vendor-provided RTSJ implementations is that the time-complexity and expected behavior is specified for a bunch of these primitive operations. The time cost is part of a method/operator's contract. – andersoj Nov 03 '10 at 00:22
  • @sleske, Just google . The techniques are general, it can be done with JavaScript too. – Pacerier Sep 19 '17 at 22:24

6 Answers6

15

Update (March 2012): While there is no public announcement available, it appears Oracle has shut down their JavaRTS group.

The two topics you should look at are the Real-Time Specification for Java (JSR-1, JSR-282) and the Safety-Critical Specification for Java (JSR-302), (aonix page). RTSJ provides the resources for doing the type of careful memory management you are talking about in your question, though most of the users of RTSJ probably tolerate greater jitter than an "all-primitives" approach would (and, for that tolerance, they get the benefits of GC by using an RTGC). For some examples of the kinds of limited allocation (limiting when/where/how many times you can use new) search for "rtsj scoped memory rules", and here's an example academic paper on the topic. You should also look at "eventrons" as an example of trying to solve these high-frequency, low-latency constraints in Java in a (relatively) clean way.

For a specific new story on the use of JavaRTS in trading contexts, see here. For some instructions on how to get the most deterministic behavior out of JavaRTS in particular, see here.

Safety-Critical users are more likely to have these sorts of constraints, and SC-Java is (expected to be) partially a downscoped version of RTSJ. You can see some examples of what the expert group is looking at in Doug Locke's 2007 status slides from JTRES.

Several folks produce JVMs intended for use in this environment. Aonix/Atego's PERC; aicas' JamaicaVM; Apogee's Aphelion. The Sun/Oracle JavaRTS is targeted more towards larger applications that can tolerate (and benefit from) less stringent constraints.

While the "all-primitives" approach you cite is an extreme form of this type of programming, you are most likely to find resources around this topic at one of the above references.

For the "business logic in primitives?" naysayers above me, note that substantial amounts of IRS tax business logic is implemented in IBM assembler, and a number of "enterprise" layers like Java are being looked at to wrap (not replace!) that logic. Put that in your pipe and smoke it.

I can't cite any specifics here, but there are a number of defense applications using Java (usually a flavor of RTSJ) that have very tight constraints on memory allocation, and much of the code is statically-allocated, primitive focused. Signal processing, etc... I did a search for open source materials on systems I'm aware of, and these documents (see especially the references sections) point the way to some specific examples of these applications.

andersoj
  • 22,406
  • 7
  • 62
  • 73
  • One downside of realtime VMs is that they tend to be fairly expensive – Eleco Nov 03 '10 at 08:03
  • @elec, well you can use the JavaRTS for free for limited (academic or non-commercial) uses, and most of the other vendors are generous about licensing for these purposes. But they have to turn a buck, and this is not a high-volume market. See my comments here http://stackoverflow.com/questions/4051966/which-real-time-rtsj-jvm-is-most-preferred/4083572#4083572 – andersoj Nov 03 '10 at 12:58
5

I have worked on a number of such systems. You will need to worry about creating objects if you need sub millisecond latencies. It is possible to write an application which doesn't GC all day, just to avoid any GC delays.

However, 99%+ of applications don't need this extreme.

andersoj
  • 22,406
  • 7
  • 62
  • 73
Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
3

No, I have never heard of such programs, or even of the technique.

It seems like a bad idea, because then you are effectively limited to what you can do in C or similar low-level languages. And if you want that, it's easier to write in C. Plus there are many ways to avoid long GC pauses, and in practice most low-latency requirements can be met by them.

Plus, in order to do anything useful, you'll have to use the Java platform APIs, or other 3rd party libraries, which will probably allocate plenty of objects behind your back, so avoiding all object instantiations is probably not even practical in a non-trivial program.

So I am fairly certain that this is an urban legend, or at most a niche idea.

Edit:

This technique is used to obtain realtime or low-latency behaviour. Nowadays it might be obsolete because of better GC algorithms, but this will of course depend on circumstances. So it is probably something to consider at least for hotspots of an algorithm.

As an example:

Many Java realtime environments place some restrictions on object creation. This does not mean that they can only use primitives: Use of complex objects is still possible, but for example Safety Critical Java ( http://www.aicas.com/papers/scj.pdf ) requires all object instantiations to happen during an initialization phase. Once the application is running ("mission phase"), instantiation is no longer allowed, so you have to work with the object instances you have.

This avoids the unpredictability introduced by dynamic allocation and garbage collection, but still allows the use of dynamic memory (in a limited fashion).

Thanks to andersoj & mikera for explaining this to me.

sleske
  • 81,358
  • 34
  • 189
  • 227
  • 1
    It's worse than C. C has composite data types (structs). –  Nov 01 '10 at 19:31
  • -1: There are plenty of examples of statically allocated, primitives-only programs. Just not in the general purpose computing world... – andersoj Nov 01 '10 at 23:26
  • -1 for the false assertions: this is a perfectly valid technique and useful that is applied in many low latency situations. financial market trading, multimedia/sound production and gaming are all good examples where this kind of thing is common. – mikera Nov 02 '10 at 00:09
  • @andersoj, mikera: I really appreciate learning about systems like this; it just seemed a very strange idea. Could you provide any references / pointers to systems coded like this? – sleske Nov 02 '10 at 01:02
  • @sleske: It's difficult to point to specific instances of these systems, partly because those I've worked on tend to be defense applications. Here's a publicly-released document on SCJ that points to a few. http://www.cs.purdue.edu/homes/jv/pubs/safecert09.pdf and http://www.aicas.com/papers/scj.pdf Another example that has publicly acknowledged use of RT Java is the AN/FPS-85 SpaceTrack radar. http://unix.org.in/2010/10/sun-java-real-time-system-selected-for-space-surveillance-radar-java-technology-enables-real-time-behavior-and-throughputaviation – andersoj Nov 02 '10 at 01:42
  • @andersoj: Thanks for the interesting links. I had no idea about this. Looks like a fascinating area. – sleske Nov 02 '10 at 14:33
  • @delnan, Structs can be implemented using primitives and functions. – Pacerier Sep 19 '17 at 22:27
3

Check out the Realtime specification for Java.

Though it has rather more complicated rules w.r.t. allocation.

starblue
  • 55,348
  • 14
  • 97
  • 151
2

There might be some trading systems that do this, but from my understanding (My colleagues work in Java based low-latency trading platforms) a good fast modern JVM combined with sensible coding practice removes the need to go for 'pure primitives' in code. That said I'm not a low-latency developer, our middleware team only worries about 1,000's of txn's per minute, not 10,000's+. Can't say I've found a convincing article on this, although I'm happy to be pleasantly surprised :)

Martijn Verburg
  • 3,287
  • 21
  • 26
  • 2
    Read the metronome papers here http://domino.research.ibm.com/comm/research_projects.nsf/pages/metronome.metronomegc.html. Fascinating stuff. Modern JVMs are good enough for soft real time (late is bad but not wrong) but you need something special for hard real time (nuclear reactors, fly-by-wire software...) – The Archetypal Paul Nov 01 '10 at 15:40
  • Thanks @Paul - good stuff that. I wonder how much if any of it might find its way into the openJDK/future JVM – Martijn Verburg Nov 01 '10 at 15:52
  • @Paul: Metronome is great stuff. For contrast, compare metronome to Sun/Oracle's JavaRTS garbage collection approach. – andersoj Nov 01 '10 at 23:53
2

I've used this technique in game programming, where too many long GC delays can degrade the user experience. In practice it's not necessary to eliminate all "new" instructions, just keep them to a sensible level.

A good example of a library that uses a lot of these techniques is Javolution - well worth a look if you are either interested in how it is coded or want to use ready-built data structures and algorithms that support operation with minimal allocations.

mikera
  • 105,238
  • 25
  • 256
  • 415