7

I've worked on a number of products that make use of code generation. It seems to be the only way to achieve both a high degree of user-customizability and high execution speed.

The downside is that we are requiring users to install a compiler (primarily on MS Windows).

This has been an on-going headache, because vendors like MS keep obsoleting compilers, and some users tend to have more than one compiler installed.

We're considering using GNU C, and possibly C++, but even there, there are continual version issues.

I've considered possibly generating assembly language, in an effort to get off the compiler-version-treadmill, but assembly languages are all machine-specific.

Ideally there would be some way to produce generated code that would be flexible, run fast, and not expose us to the whims of third-party providers.

Maybe I'm overlooking something simple, like Java. Any ideas would be appreciated. Thanks.

Mike Dunlavey
  • 40,059
  • 14
  • 91
  • 135

12 Answers12

8

If you're considering C and even assembler, take a look at LLVM first: http://llvm.org

Shannon Weyrick
  • 96
  • 3
3

I might be missing some context here, but could you just pin yourself to a specific version? E.g., .NET 2.0 can be installed side by side with .NET 1.1 and .NET 3.5, as well as other versions that will come out in the future. So as long as your code makes use of a specific version of a compiler, what's the problem?

Paul Stovell
  • 32,377
  • 16
  • 80
  • 108
  • That's a good suggestion, aside from being windows-specific. I will consider that. – Mike Dunlavey Feb 25 '09 at 17:53
  • Ideally, it wouldn't be windows-limited, and could be relied on to work 10-15 years from now. – Mike Dunlavey Feb 25 '09 at 17:56
  • Yes, but you could also target something like C# 1.0. That should be forwards compatible and supported on other platforms via the Mono project. – dhable Feb 25 '09 at 18:48
  • 1
    visual basic 4.0 was released about 15 years ago, and applications built with it still run (although it there is no more tech support). Why would .NET applications suddenly stop running in 10 years? – Paul Stovell Feb 25 '09 at 23:55
  • and as Dan pointed out, the mono project provides an open source implementation of .NET 2.0 that works on windows and other platforms. And C# and the CLI have been through ECMA and ISO standardisation. http://en.wikipedia.org/wiki/Microsoft_.NET#Standardization_and_licensing – Paul Stovell Feb 25 '09 at 23:58
3

I've considered possibly generating assembly language, in an effort to get off the compiler-version-treadmill, but assembly languages are all machine-specific.

That would be called a compiler :)

Why don't you stick to C90?

I haven't heard much of severe violations of standards from gcc's side, if you don't use extensions.

And you can always distribute a certain version of gcc along with your product, say, 4.3.2, giving an option to users to use their own compiler at their own risk.

As long as all code is generated by you (i. e. you don't embed your instructions into other's code), there shouldn't be any problems in testing against this version and using it to compile your libraries.

Quassnoi
  • 413,100
  • 91
  • 616
  • 614
3

If you want to generate assembly language code, you may take a look at asmjit.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
kcwu
  • 6,778
  • 4
  • 27
  • 32
2

Start here: The Code Generation conference

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
SQLMenace
  • 132,095
  • 25
  • 206
  • 225
2

One option would be to use a language/environment that provides access to the compiler in code; For example, here is a C# example.

Chris Shaffer
  • 32,199
  • 5
  • 49
  • 61
  • Thanks. That looks good, but I wonder about its longevity and/or platform independence. – Mike Dunlavey Feb 25 '09 at 18:06
  • I don't think Longevity should be a concern; As long as your application can run on the client machine, the code generation should work(since they both depend on the same framework). Platform dependence will depend on your language; For C#, you could look into Mono (Open source .net implementation). – Chris Shaffer Feb 25 '09 at 18:17
  • After reading some of your other comments regarding longevity I see I misunderstood what you were looking for. 10-15 years is going to be hard to guarantee anywhere, though I think a language that builds to intermediate code (like C# or Java) would be the safest bet... – Chris Shaffer Feb 25 '09 at 18:50
2

Why not ship a GNU C compiler with your code generator? That way you have no version issues, and the client can constantly generate code that is usable.

Beep beep
  • 18,873
  • 12
  • 63
  • 78
2

It sounds like you're looking for LLVM.

Ken
  • 5,074
  • 6
  • 30
  • 26
1

Embed an interpreter for a language like Lua/Scheme into your program, and generate code in that language.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
alvin
  • 1,176
  • 8
  • 15
  • Thanks. Not too late. I think that's a good idea. An argument against it is if performance is an issue, it is better to generate a language that can be dynamically compiled. Of course, the problem with that is my app isn't complete without such a compiler/linker. I've always been on the lookout for a middle ground. But if performance is not such an issue, I like your idea. – Mike Dunlavey Oct 08 '10 at 17:45
1

In the spirit of "might not be to late to add my 2 cents" as in @Alvin's answer's case, here is something I'd think about: if your application is meant to last for some years, it is going to face several changes in how applications and systems work.

For instance, let's say you were thinking about this 10 years ago. I was watching Dexter back then, but I guess you actually have memories of how things were at that time. From what I can tell, multithreading was not much of an issue to developers of 2000, and now it is. So Moore's law broke for them. Before that people didn't even care about what will happen in "Y2K".

Speaking of Moore's law, processors are indeed getting quite fast, so maybe certain optimizations won't be even that necessary. And possibly the array of optimizations will be much bigger, some processors are getting optimizations for several server-centric stuff (XML, cryptography, compression and regex! I am surprised such things can get done on a chip) and also spend less energy (which is probably very important for warfare hardware...).

My point being that focusing on what exist today as a platform for tomorrow is not a good idea. Make it work today, and surely it will work tomorrow (backward-compatibility is especially valued by Microsoft, Apple is not bad it seems and Linux is very liberal about making it work as you want).

There is, yes, one thing that you can do. Attach your technology to something that just won't (likely) die, such as Javascript. I'm serious, Javascript VMs are getting terribly efficient nowdays and are just going to get better, plus everyone loves it so it's not going to dissappear suddenly. If needing more efficiency/features, maybe target the CRL or JVM?

Also I believe multithreading will become more and more of an issue. I have a gut feeling the number of processor cores will have a Moore's law of their own. And architectures are more than likely to change, from the looks of the cloud buzz.

PS: In any case, I belive C optimizations of the past are still quite valid under modern compilers!

Camilo Martin
  • 37,236
  • 20
  • 111
  • 154
  • I appreciate your views (though you are giving me a chuckle :). There's another law: Nature abhors a vacuum. Good old Moore will never outrun coders. If there are more cycles to burn, those could be used for added wonderfulness, but what usually happens is they get glommed by programmers out of sheer carelessness, and there's no fundamental limit to how wasteful software can be. Never fear, no amount of threads, cores, clouds, compiler wizardry, quantums, can outrun any programmer of average foolishness. – Mike Dunlavey Nov 21 '10 at 14:00
  • ... Not to give you the impression that I'm being negative :-) There is a skill of performance tuning which, if people actually use it (which they talk about but don't do, really), can remove performance bugs, and put all those wonderful cycles to *good* use: http://stackoverflow.com/questions/926266/performance-optimization-strategies-of-last-resort/927773#927773 – Mike Dunlavey Nov 21 '10 at 14:06
  • @Mike Well, you are right in that, and the actual code that CPUs process nowdays is far from the efficiency of old code. I was surprised that a very old (90's) GIF maker I used recently was about twice as fast and memory-efficient as Phtoshop CS3 (in my case it mattered, because the GIF had ~600 frames), So I'd guess even codebases as Photoshop's aren't well optimized. Seriously, I'm relatively new at programming, and I'm starting to feel a bit like you regarding how slow software is compared to what it could be if companies (and, hence, programers) wanted to really optimize for speed. – Camilo Martin Nov 22 '10 at 05:59
  • ...But to think about it, it's easier being a one-eyed king when people are closing their eyes. :) I hope one day to have your coding skills. – Camilo Martin Nov 22 '10 at 06:04
  • Is that a Portugues concept? It sounds a bit like one of my favorites, the "Emperor's new clothes", but I'll have to think about it. – Mike Dunlavey Nov 22 '10 at 13:08
  • I was referring to this proverb: http://en.wiktionary.org/wiki/in_the_land_of_the_blind,_the_one-eyed_man_is_king - I've always thought it to be more common in english-speaking countries. What I meant is, if everyone starts relying too much on leaky abstractions, knowing what's under the hood becomes more of an advantage. I'm sure you feel good when you solve an obscure problem easily by knowing where to look. :) – Camilo Martin Nov 22 '10 at 19:00
1

I would stick to that language that you use for generating that language. You can generate and compile Java code in Java, Python code in Python, C# in C#, and even Lisp in Lisp, etc.

But it is not clear whether such languages are sufficiently fast for you. For top speed I would choose to generate C++ and use GCC for compilation.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Jiri
  • 16,425
  • 6
  • 52
  • 68
  • Thanks. We're considering gcc. We want it to be a compiler/linker/libraries that can still be installed and will work years and years from now. – Mike Dunlavey Feb 25 '09 at 18:24
  • FWIW, some Lisp implementations are extremely fast (such as CMU CL). They may be unsuitable for other reasons (like lack of libraries, or bad integration with the OS). – David Thornley Feb 25 '09 at 18:27
  • @David: Good suggestion. A little far-out for this outfit, though. – Mike Dunlavey Feb 25 '09 at 18:45
1

Why not use something like SpiderMonkey or Rhino (JavaScript support in Java or C++). You can export your objects to JavaScript namespaces, and your users don't have to compile anything.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Chris K
  • 11,996
  • 7
  • 37
  • 65