59

I am working on an embedded system running Linux on a DSP. Now we want to make some parts of it scriptable and we are looking for a nice embeddable scripting language. These scripts should integrate nicely with our existing C++ code base, be small and fast.

I understand that Lua is the industry choice for problems like this. We will probably go with Lua because it is tried-and-true and proven to be stable and so on. However, as a programming language it has some rather quirky corners.

So, what alternatives are out there for embeddable languages?

EDIT:

This is about a year later.

We actually used Lua on our embedded system and it performs marvelously well. Over time, we added more and more scripting support to more and more parts of the project and that really helped to bring it along.

Performance is outstanding, really. Even rather complex operations that involve searching through long arrays or fancy string operations perform surprisingly well. We basically never ran into Lua related performance problems at all.

Interfacing with C functions is very straightforward and works really well. This allowed us to grow the scripting system painlessly.

Finally, we were astounded at how flexible Lua proved to be. Our Lua interpreter has to run on a system with a nonstandard memory allocator and without support for the double data type. There are two well-documented places in one header file we had to modify to make Lua work on that system. It is really well suited for embedding!

bastibe
  • 16,551
  • 28
  • 95
  • 126

8 Answers8

39

Since you say "embedded system" and "small and fast" and "integrate nicely" I would say you are correct that Lua is the number one if not the only choice. But I no longer agree that the programming language has "quirky corners". Firstly, the book Programming in Lua is simply splendid, one of the best books I have ever read. Secondly, some of the "quirky corners" come from the fact that the language is very orthogonal and clean, which in the long run is an asset, not a drawback. I find for example JavaScript much worse. If you read "Javascript the good parts" the author explains in length why some constructs in the language are design mistakes and why one should avoid the new operator. Not so in Lua, the bad parts have been removed, for example the quirky upvalue stuff was replaced with standard syntactic scoping in version 5.x.

My view is actually that Lua is a language with far less quirky corners than most other languages! We use it in a commercial project and we are more than happy with it.

AndersH
  • 808
  • 6
  • 7
  • I actually read that book already. A book like that probably makes for a better language choice in itself. – bastibe Dec 15 '10 at 13:08
  • 6
    I would point out that Lua still has some quirks - but I do agree that it has less than most. +1 – kikito Dec 15 '10 at 17:24
  • I very much agree with this assessment. I yearn for Lua in the (thankfully) rare event that I need to write some JavaScript for a web page. – Judge Maygarden Dec 15 '10 at 20:56
  • 28
    Some quirks in Lua that have led me to search for an alternative language: 1-based arrays, lack of Unicode support, inability to use nil as a table key, unspecified behaviour of the # (length) operator in that it *may* stop counting when it finds a nil (so inserting a nil into an array produces undefined results), the fact that `ipairs` may produce fewer results than #, no way to tell the number of keys in a table (# only works on arrays), extremely weak typing (`"abc" + "456"` is an error, `"123" + "456"` is 579). Next to Lua, Python is a dream (but not really embeddable, unfortunately). – mgiuca Aug 26 '11 at 02:00
  • @mgiuca why do you say that python is not embeddable? Is security one of the reasons? – uniqueid May 01 '21 at 00:08
  • 1
    @uniqueid Python assumes much more direct access to OS resources, such as threading and filesystems. Can you really remove those from a Python runtime? – SOFe Apr 07 '22 at 08:46
14

I wholeheartedly recommend Lua for your use case. However, Forth is an alternative--especially for resource constrained embedded devices--that has not yet been mentioned.

Judge Maygarden
  • 26,961
  • 9
  • 82
  • 99
12

There's always Lisp. :) But that underscores the fact that Lua is in fact less "quirky" than most languages. It was designed for non-programmers and reads like pseudocode. It has clean, uniform semantics (first class nested functions with lexical scoping; multiple assignment; multiple return values; a single, flexible data structuring mechanism with clean constructor syntax; etc.) which make it very easy to learn, read, write, etc. It also happens to be unexpectedly powerful and expressive (proper tail calls, continuations, metaprogramming, etc.)

The only really "quirky" aspect of Lua is that arrays index from 1, and that fact that it doesn't borrow C's conventions like everybody else (~= rather than !=, -- rather than //, etc.), but these are mostly quirky through the eyes of programmers habituated to C-like languages.

An alternative might be Squirrel, which is inspired by Lua, has similar goals, but C-like syntax. I've not used it though, so I don't know well it meets it's goals.

Mud
  • 28,277
  • 11
  • 59
  • 92
  • 2
    See my comment above (on the accepted answer) regarding the many, many quirks of Lua. Lisp has many different dialects with various issues, but Scheme (probably the second most popular Lisp dialect) is about the cleanest language I know of. I agree that first-class nested functions with lexical scoping is ideal in Lua, but that's also true of Scheme and Python and many others. – mgiuca Aug 26 '11 at 02:03
  • 7
    IMO the quirks you mention are rather superficial quirks. Lua strings can hold arbitrary data and I've used Unicode with external libraries that require it without issues. # *is* well specified; you just don't want to use it with sparse arrays. Coercion of string to number arithmetical expressions is a (mis)feature of many dynamic languages, but it's less of an issue in Lua because Lua doesn't use + for concatenation (much less overload other operators, like *, as in Ruby). Lua is less quirky where it counts, such as the uniform handling of scope, the generality of tables, etc. – Mud Aug 26 '11 at 05:24
  • 2
    Lack of Unicode is not a "quirk" but it is poorly abstracted. # is *explicitly* specified as having undefined results in some cases ("The length of a table t is defined to be any integer index n such that t[n] is not nil and t[n+1] is nil"). That is a quirk by any definition. I agree that it's great that Lua has a separate concatenation operator. I'm aware that other dynamic languages have the same "(mis)feature", but coming from a Python background this seems very backwards. I agree, other than what I mentioned, Lua's semantics are very nice. – mgiuca Aug 28 '11 at 06:56
  • 7
    (1) It seems you have little idea what Unicode support entails and/or what Lua's design goals are. Lua strings can hold arbitrary data, including Unicode, and can be extended via metatables. If you need support you can get it. The out-of-the box standard library is ultra-minimal *on purpose*. (2) You say # has *unspecified* behavior, then quote the *specification*. \*lol\* The circumstances where behavior is **undefined** is **well specified**, much [as in C](http://en.wikipedia.org/wiki/Undefined_behavior). (3) Python is a lot quirkier than Lua, in much more important areas. – Mud Aug 28 '11 at 07:58
  • Believe me, as a language implementer I am quite aware of the design issues surrounding Unicode (you might for example search for "uri encoding" on Google and click the third link to find a blog post of mine on the matter). I am aware that Lua's strings are designed to be 8-bit clean, and that's a useful data type, but not a string. Python's strings were originally designed that way until the language designers realised their mistake and fixed it in Python 3 with "bytes" (8-bit clean binary data) and "str" (properly abstracted Unicode strings). JavaScript similarly has string and Uint8Array. – mgiuca Aug 28 '11 at 08:08
  • 3
    (2) I don't see the irony in quoting the specification. The specification does not specify the behaviour in certain circumstances. That is the definition of unspecified behaviour. Yes, there are well-specified situations in which it will work, but there are also other situations in which it is unspecified. Most high-level languages do not have unspecified behaviour -- C does, but that's because it's a very low-level language. (3) What would you say are quirks of Python? Sure it is a much more *complex* language than Lua (and not appropriate for embedding), but I wouldn't say more quirky. – mgiuca Aug 28 '11 at 08:12
  • @mgiuca let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/2933/discussion-between-mud-and-mgiuca) – Mud Aug 28 '11 at 08:54
  • 1
    Sorry, I was offline when you sent that last message. – mgiuca Aug 31 '11 at 04:14
4

Tcl was designed from the ground up to be an embedded language, and has been around for decades. Plus, it is a perfect choice for developing a domain-specific language because of its highly extensible nature.

I don't know a lot about the DSP world, but when you google "dsp lua" and "dsp tcl" you get twice as many hits for Tcl.

Bryan Oakley
  • 370,779
  • 53
  • 539
  • 685
  • 8
    You get more hits for Tcl because it had a half a decade head start. The author needs a language for an embedded system, and [Tcl is bloated and slow language compared to Lua](http://dada.perl.it/shootout/craps.html), and he's concerned about "quirkiness", an area where Tcl trumps almost anything, while Lua reads almost like pseudocode. – Mud Dec 15 '10 at 21:42
  • 2
    @Mud: *Simple* Lua reads like pseudocode. More complex Lua reads like... well, it actually reads all right, but sometimes expresses some quite unexpected things. Which is not a bad thing, only it is not very pseudocode-y. – bastibe Dec 16 '10 at 09:21
  • @mud: "bloated and slow" may not be a factor. For one, "bloated" can also be interpreted as "feature rich". As for slow -- embedded languages don't necessarily have to be fast (though Tcl is plenty fast enough for most tasks). In many cases speed has nothing to do with it since evaluating code in any scripting language is inherently considerably slower than staying in the compiled language. Embedded languages are more about convenience for the user rather than speed. Don't let your prejudices sway you without having all the facts. – Bryan Oakley Dec 16 '10 at 12:02
  • 5
    @Bryan: It's not an unfounded prejudice, I worked with Tcl for 5 years. [This is how I felt about it then](http://compilers.iecc.com/comparch/article/01-09-007). Nothing's happened in the interim to change my mind. In particular, your "count the Google hits" comment underscores my the point that "TCL is perpetuated by the momentum of its large user-base, and not by the technical merits of the language". Now that languages like Lua exist, Tcl can die in a ditch, AFAIC. :) – Mud Dec 16 '10 at 15:15
  • 3
    @Mud "Now that languages like Lua exist, Tcl can die in a ditch, AFAIC." That's very well put! ;) – Judge Maygarden Dec 16 '10 at 16:14
  • @Mud: your opinion was formed nine years ago. A lot has changed in that time. Plus, your rant seems to boil down to one point: you don't like that tcl requires `expr` to do math. That is such a minor point about Tcl that it's hardly worth mentioning. Math is not a strong suit for Tcl -- who cares? An embedded language is rarely used to do much real math. It's unfortunate you had such a bad experience with Tcl, but I think the fact that it continues to be used to this day in many different industries speaks well of its relevance. – Bryan Oakley Dec 16 '10 at 16:39
  • 2
    That wasn't my only point, and it's hardly a *minor* point, especially in the context of this question. It's a *major* bit of "quirkiness" which it's now numerous alternatives do not possess. The reason it "continues to be used" today has not changed since I wrote that post: the momentum of it's userbase. COBOL is still used to this day, too. – Mud Dec 16 '10 at 17:26
  • I'v been used (in the past, about 10 years long) big business application with embedded TCL. One of the worst languages, like very fat macrogenerator, maybe simple during embedding (?) but not for user – Jacek Cz Aug 13 '15 at 14:07
4

With your requirements (small footprint, little quirks and integration with C++), the only option I can think about is Common Lisp.

Some people in this other SO question are recommending CFFI for integrating it with C.

But I'd stick with Lua if I where you.

Community
  • 1
  • 1
kikito
  • 51,734
  • 32
  • 149
  • 189
  • 3
    I actually thought about Lisp. But then again, I would probably be the only one who could read or write those scripts and that is kind of a deal breaker. – bastibe Dec 16 '10 at 09:23
4

A more recent alternative is wren:

Wren is a small, fast, class-based concurrent scripting language

Wren is a scripting language. Wren is intended for embedding in applications. It has no dependencies, a small standard library, and an easy-to-use C API. It compiles cleanly as C99, C++98 or anything later.

Real
  • 192
  • 1
  • 8
2

Have you considered Python? There's a nice extending and embedding guide available. If you're using Boost, Boost Python is a library for seemless integration between C++ and Python.

darioo
  • 46,442
  • 10
  • 75
  • 103
  • 22
    I like python and all, but since when is embedding it _small_? – Theo Belaire Dec 15 '10 at 22:23
  • 2
    I had a go at embedding Python in a home project a few years ago and it was not easy - even on desktop machines. It insisted on drawing in a lot of modules to function at even a minimal level. IIRC, I managed to whittle the total amount of baggage down to about 15MB of files. I can tell you that it IS possible but surely impractical for many smaller systems and much harder than embedding Lua. [Here's hoping things have improved.](http://wiki.python.org/moin/EmbeddedPython) – John McFarlane Aug 23 '13 at 23:47
  • One of questions by embedding (in many projects) is security, running interpreted fragments in sandbox (aka jail). Last Python running good in sandbox (i.e without files, io etc if author want) was ancient 1.5. Lua and many JavaScript engines realize very good this criteria. Newer Pythons are good general purpose language, in my opinion not for embedding (btw. python embedding doc is still present in distro) – Jacek Cz Aug 13 '15 at 14:12
  • Please don't embed Python, it's too general purpose. – J. M. Becker Jan 11 '20 at 16:57
2

I understand that Lua is the industry choice for problems like this.

A bold claim! I would suggest that if you are already running Linux, the choice is wide open. Linux itself is by no means the "industry choice" for embedded systems; the 'industry' is far more fragmented than that.

A language that is implementable on virtually any system regardless of performance, size, and OS (or even no OS), is Forth. Not the most fashionable language perhaps, but easily implementable and extensible.

Another candidate might be ch, which is an embedable C/C++ interpreter, so you can use the same language for compiled and scripted code.

Clifford
  • 88,407
  • 13
  • 85
  • 165