10

I want to make a mini programming language in mma. From a textfile to Module(s) in a Package. Ideally I should be able to generate the package and modules from within Mathematica by functions in another package.

Question: Is this possible? I am looking for a reference or an example to get this started.

EDIT: For example:

Imagine a memory bank with n integer typed registers.

The instructions are:

1 Z(n)

2 C(m,n)

3 J(m,n,q)

4 S(n)

Each line has an address. First line 1, second 2, etc. Z(n) store 0 in register n. C(m,n) store value of register m in register n. J(m,n,q) if value register m equals value of register n then jump to line with address q. S(n) add 1 to value in register n.

Then given two working programs P and Q I want to generate the concatenated program P+Q.

Then given two working programs P and Q I want to generate the substitution Q after P.

Finally I want to start experimenting with recursion... the purpose of this 'mini-project'.

nilo de roock
  • 4,077
  • 4
  • 34
  • 62
  • 1
    It is not very clear what you want to accomplish. Please clarify. Once you have created some definitions for a symbol, you can save them to a file ('package' if you like) using `Save`. Or is your question how to create those definitions programmatically? Please give specific examples. – Szabolcs Jun 02 '11 at 13:28

2 Answers2

13

Your question has several parts. First, if you want to use some non-mma syntax for your language, you need to make a parser from your language to mma expression (AST of your code). I will leave this one out (since this is a separate topic) and assume that you are willing to use mma syntax or have means to transfer your program to some mma expression.

Regarding the mma code generation, Mathematica is very well suited for it since it embraces the code-is-data paradigm. The hardest part here is the evaluation control - we want to make sure that none of our generated code pieces evaluates during the code-generation process. The standard techniques of evaluation control can be successfully used for that, but this will generally make things rather complicated. I will illustrate one technique of mma code generation, which is not the best/most powerful one, but the easiest.

Consider a toy language created by these definitions:

SetAttributes[testSet, HoldFirst];
SetAttributes[testIf, HoldRest];
SetAttributes[testVar, HoldAll];
SetAttributes[module, HoldAll];
SetAttributes[{package, inContext}, HoldRest];
testPlus[x_, y_] := Plus[x, y];
testTimes[x_, y_] := Times[x, y];
testDivide[x_, y_] := If[y == 0, Inf, Times[x, Power[y, -1]]];
testPower[x_, y_] := If[x == 0 && y < 0, Inf, Power[x, y]];
testSet[HoldPattern[testVar[x_]], expr_] := Set[x, expr];
testVar[x_] := If[ValueQ[x], x, Throw[$Failed, {"varundef", x}]];
testIf[cond_, expr_] := If[cond, expr];
testIf[cond_, expr_, else_] := If[cond, expr, else];
module[{vars__}, body_] := Module[{vars}, body];
package[name_, code_] := (BeginPackage[name]; code; EndPackage[]);
inContext[name_, code_] := (Begin[name]; code; End[]);

Here is a small code snippet in this new language (wrapped in Hold):

cd = 
Hold[module[{a}, testSet[testVar[a],
  testPlus[testTimes[testTimes[testPlus[1, 2],
    testPower[testPlus[3, 4], -1]], testPlus[5, 6]], -7]]; testVar[a]]]

It corresponds to this mma code:

Module[{a},a = (1 + 2)/(3 + 4)*(5 + 6) - 7; a]

Our code-generator is based on a very simple idea - we will repeatedly apply local rules to our held code. The local rules will be extracted from the definitions of our functions, like so:

ClearAll[expansionRules];
expansionRules[heads : {__Symbol}] := Flatten[DownValues /@ heads]

We need to supply a list of heads for our language. I will do that manually, but it is easy to automate, by creating custom assignment operators.

allHeadsToExpand[] := {testIf, testVar, testPlus, testTimes, testDivide, 
      testPower, testSet, testIf,module,package, inContext}

Now, we generate our code:

In[195]:= expanded = cd//.expansionRules[allHeadsToExpand[]]

Out[195]= 
 Hold[Module[{a}, 
    a = ((1 + 2) If[3 + 4 == 0 && -1 < 0, Inf, 1/(3 + 4)]) (5 + 6) - 7; 
    If[ValueQ[a], a, Throw[$Failed, {"varundef", a}]]]]

To execute it, you can simply use ReleaseHold:

In[197]:= ReleaseHold[expanded]

Out[197]= -(16/7)

The advantage of our construction is that we can also execute our AST directly:

In[198]:= ReleaseHold[cd]

Out[198]= -(16/7)

To save this to a package, you can simply use Put command. It is also easy to extend the language in any way you want. Of course, the way the code in this language looks is not pretty, since it is essentially the AST expressed as mma expression. To make it prettier, you'd need to introduce your own syntax and write a parser from it to mma AST, but that is another story.

EDIT

Regarding automating of code-generation and saving the generated code into a package: here are a couple of utilities to do that.

Clear[generateCode];
generateCode[code_Hold] :=
  code //. expansionRules[allHeadsToExpand[]] //.
   HoldPattern[
      CompoundExpression[left___, CompoundExpression[middle___], right___]] :> 
       (left; middle; right);

Clear[formatCode];
formatCode[code_Hold] :=
  StringReplace[Function[Null, ToString[Unevaluated[#], InputForm], HoldAll] @@ 
     code, ";" :> ";\n"];

Clear[saveCode];
saveCode[file_, generatedCode_] :=
 With[{result = BinaryWrite[file, formatCode@generatedCode]},
   Close[file];
   result];

Here is the same example but placed in a package:

cdp = Hold[
   package["myPackage`",
     inContext["`Private`",
       module[{a}, 
         testSet[testVar[a],
           testPlus[testTimes[testTimes[testPlus[1, 2],
            testPower[testPlus[3, 4], -1]], testPlus[5, 6]], -7]]; 
         testVar[a]]]]]

We generate and save the code as follows:

In[101]:= file = FileNameJoin[{"C:","Temp","myPackage.m"}]
Out[101]= C:\Temp\myPackage.m

In[106]:= saved =saveCode[file,generateCode[cdp]]
Out[106]= C:\Temp\myPackage.m

We can Import it to test:

In[107]:= Import[file,"Text"]

Out[107]= 
BeginPackage["myPackage`"];
 Begin["`Private`"];
 Module[{a}, a = ((1 + 2)*If[3 + 4 == 0 && -1 < 0, Inf, (3 + 4)^(-1)])*(5 + 6) - 7;
  If[ValueQ[a], a, Throw[$Failed, {"varundef", a}]]];
 End[];
 EndPackage[]

EDIT 2

Regarding the way the code in your language will look, you can make this prettier without going all the way to create your own parser, by using the Notation package to alter the way you can input code and Format/FormatValues to control how it is rendered by the FrontEnd.

Leonid Shifrin
  • 22,449
  • 4
  • 68
  • 100
  • 2
    Leonid's answer /.{AST->"Abstract Syntax Tree"} – Dr. belisarius Jun 02 '11 at 13:55
  • @belisarius Thanks! I will probably have to edit this and add more explanations. Oops... It took me a while to understand the deep code-generating nature of your comment :) – Leonid Shifrin Jun 02 '11 at 13:56
  • 2
    I didn't even bother answering this question, as I knew you'd be by to post something much better. – Mr.Wizard Jun 02 '11 at 15:48
  • 1
    @Mr.Wizard Thanks. This is the topic of real interest to me. – Leonid Shifrin Jun 02 '11 at 16:39
  • 3
    @Leonid: I _cannot unsee_ this: ``Hold[ package["myPackage`", inContext["`Private`"...`` ಠ_ಠ ... If that was intentional, you're a frickin' genius! – abcd Jun 03 '11 at 02:50
  • @ndroock1 Even if you want to write your own parser, the code-generation part of my advice should still be relevant. Regarding the parser: if your language is simple enough, it woould probably be easiest to write a recursive descent parser in Mathematica manually. A very good example, simple and non-trivial, is in the book of Wellin, Gaylord and Kamin. If you need a really fast and production-quality parser, you can use parser generators in C or Java, and link them with Mathematica. Not too difficult either, especially given your programming background. – Leonid Shifrin Jun 03 '11 at 10:14
  • @yoda Thanks! Although I don't get what was it in that code snippet that got you so excited - perhaps I am missing something. – Leonid Shifrin Jun 03 '11 at 16:27
  • @Leonid: Umm... Don't let my warped mind corrupt you. Please stay innocent and may the force be with you! – abcd Jun 03 '11 at 16:33
  • @yoda get it I do now, although late was it :). Not being a native speaker sometimes sucks. Although, the fact that I wrote this unintentionally is also funny in a way. – Leonid Shifrin Jun 03 '11 at 16:54
  • @Leonid - I'll accept your answer because of your reference to 'An Introduction to Programming with Mathematica, by Wellin, Gaylord and Kamin, Cambridge UP 2005.' - What I want to know is all in Chapter 11.4. Implementing Languages. - Shame on me because I have that book but skipped reading chapter 11 for some reason. – nilo de roock Jun 04 '11 at 18:08
  • @ndroock1 Well, thanks, although it is a bit of a pity that the main body of my answer wasn't of much use to you. By the way, if you feel like updating your question with one or a few concrete examples of some (toy) programs in your language , and also what you want to use Mathematica for, please do, it would be great. Your current goals are still a bit unclear (to me at least). – Leonid Shifrin Jun 04 '11 at 18:27
  • @Leonid - What you wrote remains online and may be useful to me or others in the future. - I am not a professional Mathematica user. I use it in my role as math student / amateur mathematician. - I think I will produce quite some mma code in the forthcoming months for a study project. - I will post code for review. Most likely starting with the code for the tree data structure which I am currently testing. – nilo de roock Jun 04 '11 at 20:47
  • @ndroock1 I see. Regarding the tree data structure code, a non-trivial example of its use can be found in this answer of mine: http://stackoverflow.com/questions/6138540/code-manipulation-via-interactive-tree-for-mathematica/6140400#6140400, so if you didn't see it yet, you may want to have a look at it (otherwise just ignore this) – Leonid Shifrin Jun 04 '11 at 20:54
4

It is tangential to the question, but you may find important utility in the setting CellEvaluationFunction as described in a post by WReach.

Community
  • 1
  • 1
Mr.Wizard
  • 24,179
  • 5
  • 44
  • 125
  • That's one of my favorites from the 'tool bag'. I really wish something like that were covered in the documentation. (+1 for spreading the info) – telefunkenvf14 Jun 30 '11 at 18:33