22

I used to think that metaprogramming involved modifying the program, and (as do some answers to What is reflection and why is it useful? ) that reflection merely consisted of introspection of a program. However, the reflection tag wiki says

Reflection is the process by which a program can observe and modify its own structure and behavior at runtime.

Reflection is the process by which a program can perform introspection. This introspection usually involves the ability to observe and modify its own structure and behavior at runtime. From theoretical perspective reflection relates to the fact that program instructions are stored as data. The distinction between program code and data is a matter of how the information is treated. Hence programs can treat their own code as data and observe or modify them.

[emphasis added]

And the description for metaprogramming is

Metaprogramming is writing programs that write or manipulate other programs as their data.

Metaprogramming is useful because it can save programmers valuable time. Some languages have support to metaprogram themselves and this allows to create code with great expressive power.

(I assume that "write" doesn't mean writing source code to a file, because that'd be code generation.)

Would this make metaprogramming merely a subset of reflection?

Or are the terms distinct because some programming languages are metaprogrammed by another language, in which case metaprogramming occurs but not reflection? (There was a single uncited sentence claiming this in the metaprogramming Wikipedia article)

Or do the terms "reflection" and "metaprogramming" get used differently depending on what programming language the person is using?

Community
  • 1
  • 1
Andrew Grimm
  • 78,473
  • 57
  • 200
  • 338

2 Answers2

19

No. Rather, reflection provides facilities that are a subset of what metaprogramming can do.

Metaprogramming is "programs which write programs". This includes programs that read the text of programs (arguably including themselves but that is rather rare), analyze that code, and make changes. Yes, it includes writing source text to files. Code generation is a special case of metaprogramming.

Reflection as I understand it is the ability for a program to inquire about its own structure. In virtually every system I have seen in which reflection is possible (with the really exceptional case of Lisp and equivalent variants), the reflection machinery provided only a limited means of introspection. Java and C# will let you find out the names of classes and methods, but you cannot ask these systems for the content of a method, statement or local declaration. Nor can you ask most such reflective langauges to actually change their structure, that is, you can't add new classes or fields using the reflection facilities. Most langauges (e.g., C++) have basically no built-in ability to "reflect". While the reflection utilities built into langauges can be useful, they tend to be idiosyncratic with respect to what the language designers/compiler builders decided to keep around at runtime.

You end up with a much more powerful "reflection" capability if you step outside the language and the set of restrictions the langauge designers built into it. A really good metaprogramming system has access to the entire program structure, and thus can answer arbitrary questions about the program structure (modulo Turing limitations).

As an example, our DMS Software Reengineering Toolkit is a program transformation tool that has complete access to the abstract syntax tree of the program and many other facts derived by the various DMS language front ends. So DMS can "reflect" (inspect/analyze/reason) rather arbitrarily about the language which it is processing. And it can do so for C, COBOL, Java, C# and C++; for many of these langauges, it can not only provide access to the AST, but access to symbol table information and various forms of control and data flow, which no reflection facilities I've ever seen offer you.

Additionally, a program transformation tool like DMS can modify the code based on the "reflection" to generate new code, optimize, restructure, instrument, ... The variety of effects achievable this way is surprisingly broad.

[Since DMS is implemented as set of DSLs, it in fact can and does reason ("reflect") about its own code. We use DMS to synthesize large parts of itself from its DSLs, including code generation with some pretty interesting optimizations, including working parallelization.]

Ira Baxter
  • 93,541
  • 22
  • 172
  • 341
  • You said the in C# `ava and C# will let you find out the names of classes and methods, but you cannot ask these systems for the content of a method, statement or local declaration`. That is not true, you can get the method information, including itself body implementation, it will not be in C# code, or VB/F# but in MSIL https://learn.microsoft.com/en-us/dotnet/api/system.reflection.methodbase.getmethodbody?view=netframework-2.0 Btw, this is possible from .NET 2.0 version, that is very old (Version 2.0 was released on 22 January 2006) – Alberto Monteiro Feb 05 '20 at 00:07
  • That's right, you can get a represention of the program that is very hard to interpret from a C# or Java source code point of view. You can arguably do the same thing in a C++ program by taking the address of a function and extracing a sequence of bytes. You can, with enough effort, knowing the machine and writing tons of special code, even find the machine instructions that make up your function, but they are hellaciously hard to interpret as source code. – Ira Baxter Feb 05 '20 at 00:29
  • ... The fact that MS has implemented something similar for MSIL saves you the trouble of figuring what bytes make up your function, ... but that doesn't make them easier to interpret for the benefit of your program, or easy change to improve your program. So I don't consider "getting the object code" of your file as serious metaprogramming. – Ira Baxter Feb 05 '20 at 00:32
  • You can use mono cecil a nuget package that already does have this kind of feature, to get binary data and convert to MSIL opcodes, once you have it, you can also use dnspy features to decompile the source code and have good C# or other .NET language representation – Alberto Monteiro Feb 05 '20 at 12:41
  • OK, I'll concede your point that you can get C# source code from a C# program. But this is hardly "reflection built into the language", which was the point of the question. It is really "just" a massive library somebody wrote that can decompile the object code back to source; there's nothing special in the C# *language* that enables this. I'm sure you could do this for x86 optimized machine code and C++ if you put enough effort into it (there are x86 disassemblers), but doing that would not qualify C++ as a language with reflection capabilities provided by the language. – Ira Baxter Feb 05 '20 at 14:16
  • If we speak about C# specific, itself does nothing about it, because `System.Reflection` is a part of the entire .NET Framework, so not only C# on .NET will have this benefit, but F#, VB or whatver other language that compiles to .NET's MSIL. In same namespace, you have the Emit subset that allow you write MSIL code on the fly and execute it, with System.Linq.Expressions you can achieve something like that, but on Emit you have more powerfull features, like creating a class, methods and so on – Alberto Monteiro Feb 05 '20 at 16:37
4

Reflection can also happen during execution, when for instance a program is able to inspect its own call stack (by looking into call frames inside) And metaprogramming could even change the functions which have been called (that is which have frames lower in the call stack) together with their call frames.

You might consider reading Jacques Pitrat's book on Artificial Beings - the conscience of a conscious machine which explain in details why this is useful and how it can be implemented. See also his blog.

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547