5

We're building a small REPL that evaluates (with eval) javascript expressions as they are being entered by the user. Since the whole thing is event-driven, evaluation must take place in a separate function, but the context (that is, all declared variables and functions) must be preserved between the calls. I came up with the following solution:

function* _EVAL(s) {
    while (1) {
        try {
            s = yield eval(s)
        } catch(err) {
            s = yield err
        }
    }
}

let _eval = _EVAL()
_eval.next()

function evaluate(expr) {
    let result = _eval.next(expr).value
    if (result instanceof Error)
        console.log(expr, 'ERROR:', result.message)
    else
        console.log(expr, '===>', result)
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('console.log("SIDE EFFECT")')
evaluate('let twenty = 20')
evaluate('twenty + 40') // PROBLEM

As you can see it works fine with function-scoped variables (var and function), but fails on block scoped ones (let).

How can I write a context-preserving eval wrapper that would also preserve block-scoped variables?

The code runs in a browser, DOM and Workers are fully available.

It should be mentioned that the desired function must handle side effects properly, that is, each line of code, or, at least, each side effect, should be performed exactly once.

Links:

JavaScript: do all evaluations in one vm | https://vane.life/2016/04/03/eval-locally-with-persistent-context/

georg
  • 211,518
  • 52
  • 313
  • 390
  • If you're in a front-end environment, an alternative method is to append a ` – CertainPerformance Apr 29 '21 at 18:47
  • You'll want to have a look at how the Chrome console is implemented. They employ several tricks, including TDZ avoidance. – Bergi Apr 29 '21 at 20:21
  • Would you be fine with the statements running in the global scope? Or: *a* global scope, like a web worker? – Bergi Apr 29 '21 at 20:22
  • @CertainPerformance: yes, this is in a browser – georg Apr 30 '21 at 06:55
  • @Bergi: I tried Workers, but don't seem to be able to escape the `onmessage` jail. – georg Apr 30 '21 at 06:56
  • @georg Did you try [global eval](http://perfectionkills.com/global-eval-what-are-the-options/) with `(1, eval)(…)`? – Bergi Apr 30 '21 at 09:55
  • @Bergi: no luck with that either – georg Apr 30 '21 at 10:34
  • 1
    @georg Ah, that might not work for `let`, it's quite possible lexical variables are always constrained to the scope of the eval'd expression :-/ – Bergi Apr 30 '21 at 10:36
  • @georg could you explain what is left to be refined in my solution? This returns the correct/expected values with each run. The only case this wouldn't work for is promises, (e.g. `fetch`, `async`/`await`, etc.) but those cases are impossible to account for in your current setup as it runs synchronously at its core. – Brandon McConnell May 05 '21 at 00:30
  • @georg `fetch` `async`/`await` are all perfectly possible and reasonable requirements. Please check my solution which works for all expressions including `fetch` –  May 05 '21 at 07:13

4 Answers4

3

The article you linked contains a crazy approach that actally works: during each eval() call, we create a new closure inside that eval scope and export it so that to we can use it evaluate the next statement.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

function evaluate(expr) {
    try {
        const result = __EVAL(expr);
        console.log(expr, '===>', result)
    } catch(err) {
        console.log(expr, 'ERROR:', err.message)
    }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('console.log("SIDE EFFECT")')
evaluate('let twenty = 20')
evaluate('twenty + 40') // NO PROBLEM :D
Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • Of course!!! I tested the Vilic's code, but never came to the idea to use ONE eval instead of two. That seems to be the crucial point. Would you mind elaborating on how exactly this works, for my and others' enlightenment? – georg May 05 '21 at 07:48
  • @georg did you see the solution by user15246403 below. It's essentially a deep dive into Bergi's answer. – Brandon McConnell May 11 '21 at 00:44
  • @BrandonMcConnell: yes, I did. It's a quality answer, but like in science, whoever comes up first with an idea, gets the credit. – georg May 11 '21 at 04:46
  • @georg I'm not trying to sway your selection here. My answer wasn't the best answer here either. Bergie's concept was clearly the right direction here, and deserves the credit, though I think user15246403's answer could actually help you decide how best to implement your version, especially as it discusses in detail the different ways to implement a fetch command. – Brandon McConnell May 11 '21 at 12:53
3

TL;DR

Here is the recommended, best solution I came up with below, supporting all expressions including promise-based expressions like fetch(), making use of async/await and nesting evaluate() in the final then() of my fetch().

Note (also mentioned in full post below)
The result of the nested evaluate() expression is logged first. This is correct and to be expected as that nested expression runs within the fetch() that runs it. Once the entire fetch runs, it will return undefined just as a variable assignment would. For every other [non-recommended] solution in my answer below, the title variable will be evaluated if and after the fetch() statement has been fully evaluated successfully. This is because we are either forcefully deferring the evaluation of the title variable by means of setTimeout() or a pre-processed then(), or by means of forced sequential loading in the "BONUS" solution at the bottom of this solution.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    console.log(expr, '===>', result)
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title).then(() => evaluate("title"))')

The madness explained

A few other solutions came very close here, so I must give credit to both Bergi and Brandon McConnell— Bergi for his/her clever use of closures with eval() and Brandon for his ingenuity in using a "stepped" result.

The correct solution does exist, and it does work with promises. For ease of use, I did use Bergi's solution as a foundation for my own, and if you do select my answer, I will gladly split the reputation bonus evenly with them.

Simply by making the evaluate() function async/await, you allow it to work with promises. From here, you have to decide how you would like for it to run— either organically, where fetch() statements run asynchronously as they normally would, or synchronously and wait for any Promise to be settled before proceeding to the next evaluate() call.

In my solution here, I chose to go the organic route as this is how JavaScript does actually work natively. To force all promises to run before proceeding would be to circumvent the nature of JavaScript. For example, if you were using this code to build a JavaScript engine or compiler, you would want the code to run the same with your engine as it would on the web for other users, so organic would be the wait to go.

BONUS ✨✨
If you would like to explore the non-organic, forced-sequential ordering idea I mentioned above, please scroll down to the bottom of this solution where I explain that concept in detail, link to an external resource that explains how to forcefully execute promises sequentially, and show a live working prototype of that idea in action.

If someone using your engine wants to wait for a fetch() to finish loading before proceeding, then they should adhere to the proper usage of then() as they would in other projects.

We can accomplish this by one of a few methods for fetch():

  1. Include evaluate() in an actual then() statement ORGANIC & RECOMMENDED
  2. Add a chaining command to our evaluate expression which will allow us to run one once another has completed. This can work out nicely for us in pure execution but it is not NOT RECOMMENDED as this adds special logic to the actual evaluation logic rather than the code being evaluated, so it is more of a server-side or back-end evaluation in a sense than the actual JS code running.
  3. Add a delay on the next line using setTimeout() to provide time for the fetch() to complete. NOT RECOMMENDED as this does not guarantee the promise has been settled, whether resolved or rejected. Fetch and async/await both deal with promises, so we should use promises to wait for them as well.

Here are examples for all three methods:

1. Including a nested evaluate() expression in our fetch().then()

Note: One important note here is that you will see the result of the nested evaluate() expression first. This is correct and to be expected as that nested expression runs within the fetch() that runs it. Once the entire fetch runs, it will return undefined just as a variable assignment would.

For every other [non-recommended] solution in my answer below, the title variable will be evaluated if and after the fetch() statement has been fully evaluated successfully. This is because we are either forcefully deferring the evaluation of the title variable by means of setTimeout() or a pre-processed then(), or by means of forced sequential loading in the "BONUS" solution at the bottom of this solution.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    console.log(expr, '===>', result)
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title).then(() => evaluate("title"))')

2. Chaining then() onto evaluate() (NOT RECOMMENDED) ⚠️

Note: In order to add this chaining method to our evaluate() expressions, we must return a new promise each time we run then. These promises, however, can/will be self-settling, so they simply allow us to chain the then() statement to the end of any evaluate() calls. Here is how that would work:

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    return new Promise(_ => _(console.log(expr, '===>', result)))
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title)')
  .then(() => evaluate('title'))

3. Using setTimeout() (NOT RECOMMENDED) ⚠️

Note: The issue here which I mentioned previously is that there is no guarantee of when the promise will be settled without waiting for the promise itself. Using setTimeout() grants the advantage over option #2 in that this runs as pure JS and does not work around the JS by running extra processes in the background, but this solution would require you to guess at how long your fetch might take to complete. This is not recommended, whether for this evaluate() function or in practice on actual projects. Option #1 using fetch().then() is the only solution that offers the flexibility of waiting for the promise to settle within the actual code that is entered and also waits until the promise has successfully settled.

❗ Even sometimes when running the snippet below, after waiting a full second, the fetch() still has not completed, and the setTimeout() executes first producing a blank string rather than the actual title string as desired. After repeated testing, this appears to work most of the time but not all of the time.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    console.log(expr, '===>', result)
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title)')
evaluate('setTimeout(() => evaluate("title"), 1000)')

BONUS: Forced-Sequential Execution (NOT RECOMMENDED) ⚠️

If you do want to force all evaluate functions to wait for the previous one to fully complete before proceeding, which I would highly discourage as this is not how JS will work in an actual browser, there is a great article on this exact concept of continuous/recursive promise chaining by James Sinclair which you can read here under the section labeled "A Sequential Solution". It is important if you choose to go this route that you do not simply use Promise.all() as that will not guarantee the order of execution of each promise. Instead, use a recursive chain of promise().then().

Here is how this might look in practice:

const exprs = [];
const evaladd = expr => exprs.push(expr);

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    return new Promise(_ => _(console.log(expr, '===>', result)))
  } catch(err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

function evaluateAll(exprs = []) {
  evaluate(exprs[0]).then(() => exprs.length > 1 && evaluateAll(exprs.slice(1)));
}

evaladd('var ten = 10')
evaladd('function cube(x) { return x ** 3 }')
evaladd('ten + cube(3)')
evaladd('let twenty = 20')
evaladd('twenty + 40')
evaladd('let title = ""')
evaladd('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title)')
evaladd('title')
evaluateAll(exprs)
1

If the user-entered code isn't meant to have any side-effects outside of their uses of evaluate, one approach is to concatenate the new input strings onto the old input strings. So, for example:

evaluate('ten + cube(3)')
evaluate('let twenty = 20')

results in the following being run. First time:

ten + cube(3)

Second time:

ten + cube(3)
let twenty = 20

This isn't very elegant since the code will have to run all code previously entered every time, but it'll at least make the repl functional.

function* _EVAL(codeToTry) {
    let userCode = '';
    while (1) {
        while (!codeToTry) {
            codeToTry = yield null;
        }
        try {
            const newCode = userCode + ';' + codeToTry;
            const result = eval(newCode)
            // No error, so tack onto userCode:
            userCode = newCode;
            codeToTry = yield result;
        } catch(err) {
            // Error, don't tack onto userCode:
            codeToTry = yield err
        }
    }
}

let _eval = _EVAL()
_eval.next()

function evaluate(expr) {
    let result = _eval.next(expr).value
    if (result instanceof Error)
        console.log(expr, 'ERROR:', result.message)
    else
        console.log(expr, '===>', result)
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40') // PROBLEM
CertainPerformance
  • 356,069
  • 52
  • 309
  • 320
0

It is native JavaScript behavior to return 5 if you run 5; let a = 2 which is essentially what is happening here in the final statement of your program when you check for the value of twenty + 40. However, a quick workaround for this would be to gather both results, the full result fullResult and the result of just that step (stepResult). With both of these, once a success is met in your evaluate() function, we can check to see if stepResult is equal to undefined which occurs when assigning a new variable value.

If this is the case, we use that value undefined. Otherwise, we use the value of fullResult, which works in every case of the provided code in your question:

const pastEvals = [];

function* _EVAL(s) {
    while (1) {
        try {
            s = yield eval(s)
        } catch(err) {
            s = yield err
        }
    }
}

let _eval = _EVAL()
_eval.next()

function evaluate(expr) {
    pastEvals.push(expr)
    const fullResult = _eval.next(pastEvals.join(';')).value
    const stepResult = _eval.next(expr).value
    if (fullResult instanceof Error)
        console.log(expr, 'ERROR:', result.message)
    else
        console.log(expr, '===>', stepResult === undefined ? stepResult : fullResult);
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')

You will run into problems with this when trying to use more advanced JS functions such as async/await and fetch, but it should work just fine for these simpler use-cases. If you need to build something that works for more advanced uses, you may need to create a virtual DOM behind the scenes, create and destroy a new virtual DOM with each run, and also wait until any created promises are fulfilled and completed between iterations, as that is what would be required for any fetch-related operations.

Brandon McConnell
  • 5,776
  • 1
  • 20
  • 36
  • another problem is that the VM that I'm working on, also supports console functions like `console.log()`. so with your solution, each time the code executes it will log, if any. you can see source code here https://github.com/MohammadMD1383/js-interactive – Mohammad Mostafa Dastjerdi May 04 '21 at 06:37
  • 2
    Indeed, like another solution, side effects are a big problem here. – georg May 04 '21 at 07:26
  • @georg I wouldn't personally call `fetch` not working here a "side effect". `fetch` cannot be run synchronously, so the nature of your function would need to change to use promises in order to support `fetch`. **Still, this solution resolves the issue from the other solution where `let twenty = 20 ===> 37`. Here, every line has its correct output value `let twenty = 20 ===> undefined` – Brandon McConnell May 04 '21 at 12:03
  • @BrandonMcConnell: what I meant here, is that if some expression has a side effect (not necessarily async, it can be just logging something), this effect will be performed as many times as there are expressions. For example, if you add `console.log("SIDE EFFECT")` like in my (edited) snippet, you'll see it logged 3 times, instead of expected 1. Sorry I wasn't clear about this requirement from the start. – georg May 05 '21 at 06:50
  • @georg I see what you mean now. I was working on a solution to add support for promises but it looks like seanmitchell87 beat me to the punch. Good explanations there and honestly much cleaner and clearer than what I was putting together. Digging into his solution and Bergi's foundation to see how it works – Brandon McConnell May 05 '21 at 07:18