0

I'm trying to write a simple replacer/reviver combination that lets me serialize and deserialize ES6 classes properly (I need to deserialize them as ES6 classes, not Objects) for a TypeScript project I'm working on.

I got it working easily enough by storing the class names in the JSON output under a special key/value pair (i.e. "$className":"Position").

The only problem is that the only way I could find to instantiate these classes later was to use:

const instance = eval(`new ${className}()`);

I was writing the rough draft for the code in a test file using Vitest to make sure it worked, and everything was working great until I was done and moved it to another file. If I import the reviver function that uses eval() under the hood, it gives me the following error:

ReferenceError: Position is not defined

Note: Position is an ES6 class in my test file

If I copy/paste the reviver code back in the .test.ts file, the tests pass. If I remove that code and use import, it complains about Position not being defined.

I'm kind of confused about what might be causing this. Is it a known limitation of modules? Is it a TypeScript or Vite thing where the file imports mess with the hoisting, maybe? Is it an issue with eval() when imported?

Any help would be appreciated.

Edit: Added full code below. Moving the contents of serialization.ts before or after the class declarations in serialization.test.ts doesn't cause the ReferenceError: Position is not defined error.

serialization.ts

type JsonCallback = (key: string, value: any) => any;

export const replacerWithMap = (): [replacer: JsonCallback, idsToClassNames: Map<number, string>] => {
  let counter = 0;
  const classNamesToIds = new Map<string, number>();
  const idsToClassNames = new Map<number, string>();

  const replacer = (_: string, value: any): any => {
    if (typeof value === 'object' && !Array.isArray(value) && !!value.constructor) {
      const className = value.constructor.name;
      if (!className) {
        throw new Error(`Expected value to be class instance but was: ${value}`);
      }

      if (Object.hasOwn(value, '$_')) {
        throw new Error(`Illegal property name "$_" found on object during serialization: ${className}`);
      }

      let id = classNamesToIds.get(className);
      if (!id) {
        id = counter;
        classNamesToIds.set(className, id);
        idsToClassNames.set(id, className);
        ++counter;
      }

      return {
        '$_': id,
        ...value
      };

    }

    return value;
  }

  return [replacer, idsToClassNames];
}

export const reviverFromMap = (idsToClassNames: Map<number, string>): JsonCallback => {
  const reviver = (_: string, value: any): any => {
    if (typeof value === 'object') {
      if (Object.hasOwn(value, '$_')) {
        const className = idsToClassNames.get(value.$_);
        const instance = eval(`new ${className}()`); // <-------- eval() here
        for (const [key, val] of Object.entries(value)) {
          if (key !== '$_') {
            instance[key] = val;
          }
        }
        return instance;
      }
    }

    return value;
  }

  return reviver;
}

serialization.test.ts

import { describe, expect, it } from 'vitest';
import { replacerWithMap, reviverFromMap } from './serialization';

class Position {
  public x: number;
  public y: number;
  private _z: number;

  public constructor(x: number, y: number) {
    this.x = x;
    this.y = y;
    this._z = x * y;
  }

  public get z(): number {
    return this._z;
  }

  public set z(value: number) {
    this._z = value;
  }
}

class Transform {
  // Testing to see if it handles nested instances properly.
  public position = new Position(3, 5);

  public name: string = '';
}


describe('Serialization helpers', () => {
  it('Serializes objects and maintains their class names in a map', () => {
    // Arrange
    const transform = new Transform();
    transform.name = 'rotation';

    // Act
    const [replacer, idsToClassNames] = replacerWithMap();
    const reviver = reviverFromMap(idsToClassNames);

    const str = JSON.stringify(transform, replacer);
    console.log(str, idsToClassNames);
    const deserialized = JSON.parse(str, reviver) as Transform;

    // Assert
    expect(deserialized).toEqual(transform);
    expect(deserialized).toBeInstanceOf(Transform);
    expect(deserialized.position).toEqual(transform.position);
    expect(deserialized.position).toBeInstanceOf(Position);
    expect(deserialized.position.x).toBe(3);
    expect(deserialized.position.y).toBe(5);
    expect(deserialized.position.z).toBe(deserialized.position.y * deserialized.position.x);
  });
});
micka190
  • 742
  • 2
  • 8
  • 20
  • 1
    Please post the full code - where is `eval` called, and where is `Position` defined? – Bergi May 29 '23 at 20:52
  • 1
    `eval`ing arbitrary `$classname` strings is insecure anyway. You should be very explicit about which classes are allowed to be instantiated, and [pass them in a dictionary](https://stackoverflow.com/q/31776949/1048572) to your reviver function – Bergi May 29 '23 at 20:54
  • @Yogi I've spent most of the day doing just that. Every single answer I've found is "use `eval()`" or "know your classes beforehand and put them in a dictionary" (which isn't an option for my use case). If you know how I can instantiate arbitrary classes using only their class names as strings, feel free to share! I might have missed a post. – micka190 May 29 '23 at 21:17
  • @Bergi Updated the question with the code. I won't know the classes beforehand, so pre-emptively storing them in a map won't work. The consensus online seems to be that `eval()` is the only way to do it. – micka190 May 29 '23 at 21:18
  • 1
    Thanks for adding the code. The `eval` call in *serialization.ts* cannot work, as `Position` is not in scope there, it is declared in *serialization.test.ts*! You would have needed to `import` it, but of course you wouldn't want to import all the classes that should be usable in the serialisation library. – Bergi May 29 '23 at 21:20
  • Why are you storing the class *names* in `idsToClassNames` instead of simply storing the classes itself? – Bergi May 29 '23 at 21:21
  • @Bergi I'm storing the class names because the serialized string is going to be saved for later use (either off the client or in something like localstorage). Serializing those won't work. – micka190 May 29 '23 at 21:23
  • 2
    @micka190 "*I won't know the classes beforehand*" - surely there is a distinct set of known classes that are defined in your code (and might be used by the serialised objects)? How else would you attempt the deserialise instances whose classes are not defined in your code?! "*I'm storing the class names because the serialized string is going to be saved*" - I'm not talking about the serialised `str`ing, I'm talking about the `idsToClassNames` map. Put the classes themselves in that map. Put their ids in the serialised value that is stored to disk or something. – Bergi May 29 '23 at 21:26
  • The code is intended for use in a library we'll be using on various projects, which is why I can't know what classes will and won't be serialized beforehand. I understand that you weren't talking about `str`, but my goal with the code above was to avoid having the user define every class that might be serialized/deserialized. If there's a limitation in how `eval()` works like you've said, then you've basically answered my question (feel free to post an answer) and I'll have to change the design and have the users provide the classes to the serialization code beforehand. – micka190 May 29 '23 at 21:33
  • 1
    Every project *must* provide its classes to your library, there is no way around that. Even your current approach, where the replacer fills the Map, would require the project to serialise a value containing all possible classes before attempting to deserialise any string. – Bergi May 29 '23 at 21:36

1 Answers1

1

I'm kind of confused about what might be causing this. Is it a known limitation of modules?

Yes. The class declaration is scoped to the module, it is not global, which is precisely the advantage of modules. So with Position declared in serialization.test.ts, the eval call in serialization.ts cannot work, as Position is not in scope there! You would have needed to import it, but of course you wouldn't want to import all the classes that should be usable in the serialisation library.

Instead, pass the classes, keyed by name, as a parameter to the (de)serialisation library, e.g.

const classes = new Map(Object.entries({
  Position,
  Transform,
}));
const {replacer, reviver} = serialisation(classes);
Bergi
  • 630,263
  • 148
  • 957
  • 1,375