15

I'm looking for a way to have all keys / values pair of a nested object.

(For the autocomplete of MongoDB dot notation key / value type)

interface IPerson {
    name: string;
    age: number;
    contact: {
        address: string;
        visitDate: Date;
    }
}

Here is what I want to achieve, to make it becomes:

type TPerson = {
    name: string;
    age: number;
    contact: { address: string; visitDate: Date; }
    "contact.address": string;
    "contact.visitDate": Date;
}

What I have tried:

In this answer, I can get the key with Leaves<IPerson>. So it becomes 'name' | 'age' | 'contact.address' | 'contact.visitDate'.

And in another answer from @jcalz, I can get the deep, related value type, with DeepIndex<IPerson, ...>.

Is it possible to group them together, to become type like TPerson?

Modified 9/14: The use cases, need and no need:

When I start this question, I was thinking it could be as easy as something like [K in keyof T]: T[K];, with some clever transformation. But I was wrong. Here is what I need:

1. Index Signature

So the interface

interface IPerson {
    contact: {
        address: string;
        visitDate: Date;
    }[]
}

becomes

type TPerson = {
    [x: `contact.${number}.address`]: string;
    [x: `contact.${number}.visitDate`]: Date;
    contact: {
        address: string;
        visitDate: Date;
    }[];
}

No need to check for valid number, the nature of Array / Index Signature should allow any number of elements.

2. Tuple

The interface

interface IPerson {
    contact: [string, Date]
}

becomes

type TPerson = {
    [x: `contact.0`]: string;
    [x: `contact.1`]: Date;
    contact: [string, Date];
}

Tuple should be the one which cares about valid index numbers.

3. Readonly

readonly attributes should be removed from the final structure.

interface IPerson {
    readonly _id: string;
    age: number;
    readonly _created_date: Date;
}

becomes

type TPerson = {
    age: number;
}

The use case is for MongoDB, the _id, _created_date cannot be modified after the data has been created. _id: never is not working in this case, since it will block the creation of TPerson.

4. Optional

interface IPerson {
    contact: {
        address: string;
        visitDate?: Date;
    }[];        
}

becomes

type TPerson = {
    [x: `contact.${number}.address`]: string;
    [x: `contact.${number}.visitDate`]?: Date;
    contact: {
        address: string;
        visitDate?: Date;
    }[];
}

It's sufficient just to bring the optional flags to transformed structure.

5. Intersection

interface IPerson {
    contact: { address: string; } & { visitDate: Date; }
}

becomes

type TPerson = {
    [x: `contact.address`]: string;
    [x: `contact.visitDate`]?: Date;
    contact: { address: string; } & { visitDate: Date; }
}

6. Possible to Specify Types as Exception

The interface

interface IPerson {
    birth: Date;
}

becomes

type TPerson = {
    birth: Date;
}

not

type TPerson = {
    age: Date;
    "age.toDateString": () => string;
    "age.toTimeString": () => string;
    "age.toLocaleDateString": {
    ...
}

We can give a list of Types to be the end node.

Here is what I don't need:

  1. Union. It could be too complex with it.
  2. Class related keyword. No need to handle keywords ex: private / abstract .
  3. All the rest I didn't write it here.
Val
  • 21,938
  • 10
  • 68
  • 86
  • Oof these "flatten subproperties" questions never seem to consider edge cases enough and they drive me spiraling into madness. Based on what principle would you decide not to go drilling way down into `Date` here? See [this code](https://tsplay.dev/wO8dEN). What would you like to change here? How do I know that `Date` should just be `Date` but other object types should be flattened? – jcalz Sep 10 '21 at 03:47
  • I suppose I could hardcode `Date` as an exception, like [this](https://tsplay.dev/mLRVaw), but that's pretty ad-hoc. – jcalz Sep 10 '21 at 03:51
  • 1
    Do you care about arrays? Like [this](https://tsplay.dev/WK8Q8w)? There are just so many caveats. – jcalz Sep 10 '21 at 03:56
  • @jcalz It's good to me to hardcode `Date` and other object as an exception, I recognized there could be no better way to achieve that – Val Sep 10 '21 at 04:02
  • @jcalz thanks for your excellent example! I learned a lot from it. It already perfectly answered my question, also take array into consideration which I haven't think of. You're really genius of typescript, please leave it as an answer. – Val Sep 10 '21 at 04:20
  • @jcalz I didn't see any typescript warning / caveats in vscode, and also it looks flawless to me. Is there anything I was missing? – Val Sep 10 '21 at 04:23
  • 1
    I mean, there are loads of potential edge cases: index signatures, readonly properties, optional properties, pattern template literal keys... I'm happy to write up what I have, but the failure modes of this thing could fill a book. Ideally you would tell me the sorts of things you need to support; sounds like arrays are important (which have index signatures)... anything else? – jcalz Sep 10 '21 at 13:30
  • @jcalz Honestly, I haven't think of so many edge cases. What I thought was just exactly the same as original type. I already start a bounty for it, looking forward to your great answer. thank you! =) – Val Sep 13 '21 at 03:50
  • Hmm I'm not sure that it's even worth 200 points to me to go back and forth on edge cases (looking at your exchange with @captain-yossarian). I'm not familiar with mongodb so I don't really understand the use case well enough to intuit what to do for unions, intersections, index signatures, properties with dots in them, etc., and without a list of your intended use cases in advance then I'm concerned that this question becomes a game of [whack-a-mole](https://en.wiktionary.org/wiki/whack-a-mole#Noun). – jcalz Sep 13 '21 at 13:32
  • Unions are actually quite interesting; if you ever have optional properties or properties of union types, then you need to decide if you want to see the intersection of all possible properties (even though some will surely not exist) or just a union of them as if you were reading from the object itself (which is safer but could be annoying). A union-to-union version is [here](https://tsplay.dev/Nd4xMN). – jcalz Sep 13 '21 at 15:49
  • At this point I won't engage with the question anymore unless you edit the post to describe how you'd like it to behave in sufficient detail so that an answer can be accepted or not based only on the question text and not information based only in comments (if you care about arrays, put in the question what you want to see in and out for arrays; if you care about tuples, or unions, or optionals, etc., write it). If you only care about the example listed directly in your question, then @captian-yossarian's answer should be fine and you should ask him to explain how it works. Good luck to you!! – jcalz Sep 13 '21 at 15:52
  • 1
    @jcalz I modified my question, with a list of use cases. Your sample code helped me so much than other answers. Before I think over it carefully, I almost think it's already a perfect answer. I'm busy working in my daily life currently, you get me back to the rail to ask a good question =) – Val Sep 14 '21 at 04:10
  • ``{[x: `contact.${number}.visitDate`]?: Date;}`` is not valid TypeScript. :-( – jcalz Sep 14 '21 at 20:25

2 Answers2

9

Below is the full implementation I have of Flatten<T, O> which transforms a type possibly-nested T into a "flattened" version whose keys are the dotted paths through the original T. The O type is an optional type where you can specify a (union of) object type(s) to leave as-is without flattening them. In your example, this is just Date, but you could have other types.

Warning: it's hideously ugly and probably fragile. There are edge cases all over the place. The pieces that make it up involve weird type manipulations that either don't always do what one might expect, or are impenetrable to all but the most seasoned TypeScript veterans, or both.

In light of that, there is no such thing as a "canonical" answer to this question, other than possibly "please don't do this". But I'm happy to present my version.

Here it is:


type Flatten<T, O = never> = Writable<Cleanup<T>, O> extends infer U ?
    U extends O ? U : U extends object ?
    ValueOf<{ [K in keyof U]-?: (x: PrefixKeys<Flatten<U[K], O>, K, O>) => void }>
    | ((x: U) => void) extends (x: infer I) => void ?
    { [K in keyof I]: I[K] } : never : U : never;

The basic approach here is to take your T type, and return it as-is if it's not an object or if it extends O. Otherwise, we remove any readonly properties, and transform any arrays or tuples into a version without all the array methods (like push() and map()) and get U. We then flatten each property in that. We have a key K and a flattened property Flatten<U[K]>; we want to prepend the key K to the dotted paths in Flatten<U[K]>, and when we're done with all that we want to intersect these flattened objects (with the unflattened object too) all together to be one big object.

Note that convincing the compiler to produce an intersection involves conditional type inference in contravariant positions (see Transform union type to intersection type), which is where those (x: XXX) => void) and extends (x: infer I) => void pieces come in. It makes the compiler take all the different XXX values and intersect them to get I.

And while an intersection like {foo: string} & {bar: number} & {baz: boolean} is what we want conceptually, it's uglier than the equivalent {foo: string; bar: number; baz: boolean} so I do some more conditional type mapping with { [K in keyof I]: I[K] } instead of just I (see How can I see the full expanded contract of a Typescript type?).

This code generally distributes over unions, so optional properties may end up spawning unions (like {a?: {b: string}} could produce {"a.b": string; a?: {b: string}} | {"a": undefined, a?: {b: string}}, and while this might not be the representation you were going for, it should work (since, for example, "a.b" might not exist as a key if a is optional).


The Flatten definition depends on helper type functions that I will present here with various levels of description:

type Writable<T, O> = T extends O ? T : {
    [P in keyof T as IfEquals<{ [Q in P]: T[P] }, { -readonly [Q in P]: T[P] }, P>]: T[P]
}

type IfEquals<X, Y, A = X, B = never> =
    (<T>() => T extends X ? 1 : 2) extends
    (<T>() => T extends Y ? 1 : 2) ? A : B;

The Writable<T, O> returns a version of T with the readonly properties removed (unless T extends O in which case we leave it alone). It comes from TypeScript conditional types - filter out readonly properties / pick only required properties.

Next:

type Cleanup<T> =
    0 extends (1 & T) ? unknown :
    T extends readonly any[] ?
    (Exclude<keyof T, keyof any[]> extends never ?
        { [k: `${number}`]: T[number] } : Omit<T, keyof any[]>) : T;

The Cleanup<T> type turns the any type into the unknown type (since any really fouls up type manipulation), turns tuples into objects with just individual numericlike keys ("0" and "1", etc), and turns other arrays into just a single index signature.

Next:

type PrefixKeys<V, K extends PropertyKey, O> =
    V extends O ? { [P in K]: V } : V extends object ?
    { [P in keyof V as
        `${Extract<K, string | number>}.${Extract<P, string | number>}`]: V[P] } :
    { [P in K]: V };

PrefixKeys<V, K, O> prepends the key K to the path in V's property keys... unless V extends O or V is not an object. It uses template literal types to do so.

Finally:

type ValueOf<T> = T[keyof T]

turns a type T into a union of its properties. See Is there a `valueof` similar to `keyof` in TypeScript?.

Whew!


So, there you go. You can verify how closely this conforms to your stated use cases. But it's very complicated and fragile and I wouldn't really recommend using it in any production code environment without a lot of testing.

Playground link to code

jcalz
  • 264,269
  • 27
  • 359
  • 360
  • Excellent answer! It's already elegant enough to me. Thank you! – Val Sep 19 '21 at 17:48
  • Thanks for the elegant solution! I wonder if there is a way to improve it to infer the type of the value of the nested property? I've tried to do some simple manipulations with your code, but it didn't work out as I would like it to. `lookupWithReturn(foo, "a.0", 20) // OK` `lookupWithReturn(foo, "a.0", "20") // ERROR! string !== number` `lookupWithReturn(foo, "a.0", false) // ERROR! boolean !== number` – Mark Dolbyrev Dec 30 '22 at 08:09
  • It would be awesome if you could answer my question, so I will be able to give you bounty (I plan to add it in a couple of days, when I can): https://stackoverflow.com/questions/74956397/typescript-change-value-of-object-with-deep-key-and-typings?noredirect=1#comment132275074_74956397 – Mark Dolbyrev Dec 30 '22 at 08:11
  • @MarkDolbyrev Hmm, well you deleted that question, and I had already commented and suggested [this playground link](https://tsplay.dev/w1AG2w) as a possible approach. If that works for you I'd be happy to write up an answer explaining it... as long as the question is not deleted, of course. Not sure what else I can do here. – jcalz Dec 30 '22 at 16:03
8

In order to achieve this goal we need to create permutation of all allowed paths. For example:

type Structure = {
    user: {
        name: string,
        surname: string
    }
}

type BlackMagic<T>= T

// user.name | user.surname
type Result=BlackMagic<Structure>

Problem becomes more interesting with arrays and empty tuples.

Tuple, the array with explicit length, should be managed in this way:

type Structure = {
    user: {
        arr: [1, 2],
    }
}

type BlackMagic<T> = T

// "user.arr" | "user.arr.0" | "user.arr.1"
type Result = BlackMagic<Structure>

Logic is straitforward. But how we can handle number[]? There is no guarantee that index 1 exists.

I have decided to use user.arr.${number}.

type Structure = {
    user: {
        arr: number[],
    }
}

type BlackMagic<T> = T

// "user.arr" | `user.arr.${number}`
type Result = BlackMagic<Structure>

We still have 1 problem. Empty tuple. Array with zero elements - []. Do we need to allow indexing at all? I don't know. I decided to use -1.

type Structure = {
    user: {
        arr: [],
    }
}

type BlackMagic<T> = T

//  "user.arr" | "user.arr.-1"
type Result = BlackMagic<Structure>

I think the most important thing here is some convention. We can also use stringified `"never". I think it is up to OP how to handle it.

Since we know how we need to handle different cases we can start our implementation. Before we continue, we need to define several helpers.

type Values<T> = T[keyof T]
{
    // 1 | "John"
    type _ = Values<{ age: 1, name: 'John' }>
}

type IsNever<T> = [T] extends [never] ? true : false;
{
    type _ = IsNever<never> // true 
    type __ = IsNever<true> // false
}

type IsTuple<T> =
    (T extends Array<any> ?
        (T['length'] extends number
            ? (number extends T['length']
                ? false
                : true)
            : true)
        : false)
{
    type _ = IsTuple<[1, 2]> // true
    type __ = IsTuple<number[]> // false
    type ___ = IsTuple<{ length: 2 }> // false
}

type IsEmptyTuple<T extends Array<any>> = T['length'] extends 0 ? true : false
{
    type _ = IsEmptyTuple<[]> // true
    type __ = IsEmptyTuple<[1]> // false
    type ___ = IsEmptyTuple<number[]> // false

}

I think naming and tests are self explanatory. At least I want to believe :D

Now, when we have all set of our utils, we can define our main util:

/**
 * If Cache is empty return Prop without dot,
 * to avoid ".user"
 */
type HandleDot<
    Cache extends string,
    Prop extends string | number
    > =
    Cache extends ''
    ? `${Prop}`
    : `${Cache}.${Prop}`

/**
 * Simple iteration through object properties
 */
type HandleObject<Obj, Cache extends string> = {
    [Prop in keyof Obj]:
    // concat previous Cacha and Prop
    | HandleDot<Cache, Prop & string>
    // with next Cache and Prop
    | Path<Obj[Prop], HandleDot<Cache, Prop & string>>
}[keyof Obj]

type Path<Obj, Cache extends string = ''> =
    // if Obj is primitive
    (Obj extends PropertyKey
        // return Cache
        ? Cache
        // if Obj is Array (can be array, tuple, empty tuple)
        : (Obj extends Array<unknown>
            // and is tuple
            ? (IsTuple<Obj> extends true
                // and tuple is empty
                ? (IsEmptyTuple<Obj> extends true
                    // call recursively Path with `-1` as an allowed index
                    ? Path<PropertyKey, HandleDot<Cache, -1>>
                    // if tuple is not empty we can handle it as regular object
                    : HandleObject<Obj, Cache>)
                // if Obj is regular  array call Path with union of all elements
                : Path<Obj[number], HandleDot<Cache, number>>)
            // if Obj is neither Array nor Tuple nor Primitive - treat is as object    
            : HandleObject<Obj, Cache>)
    )

// "user" | "user.arr" | `user.arr.${number}`
type Test = Extract<Path<Structure>, string>

There is small issue. We should not return highest level props, like user. We need paths with at least one dot.

There are two ways:

  • extract all props without dots
  • provide extra generic parameter for indexing the level.

Two options are easy to implement.

Obtain all props with dot (.):

type WithDot<T extends string> = T extends `${string}.${string}` ? T : never

While above util is readable and maintainable, second one is a bit harder. We need to provide extra generic parameter in both Path and HandleObject. See this example taken from other question / article:

type KeysUnion<T, Cache extends string = '', Level extends any[] = []> =
  T extends PropertyKey ? Cache : {
    [P in keyof T]:
    P extends string
    ? Cache extends ''
    ? KeysUnion<T[P], `${P}`, [...Level, 1]>
    : Level['length'] extends 1 // if it is a higher level - proceed
    ? KeysUnion<T[P], `${Cache}.${P}`, [...Level, 1]>
    : Level['length'] extends 2 // stop on second level
    ? Cache | KeysUnion<T[P], `${Cache}`, [...Level, 1]>
    : never
    : never
  }[keyof T]

Honestly, I don't think it will be easy for any one to read this.

We need to implement one more thing. We need to obtain a value by computed path.


type Acc = Record<string, any>

type ReducerCallback<Accumulator extends Acc, El extends string> =
    El extends keyof Accumulator ? Accumulator[El] : Accumulator

type Reducer<
    Keys extends string,
    Accumulator extends Acc = {}
    > =
    // Key destructure
    Keys extends `${infer Prop}.${infer Rest}`
    // call Reducer with callback, just like in JS
    ? Reducer<Rest, ReducerCallback<Accumulator, Prop>>
    // this is the last part of path because no dot
    : Keys extends `${infer Last}`
    // call reducer with last part
    ? ReducerCallback<Accumulator, Last>
    : never

{
    type _ = Reducer<'user.arr', Structure> // []
    type __ = Reducer<'user', Structure> // { arr: [] }
}

You can find more information about using Reducein my blog.

Whole code:

type Structure = {
    user: {
        tuple: [42],
        emptyTuple: [],
        array: { age: number }[]
    }
}


type Values<T> = T[keyof T]
{
    // 1 | "John"
    type _ = Values<{ age: 1, name: 'John' }>
}

type IsNever<T> = [T] extends [never] ? true : false;
{
    type _ = IsNever<never> // true 
    type __ = IsNever<true> // false
}

type IsTuple<T> =
    (T extends Array<any> ?
        (T['length'] extends number
            ? (number extends T['length']
                ? false
                : true)
            : true)
        : false)
{
    type _ = IsTuple<[1, 2]> // true
    type __ = IsTuple<number[]> // false
    type ___ = IsTuple<{ length: 2 }> // false
}

type IsEmptyTuple<T extends Array<any>> = T['length'] extends 0 ? true : false
{
    type _ = IsEmptyTuple<[]> // true
    type __ = IsEmptyTuple<[1]> // false
    type ___ = IsEmptyTuple<number[]> // false
}

/**
 * If Cache is empty return Prop without dot,
 * to avoid ".user"
 */
type HandleDot<
    Cache extends string,
    Prop extends string | number
    > =
    Cache extends ''
    ? `${Prop}`
    : `${Cache}.${Prop}`

/**
 * Simple iteration through object properties
 */
type HandleObject<Obj, Cache extends string> = {
    [Prop in keyof Obj]:
    // concat previous Cacha and Prop
    | HandleDot<Cache, Prop & string>
    // with next Cache and Prop
    | Path<Obj[Prop], HandleDot<Cache, Prop & string>>
}[keyof Obj]

type Path<Obj, Cache extends string = ''> =
    (Obj extends PropertyKey
        // return Cache
        ? Cache
        // if Obj is Array (can be array, tuple, empty tuple)
        : (Obj extends Array<unknown>
            // and is tuple
            ? (IsTuple<Obj> extends true
                // and tuple is empty
                ? (IsEmptyTuple<Obj> extends true
                    // call recursively Path with `-1` as an allowed index
                    ? Path<PropertyKey, HandleDot<Cache, -1>>
                    // if tuple is not empty we can handle it as regular object
                    : HandleObject<Obj, Cache>)
                // if Obj is regular  array call Path with union of all elements
                : Path<Obj[number], HandleDot<Cache, number>>)
            // if Obj is neither Array nor Tuple nor Primitive - treat is as object    
            : HandleObject<Obj, Cache>)
    )

type WithDot<T extends string> = T extends `${string}.${string}` ? T : never


// "user" | "user.arr" | `user.arr.${number}`
type Test = WithDot<Extract<Path<Structure>, string>>



type Acc = Record<string, any>

type ReducerCallback<Accumulator extends Acc, El extends string> =
    El extends keyof Accumulator ? Accumulator[El] : El extends '-1' ? never : Accumulator

type Reducer<
    Keys extends string,
    Accumulator extends Acc = {}
    > =
    // Key destructure
    Keys extends `${infer Prop}.${infer Rest}`
    // call Reducer with callback, just like in JS
    ? Reducer<Rest, ReducerCallback<Accumulator, Prop>>
    // this is the last part of path because no dot
    : Keys extends `${infer Last}`
    // call reducer with last part
    ? ReducerCallback<Accumulator, Last>
    : never

{
    type _ = Reducer<'user.arr', Structure> // []
    type __ = Reducer<'user', Structure> // { arr: [] }
}

type BlackMagic<T> = T & {
    [Prop in WithDot<Extract<Path<T>, string>>]: Reducer<Prop, T>
}

type Result = BlackMagic<Structure>

Playground

This implementation is worth considering

  • Thanks to provide an alternative (also brilliant) way. I did some test with it, found it fails to detect array type. Is it possible to work with it? – Val Sep 10 '21 at 08:49
  • I tried the playground, I found a few not working caveats / edge cases in the code. for example, it won't work if I change "bar.baz" to array type (and also bar.gaz.zaz). And to `ExludeHighLevel` from `KeysUnion`, `Structure &` right after looks not so good to me (and may cause more exceptions). Any chance to improve it? – Val Sep 13 '21 at 04:15
  • @Val why do you think it does not work with extra array? What makes you think that? – captain-yossarian from Ukraine Sep 13 '21 at 06:40
  • it's by experiment. after I add `[]` after `baz`, (so it becomes `baz: { zaz: 2 }[]`) it disappeared from the final `Result` type. – Val Sep 13 '21 at 06:46
  • 1
    I'm not sure how do you want to handle arrays. Assume you have an empty array. Do you want me to generate: `baz.0`? Regarding the `{ zaz: 2 }[]`. Neither I nor typescript is unable to figure out what indexes are allowed and what are disallowed. How many indexes should I generate in case of array? From the other hand, it is much easier to work with tuples because TS is able to figure out the length – captain-yossarian from Ukraine Sep 13 '21 at 08:18
  • 1
    You're right, I should list down the use cases that I need, or there could be too many edge cases there. The handle of array / index signature by `$number` is good enough. I've added a list of use cases into the question, please have a look with it =) Thanks for the answer and patient explanation, I'll read into it after work (its complexity explode my head, oops) – Val Sep 14 '21 at 04:19
  • 1
    @captain-yossarian This is absolutely wonderful. I noticed that the final BlackMagic type has a typo, it is hard-coded to Structure :) – Thijs Koerselman Nov 29 '21 at 21:09
  • @ThijsKoerselman please see [this](https://github.com/react-hook-form/react-hook-form/blob/274d8fb950f9944547921849fb6b3ee6e879e358/src/types/utils.ts#L86). This implementation does not take into account keys of empty array, however the code is cleaner – captain-yossarian from Ukraine Dec 02 '21 at 09:32
  • @captain-yossarian Thanks I'll check it out! I'm currently getting swamped by other things so it might take me a while to continue with that project... – Thijs Koerselman Dec 05 '21 at 08:44