How does one deduplicate a union

Question

Given a union type that contain duplicates:

type UnionWithDuplicates = ((...args:any)=>void) | ((...args:any)=>void) | ((...args:any)=>void) | ((a: string)=>number)

code

How would one deduplicate or flatten it so that resultant type is:

type UnionWithOutDuplicates = ((...args:any)=>void) | ((a: string)=>number)

The use case for this is simplify the type and make it easier to read and understand - even though in practice these types are probably identical.

I don't think there's a *great* way to do this without some inadvisable coercing of a union into a tuple or other related type. Something like [this](https://tsplay.dev/mbKQ3W) maybe. Does that meet your needs? If so I could write up an answer explaining; if not, what am I missing? — jcalz, Dec 06 '22 at 18:37
@jcalz: you are amazing ... thank you ... I thought this was going to not be possible, but I'm so glad you it doable! — TrevTheDev, Dec 07 '22 at 02:15
@jcalz: how does the LastOf type work in your solution? It is a strange beast as [this](https://tsplay.dev/wXzqJm) shows — TrevTheDev, Dec 07 '22 at 02:40
If you try to "inline" a [distributive conditional type](https://www.typescriptlang.org/docs/handbook/2/conditional-types.html#distributive-conditional-types) it won't be distributive anymore. Most of this comes from [this answer](https://stackoverflow.com/a/55128956/2887218) I'm happy to write up an answer here, but it's still not advisable to use it. It's not *horrendous* because we are immediately discarding the ordering, but I would be wary of it (e.g., if a future TS update breaks it, the TS team might not feel like fixing it based on this use case) — jcalz, Dec 07 '22 at 02:45
@jcalz: my type was an overflow of redundant duplicates, post this fix it simplifies to `((...args:any)=>void)[]` and I see no conceptual problems with your solution. You raise a valid practical concern and one that I have no control over. My choice is: leave it bad, or fix it, but the TS team may break it in the future. Thank you for the info re distributive types. Someone else will want to de-duplicate a union in the future - you've provide enough info to solve it - so you decide what's best. — TrevTheDev, Dec 07 '22 at 03:13

jcalz · Accepted Answer · 2022-12-07T19:14:20.263

⚠ WARNING ⚠

THIS TECHNIQUE IS POTENTIALLY FRAGILE AND UNLIKELY TO BE SUPPORTED BY TYPESCRIPT. USE AT YOUR OWN RISK

I think the only way something like this could work would be to give the union members some sort of temporary order, so you could do something different with two identical union members. If there were an order, then you could just keep the first union member of any particular type. Of course the final output would be independent of that order, but you need an ordering to figure out which one of the set of identical members to keep.

But unions are inherently unordered: the type A | B is the same as B | A, and if there is an underlying order, it's a pure implementation detail, and not meant to be discoverable to developer code. An operation that would discover such an order, or impose an external one, would not be supported. The request to convert unions to tuple types, at microsoft/TypeScript#13298 was explicitly declined.

Most operations we can perform on unions are purely symmetric and do not have different outputs that depend on order. For example, distributive conditional types let you break a union into pieces, but you have to do the same operation on each one without knowing anything about its location in the union. If you choose to keep an element of the union, you will end up keeping all elements of the union of the same type, which doesn't help us.

I did say most operations are purely symmetric. But not all are. You can turn a union into an intersection of functions, and an intersection of functions is equivalent to an overloaded function which inherently has an order (since the call signatures are consulted in order when resolving a call), and you can indeed, shamefully, tease the implementation-defined order from the compiler. See this answer for more details.

The final deduplication code is here:

type UnionToIntersection<U> =
    (U extends any ? (k: U) => void : never) extends
    ((k: infer I) => void) ? I : never

type LastOf<T> = UnionToIntersection<
    T extends any ? () => T : never> extends
    () => (infer R) ? R : never

type Dedupe<T, L = LastOf<T>, N = [T] extends [never] ? true : false> =
    true extends N ? never : Dedupe<Exclude<T, L>> | L;

So LastOf<T> grabs what the compiler considers the last element of the tuple, and then we perform Dedupe by adding the last element to our output, and excluding it from the input, and then recursing. Let's test it on your output:

type UnionWithDuplicates =
    ((...args: any) => void) |
    ((...args: any) => void) |
    ((...args: any) => void) |
    ((a: string) => number)

type Unduped = Dedupe<UnionWithDuplicates>;
/* type Unduped = 
    ((...args: any) => void) | 
    ((a: string) => number) 
 */

So, yay, it works!

But the techniques used are not really supported by TypeScript. If you run into an edge case where it behaves oddly, there's not much I can do about it. If a future version of TypeScript spoils the trick that makes LastOf work, the TypeScript team is probably not going to be very sympathetic if you open a bug that says "hey, my unsanctioned union-order hack is broken".

So, I wouldn't recommend putting this in any kind of production code I needed to support. Use it at your own risk.

Playground link to code

How does one deduplicate a union

1 Answers1

⚠ WARNING ⚠