The cost of Excel interop is the overhead of making a call into an out-of-process component. Work needs to be done to marshal the data across the process boundary. But that's a constant cost that depends on the amount of data. Less data is better but that's not a typical choice you have.
So the true cost is in the overhead of making the call in the first place. It is very substantial compared to the cost of making an in-process method call. Which takes a handful of cpu cycles, at worst. An out-of-process call requires two thread context switches, something that costs between 2000 and 10000 cycles at minimum, just for the processor overhead. You're on the high end of that one because the thread that needs to run inside Excel is owned by another process so the processor caches are junk. Add the latency of that thread responding, highly unpredictable.
So you've got a budget of tens of thousands of cpu cycles to make the conversion and get ahead. Spend them wisely. And of course, actually try it, nobody can give you a guarantee.