The only way to do that (at least in my previous experiences) would be to have the chart on the same level as the Route (with some conditionals...), if you don't need it on top you can change the route place ... the thing is that it doesn't give much flexibility for grid/flex but it works :P (in my case I had the selected graph always on top so it was perfect fit, you may try some conditional styling based on the location but I am afraid it would rerender)
<Route path="/charts" >
{history.pathname === '/charts/first' && <Chart data ..../>}
{history.pathname === '/charts/second' && <Chart data ..../>}
<Route path="/charts/first" >
<h1> Up you have the unRendered graph </h1>
</Route>
</Route>
*
Chart is a wrapper of any rechart graph(bar, pie ...) and has the REACT MEMO IMPLEMENTED
*
I spent days with this issue if you need more help just comment here