Given two dataframes A and B, and X amount of shared columns to merge on (B is merged to A), is there a way to pre-calculate/estimate in real-time how much memory the operation will require?
Asked
Active
Viewed 80 times
1
-
may be useful: http://stackoverflow.com/questions/110259 – behzad.nouri Dec 14 '13 at 02:29
-
Thanks and sorry I forgot to mention that the estimation is needed in real time. Couldn't find a reference to that in that link. – user2808117 Dec 14 '13 at 02:40
-
Then no, not really. Pandas would at least need to keep around column statistics to estimate that (and it doesn't). If your merges are reasonably consistent, you can probably make your own estimation function. – U2EF1 Dec 14 '13 at 07:31