Using %%timeit results in permission error for large datasets

Question

I'm reading a 1.5 GB CSV file and performing group by operation using jupyter notebook. It works fine when I run the following -

path = r"C:\Users\Downloads\Answers.csv"
df = pd.read_csv(path, encoding = "ISO-8859-1")
results2 = df.groupby(['Score']).mean()

However, if I use %%timeit (ipython magic function) as follows-

%%timeit
path = r"C:\Users\Downloads\Answers.csv"
df = pd.read_csv(path, encoding = "ISO-8859-1")
results2 = df.groupby(['Score']).mean()

The cell runs successfully but I get the following permission error-

Future exception was never retrieved
future: <Future finished exception=PermissionError(13, 'Access is denied', None, 5, None)>
Traceback (most recent call last):
  File "C:\Users\shradh\Softwares\Anaconda\lib\site-packages\distributed\process.py", line 34, in _call_and_set_future
    res = func(*args, **kwargs)
  File "C:\Users\shradh\Softwares\Anaconda\lib\multiprocessing\process.py", line 124, in terminate
    self._popen.terminate()
  File "C:\Users\shradh\Softwares\Anaconda\lib\multiprocessing\popen_spawn_win32.py", line 119, in terminate
    _winapi.TerminateProcess(int(self._handle), TERMINATE)
PermissionError: [WinError 5] Access is denied
50.8 s ± 6.62 s per loop (mean ± std. dev. of 7 runs, 1 loop each)

Here's the CSV file I'm using.

Note- %%timeit is working fine for smaller CSV files and other python statements.

SInce the error occurs when you try to use `timeit`, I would think that it is related with `timeit` not the CSV file. — Asocia, Jan 20 '21 at 19:07
How you tried running a simple `print` statement with `timeit`? — Asocia, Jan 20 '21 at 19:08
@Asocia - `timeit` works fine with smaller CSV files. I've tried print statements as well. — Shradha, Jan 20 '21 at 19:11
I run your code without any issues. Try updating at the Anaconda prompt. `conda update --all`. — Trenton McKinney, Jan 20 '21 at 22:59

Using %%timeit results in permission error for large datasets

0 Answers0