Declaring path in pycharm not working

Question

I am using Pycharm and I created the project in a folder called collaborative filtering. I have some csv in a folder called ml-latest-small that I also placed in the collaborative filtering folder that has the .py file I am working from.

I am getting the following errors:

Traceback (most recent call last):
  File "/Users/usernamehere/Desktop/Machine Learning/Lesson 5/CollaborativeFiltering/movies.py", line 32, in <module>
    cf = CollabFilterDataset.from_csv(path, 'ratings.csv', 'userId', 'movieId', 'rating')
  File "/Users/usernamehere/Desktop/Machine Learning/Lesson 5/CollaborativeFiltering/venv/lib/python3.6/site-packages/fastai/column_data.py", line 146, in from_csv
    df = pd.read_csv(os.path.join(path,csv))
  File "/Users/usernamehere/Desktop/Machine Learning/Lesson 5/CollaborativeFiltering/venv/lib/python3.6/site-packages/pandas/io/parsers.py", line 709, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/Users/usernamehere/Desktop/Machine Learning/Lesson 5/CollaborativeFiltering/venv/lib/python3.6/site-packages/pandas/io/parsers.py", line 449, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/usernamehere/Desktop/Machine Learning/Lesson 5/CollaborativeFiltering/venv/lib/python3.6/site-packages/pandas/io/parsers.py", line 818, in __init__
    self._make_engine(self.engine)
  File "/Users/usernamehere/Desktop/Machine Learning/Lesson 5/CollaborativeFiltering/venv/lib/python3.6/site-packages/pandas/io/parsers.py", line 1049, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/Users/usernamehere/Desktop/Machine Learning/Lesson 5/CollaborativeFiltering/venv/lib/python3.6/site-packages/pandas/io/parsers.py", line 1695, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 402, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 718, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: File b'/Users/usernamehere/Users/usernamehere/Desktop/Machine Learning/Lesson 5/ratings.csv' does not exist

I am not sure what is wrong with the way I am declaring the path. Here is my code:

import torch
from fastai.learner import *
from fastai.column_data import *

path = '~/Users/usernamehere/Desktop/Machine Learning/Lesson 5'

ratings = pd.read_csv(path+'ratings.csv')
#print(ratings.head())

movies = pd.read_csv(path+'movies.csv')
#print(movies.head())

# Crete a subset for Excel
g = ratings.groupby('userId')['rating'].count()
topUsers = g.sort_values(ascending=False)[:15]

g = ratings.groupby('movieId')['rating'].count()
topMovies = g.sort_values(ascending=False)[:15]

top_r = ratings.join(topUsers, rsuffix='_r', how='inner', on='userId')
top_r = top_r.join(topMovies, rsuffix='_r', how='inner', on='movieId')

# pd.crosstab(top_r.userId, top_r.movieId, top_r.rating, aggfunc=np.sum)

# Collaborative Filtering - High Level

# Get a valisation indexes
val_idxs = get_cv_idxs(len(ratings))
wd = 2e-4
n_factors = 50

cf = CollabFilterDataset.from_csv(path, 'ratings.csv', 'userId', 'movieId', 'rating')

Edit:

Changing the path to path='ml-latest-small/' seemed to work.

@jhpratt In the middle it was my name so I removed it for the post — , Apr 21 '18 at 19:29
Ok, cool. I just edited in `usernamehere` so others don't get confused. — jhpratt, Apr 21 '18 at 19:30
@jhpratt Thanks, so besides that do you know what I am doing wrong? — , Apr 21 '18 at 19:31
You have: `path = '~/Users/usernamehere/Desktop/Machine Learning/Lesson 5' ratings = pd.read_csv(path+'ratings.csv')` How are you getting that, considering you're not concatenating a `/` after `path`? — Colin Ricardo, Apr 21 '18 at 19:31
@Colin I took away the tilde, but I get that from going into the folder where the .py file is located and clicked get info and copied everything after where: — , Apr 21 '18 at 19:41
No, I mean in your code, you don't add a `/` so why is it in the error output? — Colin Ricardo, Apr 21 '18 at 19:41
e.g. on lines 5-7 you're doing `'a' + 'b'` and getting `'a/b'`. How is that? — Colin Ricardo, Apr 21 '18 at 19:45

Colin Ricardo · Answer 1 · 2018-04-21T19:32:58.360

0

Here, the ~ means $HOME (read here):

which is why you end up with: /Users//Users/usernamehere/Desktop/Machine Learning/Lesson 5/ratings.csv' which is not a valid path.

edited Apr 21 '18 at 19:32

answered Apr 21 '18 at 19:23

Colin Ricardo

16,488
11
47
80

If I take away the tilde I still get the same error: FileNotFoundError: File b'/Users//Desktop/Machine Learning/Lesson 5ratings.csv' does not exist – Apr 21 '18 at 19:25
Are you sure you're meant to have the `//` in your path? – Colin Ricardo Apr 21 '18 at 19:27
No, it contained my name so I just removed it. – Apr 21 '18 at 19:28
Seems like you get error because your path isn't ending with slash. Try this: path = '~/Users/usernamehere/Desktop/Machine Learning/Lesson 5/'. It's exactly what Colin pointed out in comments – Alex K. Apr 21 '18 at 20:12
Because it's unlikely that your filename is 'Lesson 5ratings.csv'. You probably want to get file 'ratings.csv' from 'Lesson 5' folder, so it should be ''Lesson 5/ratings.csv' in the end of your path/ – Alex K. Apr 21 '18 at 20:14

score 0 · Accepted Answer · answered Apr 21 '18 at 20:05

Since you are on a *nix-based system, I would recommend you escape your spaces with \. Here's a simple test on Mac to show a scenario with and without escaping:

$ pwd
/tmp
$ mkdir "Machine Learning"
$ cd Machine Learning
-bash: cd: Machine: No such file or directory
$ cd Machine\ Learning
$ pwd
/tmp/Machine Learning

Declaring path in pycharm not working

2 Answers2