how to remove auto indexing in csv when converting to pandas dataframe

Question

I'm converting my csv into dataframe using pandas but when printing it it prints index automatically I tried to use

index=false

and

dataframe.drop(['Unnamed 0']

but non worked this is the code

data = pd.read_csv('testlast.csv', sep='\t', index=False)
print(data)

the output looks like

that is how the sentence look like in csv

how to remove those auto generated numbers

What is reason for it? Because in pandas it is called index and cannot be removed. Or need remove if write to file like `df.to_csv(file, index=False)` ? — jezrael, Jan 07 '20 at 13:30
because I'm trying to find the frequency of the words and it will count the numbers for some reason as well — programming freak, Jan 07 '20 at 13:32
also use utf encoding to read the dataset `pd.read_csv('testlast.csv', encoding='utf8',index_col='false')` index_col=False can be used to force pandas to not use the first column as the index — Sebin Sunny, Jan 07 '20 at 13:34
OK, if use `print (df['sentences'].iat[0])` there is `0` values? If not, just ignore it. If yes, there is no separator tab in file. — jezrael, Jan 07 '20 at 13:35
but this one is just printing one sentence from the whole set not all of it — programming freak, Jan 07 '20 at 13:37
I added the csv picture as well but when printing the index appears which I dont want, I don't understand why pandas is adding it automatically @jezrael — programming freak, Jan 07 '20 at 13:40
pandas working this way, always generated some index values, here are defaul index values. — jezrael, Jan 07 '20 at 13:41
this time it says TypeError: unhashable type: 'list' @jezrael — programming freak, Jan 07 '20 at 13:43
@jezrael is there a way to save the created df into a txt or csv file with words and the frequency next to it like an easy go around way ? — programming freak, Jan 07 '20 at 14:18
perfectly worked, thank you, can you post that as an answer so I can accept it — programming freak, Jan 07 '20 at 14:31

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

1

Series and DataFrame in pandas have always index, reasons are docs:

The axis labeling information in pandas objects serves many purposes:

Identifies data (i.e. provides metadata) using known indicators, important for analysis, visualization, and interactive console display.
Enables automatic and explicit data alignment.
Allows intuitive getting and setting of subsets of the data set.

So for working with data you can ignore it (if not necessary for data processing):

df1 = (df['sentences'].str.split(expand=True)
                      .stack()
                      .value_counts()
                      .rename_axis('a')
                      .reset_index(name='b'))

Last for avoid write index to file add index=False parameter to DataFrame.to_csv:

df1.to_csv(file, index=False)

edited Jun 20 '20 at 09:12

Community

1
1

answered Jan 07 '20 at 15:02

jezrael

822,522
95
1,334
1,252

hey @Jezreal is there a way to remove the string punctuation inside your code ? – programming freak Jan 08 '20 at 12:44
@programmingfreak - You can check [this](https://stackoverflow.com/questions/50444346/fast-punctuation-removal-with-pandas) – jezrael Jan 08 '20 at 12:46
I tried this ``` df2=set([x for i in data['sentences'].str.split().apply(lambda x: [''.join([y for y in i if y not in string.punctuation]) for i in x]).tolist() for x in i])``` it gives me the list of words correctly without punctuation but it doesnt print the frequency as yours – programming freak Jan 08 '20 at 12:48
@programmingfreak - Maybe need `pd.Series(df2).value_counts()` – jezrael Jan 08 '20 at 12:50
it says TypeError: 'set' type is unordered – programming freak Jan 08 '20 at 12:52
1

@programmingfreak - hmm, then use `pd.Series(list(df2)).value_counts()` – jezrael Jan 08 '20 at 12:53

how to remove auto indexing in csv when converting to pandas dataframe

1 Answers1