0

I am relatively new to data science and machine learning and I am currently working on my first project with a very large data set, with over a million rows and 88 columns`.

I am currently in the process of cleaning the data and trying to use features like data.isnull(), .sum() and data[data.isnull().values.any(axis=1)].head() but my Jupiter notebook file will only show the first ten and last ten columns.

Just looking for the best way to view the data or to be directed to any resources that may help.

Nuno André
  • 4,739
  • 1
  • 33
  • 46
Anthony Gonzalez
  • 3
  • 2
  • 2
    Does this answer your question? [Pretty-print an entire Pandas Series / DataFrame](https://stackoverflow.com/questions/19124601/pretty-print-an-entire-pandas-series-dataframe) – G. Anderson Feb 24 '20 at 20:06
  • You are using `.head()` in your code. that will give you first 10 columns only(by default). Also check if you are using `.tail()`, this will give only last 10. – Prashant Kumar Feb 24 '20 at 20:08
  • Welcome to stack overflow! It's worth noting that this is a display feature, not a bug. Do you really want to display a million rows every time you look at the data? That's a lot of scrolling, and there are better ways to verify that what you're doing is taking effect without visually inspecting the DF – G. Anderson Feb 24 '20 at 20:08

3 Answers3

0

You are using .head() in your code,

data[data.isnull().values.any(axis=1)].head().

This will give you first 10 columns by default. You can specify number of columns you want to see as argument to .head().

If you remove .head() from above line you will see the full dataframe.

Similarly, check if you are using .tail() somewhere. That will give you last 10 columns.

Prashant Kumar
  • 2,057
  • 2
  • 9
  • 22
0

You can change number of columns displayed by putting:

#without restriction
pd.set_option('display.max_columns', None)
Renaud
  • 2,709
  • 2
  • 9
  • 24
0

Try to using this 

   pd.set_option('display.max_columns', 30)

You can change the number according to your need. using None for full list.

DSAnup
  • 23
  • 7