How to preview a part of a large pandas DataFrame, in iPython notebook?


Question

I am just getting started with pandas in the IPython Notebook and encountering the following problem: When a DataFrame read from a CSV file is small, the IPython Notebook displays it in a nice table view. When the DataFrame is large, something like this is ouput:

In [27]:

evaluation = readCSV("evaluation_MO_without_VNS_quality.csv").filter(["solver", "instance", "runtime", "objective"])

In [37]:

evaluation

Out[37]:

<class 'pandas.core.frame.DataFrame'>
Int64Index: 333 entries, 0 to 332
Data columns:
solver       333  non-null values
instance     333  non-null values
runtime      333  non-null values
objective    333  non-null values
dtypes: int64(1), object(3)

I would like to see a small portion of the data frame as a table just to make sure it is in the right format. What options do I have?

1
37
11/13/2017 8:58:08 PM

Accepted Answer

In this case, where the DataFrame is long but not too wide, you can simply slice it:

>>> df = pd.DataFrame({"A": range(1000), "B": range(1000)})
>>> df
<class 'pandas.core.frame.DataFrame'>
Int64Index: 1000 entries, 0 to 999
Data columns:
A    1000  non-null values
B    1000  non-null values
dtypes: int64(2)
>>> df[:5]
   A  B
0  0  0
1  1  1
2  2  2
3  3  3
4  4  4

ix is deprecated.

If it's both wide and long, I tend to use .ix:

>>> df = pd.DataFrame({i: range(1000) for i in range(100)})
>>> df.ix[:5, :10]
   0   1   2   3   4   5   6   7   8   9   10
0   0   0   0   0   0   0   0   0   0   0   0
1   1   1   1   1   1   1   1   1   1   1   1
2   2   2   2   2   2   2   2   2   2   2   2
3   3   3   3   3   3   3   3   3   3   3   3
4   4   4   4   4   4   4   4   4   4   4   4
5   5   5   5   5   5   5   5   5   5   5   5
37
10/7/2017 7:36:28 AM

# Say you have a df object containing your dataframe
df.head(5) # will print out the first 5 rows
df.tail(5) # will print out the 5 last rows
# Note: it is similar to R

Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Icon