Histogram values of a Pandas Series


Question

I have some values in a Python Pandas Series (type: pandas.core.series.Series)

In [1]: series = pd.Series([0.0,950.0,-70.0,812.0,0.0,-90.0,0.0,0.0,-90.0,0.0,-64.0,208.0,0.0,-90.0,0.0,-80.0,0.0,0.0,-80.0,-48.0,840.0,-100.0,190.0,130.0,-100.0,-100.0,0.0,-50.0,0.0,-100.0,-100.0,0.0,-90.0,0.0,-90.0,-90.0,63.0,-90.0,0.0,0.0,-90.0,-80.0,0.0,])

In [2]: series.min()
Out[2]: -100.0

In [3]: series.max()
Out[3]: 950.0

I would like to get values of histogram (not necessary plotting histogram)... I just need to get the frequency for each interval.

Let's say that my intervals are going from [-200; -150] to [950; 1000]

so lower bounds are

lwb = range(-200,1000,50)

and upper bounds are

upb = range(-150,1050,50)

I don't know how to get frequency (the number of values that are inside each interval) now... I'm sure that defining lwb and upb is not necessary... but I don't know what function I should use to perform this! (after diving in Pandas doc, I think cut function can help me because it's a discretization problem... but I'm don't understand how to use it)

After being able to do this, I will have a look at the way to display histogram (but that's an other problem)

1
58
3/19/2019 2:33:34 PM

Accepted Answer

You just need to use the histogram function of NumPy:

import numpy as np
count, division = np.histogram(series)

where division is the automatically calculated border for your bins and count is the population inside each bin.

If you need to fix a certain number of bins, you can use the argument bins and specify a number of bins, or give it directly the boundaries between each bin.

count, division = np.histogram(series, bins = [-201,-149,949,1001])

to plot the results you can use the matplotlib function hist, but if you are working in pandas each Series has its own handle to the hist function, and you can give it the chosen binning:

series.hist(bins=division)
71
3/20/2019 8:37:34 AM

Inorder to get the frequency counts of the values in a given interval binned range, we could make use of pd.cut which returns indices of half open bins for each element along with value_counts for computing their respective counts.

To plot their counts, a bar plot can be then made.

step = 50
bin_range = np.arange(-200, 1000+step, step)
out, bins  = pd.cut(s, bins=bin_range, include_lowest=True, right=False, retbins=True)
out.value_counts(sort=False).plot.bar()

enter image description here

Frequency for each interval sorted in descending order of their counts:

out.value_counts().head()
[-100, -50)    18
[0, 50)        16
[800, 850)      2
[-50, 0)        2
[950, 1000)     1
dtype: int64

To modify the plot to include just the lower closed interval of the range for aesthetic purpose, you could do:

out.cat.categories = bins[:-1]
out.value_counts(sort=False).plot.bar()

enter image description here


Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Icon