I have multiyear data (14 years) with 15 mins internals and I am trying to make multiple plots comparing short term and long term CDF values.
I am computing the cumulative distribution function (CDF) of one variable (Energy values) for each month:
one cumulative distribution function for this variable, each month and each year of data (e.g. one for Jan. 2007, one for Jan. 2008, one for Jan 2009, ... and for each month.).
one long-term cumulative distribution function for each variable and each month (e.g. one for Energy values for January containing all daily values from 2007 to 2020 and similarly for the other months).
Now I want to see the results for each month. I want to plot both short term and long term cdf on a single plot and I need 12 plots, one for each month.
Can anyone guide me how can I do this using python. I am little lost on this.
So far I have reached until CDF functions.
import pandas as pd
import numpy as np
columns_to_read = ['DateTime', 'PLANT ENERGY MWh']
df = pd.read_excel(data, skiprows=0, usecols=columns_to_read)
df['DateTime'] = pd.to_datetime(df['DateTime'])
df.set_index('DateTime', inplace=True)
monthly_yearly_cdfs = df.groupby([df.index.month, df.index.year]).apply(lambda x: (x.rank() - 1) / len(x))
# Compute the long-term CDF for each month
long_term_cdfs = df.groupby([df.index.month]).apply(lambda x: (x.rank() - 1) / len(x))
Sample Data
Data looks like this: Data
df.sample(n=400, random_state=2023).sort_index()
DateTime,PLANT ENERGY MWh
1/3/2007 10:37:00 AM,1697.15308896009
1/13/2007 2:37:00 PM,2001.19395988835
1/18/2007 2:22:00 AM,0.0
1/26/2007 4:07:00 PM,1188.33577403658
1/29/2007 11:07:00 AM,2127.49307914357
3/7/2007 5:22:00 PM,334.448532482322
3/12/2007 6:52:00 PM,0.0
4/14/2007 1:22:00 PM,2233.7697588734
4/15/2007 5:22:00 PM,317.205656506989
4/28/2007 6:37:00 AM,488.824535055448
5/7/2007 7:52:00 AM,1275.23932269807
5/14/2007 10:07:00 AM,2239.92631025772
5/18/2007 7:07:00 AM,508.144515654771
5/21/2007 10:37:00 AM,2229.80792618988
5/24/2007 12:52:00 AM,0.0
6/19/2007 6:22:00 PM,7.88971689302022
6/23/2007 11:52:00 PM,0.0
6/26/2007 4:07:00 AM,0.0
7/6/2007 2:22:00 PM,864.618295502631
7/9/2007 5:52:00 AM,57.3243582278141
7/15/2007 4:52:00 PM,810.558329649709
7/30/2007 2:07:00 AM,0.0
8/15/2007 6:52:00 AM,371.352632081607
8/25/2007 9:22:00 PM,0.0
10/2/2007 4:37:00 PM,382.184867573695
10/14/2007 5:22:00 AM,0.0
11/21/2007 11:07:00 AM,423.203318513767
12/31/2007 12:37:00 AM,0.0
1/31/2008 3:37:00 PM,1609.05426883337
3/2/2008 7:37:00 PM,0.0
3/13/2008 11:37:00 AM,2266.95967476719
4/21/2008 9:52:00 PM,0.0
5/11/2008 9:22:00 AM,1829.80960610426
6/1/2008 11:37:00 AM,2245.96073624474
6/21/2008 4:07:00 AM,0.0
6/22/2008 11:07:00 PM,0.0
6/24/2008 12:07:00 PM,2259.96206542845
7/17/2008 5:52:00 AM,51.5973221015606
8/1/2008 4:22:00 AM,0.0
8/6/2008 6:07:00 PM,19.7647009486228
9/17/2008 6:07:00 AM,180.287417415929
11/8/2008 10:52:00 PM,0.0
11/15/2008 11:07:00 AM,545.990451766781
11/17/2008 9:22:00 PM,0.0
11/28/2008 12:37:00 PM,1719.4663563644
12/10/2008 1:52:00 AM,0.0
12/27/2008 6:22:00 PM,0.0
1/3/2009 5:52:00 PM,0.0
1/5/2009 7:22:00 AM,335.136147656082
1/17/2009 2:52:00 PM,1909.33403627444
1/25/2009 11:52:00 PM,0.0
1/28/2009 9:52:00 PM,0.0
2/25/2009 11:07:00 AM,1619.45530002807
3/16/2009 1:37:00 PM,2246.82725911847
3/18/2009 9:52:00 AM,2261.26512617316
3/29/2009 4:52:00 PM,701.565265876862
3/31/2009 9:07:00 AM,1046.46520763076
4/1/2009 1:07:00 PM,2064.36376234579
4/8/2009 5:52:00 AM,12.972388116731
4/17/2009 7:37:00 AM,692.358096524199
4/25/2009 10:37:00 AM,1854.46955717815
5/5/2009 10:52:00 PM,0.0
5/25/2009 8:22:00 AM,1697.85769768111
5/28/2009 8:52:00 PM,0.0
6/26/2009 4:37:00 AM,0.0
7/27/2009 10:22:00 PM,0.0
8/9/2009 11:22:00 PM,0.0
8/29/2009 7:22:00 AM,594.861006878339
9/17/2009 1:52:00 PM,690.529124708089
9/27/2009 1:07:00 PM,1329.97088539607
10/2/2009 9:22:00 PM,0.0
10/7/2009 4:52:00 AM,0.0
10/13/2009 9:07:00 AM,2103.38141683629
10/24/2009 1:22:00 AM,0.0
11/11/2009 10:22:00 AM,2251.0384034407
11/22/2009 9:22:00 AM,1342.49452643981
11/27/2009 2:37:00 PM,1813.79712527398
11/30/2009 9:52:00 PM,0.0
12/9/2009 12:22:00 PM,2189.95023445697
12/27/2009 6:22:00 PM,0.0
1/9/2010 7:37:00 AM,780.595688944353
1/13/2010 11:07:00 PM,0.0
1/14/2010 2:22:00 PM,1730.47123769195
2/13/2010 6:22:00 AM,2.5718700880953
2/21/2010 4:07:00 AM,0.0
3/8/2010 4:07:00 PM,1408.94018629172
3/21/2010 12:52:00 AM,0.0
3/21/2010 2:52:00 PM,2178.13783101468
3/27/2010 1:37:00 PM,2221.39248545497
4/6/2010 5:22:00 PM,384.551199703043
4/10/2010 10:07:00 AM,2231.68051156408
5/1/2010 8:22:00 AM,1539.37393822866
5/2/2010 2:07:00 AM,0.0
5/25/2010 3:22:00 AM,0.0
6/9/2010 10:37:00 PM,0.0
6/15/2010 10:22:00 AM,2221.49331162576
6/19/2010 3:22:00 AM,0.0
7/11/2010 2:37:00 AM,0.0
7/11/2010 5:22:00 AM,0.0
7/18/2010 1:07:00 PM,177.234158020317
7/20/2010 1:52:00 PM,1617.79156092834
8/4/2010 7:07:00 PM,0.0
8/15/2010 1:22:00 AM,0.0
8/23/2010 11:22:00 AM,2240.11034099836
10/5/2010 11:37:00 AM,1015.36807551329
10/11/2010 10:37:00 AM,2233.80329680852
10/23/2010 6:37:00 AM,364.89510315923
11/20/2010 10:37:00 PM,0.0
2/15/2011 11:37:00 AM,2264.61961430358
2/15/2011 5:22:00 PM,325.880976602414
2/27/2011 12:37:00 AM,0.0
3/14/2011 11:52:00 AM,2252.7096856352
3/15/2011 12:37:00 AM,0.0
4/6/2011 12:37:00 PM,1849.3643869394
4/20/2011 2:37:00 PM,2126.29455289231
5/12/2011 10:22:00 PM,0.0
5/13/2011 2:37:00 PM,2128.63365958903
5/29/2011 9:37:00 AM,1293.39647346765
6/5/2011 11:52:00 PM,0.0
6/13/2011 10:52:00 PM,0.0
7/19/2011 4:22:00 AM,0.0
8/15/2011 3:22:00 PM,906.023563345153
8/30/2011 6:37:00 AM,231.904817099589
9/23/2011 11:52:00 PM,0.0
10/12/2011 9:37:00 PM,0.0
10/31/2011 3:52:00 PM,930.09465715263
11/8/2011 3:07:00 PM,977.406939503701
11/13/2011 7:07:00 PM,0.0
11/23/2011 2:52:00 AM,0.0
11/30/2011 5:07:00 PM,27.298205385358
12/1/2011 3:07:00 AM,0.0
12/12/2011 10:37:00 AM,1263.45669467322
12/15/2011 12:37:00 PM,2246.92287639293
12/25/2011 4:52:00 PM,246.438785724009
12/31/2011 7:37:00 PM,0.0
1/18/2012 11:07:00 PM,0.0
2/15/2012 4:07:00 AM,0.0
3/3/2012 3:07:00 AM,0.0
3/14/2012 9:37:00 PM,0.0
4/11/2012 10:07:00 PM,0.0
4/26/2012 11:22:00 PM,0.0
5/13/2012 8:37:00 AM,1838.25097018283
5/14/2012 1:37:00 AM,0.0
5/15/2012 2:52:00 AM,0.0
6/4/2012 11:07:00 AM,1510.79288480949
6/22/2012 12:22:00 AM,0.0
6/30/2012 11:22:00 AM,849.138608900924
7/8/2012 12:07:00 AM,0.0
7/23/2012 3:22:00 AM,0.0
7/25/2012 10:52:00 AM,1051.97121880218
7/27/2012 9:37:00 AM,2069.59700360015
8/29/2012 11:22:00 AM,1344.68789088748
8/29/2012 12:22:00 PM,1320.87610528159
9/1/2012 1:37:00 PM,1616.38568269662
9/4/2012 5:52:00 AM,65.7313698615252
10/3/2012 11:37:00 AM,210.717982319463
11/3/2012 1:22:00 AM,0.0
11/25/2012 2:22:00 AM,0.0
12/3/2012 5:22:00 AM,0.0
12/11/2012 8:07:00 AM,1264.75635450344
12/15/2012 3:22:00 PM,1144.96069403523
12/29/2012 7:37:00 PM,0.0
1/14/2013 4:22:00 PM,493.200390367576
1/15/2013 8:07:00 AM,1017.8240973095
2/2/2013 4:52:00 AM,0.0
2/5/2013 11:52:00 PM,0.0
2/25/2013 3:07:00 AM,0.0
2/25/2013 8:07:00 AM,751.762960018433
3/8/2013 11:07:00 AM,2253.25776764632
3/25/2013 12:22:00 PM,1784.09991583734
5/1/2013 11:37:00 PM,0.0
6/3/2013 5:22:00 PM,385.627924290769
6/6/2013 1:22:00 AM,0.0
6/20/2013 3:22:00 PM,645.051467394196
6/29/2013 12:07:00 PM,1102.03805855432
6/30/2013 5:07:00 AM,0.0
7/2/2013 5:52:00 PM,136.203682460618
7/12/2013 7:22:00 AM,779.231336832705
7/20/2013 9:52:00 PM,0.0
7/31/2013 11:37:00 AM,1744.3572678315
7/31/2013 5:52:00 PM,100.03550973256
8/5/2013 4:07:00 PM,124.45735342751
8/31/2013 6:52:00 AM,440.389115692219
9/7/2013 6:37:00 AM,334.536499194091
9/26/2013 12:22:00 PM,1112.06280915502
9/28/2013 2:52:00 AM,0.0
9/29/2013 9:37:00 AM,2244.28200544644
10/5/2013 4:52:00 AM,0.0
10/25/2013 2:22:00 PM,1879.49706658889
10/30/2013 6:07:00 AM,91.9530243255773
11/22/2013 12:22:00 AM,0.0
11/23/2013 6:07:00 AM,16.8277390161346
11/28/2013 11:22:00 PM,0.0
12/4/2013 10:22:00 PM,0.0
12/8/2013 5:37:00 AM,0.0
12/14/2013 9:22:00 PM,0.0
12/22/2013 2:07:00 AM,0.0
1/30/2014 7:07:00 PM,0.0
2/13/2014 11:37:00 AM,2247.95944038382
2/13/2014 5:37:00 PM,105.199078964879
3/9/2014 2:22:00 AM,0.0
4/12/2014 12:22:00 AM,0.0
4/24/2014 8:22:00 AM,1732.60647545131
5/1/2014 7:07:00 AM,849.041096679454
5/14/2014 8:52:00 PM,0.0
5/21/2014 2:52:00 PM,1364.73057626358
5/24/2014 11:37:00 PM,0.0
6/2/2014 5:37:00 PM,31.3122284453889
6/8/2014 12:07:00 PM,2189.03708035709
6/16/2014 10:37:00 AM,1991.5811944257
6/19/2014 4:07:00 AM,0.0
6/24/2014 4:22:00 AM,0.0
6/29/2014 12:52:00 PM,1666.75943254743
7/5/2014 10:52:00 PM,0.0
7/10/2014 9:22:00 PM,0.0
7/13/2014 4:22:00 AM,0.0
7/18/2014 10:37:00 PM,0.0
7/24/2014 8:22:00 PM,0.0
7/31/2014 11:07:00 PM,0.0
8/7/2014 11:22:00 AM,2072.96745414091
8/21/2014 4:37:00 PM,722.767349369365
8/23/2014 8:07:00 AM,1500.44231651125
8/24/2014 4:07:00 AM,0.0
9/11/2014 4:52:00 AM,0.0
9/11/2014 9:22:00 AM,1869.30516413828
9/12/2014 8:37:00 AM,1710.73033450055
10/20/2014 5:52:00 AM,19.4205739999711
10/21/2014 6:07:00 AM,158.634771951938
10/25/2014 2:52:00 PM,1803.95482753973
10/29/2014 6:37:00 AM,162.919158884821
11/1/2014 2:07:00 PM,1970.73022590591
11/10/2014 3:07:00 PM,911.160075032066
12/12/2014 8:52:00 AM,1768.89711015088
12/29/2014 9:52:00 AM,1996.11311231376
1/17/2015 5:07:00 PM,116.278784714271
1/20/2015 3:07:00 PM,1515.88857937741
1/24/2015 6:07:00 AM,0.0
2/12/2015 9:52:00 PM,0.0
2/21/2015 4:37:00 AM,0.0
2/23/2015 11:07:00 PM,0.0
3/8/2015 8:07:00 AM,1441.39903094257
3/11/2015 12:52:00 PM,1360.3205244187
3/25/2015 12:22:00 PM,1613.28196521365
3/25/2015 9:52:00 PM,0.0
3/25/2015 11:22:00 PM,0.0
3/29/2015 6:37:00 AM,322.274273699383
4/8/2015 6:07:00 AM,106.374454642966
5/20/2015 5:22:00 AM,0.0
5/22/2015 6:22:00 AM,312.223354265717
6/14/2015 1:52:00 AM,0.0
6/27/2015 3:37:00 PM,1581.06172254746
7/27/2015 12:52:00 AM,0.0
8/4/2015 7:37:00 PM,0.0
8/24/2015 1:22:00 AM,0.0
8/24/2015 4:22:00 PM,399.294694339561
9/5/2015 11:07:00 AM,2215.84603206964
10/15/2015 10:52:00 AM,2229.46768062446
11/28/2015 7:52:00 AM,385.100835443082
1/2/2016 11:22:00 AM,2237.30043095103
2/14/2016 10:52:00 AM,2228.87777153332
2/24/2016 12:07:00 PM,2224.84413280813
3/1/2016 7:52:00 AM,805.881212801799
3/12/2016 7:37:00 PM,0.0
5/8/2016 5:37:00 AM,0.0
5/12/2016 4:22:00 AM,0.0
7/9/2016 8:37:00 AM,898.113346822966
8/29/2016 10:52:00 PM,0.0
9/22/2016 11:07:00 AM,1381.94315896979
10/4/2016 8:22:00 AM,1456.48740626995
10/20/2016 11:22:00 AM,1158.60905438133
11/16/2016 9:22:00 AM,785.844444160097
11/20/2016 2:52:00 AM,0.0
12/15/2016 2:37:00 AM,0.0
12/16/2016 11:52:00 AM,417.89954596086
12/28/2016 4:37:00 AM,0.0
1/13/2017 11:52:00 AM,2223.7194438618
1/14/2017 7:52:00 PM,0.0
1/15/2017 11:37:00 PM,0.0
1/25/2017 10:52:00 AM,2084.48118540304
2/3/2017 12:22:00 AM,0.0
2/5/2017 5:22:00 AM,0.0
2/18/2017 2:37:00 PM,490.975221341271
2/19/2017 9:07:00 AM,472.368352031344
2/20/2017 1:52:00 AM,0.0
3/8/2017 6:52:00 PM,0.0
3/10/2017 1:07:00 AM,0.0
3/31/2017 6:37:00 PM,0.0
4/7/2017 1:37:00 PM,2179.86519440919
4/29/2017 7:22:00 AM,992.085502332247
5/21/2017 4:37:00 AM,0.0
6/8/2017 11:37:00 PM,0.0
6/25/2017 2:07:00 PM,1802.34173117944
6/29/2017 8:07:00 PM,0.0
6/30/2017 12:52:00 PM,1988.17404515818
6/30/2017 7:22:00 PM,0.0
7/1/2017 1:37:00 PM,2171.13733947046
7/10/2017 1:52:00 AM,0.0
7/18/2017 5:22:00 AM,0.0
7/23/2017 4:37:00 PM,376.674126050257
7/26/2017 5:52:00 AM,10.6644177186485
8/5/2017 12:37:00 AM,0.0
8/17/2017 6:22:00 AM,212.032074357397
8/29/2017 12:22:00 AM,0.0
9/14/2017 6:22:00 PM,0.0
9/25/2017 2:07:00 PM,429.394672110146
9/26/2017 4:07:00 AM,0.0
10/1/2017 3:22:00 AM,0.0
11/5/2017 9:07:00 AM,84.7524925739651
11/20/2017 8:22:00 PM,0.0
12/27/2017 4:07:00 AM,0.0
3/16/2018 12:37:00 AM,0.0
3/23/2018 10:07:00 PM,0.0
3/27/2018 11:22:00 AM,2212.90966758228
4/19/2018 7:22:00 PM,0.0
4/21/2018 6:22:00 AM,291.299618596293
4/22/2018 5:37:00 PM,192.633704823958
5/10/2018 1:22:00 PM,2151.96101205623
5/17/2018 4:37:00 PM,889.112694648581
6/4/2018 6:37:00 PM,0.0
7/20/2018 3:52:00 AM,0.0
7/22/2018 12:37:00 PM,660.596136467228
7/24/2018 10:37:00 PM,0.0
8/10/2018 6:52:00 AM,215.745023484612
8/26/2018 1:07:00 AM,0.0
8/27/2018 9:37:00 AM,1631.35546624633
9/3/2018 6:37:00 PM,0.0
9/14/2018 8:07:00 AM,179.988860609959
9/19/2018 9:22:00 PM,0.0
9/22/2018 8:22:00 PM,0.0
10/9/2018 3:22:00 AM,0.0
10/17/2018 1:52:00 AM,0.0
10/24/2018 8:22:00 PM,0.0
11/2/2018 4:52:00 PM,225.983282567867
11/18/2018 2:07:00 PM,2100.69577404248
12/16/2018 8:37:00 PM,0.0
12/17/2018 9:37:00 PM,0.0
1/20/2019 8:07:00 AM,1166.56731508099
2/10/2019 7:37:00 AM,815.777148505311
3/9/2019 11:07:00 AM,2091.1494293728
3/21/2019 4:22:00 AM,0.0
3/23/2019 8:37:00 AM,1741.87476037795
3/28/2019 2:52:00 AM,0.0
3/28/2019 4:22:00 AM,0.0
4/10/2019 5:37:00 AM,0.0
4/13/2019 5:37:00 AM,0.0
4/14/2019 6:22:00 AM,115.501506903225
4/28/2019 12:52:00 PM,2155.65013095046
5/3/2019 8:22:00 AM,1601.18794707181
5/6/2019 1:37:00 PM,2218.31975219021
5/9/2019 1:37:00 AM,0.0
5/26/2019 2:07:00 PM,1159.95842527914
5/28/2019 5:22:00 AM,0.0
6/16/2019 4:52:00 AM,0.0
6/29/2019 7:37:00 PM,0.0
7/13/2019 11:37:00 PM,0.0
8/27/2019 5:37:00 PM,70.297367492205
8/28/2019 6:22:00 AM,174.444270586398
8/29/2019 11:52:00 PM,0.0
9/17/2019 7:52:00 AM,849.831780980663
9/29/2019 1:37:00 AM,0.0
9/30/2019 10:52:00 AM,1960.68727802557
10/6/2019 12:07:00 PM,2195.98004768548
10/14/2019 4:52:00 PM,384.731147656982
10/15/2019 11:07:00 AM,2185.13478503396
10/23/2019 7:52:00 PM,0.0
11/8/2019 5:22:00 PM,0.0
11/22/2019 2:37:00 PM,272.810073966629
12/26/2019 8:22:00 PM,0.0
1/10/2020 7:52:00 AM,644.628312963384
1/27/2020 11:22:00 AM,2207.87188724061
1/31/2020 3:07:00 PM,1377.46407332282
2/1/2020 11:37:00 AM,2214.19474593128
3/4/2020 1:07:00 AM,0.0
3/9/2020 9:07:00 PM,0.0
3/19/2020 2:37:00 PM,2165.83856381109
3/23/2020 7:07:00 PM,0.0
4/6/2020 10:22:00 PM,0.0
4/21/2020 5:37:00 AM,0.0
6/1/2020 6:37:00 AM,541.601057136189
6/3/2020 4:37:00 PM,692.598128405066
6/5/2020 11:07:00 AM,2159.80323674935
6/15/2020 5:22:00 AM,0.0
6/27/2020 8:52:00 AM,1801.0129423256
6/29/2020 12:52:00 PM,2171.07665746352
6/30/2020 10:37:00 AM,2176.04432414765
7/4/2020 2:52:00 AM,0.0
8/9/2020 7:37:00 PM,0.0
8/18/2020 3:52:00 PM,1179.07868394892
8/22/2020 9:52:00 PM,0.0
8/28/2020 5:37:00 PM,83.867067156831
9/2/2020 5:22:00 PM,289.017298139724
9/3/2020 3:22:00 AM,0.0
9/5/2020 7:07:00 AM,335.582724269764
9/11/2020 2:07:00 PM,2156.50505885321
9/24/2020 1:52:00 PM,1291.82123782733
9/29/2020 9:52:00 PM,0.0
10/24/2020 3:22:00 AM,0.0
10/25/2020 5:22:00 AM,0.0
11/16/2020 10:37:00 AM,2175.28229042033
12/7/2020 5:07:00 PM,11.7636367947525
